Can i remove doors that are not 'mine'?

I found an abandoned base and moved in it, it was fully empty expect for a wooden door that has a lock on it and left wide open by its previous over. I can build around the area and all, ik i can build a 3rd door outside and the problem is solved, but if someone's camping outside while i want to build, my base will be wide open for them.