Parking across an unused drive

I should start by saying I have been a bit stupid.

Parking near me is rubbish, so you park where you can. There is a house with a gate and a drive and in the 4 years I have been here I have never seen a car on the drive or the gate open. In fact until recently a car would not fit due to some large plant pots. There is a drop curb and a white line but there are regularly cars parked there so I park there if it's to free

Today I was beeped at a man told me off, I was blocking him from getting on his drive. I apologised, popped my kid back in the car and moved (ended up closer to my house). Went to check it I had been stupid and he did park on his drive but he was in front of it. Seems he doesn't used the drive, except as a private space and it's likely that where I assume people park there it's just his car and I haven't paid attention.

It made me think, if you don't use your drive is parking across it fair game or is it essentially a reserved space for you?