Moved house and cat's behaviour changed

I have an almost 3yr old indoor cat. He's male but was neutered quite young. He's always occasionally been a bit aggressive when playing but could be easily redirected. It was always just me and him in my old flat, when my boyfriend would visit he would be skittish but was slowly getting used to him.

I moved in with my boyfriend in late April. He has an adult daughter and two teenage sons. They've been told to leave the cat alone unless he instigates play or fuss to avoid stressing him out.

However, he's now been swiping at my boyfriend and his daughter (he's mostly ok with the two sons) and hissing. He's never hissed at anyone before so I have no clue what to do! I know it's early days but it feels like he's got worse not better.

I've got a calming plug in which I feel works to an extent but I don't know what else I can do to help him. He's still super friendly with me and is more relaxed when I'm in the house so I don't see the worst of it myself.

Has anyone else had this experience and found anything that helped? Or can just give me some hope??