Randoms joining invite only parties?

So for the past few hours a friend and I have been playing Destiny 2 together. Just us two. The party was started through a direct invite so it was invite only from the start, but on two occasions we had people join for a few seconds and leave. Neither of us knew who they were and after the second intrusion we capped the party to 2.

Has anybody had similar situations happen? I certainly haven't ever and felt it may have had something to do with the psn issues today(though I really can't think of an actual reason they would be connected) but I also assume whatever is allowing people to do this is more nefarious in mature.