PS3 crappy control problems?

So I understand that playing games on anything other than a PC is wrong, I'm going to hell, and I don't deserve to have fun. But I've been playing Skyrim on PS3 and having a pretty serious problem with the favorites menu.

Whenever I'm fighting something the favorites menu will pop up randomly. It isn't terrible sometimes because it's like a random pause. Disruptive, but not damaging. However, if I'm holding down X or any trigger or the L stick, then random things will equip or unequip. I'll end up with madness in one hand and a shield in the other when half a second before I'd had my arm back to smack a dragon with an axe.

Is anyone else having this problem? It's extremely frustrating and my parents are getting sick of hearing me swear loudly all the time.