Guitars work fine, Controllers do not. PS2 Slim. Suggestions?

The title says most of what information I have. I have 2 Guitar hero guitars that both work with any guitar hero game I play but neither of my controllers work 100%. Neither of the controllers are new but I just got one from Gamestop and even it doesn't work. It seems like the button configuration might be off? Sometimes when I hit the analog button, it seems to act like "X" or if I hit square sometimes it will back out.