Expansion cards and saved games

My xbox is saying low memory. I need an expansion card. But how does it work if you have some games saved on an expansion card already? Do you have to put in the right card for each game? Does constant removal and reinsertion cause issues?