Is it safe to wear my amulet at home?

I'm getting into amulets, since carrying my staff with me is cumbersome and gets me looks from the common folk. I don't cast at home, but i'm worried about the residual energy in the crystal (yes i bought a cheap amulet). Would that affect my humors?
Any amulet or pendant users with advice to offer?