Should I keep my gemstone rings on when I wash my hands?

Hi! I recently got a new ring with a zircon gemstone. I always take it off when I'm washing my hands at home but I'm worried that if I take it off at work or somewhere else, I'd lose it. Last year I got a ring with a zultanite gem and I washed my hands with it as normal but the soap residue that built up on the bottom part of the gem made the gem seem more...dull? Definitely not as shiny as my new zircon ring. I'm worried that if I wash my hands with my new ring still on, it'll end up looking dull like that. I know I could get it cleaned but the zultanite gem became dull super quickly. Any thoughts on this? Thank you in advance! :)