How do I stop being scared to hold gem?

New player here, started playing shortly after the compendium was available, calibrated at 2.2k for season mmr, and 2.3k for regular mmr.

I never want to hold gem, unless I am a kotl/ns with aghanims. I am always worried im going to get blink-stunned and lose it. Even if we're ahead 20 kills, I think dying and losing it would balance out the gold difference. What can I do about this?