Good feelings triggering fear causing anger

I was working on doing something I enjoy doing and I hate how sometimes when that happens it feels good but the good feeling triggers an intense sense of fear and I have to slow down. I want to just push through it but the fear just makes it more difficult to continue my hobby and I don't want to associate terror with something I love. It makes me so, so angry though, and I don't know exactly who or what I'm angry at but having to restrain myself and/or redirect activities because of my fear makes me feel intense rage. I shouldn't have to feel scared anymore, I'm somewhere safe, it's so frustrating. I have no idea what to do with all this anger, though.