Does fear stop some perks from working? Or is the restful perk bugged?

I have the restful perk(faster stamina regeneration), but for some reason, whenever I get the slightest bit scared, my stamina will stop charging faster like it should.

Is this supposed to happen?