TERF beliefs require one to think that women are privileged

A thought occurred to me today. So many TERFs and other transphobes act like trans women are taking away some kind of societal position that rightly belongs to women. This is (obviously) wrong for the reason it fails to recognize trans women as women, but I think there's something deeper to it. It ultimately comes from the mindset that women are privileged.

Consider the idea of "stolen valor". Stolen valor is what it's called when someone who's not a veteran pretends that they are to get the social benefits that are normally conferred on veterans, whether they're official (like discounts) or unofficial (like people just being more kind/generous/thankful towards you). People care about stolen valor because they see it as taking the position that rightfully belongs to a group that you are not part of. They see it as benefiting from the advantages you weren't meant to benefit from.

Contrast that with if someone, for example, someone lied and said that they were someone who litters a lot, even though they don't. Or some other kind of person that people generally dislike and treat worse. People might be upset that you are part of that group, but nobody is going to feel the need to be "on guard" against anyone saying they're part of that group when they aren't. If someone finds out that you weren't part of that group when they thought you were, they wouldn't get mad at you for trying to take something that's not yours. Because if there's no benefit to being seen as part of this group, people don't really see it as a problem to choose to be seen as part of it.

Now, between these two scenarios, how do TERFs act towards trans women? Much more like the first one, because being a TERF requires seeing women as a privileged group.