Face swells when I don't purge

I purge every day and there's a certain baseline of puffiness that I've gotten used to, but on the rare occasion that I manage a day without purging, my jaw swells like crazy and it looks awful. It's almost demotivating me from stopping because I know my face is going to look insane until I purge again