Trouble changing Heng's loyalty

Ok so, I'm settled in the Great Desert and currently trying to extinct the UC. I killed the emperor and Heft is now Rebel Swordsmen, killed Sho-Battai's noble which is now Empire Peasants... but I also tried for Heng, which was half destroyed and with no noble in sight the first time I came.

I killed everyone flagged UC hoping it'll do something, and left the city with just some Empire Peasants in it, but it desperately remains UC. I left twice and came back, first because I believed I left one or two guys alive, next to see if there was any change, and it's now been a game month since everyone from the UC is dead there.

The city is now a ghost town with only 2 Peasants left (extra question: is it normal that it's not refilling ? first playthrough so IDK).

So, am I missing something? Did I do something wrong ? I'd really like to fix that.