Semi-ferals homed

Hey there. I spent a few years befriending some feral cats at my job. Several got friendly enough to pick up and snuggle; I brought one home first(Phantom), and then a yearish later I brought home his sister and nephew(Ifrit, and Sodo). I've known all 3 of these cats since they were kittens, known their parents and grand parents.

Phantom adjusted super well; you'd hardly know he was feral(and I mean he started off hissing like he'd send me to the hospital, to coming running for snuggles and following me around everywhere.) He lives in the main-house with me, my 2 dogs, 2 roomies, 1 roomy's two cats, and occasionally the other roomy's 2 dogs when he brings them over. He doesn't particularly like anyone but me, but he'll tolerate the other animals.

Ifrit was friendly when I brought her home, Sodo was skittish but also friendly. I was able to hold him a little, but he preferred to be close and not touched. Ifrit and Sodo now live in what is essentially their own house separate from the main house(a converted pig barn probably not suitable for human living but more than decent enough for 2 cats.) Inside this house which could pass for a normal sized single person home, I had a catio so they could adjust and realize they were safe. (I figured letting them have their own big space would be FAR less traumatizing than being with the mess in the main-house)

Initially, Ifrit would accept pets and get a little excited about it, she would purr and rub on me. Sodo would hide all the time*(and I mean all the time to the point where he burnt off his fur in the insulated electric warm box I gave them for winter. I see my mistake now in leaving that on for him and it wont happen again.)* SOMETIMES he'd come out for pets when he saw his mother was cozy with it.

I see/saw them daily, feed and water them regularly, check up on them, give them treats and toys, I speak to them exactly how I did before I brought them home, even bring Phantom out sometimes to show them I'm safe(Ifrit chills out a little when she sees Phantom, Sodo chills out when he sees his mother chill). I've let the catio stay open, and left their food in there so mice can't get to it, and now they have full run of the house(because Ifrit was getting really really fat and needed exercise).

I brought these two home in November and they were a LITTLE skittish, but now they run away and tremble as if I've beaten them to an absolute pulp and exposed them to the world's worst horrors possible. (legit their own mansion, free of rent.)

The only factors I can come up with aside from removing them from the area(because they were in danger of being removed there by pest control), is that they lost their trust in me when I brought them to get flea and wormer medication. I had them fixed long before with TNR and they still liked me then. Potentially once because I had to bring the dogs into the same space when the house was being sprayed for fleas..? She looked absolutely feral that day but I didn't think much of it. Truth be told, she'd already begun disliking me at that point.

They have tons of spaces to hide, plenty of things to climb; I admit that when it gets windy, the building can be eerie sounding but there's.. not much I can do about that.

How do I earn their trust back so I can eventually move them into the main house when I no longer have housemates? What did I do wrong to make them so terrified of me?