In the second picture is one of my babies, Puck. We rescued him along with his 2 sisters and his brother from outside this past summer, and by now they're all mostly used to us. Dom and their sister Rhea (not pictured) are SO loveable. Rhea follows us around the house and LOVES being pet, and Dom is super playful and inquisitive, and loves to curl up next to you. Casca (the cat on the far left) is a little skidish, but she loves pets. Puck however has moments where he'll come to me(only me he's terrified of my parents and sister for some reason) and he'll rub on my legs and when I bend down to pet him he lays and stretches out and purs like crazy. Unfortunately most of the time he runs! I'll reach down to pet him, and he just darts away. At first I would just leave him be when that happened, but was told he might be trying to play with me, so i tried going after him once, but i could tell he was getting scared immediately so i'm back to just backing off. 
He still has his moments where he runs to me whenever i come out of my room and he gets all lovey like I mentioned before, but most of the time he just runs away. I grew up with cats, i'm 30, and we never had to work this hard to gain their trust. We have 2 older cats Willy and Barney, and they just loved us from the start! Barney especially is very lovable. I love laying by them and having them lay by me, but i can't even do that with this lil guy, and he's so adorable 😭 fell in love with him the second we found them (they were 12 weeks old) and wanted him to be my lil side-kick because as loveble as the older 2 are, they aren't the type to let you hold them or sit on your lap while you watch tv like some of the cats i had as a kid were, and I miss it 😭
Anything I could do to encourage him to warm up to me?

Why is he so scared most of the time?