How long will it take to gain the trust back from family??

This is my first time quitting. My brother told me about a year ago that he'd never let his son spend the night at my house. Everyone is different. I am wondering if I should test him and ask for a sleepover or give it more time. It really sucks that someone would not trust their kid with me especially when I have 3 of my own. Anyway, feeling sorry for myself I guess.