Does the Lord of Crime organization live under one roof?

We know that the Moriarty manor has a floor dedicated to having a servants room and changing rooms as shown in the manga and anime. But in the manga there is also a scene where Fred comes to inform William about the van that kidnaps children at night but is hesitant to visit the house so late? If he lived there, he should be far more comfortable right? Other than that- the manga chapter with the bit where they show the floor is called "The Adventure of the Four Servants" who are basically Fred, Moran, James and apparently Louis? So in short I want to know, does the Lord of Crime organization live under a roof almost like a family or do they just meet up when needed?