Crystal Dwelling and underground starts

Am I alone to feel like Crystal dwelling is impairing me when I am playing underground start ?

It takes so much place and 90% of the time I have to delay my expansion, my outpost network, everything is slower.

why does he have to spawn in the middle everytime ?