How did the scourge virus hit *all* Viltrumites?

First off, I only watched the show and didn't read the comics - so bear with me if it is clearly explained in the comics!

When Thadeus hit Planet Viltrum with the virus, the Viltrumites were already busy conquering civilizations all over the galaxy, correct? And with an undiminished force no less. Thus, I would expect tens of thousands, if not millions, of the Viltrumites to be off-planet due to the war effort.

So, how come every last Viltrumite was eventually infected with the virus? Wouldn't the on-planet Viltrumites have warned their off-planet people? And even if not, how probable is it that every last Viltrumite returned home at some point from their galaxy-spanning empire/war?

My point being is, I find it highly unlikely that only around 50 Viltrumites survived the scourge, because there should be immense numbers of them distributed throughout the Viltrum empire.

Looking forward to see your couter points!