Why does the task bar freeze whenever something is happening with the network?

When I click the button at the top of the network panel in the task bar to turn off the network, the entire task bar freezes until it's done. I've had issues with my network driver or something taking a long time to start, and I'll have an unresponsive task bar for 2 minutes post boot. Seems this should not be tied together like that.