Server transfer or no?

Looked on iron forge and says the raiding population is 2900 on whitemane. The top 3 population servers are 6500-7500.

I was wondering if I’m underestimating the server size I’m on (whitemane) or if I should plan to pay for a transfer to a bigger server if I want to play in a more competitive guild this go around.

Any info is appreciated thank you!