Assistance with world building

I have a map rivers and mountains , capitals, populations, regional names for the source of my magic system and how the magic its self effects the different areas, i have main cast mostly and a chapter by chapter plan with what i want to do , however i am struggling with chosing in how much detail i should do the world building , for example do i make every city and villages or forrests i plan to focus on the journey and then as i go i write about the rest of the world with whatever character i think suits the moment, should i make extra cities in key points in the map that i can then to create characters that serve the plot and can perhaps re appear or i make characters that never appear in the story but are the reason the plot moves. Or i make have things go paralel to the story that then effect directly and indirectly in other countries maybe a famine due to a war causes by characters that will meet the mc in book 2( the fact i am thinking of book 2 when i have writen nothing is ironic). I find my self stuck. excuse my rambling honestly any sort of guidance would be much appreciated.