First time 20 day Japan itinerary

Looking for some advice on a 20 day trip, flying into Osaka and out of Tokyo in Sept/Oct. We are debating on 3 nights in Kanazawa with a day trip to Shirakawa-Go, or 2 nights in Kanazawa, Takayama 1 night.

We're currently wondering if it's worth the round about trip from Takayama to Tokyo, and how much time will be lost travelling between cities. Would people recommend going? Is this a good amount of time to spend in each city with enough down time as well?

------------

Osaka (5 nights)

- including day at universal studios

- day trip to Hiroshima + Himeji Castle otw

- 3 days to explore the city

Kyoto (5 nights)

- day trip to Nara

- 4 days to explore the city

Kanazawa (3 nights)

- day trip to Shirakawa-Go

- 2 days to explore the city

Tokyo (5 nights)

- day trip to Hakone

- 4 days to explore the city

OR (between Kyoto and Tokyo)

Kanazawa (2 nights)

- 2 days to explore the city

Takayama (1 night)

--------------

I really love the idea of breaking up the city travel with somewhere more rural, but I'm worried it's a bit of a packed itinerary with a lot of extra travel time.