Alternative to kyoto?

Traveling with a 3 year old and 1 year old first two weeks of November. We are starting in Tokyo for five days and then heading to Nikko/Tochigi area for 4 days for autumn foliage, nature, onsen, cultural experiences. The last 4 days we were going to spend in Kyoto. Here is my dilemma. I am really trying to see a more hidden gem part of Japan. I just dont want to be stuck in huge crowds as it takes away from the experience. Yes I know Tokyo is very touristy but we are avoiding most of the major tourist areas and taking some smaller day trips. We are big foodies so I am looking forward to some of the tokyo restaurants. If we dont do Kyoto I will be sad to miss the beautiful japanese gardens and tea rooms. I was looking at either Gifu, Nagano, or yamagata for our third stop on our trip but I can't decide which one makes the most sense. Has to be toddler friendly, beautiful scenery, would love if there is a bit of tea culture and japanese gardens (unless we should just focus on that in Tokyo), good food and overall some good cultural experiences. any of these feel different than Tochigi and check off those boxes? or is there somewhere else you recommend that is within 3 hours (by shinkansen) of Tokyo? Thank you!!