How to get through a trip with people I no longer want to travel with.

Has anyone ever in the below situation and what did you do? I will have a trip with a friend and his family in Japan next month. We did not plan anything when I booked the flight and hotel. During the planning process, this friend only prioritizes and picks the spots he wants and does not want to do the spots I want even thought they only take him 3 hours for the ones I hope to visit. He asked me to do it by myself even though I insist him to go with me because I don’t feel comfortable going by myself. Now I don’t feel like I want to go on this trip because I feel disappointed. However my bookings and flights are non refundable. How should I get through a trip with people I don’t want to enjoy spending the time with?