Panormos for a week?

I’m looking to visit crete for a week in early June with my wife our 9 month old. We’re looking for somewhere a little quieter than the cities, with a few restaurants. Is Panormos suitable for this? We’ll have a car so can travel for day trips etc.

Thanks for the help!