Is working in a setting you actually want a realistic goal?

I understand the answer to this question varies geographically, but generally speaking, is it realistic to think that I could get a job in a setting/environment I actually want? I see posts on here that say something along the lines of 'The great thing about PA is you can work where you want, however many hours you want!' The reason this is important to me is because the work/home balance is something I value above all. I do not want to absolutely dissolve my life in medicine or else I would just become a MD. My long-term goal is to make it to a structured Outpatient 8-5 M-F clinic. Have you had much problems actually finding a niche that gave you what you wanted most out of your career? I understand I will likely have to sacrifice some pay to get the schedule I want, but in my personal life my family time is what I value most. Can anyone shed some light on their experience?