Manager is forcing me to see my coworkers?

I don’t know how y’all feel about seeing your coworkers as patients but it makes me really uncomfortable when it’s chronic management (as opposed to one-off things). Especially when the specialty involves seeing their genitals and talking about their sex lives. A lot of my coworkers are patients at the clinic. I have been able to opt out of seeing them for a while but recently my boss, and his boss to whom I elevated it, is saying I have no right to opt out of seeing them. This is due to a shared-panel model where they see whoever is on-staff. I believe any provider has a right to say no to a dual-role relationship. Am I being too stubborn?