Do I have to start a new pack of pills?

I've taken the combo pill due to PCOS (as recommended by my doctor) for the past 6 years, but I'm honestly tired of the pills. I spoke to my doctor the other day about not wanting to take them anymore, and she told me to end my current pack and not to start a new one for about 3 months to see if I can go off the pill without my PCOS becoming unmanageable.

So anyways, I ended the pack and I'm currently on day 4 of the sugar pill. Problem is, I had sex with my bf this morning (no condom) and he came inside me. I don't want to risk pregnancy rn, I know i'm protected during the sugar pill week but will it still count if I don't start a new pack? Or should I just start a new one anyway and end BC on my next cycle?