When to transition from pee pads to outside?

So Ollie (almost 19 weeks already, where has the time gone?!) has been accident free in my apartment for 3 weeks now. We've been spending quite a bit of time at my parents (my dad is ill) and he's had accidents there but I chalk it up to it being a new place... though he's getting better every time we go there and only had 2 accidents in 2 days the last time we were there.

I want to eventually transition him to also go pee outside. I'm not entirely sure I want to take the pads away entirely (which seems to be how most people I know here do it... pads for when you're not home, but outside otherwise) but not entirely sure when to transition. I know he's not fully housetrained yet because it's only been 3 weeks without an accident, but is there a good time to start transitioning?