Frustrated with potty issues bc of deer

My lab puppy is 4.5 months and has been pretty consistent with potty training. He has one accident every 2-3 days at this point which is huge progress.

We hit a roadblock a few weeks ago specifically during our final potty around 8 pm. We get a bunch of deer in our yard and when we go out and there are deer his listening skills become non existent and he refuses to potty. The deer are so used to people here that they aren’t afraid at all and don’t run with loud noises which is very annoying to me.

I’m more annoyed at the deer than my pup bc it’s natural for him to be curious. The issue is, I can’t bring him back in because he’ll have an accident, but if I put him in his crate he’ll go to sleep and then I end up having to take him out an hour or two later because he still has to pee.

I thought we moved past this but we had the same issue tonight. I really don’t know how to get him used to going to the bathroom with these kinds of distractions and I feel so bad because I find myself getting annoyed at him when we won’t go.