Dog acts strange after babysitting

I have a 2.5 year old male German shepherd. I also babysit. Every time that I babysit for a specific family (one daughter, one year old), and I come back smelling like the baby, my German Shepherd dog acts strange. It’s as if he doesn’t recognize me by scent, he barks, and he runs from me in fear.
Usually he stops this behavior after I take a shower, but last night I babysat for 4 hours and his reaction has been prolonged. Even smelling the bag that I took to babysit made him bark and act afraid this morning. Why is this?

He has had no interaction with this family whatsoever. He’s never even laid eyes on the baby. I babysit for other families and he doesn’t do this.