In the rough

My fiancé and I had our first son 4 months ago. I work a lot and don’t actually get time with my boy. When I get home he’s asleep and I’m getting ready for bed. When he’s with my fiancé he’s perfectly fine but as soon as she hands him to me he starts screaming and nothing I do to soothe him helps. Any advice won’t hurt. Just feel like I don’t have that connection you know?