Not comfortable with my boss

I need advice. I’m a Japanese girl living in the US for almost two years. I’m working now as a tutor for a child in LA, teaching Japanese to a Caucasian kid. My problem is the father seems to have bad intentions toward me. Sometimes he makes me go home very late since the tutoring is onsite. The couple is nice, but lately I’ve been feeling awkward about the father’s actions. The pay is good, so it’s hard for me to quit, and he hasn’t done anything openly inappropriate yet. I’m not sure if it’s just me overthinking or if this is part of their culture. I just need advice because maybe I’m misinterpreting his actions, sometimes he acts overly sweet for someone who’s just my employer