My sim keeps leaving work.

When my sim goes to work at his 9-5, I play as his wife, who has some days off. When she has her days off I like taking her to the gym to build up her fitness skill or making her do little errands but whenever I travel, the husband comes home from work! Why does this happen? Is there a way to stop him from doing this when she travels? He is getting bad work performance because of this now