How do I get game chat to play through my headset?

I first noticed this playing The Division beta, party chat plays through the headset, but for whatever reason game chat will only play through my tv. I can't find any setting to change this and googling it has been useless.