Is there any way to have phone and Google home not respond to "hey google" at the same time?

it drives me crazy that when i try to talk to my home speakers, my phone responds. i want the feature because i like having an assistant on my phone. is there any fix for this?