When I mute my friends ingame, it ignores them in the main menu. Is there a fix for this?

I play with my friends and we use Teamspeak to talk to each other. If there's a random on the team that we talk to, I don't want the echo that comes with hearing them in the game and on Teamspeak. However, it seems when I voice mute them in game, it ignores their profile in the main menu, and I can't see their invites to party.