It used to work connecting to the controller(I used to play that way), but I guess there was update some months ago that now audio only comes if connected directly to the phone.

https://www.reddit.com/r/Stadia/comments/kin8lt/headset_support_with_stadia_controller/ggrtx1s?utm_medium=android_app&utm_source=share&context=3

Need to connect the headphones to the phone not the controller

https://www.reddit.com/r/Stadia/comments/kin8lt/headset_support_with_stadia_controller/ggrtx1s?utm_medium=android_app&amp;utm_source=share&amp;context=3

Using a phone screen (Pixel 3a), 3.5mm headphones connected to controller, I get NO audio through headphones. 

After reconnecting my controller to the same wifi as my pixel 3a, to be sure that is not the issue, I still get audio out through the phone only. 

These headphone do get audio from the controller when using a Chromecast Ultra.

No audio: Headphone jack on controller w/ phone