Magic mouse gestures in Apple Music and Apple TV

Why is it that we can't use the back swipe gesture with the magic mouse in those apps? I always have to click the arrow back. This is such a friction and bad UX