Too much Xbox

So I'm not so sure that my husband is than the wrong, but I'd like to ask someone else's opinion. All he ever wants to do is play xbox. Whenever I want to hang out with him (like working, out going on a walk, watching Netflix together) he says I'm being selfish. He will still do it with me but then just complain about it later. He got me an xbox so we can play together, But I'm not really intrested in the games he likes. He gets angry when I won't play the games he wants. He told me before that the main issue in our relationship is that we spend all of our time together. He goes to work at 6 in the morning and gets home between 3 and 5. So I think he means all of our free time together. So he told me that I should just go downstairs and find something to do and let him play xbox alone for 2 hours and maybe then he would treat me better. Btw he hasn't offered to go do something else for 2 hours so I am pretty sure it's just an excuse for him to play xbox. All we have downstairs is a livingroom with only a couch in it (all of our TVs are upsatirs) and the kitchen. I haven't asked him to stop playing, i just want him to spend time with me without making me out to be a bitch for not letting him play xbox the 4-6 hours I see him a day. How should I approach this issue? And am I in the wrong?