AITAH for wanting an anniversary trip to just be my husband and I?

Yesterday, I found out the world cup game (Switzerland v Argentina) is going to be in Kansas City (an hr drive from me). This is a bucket list item. It also happens to be just a couple weeks before my husband and my's anniversary.

Tickets were $2,000 each, and we both love soccer. I said "hey, we should go to this!" He replied that it's expensive and would seem frivolous, but he really wanted to go. I said "hey, we could make this our anniversary trip! Two birds, one bigger expense. We could stay for a couple days and make it a thing." He said "I can't do two days because of my kids (we are a blended family), but I we can do dinner and the game." I said "ok cool!" He later made several references to this being for our anniversary (eg asking things like "who do we say paid for it in our social media post about our anniversary lol" as a joke).

Some background: he is bipolar and got so depressed on our first anniversary that he couldn't even get out of bed and it entirely ruined the day. I saw this as a grand "makeup" of lost time. It was very special to me. About three hours later, after much discussion how excited we are, he asked if he could bring his brother too.

This is where I got upset. I wasn't mean. I just got quiet and uncomfortable and said "I feel like you don't care about our anniversary, and that sucks. I'm upset now." He got so mad that I was upset. We ended up talking about his feelings how he can't "just mention a comment without worrying I'll get mad." I empathized. Then we talked about my feelings. He said he doesn't empathize and cannot understand why I would be upset. He said I'm selfish and said "fuck you" and "shut the hell up" when he was really dysregulated.

Today he still maintains that I am TAH. He says he did nothing wrong other than talking to me in an "unkind" way. He won't even acknowledge that he was disrespectful and immature. He said he legitimately doesn't care about anniversaries much. Same for birthdays. And it sucks I'm upset about that, and it is unreasonable and a major overreaction.

So, would anyone be upset by this, or is it just me? AITAH?? He seems so sure that it's making me second guess myself.