My husband says I should “accept it or leave,” and I don’t know if I’m asking for too much.

Title: My husband says I should “accept it or leave,” and I don’t know if I’m asking for too much.
I (30F) have been married to my husband (30sM) for a few years. I love him, and I genuinely believe he’s a good person. He’s responsible, loyal, financially stable, and has a big heart. That’s what makes this so confusing. I’m not dealing with a cruel or abusive husband, but I feel increasingly unhappy and emotionally disconnected.
One important piece of context is that his mother lives with us in a separate in-law suite, which I fully supported. He is also very involved with his family.
His late brother’s children are a huge part of his life, and I understand why. He has taken on a father-figure role for them after his brother passed away. I have always respected that relationship and never asked him to stop being there for them.
His other brother, his wife, and their children also come over frequently. Many of these visits include overnight stays. The children are energetic and loud, which is normal for kids, but our house becomes very chaotic. I work full-time in a demanding job, and sometimes I just need a quiet evening.
The difficult part is that I feel like I have very little say over what happens in my own home. Visits often happen so frequently that it feels like our house is an extension of everyone else’s. I also feel frustrated because they rarely invite us to their home, and they rarely bring anything when they visit—not even something small for their own mother who lives here. It leaves me feeling like our home is expected to be available all the time while the effort isn’t reciprocal.
I’ve tried talking to my husband calmly many times. I haven’t yelled or demanded that he cut off his family. I’ve simply asked for things like:
Some weekends that are just ours.
Quieter work nights when I have deadlines.
More discussion before overnight visits.
Boundaries that allow me to feel like this is our home too.
Recently he promised that this work week would be quiet because I had a lot going on. The kids were still very noisy, so I reminded him about the promise.
Instead of discussing the situation, he told me he was “this close to being done” with our marriage.
Something else that’s been weighing on me is how our families have been treated differently.
My parents stayed with us for several months. Before they left, they told me they felt like my husband barely talked to them. They said he mostly stayed in his room, came downstairs to eat, and then went back upstairs. That broke my heart because I wanted them to feel welcome.
Then, shortly afterward, he criticized me for not greeting the kids enthusiastically. I immediately thought, “You expected warmth from me toward your family, but my parents left feeling ignored.”
On top of all this, I struggle with our relationship itself. He rarely plans dates, rarely flirts with me, rarely asks about my day, and doesn’t make me feel pursued or desired outside of sex. Yet I watch him put so much energy into planning activities for the kids, like water park trips, and it hurts because I can’t remember the last time he planned something just for us.
When I tell him I need more peace at home or stronger boundaries, I feel like the response is essentially, “Accept it or leave.” Whether those are his exact words or that’s the message I hear, I don’t feel like my needs are being treated as equally important.
I know I’m only sharing my perspective, and I’m sure he has his own frustrations with me. I’m not looking for people to tell me he’s a terrible person. I’m trying to understand whether I’m expecting too much.
Am I being unreasonable for wanting my home to feel peaceful and shared?
How do couples handle situations where one spouse has significant family responsibilities while the other feels like the marriage always comes second?
Has anyone successfully rebuilt a marriage after years of feeling emotionally deprioritized, or is this a sign of fundamental incompatibility?