Can I just combine systems?

So I have a stereo system I'm happy with, but in the process of upgrading. I also have a home theater system I'm building up, currently just a 2.1. In my house and with my space it's becoming awkward to have 2 very large systems in 2 different spots in my large open plan living room. I don't have the option to have a dedicated listening room unfortunately, and won't have the option to move for a while now due to work. Would it make sense then to just combine systems, using my svs ultra towers as the mains for both? Would I face any severe degradation of sound by connecting my ultra towers to my luxkit power amp AND my denon av reciever with seperate sources and only using one at a time? My stereo system sources are a pro-ject rpm 1, wiim mini streamer, emotiva pt1 preamp(I'm replacing this soon), and my av reciever is getting it's source from my Xbox. The big reason I want to move my stereo is as I learn more about this hobby, I realized placing my speakers directly in front of a massive bay window was probably a silly move from a sound perspective, and secondarily it takes up a LOT of space on that side of the house, making hosting events awkward. Any advice yall could give would be helpful