HA Noob and 30 year Home Automaton ninja

I had X10 in the 90's hooked into HomeSeer with a weather station and it worked well apart from x10 reliability. Ditched that and now have many Philips Hue lights, motion sensors and button switches plus Aqara doorbell, cameras, sensors and button switches. Everything is tied into HomeKit and I have automation running in Aqara, Hue, HomeKit and another iOS app that could automate with conditions. Conditions are the uber-power of HA. I want to consolidate this into one system and have chosen HA Yellow with a Pi Compute Module 5 and ZigBee module.
What I'd like to understand are the cons of stripping out all components from their existing bridges and make them first citizens in HA. Other than a a lot of work and loss of firmware updates which is quite a downside, any other downsides to making HA the canonical hub for everything connected and automated? Eg. Do you add back into Hue and HK components after adding to HA or do you leave them empty? Or am I making this overcomplicated and show just bridge existing Hue, Aqara and HK Hubs into HA as-is?

Another question. When Hue and Apple both introduced adaptive lighting (the bulb colour being dependent on time of day) I noticed the Hue-Homekit bridge could not use this as Matter did not support the adaptive protocol, so I rebuilt Hue-HomeKit without using matter. Is this still an issue and how does HA deal with Matter?

From what I've seen HA has more device connection options and powerful conditional option for Automation which is what I want, preferably in one place. Any tips on phasing the changeover to HA? Thanks.