Cannot connect IKEA Matter devices to my HA Server

I bought the IKEA VARMBLIXT Lamp and it came with the BILRESA remote. So I have bought myself a ZBDongle E and flashed it to OpenThread. I have installed the OTBR app and configured it accordingly and also the matter server aswell. Now I have transfered the Thread and Matter credentials to my iPhone and also my android phone. I start pairing the devices, it connects with the phones. Now on iPhone. I am stuck on "Setting Up" and it fails to pair it. And on Android, It connects and gets stuck on checking connectivity and says can't reach device. I have made sure IPV6 is all turned on; on HA server and my router. I have no idea what to do now. Any help is apprecitiated.