Giant defy advanced sizing

Looking for some bike sizing advice. im a touch over 6’ 4” and riding a cannondale synapse 60 for about 7 years. I think previous bike was a 58 cube aerial. have enjoyed the cannondale and no real issues. looking to buy a defy advanced and according to sizing guides I’m pretty much top of L and bottom of XL. ideally I’d try both but no shop anywhere nearby has both and tbh I’m not sure I’d be able to pick up what’s right in a quick ride. I’ve compared online and reach is bit longer on Xl, very minor, than current bike but overall they are both smaller. whether it’s to do with the bike or not I do suffer from lower back pain sometimes and numb hands which might be long reach? I never feel comfortable on drops but more than happy sat on hoods. I could fairly easily go and test a L but is not comparing to XL a bad idea?