Is it problematic to want to fuck a cave man?

I haven’t thought about it too hard. Obviously in this fantasy everything is consensual and clean, and this caveman is a homosapian, is it okay to fuck a cave man?