How do the colorblind know that 'normal' people aren't just trolling them?

"What you can't SEE that Dude, you should go to the doctor..." trollface

What if 'colorblindness is just one huge in-joke by all the other people that pretend to see colors like red and green, while in reality they see the same as 'colorblind' people. It's just one huge case of "The Emperor's New Clothes".