(Spoiler) Didn't Lord Tywin really hate prostitutes?

Like, so much that he kicked all of them out of Casterly Rock? And yet, Tyrion found him with Shae before the crossbow to the bowels. Has this incongruousness been explained?