Lufia - Cannot place the jewels on the pedestals

I have the Blue Jade and Green Jade from the towers, I examined the broken red jewel. Examined the three pedestals, did the boss at the end of the Red Tower. Spoke to the old lady in that town, and the jeweller in Ruan. Got Red Sapphire from the old guy in Medan Mine. Everytime I try to place the Jade in the Blue Tower, the hero just says 'Hey a Pedestal!' and nothing happens. What am I missing?