How did they hang things on walls in castles?

Every depiction I have seen of a medieval castle at least has torch holders on the walls. Most have curtains, paintings, or animal heads. Seeing as the walls are made of stone, and drills wouldn't be invited for a long time, how did they get those items to stay there?