Traveling between the US and Brazil with controlled prescription drugs?

I have a prescription for Lorazepam, my pills will be in their original bottle. Will I have any trouble flying to or from Brazil with these pills on me?

My bottle says that it contains 6 pills (as that's what came in it). Would it be risky to put more than 6 in the bottle?