Can I stop flip-open-to-answer-calls from defaulting to speaker phone?

I just blasted out my ear by mistake when my mother called me and I flipped over the phone, only for it to blast full whack into my ear on speaker. Is there a way to change that?