Gale romance questions

Hello all, I have a few play throughs of the game, but i've only ever successfully romanced astarion. This play through I chose gale, but when I got to the druid in the circus to do the love thing, all of a sudden, Gale and I have nothing going on? More specifically, I got "Always a pleasure to share a moment with you, if not in the way I once envisioned."

WHAT DO YOU MEAN? I literally JUST got to act 3 in this play through. What did I do wrong? Did I miss something in act 2?? I have no clue. Help!