I broke up with Astarion, will I have more chances?

I started romance in Act I and had that sexy scene, but during the party after defending Grove, Astarion annoyed me with his narcisstic lines so much that I didn't want to sleep with him then. Looks like game took it as me breaking up with him. Oopsie. Now I'm feeling a little bad about this. Will I have chance to reignite that spark or is it lost forever?