Can people really not beat Allanon?

It has come to my attention that people have struggled a lot with Allanon. My buddy and I have been playing through the storyline together, and when we reached Allanon's fighting scenes, it only took us three tries, which is fewer than most boss fights we complete. We are both conjurers, I use lightning, and he's plasma. Did we just have an easier time because there were two of us? This is a general curiosity, what made it easier for us?