Xbox one fps drops in boss fight (spoilers)

For the love of God, please tell me I'm not the only one getting INSANE fps drops during the Pontiff fight...

Specifically when he creates a phantom of himself. Drops to 5-20, if that high even.

Keeps tripping me up.

Yes, I'm pissed.