Did anyone else struggle with this in Act 3?

The >!Coral Tower gauntlets!< are kicking my ass! Having to go back around the map and collect any Mask Shards and Spool Fragments i missed before now, as well as probably upgrade my charms. Is anybody else finding this difficult, or do I just need to git gud?