Walking/cycling to brighouse along the canal?

Hi, does anyone know if it’s possible to walk/cycle to Brighouse from Huddersfield Centre following the River Calder/the Broad Canal without having to go along main roads? I’m aware the canal has a towpath along it for the most part but I can’t figure out a way past the Colne Bridge section without going along the main roads and around a large and busy roundabout.

Anyone know of a way round this, or maybe a good alternate route to get to Brighouse from Huddersfield avoiding most roads?