Chapter 4 must hit so much less hard if you don't do the optional secret boss.

All the other secret bosses are detached and plot-irrelevant enough that you could skip out on them without that chapter's central story being diminished much.

But Gerson? You're going to not fight Gerson, and then have the sequence with the hammer-wielding Gerson statue fighting you based on how he fights in the boss battle? You're going to not fight Gerson, and not get the climax to the arc of him mentoring Suzie that happens throughout the chapter? You're going to not fight Gerson, and randomly have him show up in a badass new armor design you'd never seen before to help you beat the Titan?