Tired of losing HR derby

I’ve won every all star game, my top power guy has 83 power, hits 90 HRs a year, I have no issues hitting 3 or 4 homers in a game with just him, but for some reason the HR derby is different. Cant hit more than like 6 in a single round. What am I doing wrong?