Cruise Control executes all leadership movements after replica movements - causes client latency spikes during large rebalances. Any workarounds?

Question

TL;DR: During blue-green broker deployments with 50K+ partition movements, Cruise Control moves all replicas over X hours, then executes all 10K+ leadership changes in a concentrated burst at the end, causing client latency spikes. Looking for ways to spread leadership movements throughout the rebalance.

Background scenario:

We run a 9-broker Kafka cluster and do blue-green deployments where we add 9 new brokers and rebalance the entire cluster. Our typical rebalance involves ~55,000 partition movements.

The execution is sequential:

  1. Move ALL replicas (X hours)
  2. Then move ALL leadership (concentrated burst)

This causes a "leadership storm" at the end where thousands of leadership changes happen rapidly, leading to client connection disruptions and request timeouts.

Questions:

  1. Is this sequential execution (replicas → leadership) fundamental to CC's architecture, or are we missing a config option?
  2. Has anyone else dealt with this during large rebalances or blue-green deployments?

Posted May 6, 2026, 11:23 PM UTC

05
Loading comments…