Landscape Connectivity and Dropout Stability of SGD Solutions for Over-parameterized Neural Networks

As neural networks grow wider the solutions obtained by SGD become increasingly more dropout stable and barriers between local optima disappear

More evidence IMO of the purely optimizational benefit of overparameterization.