Information preserving space warping functions to prevent affine collapse in neural networks

Rather than use say ReLU to prevent affine collapse, how about this:

https://archive.org/details/information-preserving-space-warping-to-prevent-affine-collapse-in-neural-networks

https://archive.org/details/global-modulation-versus-local-activation-functions-in-neural-systems

Example (maybe more later): https://archive.org/details/swnet-pn