What if we trained AI the way the human brain prunes itself?

One of the odder facts about brain development is that babies are born with far more neural connections than they'll ever need. Through childhood and adolescence, the brain gets rid of roughly half of them, a process called synaptic pruning. It's a large part of why infants learn so quickly: the brain starts wildly over-connected and gradually specialises, cutting away the pathways that don't get used and reinforcing the ones that do. Efficiency and function come from subtraction as much as addition.

Interestingly, people with autism show significantly less synaptic pruning than neurotypical brains. Postmortem studies (Tang et al., 2014, Columbia) found that by late childhood, typical brains had pruned around half their dendritic spine density, whereas autistic brains had pruned only a fraction of that, leaving considerably more excess connectivity intact. It's a useful reminder that pruning isn't incidental housekeeping. It's central to how a functional, specialised brain gets built at all.

Which raises a question. Could large language models be trained the same way? Train on a huge corpus, let the model operate, then identify parameters that go unused or barely contribute, and prune, weaken or delete them. Repeat across several rounds of training and pruning until you're left with something lean and specialised.

The good news is this isn't hypothetical. It's an active area of ML research already:

Magnitude pruning removes weights close to zero after training, on the reasonable assumption that a weight barely used isn't doing much work.

The Lottery Ticket Hypothesis (Frankle & Carbin, 2018) found that inside a large trained network sits a much smaller subnetwork that, trained alone from the same starting weights, matches the full network's performance. Rather like discovering the brain didn't need most of what it built in infancy.

Iterative magnitude pruning does exactly what you're describing: train, prune a bit, retrain, prune again, repeated across several cycles rather than one brutal cut.

Where it gets genuinely interesting for robotics is the distinction between unstructured pruning (zeroing out individual weights, which shrinks the model on paper but rarely speeds it up on ordinary hardware, since you still need to store the zeros) and structured pruning, which removes whole neurons, channels or attention heads. Structured pruning is what actually yields something smaller and faster to run, which is precisely what you'd want for a humanoid robot operating on limited onboard compute and battery.

The biological analogy has one limit worth flagging: brain pruning is heavily experience-dependent. Connections survive because they get used during real interaction with the world, not just because a static weight happened to be large after training. The ML equivalent would be pruning guided by activation patterns during actual deployment, rather than magnitude alone, which is closer to how a robot learning through embodied experience would end up shaping itself. That's arguably the more faithful version of the analogy, and the more useful one for anything that has to work efficiently in the physical world rather than just in a data centre.