How come higher spoke counts aren't more common?

My wheels are Origin8 track attacks. They're triple walled 700c 32 spoke mass produced wheels (they were cheap and super strong). I want to get into building wheels soon. When it comes to strength, how come it's common to have heavy rims and a medium spoke count? That weight at the edge of the wheel has a lot of inertia. Shouldn't the spokes be increased first, and then a heavier rim used if more strength is needed? I read somewhere that lower spoke counts only help aerodynamics at around 40+ mph. I kinda wanna build like, 48 spoke wheels with light, single-walled rims or something. But I've never really seen that so that must mean it's stupid right? Why is that?

P.S. I like strong wheels for jumping curbs, potholes, and speed bumps with 23c tires