How frustrating is it to lack darkvisision at early levels?

Title. I mainly make frontliners, and I've been avoiding any race without darkvision because it seems like such a major hindrance. You're basically giving up the ability to use a shield or two-handed weapon because you're stuck holding a torch. Or you're casting the Light cantrip, which still has pretty limited visibility.

Am I wrong?