So, aerodynamics matter more at a chill pace?

Short version, seems like you can finish every day with 15-20% more energy (or an hour earlier!) by... not wearing a t-shirt. Cycling pants / bib and a shirt that doesn't flap in the breeze gives you more free energy than any other single change you can make. If you have long hair, pony tail.

Frame packs and tail bags instead of panniers.

If you want a longer reference for my nonsense, GCN Tech just put a video a few days ago: "How Much SLOWER Are Normal Clothes For Everyday Cyclists?"