How do you personally decide whether to trust a borderline significant result on a small sample?

Ran an A/B test with a small sample (n=340 per arm) and got a p=0.04 result. Team wants to ship it. I'm nervous because I know small samples inflate false positive risk in practice even if the math is 'valid.' How do you personally decide whether to trust a borderline significant result on a small sample, bootstrap it, wait for more data, or just distrust p-values under a certain n?