Need to use 'merge by distance' on a mesh, but if I do, I can't polybuild

Messing with a downloaded model. I need to use merge by distance in order to grab a vertex and drag the nearby vertexes with it, otherwise it'll just detach the face from all the other faces. But if I use merge by distance, I can't polybuild. The edges that I polybuild from will disappear. Is there a way around this? Thank you.