Failure to Disclose material facts

Need legal advice in Minnesota. The buyers of our house that we closed on almost 2 months ago are demanding we pay for tree removal due to an Emerald Ash borer infestation. They claim our old neighbor (who disliked us) said we were aware of this issue because we had a conversation about it with her (which we never did). They also claim another neighbor "who claims to be a tree specialist" also says they had a conversation with us about the trees (which we never did and have absolutely no idea who this self-proclaimed tree specialist even is). They said that if we do not pay for the tree removal of $6,800 by October 2nd, they will have to pursue legal actions against us.

We never had any idea we had emerald ash borer issues and from what we could tell all of our trees had leaves and looked fine, minus the usual few benches that needed to be trimmed. We never had any reason to have an arborist come look at them.

I's this something we need to be concerned about? Do we need to get a lawyer involved? Or would we be OK drafting our own response? Should we request them to provide names, contact information, with written statements, and any evidence they have and a response time of 10 days for them to send us that. Is that wise or is that opening up Pandoras box? Any suggestions would be greatly appreciated.

Location: Minnesota.