Property/ Privacy on large acreage

My husband and I bought 180 acres of land - our home sits right next to our 5 acre private lake and the lake is roughly 10 feet within our property line. The neighbors next door own 80 acres and put their mobile home right on the property line (within their boundary) as close as possible to the lake. I am frustrated that we can see and hear one another, especially during the colder months when trees are bare. It feels like they are getting to enjoy our private lake for free when I paid a lot of money for it. The neighbor said he was under the impression he owned "half the lake" for years despite never getting a survey. The previous owner addressed this with him when he started cutting trees and building a dock about 10 years ago. I want to go and ask the neighbors if they will sell me their property as I don't think I can live like this anymore.

What do you think? I wanna hear other perspectives as this issue is running circles in my mind.

Thank you!