Protection from legal action from a harassing party.

There are a few people who are harassing/defaming me. Proof of this is everywhere (texts, emails, phone calls, witnesses). They threaten me and claim that if I take any action against them, they will file a restraining order against me. This seems to be a tactic sometimes used by media and stalkers against their targets. I don't want that on my record. Is there something that I can do to legally protect myself from these people and live in peace?