Heartbroken

Husband started going through my phone awhile back, which is fine. Nothing to hide. We'll, I thought what's good for the goose is good for the gander, so I went through his. Found several chicks "not connected" on another social media site, yet you can tell messages have been deleted. Found messages discussing shaving certain lady parts and generally flirting, Found messages from the whole time we have been together where he has been flirting with other women. Screenshots of adult photos that were emailed to him. Plus, he has been emotionally involved with his ex wife for our entire relationship; I always suspected but now have the proof. I'm heartbroken. Part of me wants to GTFO, part of me wants to crawl in a hole and hope he finally chooses me after all this time. Help. Idk how to handle this, idk how to not hurt

TL;DR Husband flirted with other women our whole relationship and idk what to do