For those who left the field

For anyone who completely left the field, how do you deal with feeling like you wasted your time or potential?

Studied my ass off in high school and university. Worked so hard to be successful and have a career I cared about. I left my original field of academia and fell in love with library work. Unfortunately, family circumstances made it necessary for me to move to a much smaller community with only 3 libraries within an hour of me, none of which have more than a few positions and don’t expect to have openings any time soon. Right now I’m working at the mall in a job that doesn’t even require a bachelors degree. I feel like I wasted the last 6 years and all the potential I was supposed to have. I’m so ashamed to tell my old coworkers or friends from university, never mind my professors who still sometimes ask what I’m up to.

I know a lot of people have left this field, I imagine I’m not the only person who has had this experience. Does anyone have advice?