Adult ADHD Stimulant Option?

I’m 52 and have always had ‘ADD’, never the ‘H’. part. Since I was a child I had difficulty with focus, attention, impulsivity, etc. Academically, I was fine. As I’ve gotten older, I’m noticing that it is harder to mask or manage this. I’m becoming extremely forgetful and my work is suffering as well. I also have very mild anxiety/panic and was taking Lexapro (for over 15 years) to now Prozac. I’ve been toying with the idea of trying a stimulant as I’ve been on a non-stimulant and it did nothing for me but make me tired. Over 15 years ago I was prescribed a weight loss medication that had that ‘upper’ type of profile and it caused me to have extreme panic and anxiety. A friend of mine just told me she was prescribed an ADHD stimulant and experienced the same side effects that I just described. Has anyone had any negative experiences with a stimulant while also having anxiety and panic? Like I said, I tried Straterra and it did nothing. Thanks for any advice.