Doctor recommended i try stopping my maintenance inhaler. Any advice to calm my nerves?

I was diagnosed with asthma as a little kid, maybe 4 or 5 years old. I had a handful of asthma attacks as a child and we went through a plethora of different meds. I remember proventil, serevent, singular, etc.

In my teens, I was prescribed advair, and my asthma finally felt really under control. I stopped having asthma attacks. ​​​I started being able to exercise and hike without any worry about my asthma. As I finished growing, my doctor started lowering my dosage of advair, then cutting me down to one puff per day of the lowest dose. Then I my early 20s, he switched me to the lowest dosage of flovent (44 ug) once per day. This has worked so well that I sometimes forget I have asthma. I honestly cant remember the last time I had an attack, and I very seldo. end up needing my rescue inhaler - probably less than once a year, on average. Ive been able to enjoy sports, backpacking, and mountaineering. New people are shocked the first time they see my using my flovent in the morning. I just dont register as someone who would have asthma.

Fast forward to now and im 40 years old. Ive been taking the same dosage of flovent for nearly 15 years and have had no issues. I saw my new allergist/immunologist yesterday, and she asked me to stop taking the flovent and see what happens. In theory im in favor of this - it would be great to reduce my daily meds. But in practice im kind of scared. If I forget to take my flovent, I dont get weezy, but my chest does feel slightly "heavier" until I take it. I dont know if that's real though, or all in my head. Im worried that ill have an asthma attack and not recognize the symptoms until its very late. Im worried that ill forget my rescue inhaler (like I said, I never use it anymore, so its not a habit for me yet). I'm worried that ill have an attack and regress and set myself back and be unable to do the things I love. Im worried about doing long term damage.

I mentioned this to my doctor but she sort of brushed it off and said it was still a good idea to try stopping the flovent. Have any of you gone through this process? How did you get over the mental barrier and fear? ​