Dealing with a reactive dog that seems to be getting worse?

I got my mutt at a shelter about 6 months ago. He is about 3 years old and was pretty unruly on the leash when I got him. I was able to get him to walk a lot better with me, no I wouldn't say he's completely mastered the loose leash. And that's only if he's not distracted. He gets spooked very easily by people, other dogs, wildlife, etc. I used to take him on a really long walk thinking that the more he was exposed to his environment, the quicker he would adjust stop reacting to every little thing. I would pause to get his attention, reassure him, and get exceptionally affectionate if he did stay calm. After doing some research, though, it seemed like the best approach was to work up to the tougher triggers because once he's reacting he's unable to process anything that I try to do including getting his attention with a reward. So I shortened walks and try to gauge his reaction, making sure to reroute if we hit things that trigger him. But anymore I feel like I'm having to limit him more and more, because I can't stand his barking and panicking when all we are doing is stepping right outside the door to go to the bathroom.

I know this stuff takes time and patience, but I'm absolutely in tears because he's a sweet dog and it kills me that I can't even take him for a walk that he'll enjoy and I'll be able to relax because there's always something that's got him on edge. I'm just lost for what I can do to make it a smoother process. Having patience doesn't do a lot of good if I'm not even doing the right thing in the first place.