Cutting out caffeine and adhd medication temporarily to fix sleep

I work a really stressful job, so I take adhd meds at a good time early in the morning (adderall) and I drink a coffee sometimes before 11am, sometimes I won’t take adderall and drink coffee and vice versa, either way, the days when I don’t work and don’t drink coffee or take adderall, I literally fall into the deepest best sleep ever. I think sleep is so damn important, today was like it for me, I try to sleep at the same time every night, well I actually do sleep at the same time every night at 1030pm and low and behold, im asleep but I’m not really asleep if you know what I mean. I don’t know why we have these crutches that we think we need when in reality what we really need is a good nights sleep, completely sober. I actually feel like I’m going crazy when I don’t get sleep like today I just felt so out of wack, like I was drunk or something