Sperm quality & sleep?

We’re on day 7 of stims in our first ER cycle and my husband’s sleep is concerning me. For the last several weeks he’s been working late (midnight/1am) and he’s always a bad sleeper (even on a good night he will wake up at 3am and struggle to fall back asleep). Last night at 9pm he absentmindedly took his adderal instead of his vitamins and then didn’t get to sleep until 4am. We’re doing all of this because I have stage 4 endometriosis and his initial sperm testing (in 2023) was good, but with passing IUIs last year his count dropped from the 40 million range it was at (post wash) in 2023 to 13 million (post wash). Now I have a creeping anxiety that this extra crap sleep in the last few weeks is going to lead to poor results with our embryo development - is that unfounded? Or is it really a concern?