I'm so scared there's not enough time

I've always wanted to be an author. Not as a primary job, but more as a hobby. I live to read and write, but I very much don't want to write to live. I'm finally over halfway through my first real novel-length story because I've managed to stick to writing 500-1000 words every day (though I have mercy on myself if I'm having a tough day etc). But every time I quit for the day, even when I know that I've spent my creative energy for the time being, I get terrified that unless I work myself to exhaustion right now, something horrible is going to happen and I won't get to fulfill my lifelong dream of finishing a story that's longer than 15k words. It's like if I'm not doing as much as possible right now immediately asap, my whole body freaks out.

Sigh. Wish I could just do things at a normal pace without it creating additional stress. Do too much, you get exhausted. Avoid entirely, you never get the job done. Try to do just enough, and I have to wrestle with myself every time so I don't panic and then end up avoiding or overworking again.