Panicking when I wake up

Hoping this flair is appropriate!

LO is 4 days old and I love her beyond words. She does not like sleeping in her crib, so my husband and I take turns holding her while the other person sleeps. I have so much anxiety about him or me falling asleep holding her that every time I wake up from my sleep shift I will panic and frantically search for her, thinking that I fell asleep holding her. I’m hoping this will fade in time, but I am also highly anxious and anticipate that PPA could be a real struggle for me. Has anyone had a similar experience and how long did it take to adjust and feel more confident/at ease?

Note: We are not interested in cosleeping. I understand that we should setup our sleep habitat just in case it happens accidentally, but I will not be doing it intentionally and even knowing that the bed is “safe” does not make me feel better.