Handstands

So I just started doing handstands a week ago, I feel like I should work on different parts of my body first. Should I start small like doing pull ups or can I just start with handstands. I currently last for 40-50 seconds then I would fall over