Should I transfer from physics to engineering?

I’m two years through a degree in physics and Astronomy which I really enjoy. However I’m starting to think that I want to pursue a career in engineering, specifically aerospace. I’m on a design team and I’m taking steps to learn more engineering skills so that I have a chance to do a masters in engineering. My school offers engineering, however it’s very competitive and I don’t think my average is strong enough to get in. Even if I did I would have to restart from first year. Is it worth it to start over in first year and transfer to an engineering program at a different school? I still enjoy my physics degree I just find it difficult to compete with engineering students to get co-ops and resources to learn. I’m in Ontario, Canada.