Defender tips

On the weekends my friends and I usually play casual matches between us of normally 5v5 8sometimes 6v6 or 7v7) the thing is everyone there is like 1 or 2 years older than me, stronger or taller and I like playing as a Central-back, i do know how to positionate myself but i cant win aereal duels or physical ones and im a bit scared of being agrgresive and making tackles in case I do it wrong and hurt someone, i dont know if its a confidence thing or what but if someone could give me tips to be less scared of being aggressive and some extra tips would come in handy!