Question about raspberries/blackberries

I have a few raspberry/blackberry bushes that I planted, and as is expected, there are half of the vines that actually produce fruit, and the other half are just growth with pricker needles growing on them.

Can I trim those halves of the plant that are just the prickers on them, or would that be bad for the plant?