Should I learn freediving?

I recently jumped off a boat with free dive fins and swam around a kelp paddy for the first time and I am hooked. I am super interested in learning and am thinking about taking courses from a shop near me that has good reviews. My ultimate goal would to be able to do blue water spear fishing, but I know that’s several steps and lots of learning in the future before I get there.

I’m just super nervous about the breath holding part of this. I can’t imagine being able to hold my breath for minutes. At about 45 seconds holding my breath while resting I get the panicky urge to breathe. I’m in alright physical shape, I can tackle strenuous backpacking trips but don’t do regular cardio exercise. Would I really be able to train up and learn to hold my breath that long and dive down and putz around with a spear gun without panicking and suffocating? Does this require lots of specialized training?

I have my advanced open water scuba cert and love scuba diving, I’m very comfortable in deep water and around reefs, have good buoyancy control and have never had problems equalizing. I even did a bit of lionfish spearfishing once and loved it, but that was just short snorkeling dives in a shallow reef with a short sling spear.