I replaced an AI cold email tool with my own system and realized the AI was not the hard part

I had been paying for an AI cold email tool for almost a year.

The idea sounded perfect: give it a market, let it find leads, write emails, send them, learn from the results and keep improving.

Basically, a little salesperson living inside my computer.

I work in my father’s agency, and until now most clients came from organic/inbound.

Which is great, but also a bit uncomfortable because you don’t really control it. Some months leads come in, some months they don’t.

So I got obsessed with cold outreach.

I read a line somewhere about a company doing millions a year just from cold outreach, and that sent me down the rabbit hole.

I started thinking:

What if instead of me prospecting every day, I had a system doing it constantly?

Not just sending emails, but learning:

• which ICP responds better
• which angles work
• which subject lines get opened
• which offers create replies
• which industries are a waste of time

So I used the AI tool for months.

Then, about a week ago, I built a simpler version myself.

And weirdly, it is already sending more volume and producing better emails.

But the biggest lesson was not “AI is amazing”.

The biggest lesson was that AI without a testing structure is just a fast intern with no manager.

I was telling it to write better emails, learn, improve, try things…

But I had not given it a proper system.

Then my father said something very simple:

“Wouldn’t it make more sense to test one sales angle per day for a month and see which one actually works?”

And I felt stupid because yes, obviously.

Instead of randomly trying to “improve cold emails”, the system now tests specific angles one by one.

One day it tests speed.
Another day cost.
Another day quality.
Another day reliability.
Another day “done for you”.
Another day comparison against hiring internally.

Same service, different angle.

Now the AI has something to optimize around.

I think this is where a lot of people, including me, get AI wrong.

We expect it to magically improve a messy process.

But most of the value comes from thinking clearly about the process first, then letting AI execute parts of it faster.

Curious if anyone else here is using AI for outbound or sales.

Are you letting it run freely, or are you forcing it into a testing system?