What’s the hardest part of finding new B2B customers?

I’ve been thinking about how much time small businesses spend just trying to identify companies that could actually become customers.
There are plenty of ways to find prospects, Google, LinkedIn, referrals, directories, etc. but the difficult part seems to be finding enough relevant businesses without spending hours researching each one.
I’ve been looking at Targetron's B2B Local Leads Directory as one possible way to speed up that research.
For those running a growing business, what’s currently your biggest challenge with B2B prospecting: finding the right companies, getting contact information, reaching decision-makers, or actually converting them?