Display ad - should I keep narrowing our target down?

I set up a display ad for a bunch of specific websites (competition) + keywords (our target audience). I see they are displayed mainly on the websites I chose, but also some random ones.

My last campaign failed after I kept narrowing everything down and leaving only the "best performing" ads and the websites with best CTR, so I'm wondering if I should go and narrow it only to our very specific target, or keep it more open and let Google "optimize it" for me.

I'd appreciate any tips.