Control Wards vs Stealth Wards, how many should I be putting down?

As a support who usually gets sight stone fairly quickly, I have a tendency to stop buying control wards fairly early in the game. Looking at my last 20 or matches, I am only placing on average 2 control wards, but 25 - 35 vision wards for average length games. I understand that its all relative to your control on the map, and the vision you can lay down safely at any given time, but am I giving up on control wards too soon?

I mainly use Control wards to spot out other wards, so when I upgrade to sweeper after getting SS, I dont spend my gold on wards anymore and instead focus on build. Thoughts on how to correct this if needed? Im trying to balance the lower gold generating support with what to buy/when and priority.

I main Zyra and Zilean supp if that matters.