Bigger Video file than the original?? What am I doing wrong?

Hi, I was originally use handbrake and getting really good results, like i mean a folder at 700GB with over 280 video files and coming out at like 300GB.

Now I saw a video on YT about this guy compressing his videos using Fileflows and how it saved so much space, now me thinking if a paid compression app will do a lot better job......then its worth it?

Now i have paid and set it up and seen a very limited video on what to do... my files are coming out bigger than the original, and how could this be, i have tried different flows and nothing seems to compress the way I want it to ( using handbrake). Am I missing something?