Matchmaking in groups of 4 or 5

I'm not sure if this happens to any others, but...

When we party up with 4 or 5 people, we seem to get slapped against a team full of golds (we're all mid range silvers) and cop it hard without a pillow to bite. Mind you, we're not all that bad at the game, it is our first MOBA and we're understanding what to do now, but it seems pretty unfair we can't get an evenly matched game every time we all party up. Then it comes to when we only have 2 or 3 of our usual group play, matchmaking seems pretty even and it's a fairly competitive and fun match. Does anyone else seem to have this problem or would this be solved by simply getting gud?

TL;DR - Matchmaking in a group of 5 = getting fucked on Matchmaking in a group of 2 or 3 = competitive and in for a chance