Having trouble connecting to group dedicated server?

Hiya.. I rented a dedicated server and had it set up to private, but somehow random strangers were still joining. I looked up solutions and found out how to set it to a steam group, so I did that but it's not letting anyone connect. Any help would be appreciated here.. thanks D: