Trouble with Chart on Sock Ruffle!

Hi friends! This is my first time on Reddit and I'm on here specifically to ask this question haha. So I'm working on a pair of toddler socks that have a ruffle on them and I'm having trouble understanding the knitting chart. I've included pictures of the ruffle instructions and the chart. I have 40 stitches starting out and I'm knitting in the round. From what I understand, the pattern is meant to be repeated since there are more stitches on the needles than on the chart. I did the first row fine, but the second row is holding me up. I consistently end up with 4 stitches left over and I don't know why. I've double and triple check that I have the right number of stitches and even go slow through the pattern to make sure I'm not overlooking anything and still end up with 4 extra stitches. So I know I'm doing something wrong, but I don't know what! It's really annoying to me that I can't figure it out haha but I know there are some experts on here that will know what to do! Any help or advice you have would be much appreciated!!