When people who arrive after you at a four way stop try and drive off before you

This doesn't happen all the time but I live near a lot of four way stops and it happens enough that it annoys me.