Entanglement, how do we know it affects the things at a distance ?

So I remember in some documentary there was a thing where Einstein said spooky motion at a distance is impossible and he proposed that it was like if you have a pair of gloves, put each in a box and then shuffle the boxes until you don't know which is which then send each to a different city. When someone opens one of the boxes and he sees there's a right glove in it they instantly know the other box contain the left glove but nothing has been affected it was like that all along. From what I remember it was proven that this is not what happens with entanglement but I really don't get how it is even possible to prove that. If you can't measure the particle without breaking the entanglement then how can you know it's state changed when you measured the other and it wasn't just in that state to begin with ?