RNG bias testing over time
Let's say I flip a coin and add 1 to a count if it's heads or subtract 1 if it's tails.
Is there any expectation of what the counter value will be over time? I mean like after thousands to millions of coin tosses.
I am trying to use this method as a test measurement to see if my random generator has any bias towards heads or tails over time. It's a truly random generator, not a pseudo-random sequence.
Posted Nov 9, 2018, 10:20 PM UTC
15
The expected value after any given number of flips will be the same as the initial value. So, if you start at zero, the expected value will be zero. The actual probability distribution will be a binomial distribution and, as the number of flips gets large, will approach closely a normal distribution.
Thanks! If I understand this it means that a single run of a million coin flips isn't a useful metric. I'd need to run many samples to start to see if the distribution follows the curve.
Well you can also test based on a single long run. If you draw 1000000 times and get 55% runs then there is almost certainly a bias.
Actually these kinds of tests are very important in the financial system for testing if a percentile of banks daily profit/loss (called value at risk) was correctly predicted. Though then it is based on not a 50/50 but a 99/1 chance of seeing a 1. See backtesting section from wikipedia on VaR for sources :)
Not sure what that means.
I ran 10,000 samples of 1000 tosses each and got a real pretty binomial distribution curve.
Thanks for all the help guys!
I think he meant that if you do a single sample of 1 million and the result was 55% or more one way or the other then there is likely a bias because that result would be highly improbable with an unbiased setup. That is, it would be way out in the extreme tails of the curve.