We toss a coin. If you get heads first toss, you get €2. If not, we toss again. If you get heads second toss, you get €4. In general, if you get heads on the n'th toss, you get €. What is your expected gain?
Basic probability theory tells us that the expected value is the sum of the probabilities of the possible events multiplied by the corresponding gain. One can consider the coin tossing as a discrete random variable, taking the values 0 or 1, each with a probability of . Getting heads at first toss has a probability of and earns you €2, getting heads at second toss has a probability of and earns you 4€, and so on. Thus the expected value of this game is .
This sounds surprising. After all, you cannot reasonably expect to earn an infinite amount of money - there will never be an infinite row of tails. However, in this case the gain grows just as quickly as the diminishing probability of successive tails.
A simulation of the game shows that, indeed, the (arithmetic) mean grows as the number of tosses grow, but very slow. The plot below shows the mean value up to tosses. Since the expected value is infinity, a mean gain of about €25 after ten million tosses doesn't sound much.
The plots show the same script run twice. The dissimilarity of the plots indicates the slow growth of the accuracy of the simulation (indeed, if the simulation told the real story, the plots would be too large for this world).
Let us, just for fun, change the rules of the game. Instead of getting on the n'th toss, you now get . As we know, linear functions grow substantially slower than exponential ones, so we should expect a change in the expected value. Indeed, a calculation shows that the expected value is 2. Plotting this:
We see that already at 100 tosses, we can be pretty sure what the expected value is. This is far from the case with the original rule. From the plots, the expected value could just as well be converging (albeit very slowly).
What does these considerations imply? Firstly, that simulating events with extremely low probabilities demand extremely accuracy/number of trials. Secondly, that using only expected value leads to "paradoxes". Say you were an economist - then you'd naïvely assume that your rational course of action is to play this game. After all, if you do it many times, you will certainly get rich. But as the probability of earning anything more than a few € is so low, it will take ages before you get rich. Economists solved this by considering time as a resource. Read Wikipedias's article for more information.
The Python file used to generate the plots.