Law of Large Numbers

Describing Law of Large Numbers

• The law of large numbers assumes independent, identically-distributed (IID) random variables $X_1, X_2, ..., X_n$
• We assume each random variable has a finite population mean $\mu$
• We know that the sample mean $\bar{x}$ is an unbiased estimator for the population mean $\mu$
• In other words, our best guess of the population mean $\mu$ is the sample mean $\bar{x}$
• The law of large numbers states that as the number of observations in our sample grows larger, then our sample mean will converge closer to the population mean

Weak Law of Large Numbers

• The weak law of large numbers states that the sample mean converges in probability towards the population mean
• Specifically, there is a high probability that the sample mean will be close (within some non-zero margin) to the population mean, assuming a large enough sample size

Strong Law of Large Numbers

• The strong law of large numbers states that the sample mean converges to the population mean
• Specifically, the sample mean will equal the population mean, assuming a large enough sample size
• If the strong law of large numbers holds true for a set of random variables, then the weak law of large numbers holds true as well
• However, if the weak law of large numbers holds true for a set of random variables, the the strong law of large numbers does not necessarily hold true
• Typically, the strong law of large numbers will not hold true if there isn't independent sampling

References

• Probability, Statistics and Stochastic Processes
• Strong Law of Large Numbers Wiki
• When does the Law of Large Numbers Fail