Philosophy behind Probability

Motivating Randomness and Uncertainty

• We can think of randomness as a way of expressing what we don't know
• In other words, we can think of randomness as a measure of uncertainty
• Perhaps, if we knew more about the force I flipped the coin with, the initial orientation of the coin, the impact point between my finger and the coin, the turbulence in the air, the surface smoothness of the table the coin lands on, the material characteristics of the coin and the table, and so on, we would be able to definitively say whether the coin would come up heads or tails
• However, in the absence of all that information, we cannot predict the outcome of the coin flip
• In other words, in the absence of all that information, our process becomes probabilistic, rather than deterministic
• When we say that something is random, we are saying that our knowledge about the outcome is limited, so we can't be certain what will happen

Two Conceptions of Probability

• A bayesian and frequentist define probability in the same way
• Specifically, they both believe probability is any number $p$ that obeys the axioms of probability
• However, they represent uncertainty differently, causing them to estimate probability differently
• Frequentists represent uncertainty of their estimates after they're already calculated, typically using confidence intervals
• Frequentists estimate probability by calculating the MLE of the probability
• Bayesians will represent uncertainty of their estimates within the estimates themselves, typically using prior probabilities
• Bayesians estimate probability using simulation to calculate the posterior probability

• For example, let's say we want to flip a fair coin and estimate the probability of flipping heads:

  • A bayesian and frequentist would both say the probability of the coin coming up heads is 0.5
  • However, a frequentist would estimate the probability of receiving heads by tossing the coin many times
  • However, a bayesian would estimate the probability of receiving heads by tossing the coin many times and including some prior knowledge
  • This prior knowledge could be the assumed 0.5 probability, or it could be any previous tosses, or whatever we want it to be

References

• Bayesian and Frequentist Differences
• Representations of Uncertainty in Bayesianism and Frequentism
• Contrasts of Bayesianism and Frequentism Example
• Bayesian and Frequentist Reasoning in Plain English
• Differences between a Probability and a Proportion
• Bayesians Defining and Interpreting Probability
• Chance versus Randomness
• Probability, Statistics and Stochastic Processes
• Basics of Probability Course
• Probability Interpretations Wiki