Describing Probabilistic Models
- A deterministic model is a model that returns the same outcome when given a consistent input
- A probabilistic model is a model that returns different outcomes when given a consistent input
- Sometimes, a probabilistic model is preferred over a deterministic model, because it includes an element of randomness (or uncertainty) that occurs in the real world
- Deterministic and probabilistic models are commonly used to estimate parameters and calculate probabilities
- Frequentist inference relies on using deterministic models for parameter estimation and probability calculations
- Bayesian inference relies on using probabilistic models for parameter estimation and probabilitiy calculations
Motivating Probabilistic Methods
- Although we like to think that probabilities are generated by a single deterministic function in principle, it is sometimes more accurate to think that probabilties are generated from a probabilistic function
- For example, our deterministic probability function may not use the most accurate fixed parameters (i.e. mean and variance), which leads to inaccurate probability estimates
- This can especially happen when our sample size is small
- In this case, our parameter estimates will most likely always lead to poor probability calculations, since our methods for parameter estimation are deterministic (i.e. MLE)
- Since a level of randomness is included in probabilistic methods of parameter estimation, our parameter estimates have a higher chance of being more accurate compared to using deterministic methods
Deterministic Methods of Estimation
- Maximum likelihood estimation (MLE) is an example of a deterministic method used for parameter estimation
- A probability density function (pdf) is an example of a deterministic method used for calculating probabilities
- Most deterministic models (such as pdfs) return the same solution when its input is fixed, because its parameters are fixed
- Therefore, including fixed parameters is typically characteristic of deterministic methods
Probabilistic Methods of Estimation
- MCMC is an example of a probabilistic method used for parameter estimation
- Monte Carlo is an example of a probabilistic method used for calculating probabilities
- The Monte Carlo method is one of the most popular probabilistic methods of estimation
- Most probabilistic models (such as MCMC) return different solutions when its input is fixed, because its parameters are unfixed
- Therefore, including unfixed parameters is typically characteristic of probabilistic methods
References
Previous
Next