Computational Statistics

Chapter 6 - Monte Carlo Integration and Variance Reduction

Dr. Mehdi Maadooliat

Marquette University
MATH 4750 - Spring 2025

Lecture 1: Simple Monte Carlo Integration (Part 1)

Introduction to Monte Carlo Integration

  • Basic concepts of Monte Carlo integration
  • Example: Integrating using uniform distribution
m <- 10000
x <- runif(m)
theta.hat <- mean(exp(-x))
theta.hat
[1] 0.6315141
1 - exp(-1)
[1] 0.6321206

Example: Monte Carlo integration with bounded intervals

m <- 10000
x <- runif(m, min=2, max=4)
theta.hat <- mean(exp(-x)) * 2
theta.hat
[1] 0.1172198
exp(-2) - exp(-4)
[1] 0.1170196

Lecture 2: Monte Carlo Integration for Unbounded Intervals (Part 2)

Unbounded Intervals

  • Handling unbounded intervals in Monte Carlo integration
# Plotting a function over an unbounded interval
x <- seq(.1, 2.5, length.out = 100)
y <- exp(-x^2 / 2)
plot(x, y, type="l", main="Function Plot for Unbounded Interval", col="blue")

Lecture 3: Importance Sampling (Part 3)

Introduction to Importance Sampling

  • Importance sampling and its applications
  • Example: Applying importance sampling
# Example of importance sampling
g <- function(x) exp(-x^2 / 2)
f <- function(x) dnorm(x, mean = 1)
x <- rnorm(10000, mean = 1)
theta.hat <- mean(g(x) / f(x))
theta.hat
[1] 2.54985

Visualizing Importance Sampling

# Compare the original function with the importance sampling function
curve(g, from = -3, to = 3, col = "blue", lwd = 2, main="Importance Sampling: g(x) vs f(x)")
curve(f, add = TRUE, col = "red", lty = 2, lwd = 2)
legend("topright", legend=c("g(x)", "f(x)"), col=c("blue", "red"), lty=c(1, 2), lwd=2)

Lecture 4: Variance Reduction Techniques (Part 4)

Control Variates

  • Explanation of control variates and how they reduce variance
  • Example: Using control variates in Monte Carlo integration
# Control variates example
m <- 10000
x <- runif(m)
theta.hat <- mean(exp(-x)) - (mean(x) - 0.5)
theta.hat
[1] 0.6326808

Antithetic Variables

  • Explanation of antithetic variables and how they reduce variance
  • Example: Applying antithetic variables
# Antithetic variables example
u <- runif(5000)
theta.hat <- mean((exp(-u) + exp(-(1 - u))) / 2)
theta.hat
[1] 0.6317489
1 - exp(-1)
[1] 0.6321206

Conclusion

  • Recap of Monte Carlo integration methods, importance sampling, and variance reduction techniques
  • Practice: Try implementing Monte Carlo integration with other functions

🔗 tinyurl.com/Comp-Stat