Computational Statistics

Chapter 14 - Optimization

Dr. Mehdi Maadooliat

Marquette University
MATH 4750 - Spring 2025

Lecture 1: One-Dimensional Optimization (Part 1)

Introduction to Optimization

• Overview of optimization methods
• Example: One-dimensional optimization using optimize

f <- function(x) log(x + log(x)) / log(1 + x)

# Plotting the function
curve(f(x), from = 2, to = 15, ylab = "f(x)")

# Finding the maximum
res <- optimize(f, lower = 4, upper = 8, maximum = TRUE)
abline(v=res$maximum, col = "red", lwd = 2)

res$maximum

[1] 5.792299

Lecture 2: Maximum Likelihood Estimation (Part 2)

Introduction to MLE

• Introduction to Maximum Likelihood Estimation (MLE)
• Example: MLE for Gamma distribution

# MLE for Gamma distribution
m <- 20000
est <- matrix(0, m, 2)
n <- 2000
r <- 5
lambda <- 2

obj <- function(lambda, xbar, logx.bar) {
  r <- length(xbar)
  log(lambda) - lambda * mean(xbar) + logx.bar
}

Optimizing MLE

# Optimizing the MLE
xbar <- rnorm(n, mean = r/lambda, sd = 1)
logx.bar <- mean(log(xbar))
result <- optimize(obj, lower = 1, upper = 10, xbar = xbar, logx.bar = logx.bar, maximum = TRUE)
result

$maximum
[1] 9.999954

$objective
[1] NaN

Conclusion

• Recap of one-dimensional optimization and MLE techniques
• Practice: Apply optimization to other statistical problems and models