Computational Statistics

Chapter 2 - Probability and Statistics Review

Dr. Mehdi Maadooliat

Marquette University
MATH 4750 - Spring 2025

Probability Concepts

Random Variables and Distributions

  • Overview of random variables
  • Discrete vs Continuous random variables
  • Important distributions (Binomial, Normal, Chi-Square)
# Simulating random variables
N <- 100; mu <- 5; sig <- 2
Y <- rnorm(N, mean = mu, sd = sig)
hist(Y, main="Histogram of Normal Distribution", xlab="Y values", col="lightblue")

Some Discrete Distributions

Binomial Distribution

  • rbinom(), dbinom(), pbinom()
  • Example: Generating binomial random variables
rbinom(10, size=10, prob=0.5)
 [1] 4 6 4 6 4 4 8 3 5 6

Poisson Distribution

  • rpois(), dpois(), ppois()
  • Example: Simulating Poisson-distributed data
rpois(10, lambda=2)
 [1] 2 1 0 1 3 2 3 0 2 1

Some Continuous Distributions

Normal Distribution

  • rnorm(), dnorm(), pnorm()
  • Example: Generating normal random variables
rnorm(10, mean=0, sd=1)
 [1]  0.9543824 -0.8301930 -0.6665991 -1.1280962 -0.1236944 -0.2073844
 [7]  1.7094885  1.3681418  0.6329969  0.1309925

Exponential Distribution

  • rexp(), dexp(), pexp()
  • Simulating exponential data in R

Multivariate Normal Distribution

Properties

  • Mean vector, covariance matrix
  • Multivariate normal density
library(MASS)
mvrnorm(n=10, mu=c(0,0), Sigma=matrix(c(1,0.5,0.5,1),2,2))
            [,1]       [,2]
 [1,]  0.2000364 -0.1236475
 [2,] -1.7738766  0.4607096
 [3,] -0.8840357 -1.5032048
 [4,]  1.5186149 -0.6387394
 [5,]  1.3657603  0.3467254
 [6,] -1.0686554 -1.6470091
 [7,]  0.2336637 -0.3994583
 [8,] -1.5783893  1.2409729
 [9,]  1.1107702  0.1740867
[10,]  0.5894729  0.5428613

Limit Theorems

Law of Large Numbers

  • Statement of the law
  • Demonstration using R
n <- 1000
x <- rnorm(n)
cumsum(x) / 1:n
   [1]  0.595051315  1.071322308  0.699731095  0.349435781  0.113838044
   [6] -0.003637926 -0.131728711  0.168782488  0.168115602  0.183432449
  [11]  0.233503055  0.174602136  0.186095009  0.185595571  0.098649224
  [16]  0.127296495  0.200152021  0.224863761  0.125578047  0.134643687
  [21]  0.110874348  0.101164137  0.085663931  0.060162798  0.100610072
  [26]  0.110650929  0.093749679  0.069328999  0.033626873  0.023847977
  [31]  0.026118961 -0.018055396  0.010998266  0.015329401 -0.010919107
  [36] -0.013789335 -0.005930820 -0.012451136 -0.042433061 -0.082476698
  [41] -0.110029108 -0.138115168 -0.139255736 -0.185373902 -0.169671300
  [46] -0.151080476 -0.134283977 -0.129175374 -0.158290838 -0.125813396
  [51] -0.136477952 -0.124738283 -0.127343981 -0.144777882 -0.131871776
  [56] -0.143820106 -0.148808508 -0.173175213 -0.184681688 -0.173259776
  [61] -0.179944409 -0.206045460 -0.220170849 -0.221212040 -0.211592815
  [66] -0.214680048 -0.195334759 -0.204995750 -0.209046887 -0.214639691
  [71] -0.208817505 -0.206734929 -0.191762375 -0.185991096 -0.198350975
  [76] -0.194823278 -0.190893076 -0.191363312 -0.184194162 -0.191850532
  [81] -0.181170079 -0.197003952 -0.195495155 -0.202417843 -0.214152732
  [86] -0.204674648 -0.185411423 -0.186501894 -0.182533156 -0.166554515
  [91] -0.169519532 -0.166316823 -0.151375642 -0.140352202 -0.140826908
  [96] -0.126593381 -0.126251972 -0.121226721 -0.109688892 -0.104772657
 [101] -0.090046096 -0.088052417 -0.063741836 -0.063970678 -0.060517513
 [106] -0.062860097 -0.065902021 -0.092409650 -0.091254709 -0.093857012
 [111] -0.088171029 -0.096597957 -0.087678223 -0.101858298 -0.105124708
 [116] -0.110911930 -0.102366928 -0.110327695 -0.108861068 -0.120203936
 [121] -0.125642326 -0.116886596 -0.118190710 -0.114941364 -0.119713235
 [126] -0.114561057 -0.115915227 -0.112324426 -0.113789055 -0.122462827
 [131] -0.128251683 -0.132016137 -0.125238763 -0.135039254 -0.127301167
 [136] -0.124748516 -0.125177682 -0.135027010 -0.126449749 -0.119411104
 [141] -0.135506909 -0.136672974 -0.137209183 -0.141563791 -0.148881785
 [146] -0.148741118 -0.144102171 -0.150574087 -0.144825847 -0.147327230
 [151] -0.132208796 -0.134447167 -0.134805118 -0.132966935 -0.135964153
 [156] -0.139298601 -0.135896799 -0.127583377 -0.123633633 -0.115967585
 [161] -0.110677055 -0.113441203 -0.111702861 -0.110410091 -0.103370743
 [166] -0.101922917 -0.097508063 -0.091397098 -0.079331208 -0.080729015
 [171] -0.076046160 -0.083850734 -0.083913555 -0.080582609 -0.082424636
 [176] -0.087022538 -0.082536462 -0.083782272 -0.087033173 -0.089615728
 [181] -0.094481661 -0.088701941 -0.080377690 -0.072731523 -0.059092769
 [186] -0.063999465 -0.064931612 -0.062630041 -0.060114289 -0.053582952
 [191] -0.056352848 -0.057124558 -0.062396973 -0.064506930 -0.066852920
 [196] -0.069566647 -0.059391637 -0.060361861 -0.062433344 -0.054603935
 [201] -0.056292615 -0.058776983 -0.056922251 -0.060306432 -0.058875411
 [206] -0.057709038 -0.050964109 -0.050041090 -0.051014826 -0.048118041
 [211] -0.051062270 -0.041380753 -0.039771373 -0.046265930 -0.053233067
 [216] -0.056616435 -0.069478944 -0.062742830 -0.067545644 -0.071302182
 [221] -0.070817621 -0.076720137 -0.074862786 -0.079129746 -0.082295481
 [226] -0.080177859 -0.084387651 -0.074739922 -0.067673394 -0.061545456
 [231] -0.055639972 -0.054736199 -0.056576649 -0.052908535 -0.053928247
 [236] -0.057116875 -0.054859140 -0.052403788 -0.050827755 -0.054030275
 [241] -0.052686016 -0.051358697 -0.052634377 -0.053621022 -0.052458675
 [246] -0.049886997 -0.056143874 -0.055889560 -0.046316187 -0.049981167
 [251] -0.051049905 -0.043604904 -0.040347655 -0.039740861 -0.040667005
 [256] -0.035639158 -0.039338455 -0.041359203 -0.044268503 -0.040318232
 [261] -0.036114130 -0.032077833 -0.032643191 -0.041071013 -0.039122680
 [266] -0.037942829 -0.041678357 -0.049899995 -0.052050719 -0.053858830
 [271] -0.053245373 -0.055217503 -0.054818677 -0.056308537 -0.056969242
 [276] -0.056089806 -0.060394109 -0.051432035 -0.045171793 -0.050264097
 [281] -0.047953468 -0.050518408 -0.054152783 -0.050592856 -0.050170292
 [286] -0.050469411 -0.050382098 -0.045645180 -0.047064584 -0.047670559
 [291] -0.037026752 -0.038574797 -0.037158507 -0.032357719 -0.029428945
 [296] -0.025269079 -0.017920404 -0.016734020 -0.026602916 -0.027401477
 [301] -0.022544488 -0.018964595 -0.018558277 -0.024818811 -0.026019615
 [306] -0.021470630 -0.014986744 -0.014076587 -0.008768178 -0.008114963
 [311] -0.007790471  0.001953466  0.005395080  0.009818615  0.009121612
 [316]  0.007839645  0.015009459  0.022745498  0.022490114  0.020591115
 [321]  0.020819145  0.021622285  0.024810969  0.026604751  0.028591943
 [326]  0.035949182  0.033795296  0.031911143  0.030136515  0.032107266
 [331]  0.031146581  0.033006162  0.030150896  0.032067112  0.030378368
 [336]  0.034138922  0.035087304  0.038441044  0.039656642  0.038894203
 [341]  0.044211880  0.043212601  0.039797824  0.039327959  0.033558689
 [346]  0.033720950  0.031837945  0.033458001  0.029291245  0.031515962
 [351]  0.033450300  0.035779543  0.039347007  0.038699093  0.042171250
 [356]  0.048053737  0.045022584  0.041668047  0.042897839  0.044458629
 [361]  0.047754393  0.053153078  0.048154221  0.045172343  0.048200789
 [366]  0.056005452  0.053950118  0.053509784  0.053289225  0.057303521
 [371]  0.059181564  0.058484421  0.055061051  0.061740276  0.055242383
 [376]  0.052566699  0.053669576  0.053558696  0.054397485  0.051550390
 [381]  0.051085471  0.048796602  0.049346330  0.049889738  0.048368720
 [386]  0.045803736  0.043572700  0.046088605  0.048093852  0.049092560
 [391]  0.049723806  0.045553832  0.046951185  0.047649347  0.044422855
 [396]  0.045923068  0.045684276  0.045599324  0.043353822  0.045966909
 [401]  0.043158325  0.038061431  0.039841567  0.041540098  0.043957770
 [406]  0.042664980  0.046402091  0.045236774  0.048423721  0.047295958
 [411]  0.049112921  0.045140395  0.046191093  0.040524270  0.039512794
 [416]  0.041895250  0.041676332  0.042298037  0.042818493  0.044384622
 [421]  0.041385890  0.038183380  0.037616173  0.039030501  0.038534994
 [426]  0.039838094  0.038730779  0.038270004  0.041221981  0.040831311
 [431]  0.042449048  0.040512695  0.039919398  0.042805012  0.044221014
 [436]  0.044418480  0.044843774  0.041455159  0.041030678  0.044805513
 [441]  0.042219373  0.045354309  0.042877153  0.045166410  0.048155868
 [446]  0.047844376  0.045260234  0.045075490  0.047023321  0.045507068
 [451]  0.047182271  0.050240725  0.048849966  0.045706565  0.044883825
 [456]  0.047456068  0.046820300  0.046313074  0.046492302  0.046942871
 [461]  0.047972843  0.048696278  0.052830136  0.051381464  0.050573524
 [466]  0.050085994  0.049513479  0.048560249  0.048403745  0.046400492
 [471]  0.043640591  0.042838067  0.050149103  0.047810482  0.048527366
 [476]  0.044525983  0.042664554  0.040741955  0.040743076  0.039695695
 [481]  0.038803887  0.039492289  0.043119251  0.044906490  0.042496915
 [486]  0.046086349  0.046092363  0.046813129  0.046915058  0.042337227
 [491]  0.042616246  0.043575154  0.044000668  0.044059952  0.047403688
 [496]  0.049589181  0.045048153  0.044501388  0.044506826  0.044189120
 [501]  0.043948951  0.045037937  0.040964415  0.037499970  0.036714557
 [506]  0.036273429  0.037633017  0.040465035  0.044490850  0.043323494
 [511]  0.043734745  0.045386457  0.043253595  0.041818962  0.038522794
 [516]  0.045080017  0.046705084  0.046416225  0.046602377  0.047435947
 [521]  0.049869307  0.049275575  0.049415574  0.052408807  0.055195586
 [526]  0.054504330  0.054944252  0.054400253  0.055789466  0.054215858
 [531]  0.052525985  0.051700997  0.048966511  0.048726561  0.050467247
 [536]  0.048690280  0.052702701  0.052957328  0.052464890  0.053224009
 [541]  0.051983596  0.045653155  0.046499777  0.046819421  0.048303635
 [546]  0.046113588  0.046386918  0.045027787  0.046736107  0.046692549
 [551]  0.046402387  0.050315474  0.049651379  0.049325091  0.050496543
 [556]  0.050471929  0.048794349  0.050116420  0.050836736  0.050543998
 [561]  0.048731547  0.050444974  0.049856657  0.048588818  0.050233471
 [566]  0.052161467  0.050112971  0.049439690  0.048957915  0.047307228
 [571]  0.046808940  0.047163906  0.046204694  0.044788953  0.046881172
 [576]  0.049673572  0.048676438  0.051521976  0.053138365  0.054510155
 [581]  0.053636348  0.055057188  0.054386141  0.053520828  0.052819893
 [586]  0.050059473  0.048602443  0.051381553  0.050451032  0.050968316
 [591]  0.051439592  0.051867223  0.054091116  0.052252333  0.051475515
 [596]  0.050004238  0.047957891  0.045956920  0.047637056  0.047447357
 [601]  0.052268147  0.049655277  0.050503047  0.051729972  0.050141221
 [606]  0.049305194  0.047704825  0.050887490  0.051396012  0.051152850
 [611]  0.052415719  0.053261828  0.051615112  0.051808668  0.052190708
 [616]  0.053174356  0.050679421  0.052644308  0.053407769  0.055570509
 [621]  0.055007715  0.053922969  0.055298950  0.055556715  0.054330794
 [626]  0.052839089  0.050800165  0.048661305  0.048512059  0.047226378
 [631]  0.047063901  0.046284255  0.042132873  0.040544920  0.038261891
 [636]  0.037750089  0.037707130  0.035895846  0.036344537  0.037446594
 [641]  0.036058266  0.036925155  0.037559988  0.036223072  0.035192951
 [646]  0.033809998  0.032737976  0.032016071  0.032399201  0.032214596
 [651]  0.034826365  0.033863917  0.033518527  0.032502583  0.031462303
 [656]  0.031044065  0.030945041  0.030982784  0.029121714  0.029293349
 [661]  0.028246537  0.029902067  0.030287664  0.030686764  0.034667119
 [666]  0.034848110  0.035647159  0.035145996  0.036614318  0.035166044
 [671]  0.036373596  0.031558941  0.031020821  0.028880618  0.026692639
 [676]  0.027244798  0.028031285  0.026657251  0.025651206  0.024276993
 [681]  0.026018459  0.024014668  0.026254125  0.025624654  0.023149628
 [686]  0.024837428  0.026467264  0.026922856  0.024148632  0.024221776
 [691]  0.024068600  0.025084056  0.024815740  0.025040983  0.023364144
 [696]  0.023948562  0.024534576  0.023821156  0.024985360  0.024904196
 [701]  0.022869744  0.024403718  0.022445866  0.021695777  0.024886104
 [706]  0.026922543  0.028643580  0.028710691  0.028975764  0.027740731
 [711]  0.028566082  0.029438981  0.027554111  0.026210311  0.024479713
 [716]  0.025362724  0.027830509  0.027316774  0.029271266  0.026932523
 [721]  0.025431610  0.025163407  0.027398642  0.025241101  0.025017386
 [726]  0.024726303  0.023330651  0.022028230  0.021708832  0.021355467
 [731]  0.022342771  0.024746977  0.021234164  0.019888880  0.020037527
 [736]  0.019812969  0.018027580  0.017974422  0.017638109  0.019957560
 [741]  0.021081699  0.020646262  0.022026061  0.021780154  0.021629880
 [746]  0.018519679  0.017971951  0.017733109  0.017254077  0.018094423
 [751]  0.018127700  0.017653320  0.016946121  0.017274080  0.016611984
 [756]  0.015366705  0.015203165  0.016277315  0.017420233  0.018690326
 [761]  0.018347668  0.017665537  0.018783661  0.016855701  0.018585505
 [766]  0.019630819  0.019472412  0.021395685  0.019970907  0.021723230
 [771]  0.019506611  0.019261615  0.018048524  0.016911542  0.015718702
 [776]  0.014809101  0.016075673  0.016694818  0.018053595  0.017330545
 [781]  0.017866991  0.016840282  0.015815476  0.015742928  0.014108861
 [786]  0.016291648  0.017243898  0.016989102  0.016118787  0.016726510
 [791]  0.015526618  0.015624386  0.017036849  0.016846255  0.015632331
 [796]  0.015817821  0.017413913  0.016715548  0.016815126  0.014533409
 [801]  0.012453389  0.014119822  0.016970124  0.015173613  0.015939288
 [806]  0.016145604  0.015349773  0.016892167  0.017986047  0.017831151
 [811]  0.019386086  0.021365121  0.021383071  0.021542965  0.021264244
 [816]  0.021993754  0.019325918  0.021222879  0.021716447  0.023219191
 [821]  0.023317080  0.022546989  0.022258356  0.022337260  0.018650835
 [826]  0.016965267  0.017716959  0.014810774  0.014286979  0.014984127
 [831]  0.015601112  0.015569649  0.015836719  0.016763393  0.015780051
 [836]  0.017099793  0.015735375  0.015662366  0.015942810  0.016467076
 [841]  0.016205323  0.013753248  0.013276410  0.014509523  0.014959684
 [846]  0.016576072  0.016683006  0.014573356  0.011818139  0.011748614
 [851]  0.011935509  0.010068843  0.009876527  0.010675715  0.010487100
 [856]  0.010622713  0.009554402  0.008192215  0.009577014  0.011032568
 [861]  0.010076224  0.012454494  0.011036220  0.010475851  0.009534795
 [866]  0.009288670  0.010484802  0.010063621  0.010500150  0.008423944
 [871]  0.009310678  0.010583584  0.008782987  0.008117892  0.008433304
 [876]  0.008987257  0.008490518  0.007655138  0.008273144  0.010247986
 [881]  0.010089900  0.010694872  0.010265409  0.011194024  0.011475556
 [886]  0.012414737  0.010970954  0.010899056  0.012703425  0.010903896
 [891]  0.009334735  0.009385595  0.009683508  0.010123996  0.008652601
 [896]  0.009832660  0.009934858  0.009290867  0.009627141  0.008775443
 [901]  0.010722962  0.010919747  0.010038772  0.010555332  0.010430296
 [906]  0.009157552  0.010055729  0.008599090  0.008052722  0.008856254
 [911]  0.007993544  0.006340678  0.006452957  0.006317018  0.005761975
 [916]  0.004614658  0.003601461  0.002519035  0.003294286  0.004645832
 [921]  0.005778652  0.005429985  0.006912239  0.006184370  0.005933187
 [926]  0.004822180  0.004947744  0.005754882  0.006714398  0.006827146
 [931]  0.006656415  0.005777227  0.005556656  0.006495893  0.006926562
 [936]  0.007003295  0.007640366  0.009164736  0.009455847  0.010385500
 [941]  0.011438177  0.012692034  0.013535752  0.014723482  0.015002452
 [946]  0.014927304  0.015832689  0.014851355  0.015844096  0.014133588
 [951]  0.013147051  0.012708232  0.012619866  0.012788048  0.013158192
 [956]  0.013727774  0.014028192  0.016403521  0.015859544  0.015933706
 [961]  0.017273402  0.015391638  0.017072781  0.016749551  0.016991631
 [966]  0.015849947  0.015152755  0.015843762  0.013707439  0.013784596
 [971]  0.013178389  0.013049219  0.014064011  0.014704189  0.014757816
 [976]  0.016021221  0.017840351  0.018666870  0.018403871  0.019534896
 [981]  0.018963305  0.017850193  0.016549455  0.016151557  0.017004701
 [986]  0.017609078  0.017278804  0.018170298  0.016638575  0.015155014
 [991]  0.016969276  0.014604592  0.014607545  0.015664665  0.014804075
 [996]  0.013589190  0.013275072  0.012782236  0.013019302  0.012916640

Central Limit Theorem

  • Discuss the CLT and its importance in statistics
  • Simulating the CLT with normal distributions
# Central Limit Theorem simulation
means <- replicate(1000, mean(rnorm(100, mean = mu, sd = sig)))
hist(means, main="Central Limit Theorem", col="lightgreen", border="black")

Joint and Conditional Distributions

  • Joint distributions and their importance
  • Conditional distributions with examples
# Simulating joint distributions
mu <- c(5, 10); sigma <- matrix(c(1, 0.5, 0.5, 2), 2, 2)
library(MASS)
X <- mvrnorm(1000, mu = mu, Sigma = sigma)

plot(X[,1], X[,2], main="Scatter plot of Joint Distribution", xlab="X1", ylab="X2", col="blue")

Estimators and Their Properties

  • Discuss point estimators: Mean, Variance, etc.
  • Review bias, consistency, and efficiency
# Example of calculating mean and variance from samples
set.seed(123)
sample_data <- rnorm(100, mean = 5, sd = 2)
mean(sample_data)
[1] 5.180812
var(sample_data)
[1] 3.332931

Conclusion

  • Recap of probability concepts, distributions, and hypothesis testing
  • Review key formulas and concepts for exam preparation

🔗 tinyurl.com/Comp-Stat