# Statistical properties of the OLS estimators ## Common Assumptions [[Simple Linear Regression|Simple linear regression]] and [[Multiple linear regression|multiple linear regression]] make several assumptions. Depending on which assumptions are made, the OLS estimators have different properties. They are as follows: 1. The (unobserved) errors are an [[Independent and Identically Distributed (IID) Assumption|iid]] random sample. 2. The expected value of the errors is zero: $ E(\varepsilon_i\mid \mathbf{X}) = 0 $ 3. The variance of the errors is constant and does not depend on the predictors (Homoskedasticity): $ \text{Var}(\varepsilon_i\mid \mathbf{X}) = \sigma^2 $ 1. On top of Assumption 2, and 3, the errors are Normally distributed. ## Properties - If Assumption 1 holds, then the OLS estimators are [[Unbiased estimator|unbiased]]. - If Assumption 1 and 2 hold, then the OLS estimators are Normally distributed. - Alternatively, with a large sample size, they can be approximately Normally distributed. - By the [[Gauss-Markov Theorem]], the OLS estimators have the smallest variance among all unbiased, linear estimators (read: produces the smallest confidence intervals) - If we can assume Assumptions 1, 2, and 4, then the variance estimator $\hat{\sigma}^2$ will have a [[chi-squared distribution]] with $n-2$ degrees of freedom, multiplied by a constant. $ \hat{\sigma}^2 = \frac{\sigma^2}{n-2} \cdot \chi^2_{n-2} $ - Using this expression, we can show that $\hat{\sigma}^2$ is [[Unbiased estimator|unbiased estimator]] for $\sigma^2$ - If Assumption 1 and 3 hold, then the covariance of the OLS estimators are given by: $ \text{Cov}(\hat{\beta} \mid \mathbf{X}) = \sigma^2 (X'X)^{-1} $ - The variance of $\hat{\beta}_j$ is given by the $j^\text{th}$ diagonal value in the above expression - The covariance between $\hat{\beta}_i$ and $\hat{\beta}_j$ is given by the element in the $[i,j]$ position --- # References - [[Applied Linear Regression#2. Simple Linear Regression]] - [[Applied Linear Regression#3. Multiple Linear Regression]]