# Hypothesis tests for binomial parameter Suppose that $x_1, ..., x_n$ are observations from a Bernoulli distribution with parameter $\pi$. and that $L(\pi)$ satisfies the usual [[Likelihood regularity conditions|regularity conditions]]. Let $p(x_i;\pi)$ represent the probability mass function. ## Score Test Remember that the [[Likelihood Function#The score function|score function]] is the derivative of the log-likelihood with respect to the parameter ($\pi$ in this case). We can create a statistic that is based on the score function: $ S(\pi) = \sum_i \frac{\partial}{\partial\pi} \log p(x_i; \pi) $ Remember that the value of this function is a statistic because the data itself is random. The expected value of the score function is 0, so the expected value of this statistic is also zero. $ \begin{align} E[S(\pi)] &= E\left[ \sum_i \frac{\partial}{\partial\pi} \log p(x_i; \pi) \right] \\ &= \sum_i E\left[ \log p(x_i; \pi) \right] \\ &= 0 \end{align} $ Under regularity conditions, the log-likelihood can be approximated by a Normal distribution. We know that the variance of the score function is the [[Likelihood Function#Fisher Information|Fisher Information]], so we know that the following statistic is approximately a standard normal. $ \frac{S(\pi)}{\sqrt{\mathcal{I}(\pi)}} \rightarrow N(0, 1) $ We can create a test statistic by plugging in the null parameter value $\pi_0$ for the null hypothesis $H_0: \pi = \pi_0$ vs $H_1: \pi \neq \pi_0$. ## Wald test Another test can be constructed based on the maximum likelihood estimator. This is the test that's commonly seen in R model summaries for [[Generalized linear models (GLMs)|GLMs]]. Under the right conditions, the MLE is asymptotically normally distributed with variance equal to the Fisher Information. So, the following can be used as a test statistic for a hypothesis test: $ \mathcal{I}(\pi)^{1/2}(\hat{\pi} - \pi_0) \rightarrow N(0, 1) $ ## Likelihood ratio test The second order Taylor expansion of $L(\pi)$ about the MLE $\hat{\pi}$ gives us: $ L(\pi) \approx L(\hat{\pi}) + S(\hat{\pi})(\pi - \hat{\pi}) - \frac{1}{2}I(\hat{\pi})(\pi - \hat{\pi})^2 $ Rearranging this equation, we can get: $ -2(L(\pi) - L(\hat{\pi})) \approx \frac{1}{2}I(\hat{\pi})(\pi - \hat{\pi})^2 $ Notice that the right hand side is the square of the Wald test from the above, so this expression is thus approximated by a chi-squared distribution with 1 degree of freedom. ## Notes Asymptotically, all these tests are equivalent under the null hypothesis. However, their small-sample properties will differ. --- # References - [[Categorical Data Analysis#Chapter 1 - Distributions and Inference for Categorical Data]]