Suppose that we have a random sample of $f(x;\theta )$ and $\theta \in \mathrm{\Theta }$ we would like to test $$H_0:\theta \in \Theta_0 \qquad H_1:\theta \in \Theta_1$$where ${\mathrm{\Theta }}_0,{\mathrm{\Theta }}_1\subseteq \mathrm{\Theta }$ and where ${\mathrm{\Theta }}_0$ and ${\mathrm{\Theta }}_1$ are disjoint. Usually as ${\mathrm{\Theta }}_1=\mathrm{\Theta }\setminus {\mathrm{\Theta }}_0$.

We like to generalise the ideas of the Neymann-Pearson Lemma, and get a similar sense of a ratio of likelihoods, we would like to have a quantity that becomes small when $H_0$ is false.

Test of Generalised Likelihood Ration

Def: Let $X_1,\dots ,X_n$ be a random sample of $f(x;\theta )$ and $L(\theta \mid \underset{―}{x})$ be the likelihood function, where $\theta \in \mathrm{\Theta }$. The generalised likelihood ratio is defined as $$\lambda = \dfrac{\max\limits_{\theta \in \Theta_0} L(\theta \mid \underline x)}{\max\limits_{\theta \in \Theta}L(\theta \mid \underline x)}$$
We see that the denominator of the generalised likelihood ratio, is actually just the likelihood function evaluated on the maximum likelihood estimator. Meaning that $\underset{\theta \in \mathrm{\Theta }}{max}L(\theta \mid \underset{―}{x})=L(\hat{\theta })$, with $\hat{\theta }$ is the maximum likelihood estimator.

We see that since they are nonnegative quantities, we see that $0\le \lambda \le 1$.

We see $\lambda$ is a function of $\underset{―}{x}$, if we substitute the observation with the random sample $\underset{―}{X}$ is a statistic and is denoted $\mathrm{\Lambda }$.

Test of the generalised likelihood ratio or the Principle of the generalised likelihood ratio

This test establishes the following decision rule:

The constant $k$ is specified by the size of the test and test statistic $\mathrm{\Lambda }$.

In general, we will have good tests with this method. The problem with this method is to calculate $maxL(\theta )$ or the distribution of $\mathrm{\Lambda }$, which is necessary of the evaluation of the power of the test.

The asymptotic distribution of the likelihood quotient

Unbiased Point Estimation
Convergence in distribution

Prop: Let $X_1,\dots ,X_n$ be a random sample of $f(x;\underset{―}{\theta })$ where $\underset{―}{\theta }=({\theta }_1,\dots ,{\theta }_k)$. For the hypothesis: $$H_0: \theta_1 =\theta_1', \dots, \theta_r= \theta_r', \theta_{r+1}, \dots, \theta_k$$where ${\theta }_1^{\prime },{\theta }_2^{\prime },\dots ,{\theta }_r^{\prime }$ be fixed values and ${\theta }_{r+1},\dots ,{\theta }_k$ are not specified, it satisfies $-2\ln \lambda \stackrel{d}{⟶}{\chi }^2(r)$ (converges in distribution) when $H_0$ is true

Th: For testing the hypotheses $H_0:\theta ={\theta }_0$ against $H_1:\theta \ne {\theta }_0$, where $\theta$ is a parameter, suppose that $X_1,\dots ,X_n$ is a random sample of a population of with density function $f(x;\theta )$, which satisfies the regularity conditions, and let $\hat{\theta }$ be the maximum likelihood estimator of $\theta$. Then under $H_0$, when $n\to \mathrm{\infty }$, it satisfies that $-2\ln \lambda \stackrel{d}{⟶}{\chi }^2(1)$ (converges in distribution)