Statistics How To

Likelihood-Ratio Tests (Probability and Mathematical Statistics)

Hypothesis Testing Likelihood-Ratio Tests

This article is about Likelihood-Ratio Tests used in probability and mathematical Statistics. If you’re looking for information about the ratio used to assess diagnostic tests in medicine, see this other article: What is a Likelihood Ratio?

What is a Likelihood-Ratio Test?

The Likelihood-Ratio test (sometimes called the likelihood-ratio chi-squared test) is a hypothesis test that helps you choose the “best” model between two nested models. “Nested models” means that one is a special case of the other. For example, you might want to find out which of the following models is the best fit:

  • Model One has four predictor variables (height, weight, age, sex),
  • Model Two has two predictor variables (age,sex). It is “nested” within model one because it has just two of the predictor variables (age, sex).

This theory cam also be applied to matrices. For example, a scaled identity matrix is nested within a more complex compound symmetry matrix.

The best model is the one that makes the data most likely, or maximizes the likelihood function, fn(X – 1, … , Xn|Θ).
Although the concept is relatively easy to grasp (i.e. the likelihood function is highest nearer the true value for Θ), the calculations to find the inputs for the procedure are not. Likelihood-ratio tests use log-likelihood functions, which are are difficult and lengthy to calculate by hand. Most statistical software packages have built in functions to handle them.

Running the Test

Basically, the test compares the fit of two models. The null hypothesis is that the smaller model is the “best” model; It is rejected when the test statistic is large. In other words, if the null hypothesis is rejected, then the larger model is a significant improvement over the smaller one.

If you know the log-likelihood functions for the two models, the test statistic is relatively easy to calculate as the ratio between the log-likelihood of the simpler model (s) to the model with more parameters (g):
likelihood-ratio tests formula


You might also see this equation with “s” written as the likelihood for the null model and “g” written as the likelihood for the alternative model.

The test statistic approximates a chi-squared random variable. Degrees of freedom for the test equal the difference in the number of parameters for the two models.

------------------------------------------------------------------------------

If you prefer an online interactive environment to learn R and statistics, this free R Tutorial by Datacamp is a great way to get started. If you're are somewhat comfortable with R and are interested in going deeper into Statistics, try this Statistics with R track.

Comments are now closed for this post. Need help or want to post a correction? Please post a comment on our Facebook page and I'll do my best to help!
Likelihood-Ratio Tests (Probability and Mathematical Statistics) was last modified: October 12th, 2017 by Stephanie Glen

One thought on “Likelihood-Ratio Tests (Probability and Mathematical Statistics)