Omnibus Test

Hypothesis Testing >

An omnibus test (also called a combined test) is an overall test for a whole group of results. For example, an ANOVA is an omnibus test — if you reject the null hypothesis, then one pair may have significant differences or all pairs may be significantly different. You may want to run a post hoc test after an omnibus to find out which pairs are significant and which are not.

In general, if your goal is to plan specific comparisons, the omnibus test may be of no interest — there may not be a logical reason to conduct the test at all. However, your data or hypothesis may lack the groupings needed to make these specific comparisons, it may be a good idea to run an omnibus first, then if the hypothesis is rejected (indicating there is an overall difference) you can examine the data in more detail [1].

Types of Omnibus Test

  • ANOVA: A test to find out if there are differences between groups.
  • Chi-square: an overall test to find differences between cell frequencies.
  • F omnibus test: A special case of the F test for all variables in a regression [2].
  • Kolmogorov – Smirnov: compares your data with a known distribution and lets you know if they have the same distribution. Can also be used to compare two distributions, but if the null hypothesis of equality is rejected, that conclusion can be difficult to interpret (Boos, 1986, cited in [3].

Some authors refer to an omnibus test as the complement of a directional test, which are oriented to specific kind of differences. For example, a log-rank test might be performed with the stipulation of S1(t) = [S0(t)]θ for some θ. Alternatively, the omnibus may test for all differences, e.g., the test might be based on an integral over some time interval [4].

References

[1] Keppe, G. & Zedeck, S. (1989). Data Analysis for Research Designs. Worth Publishers.
[2] Section 9 – multiple testing (f-test). Retrieved June 25, 2022 from: https://scholar.harvard.edu/files/msen/files/sen_gov2000_section9.txt
[3] Babu, G. & Padmanabhan, A. (1996).A Robust Test for Omnibus Alternatives. In E. Brunner and M. Denker (Eds), Madan Puri Festschrift, pp. 319–327.
[4] STAT331 Logrank test. Retrieved June 25, 2022 from: https://web.stanford.edu/~lutian/coursepdf/unitweek3.pdf


Comments? Need to post a correction? Please Contact Us.

Join us on our YouTube Channel