Hypothesis Testing > MANCOVA Sample Size
About the MANCOVA Sample Size
MANCOVA (Multivariate Analysis of Covariance) tests for a statistically significant difference on the effect of an independent variable on two or more dependent variables, while removing the effects of one or more covariates. The ability to remove a variety of covariates comes at a cost: One of the major problems with decreasing sample size in MANCOVA is that unequal n’s appear, violating the assumption of homogeneity of variance-covariance matrices [4].
You need a larger sample size than for other tests. If you are unable to meet this requirement, a MANOVA (i.e. the same test without looking at covariates) can be more powerful.
Calculating MANCOVA Sample Size
The measure of effect size in MANCOVA is Cohen’s f2</sup> (an extension of Cohen’s d). Sample size depends on many factors including the number of levels of the independent variable and the number of dependent variables. Typically, a power analysis (using software) is conducted to obtain a “large enough” sample.
Resources for calculating sample size for MANCOVA are hard to find. One approach is to use the freely available GPower program for MANOVA (without covariates), then adjust the denominator degrees of freedom. If k is the number of cells (independent variables * dependent variables) in your design and g is the number of covariates, then groups = k *g [1].
One size doesn’t fit all: The power analysis is specific to the different multivariate tests on the Group factor and for each covariate. For example, a MANCOVA with eight levels and three dependent variables with α = .05, power = 0.80, and a 0.40 effect size needs a sample size of just 24 [2]. If you have three groups and the number of DVs is equal to the number of DVs plus the number of covariates (which is three in this example), with α = .05, moderate power of 0.15, and a 0.20 effect size, then you need a sample size of 54 [1].
The Problem with Small Samples and Covariates
For small group size (under 20), more than three covariates becomes problematic, because power will be low for small of medium effect sizes (f2 ≤ .15) [1]. The following formula can be used to assess a limit for the number of covariates in a model with small sample sizes [3]:
C = (.10*N) – (J – 1),
where:
- C = number of covariates
- J = number of groups
- N = sample size.
For example, for three groups with 40 participants, C < 4 – 2 = 2. That means more than two covariates will make the results from any analysis suspect.
References
[1] Dattalo, P. (2013). Analysis of Multiple Dependent Variables. Oxford University Press.
[2] Statistics Solutions. (2010). Sample Size Write-up [WWW Document]. Retrieved from http://www.statisticssolutions.com/resources/sample-size-calculator/mancova-groups/mancova-8-levelsand-3-dependent-variables
[3] Huitema, B. (1980). The Analysis of Covariance and Alternatives. Wiley.
[4] Hasan, N. (2020). MANOVA/MANCOVA using SPSS.
Retrieved October 12, 2021 from: https://sites.education.miami.edu/statsu/wp-content/uploads/sites/4/2020/07/MANOVAMANCOVA-Webinar.pdf