< Hypothesis testing < MANOVA
What is MANOVA?
Multivariate ANOVA (MANOVA) expands the capabilities of analysis of variance (ANOVA) to evaluate multiple dependent variables simultaneously. ANOVA examines the differences between three or more group means ; It is basically an ANOVA that examines multiple dependent variables. It’s similar to other tests and experiments that aim to determine if manipulating the independent variable affects the response variable (i.e., the dependent variable).
This test seeks to answer research questions such as:
- What changes in independent variables have statistically significant effects on dependent variables?
- What are the interactions among dependent variables?
- What are the interactions among independent variables?
When to use MANOVA: an example
ANOVA tests have the limitation of evaluating only a single dependent variable at a time in your model. Even if you incorporate multiple independent variables into a general linear model, it still focuses on just one dependent variable. This constraint poses an issue as these models cannot detect patterns across multiple dependent variables. This constraint becomes particularly problematic in cases where a standard ANOVA fails to yield statistically significant results. Thus, you’ll want to use MANOVA for more than one dependent variable.
For example, suppose you have three different weight loss methods: low calorie, paleo, and vegan:
- You want to assess the average scores for these groups: use ANOVA.
- You want to assess the impact of different weight loss methods on men vs. women; since there are two dependent variables (men and women), MANOVA is appropriate to analyze the data.
While ANOVA provides a single (univariate) F-value, MANOVA provides a multivariate F-value. MANOVA assesses multiple dependent variables by creating new synthetic ones — called principal components — that maximize group differences. These new dependent variables are linear combinations of the measured dependent variables.
Interpreting the results of MANOVA, a statistically significant multivariate F-value suggests that something is significant. However, in the example given above, it would not determine which weight loss method worked for men or women, or if they all worked. To identify the specific contributing dependent variable(s), examine each individual component with post-hoc univariate F-tests after obtaining a significant result.
MANOVA assumptions
Assumptions for MANOVA [1], which should be present in data for valid results, include:
- Independent observations,
- Equal (or comparable) variance-covariance matrices for all groups,
- Reliable variables and drawing dependent variables from a multivariate normal distribution; If all variables meet the univariate normality requirement then its likely that the variables meet this assumption (for larger samples, the central limit theorem suggests normality).
- Linearity must be present among all pairs of dependent variables to protect from reduced power; Departure from linearity reduces power because the linear combinations of dependent variables do not maximize the difference between groups.
In sum, check for:
- Independence of observations
- Reliability of continuous variables (Cronbach’s alpha > .8)
- Multivariate normality
- Linearity among all pairs of dependent variables
- Absence of multicollinearity and singularity among dependent variables.
- Equality of variance-covariance matrices – (non-significant result from Box’s M test / Levene’s test).
Singular multivariate data happens when the determinant of the covariance matrix is zero; this means that the dependent variables are perfectly correlated and thus, cannot be distinguished from each other.
Box’s M test and Levene’s test are used to examine the equality of variance-covariance matrices. Box’s M test is more powerful in detecting differences, but is sensitive to violations of assumptions. Levene’s test is less powerful but more robust to violations. If the Box’s M test is non-significant, the variance-covariance matrices are equal. In this case, use the Levene’s test to confirm. If the Box’s M test is significant, do not use the Levene’s test. Instead, use a non-parametric MANOVA test such as the Kruskal-Wallis test.
Power and type I error
If two (or more) dependent variables are highly correlated, the probability of a Type I error — rejecting the null hypothesis when it is true — is reduced, but the trade-off is that the power of the MANOVA test is also reduced.
When two or more dependent variables are highly correlated, it indicates that they measure the same thing to some extent. This can make it challenging to differentiate the effects of the independent variable on any dependent variables in the model. Thus, the probability of a Type I error decreases because the MANOVA test is less likely to identify a significant difference between the groups if dependent variable are highly correlated.
While this might sound good in principle, in practice there is a trade-off: if dependent variables are highly correlated the power of the MANOVA test also decreases because the MANOVA is less likely to detect a genuine difference between groups. The power of a test is the probability of correctly rejecting the null hypothesis when it is false. In other words, it is the probability of identifying a statistically significant difference between groups, if a difference actually exists.
In general, having high power is preferable to low power. However, it is important to balance the test’s power with the risk of committing a Type I error.
Whether or not to use a MANOVA in the presence of highly correlated dependent variables depends on the research question. If the primary goal is to minimize the risk of making a Type I error, then the MANOVA test may suitable. But if the primary concern is high power, then alternative tests may be more appropriate such as discriminant analysis or canonical correlation.
Advantages and disadvantages of MANOVA compared to ANOVA
ANOVA’s main limitation is that it can only assess one dependent variable at a time. This restriction can be problematic in certain situations, as it may hinder the detection of existing effects. On the other hand, MANOVA is considerably more complex than ANOVA, which makes it challenging to assess which independent variables have an effect on dependent variables.
MANOVA has various advantages over ANOVA, including:
- Allows for testing multiple dependent variables.
- Can help to reduce Type I errors.
But it also comes with disadvantages, such as
- Each additional variable added results in the loss of one degree of freedom.
- Dependent variables should ideally be uncorrelated. If they are correlated, including more than one dependent variable in the test may not provide significant advantages due to the loss of degrees of freedom.
References
- MANOVA and MANCOVA. Retrieved August 16, 2023 from: https://sites.education.miami.edu/statsu/2020/10/16/manova-and-mancova/
- Manni2425, CC BY-SA 4.0 https://creativecommons.org/licenses/by-sa/4.0, via Wikimedia Commons