Statistics How To

Orthogonality in Statistics

Statistics Definitions > Orthogonality

What is Orthogonality in Statistics?

Simply put, orthogonality means “uncorrelated.” An orthogonal model means that all independent variables in that model are uncorrelated. If one or more independent variables are correlated, then that model is non-orthogonal.

orthogonality

The design on the left is balanced because it has even levels.

The term “orthogonal” usually only applies to classic ANOVA. An orthogonal ANOVA has all categorical independent variables and each cell in a two-way table has the same number of observations (called a balanced design). On the other hand, general linear models are never orthogonal, as at least one independent variable is not categorical (GLMs have one continuous variable).

Running Tests with Orthogonality

Orthogonality also makes a difference in how statistical tests are run. Orthogonal models only have one way to estimate model parameters and to run statistical tests. Non-orthogonal models have several ways to do this, which means that the results can be more complicated to interpret. In general, more correlation between independent variables means that you should interpret result more cautiously.

Whether a model is orthogonal or non-orthogonal is sometimes a judgment call. For example, let’s say you had four cells in an ANOVA: three cells have 10 subjects and the fourth cell has 9 subjects. Technically this is a non-balanced (and therefore non-orthogonal) design. However, the missing subject in one cell will have very little impact on results. In other words, you can treat this semi-unbalanced design as orthogonal.

Roots in Matrix Algebra

The term orthogonality comes from matrix algebra. Two vectors are orthogonal if the sum of the cross-element products is zero. For example the cross products of these two vectors is zero:
matrix


2(4) + 3(-4) + 4(1) + 0(2) = 0.
(See: matrix multiplication for why this works.)
The same concept (i.e. matrix multiplication) can be applied to cells in a two way table. Orthogonality is present in a model if any factor’s effects sum to zero across the effects of any other factors in the table.

------------------------------------------------------------------------------

If you prefer an online interactive environment to learn R and statistics, this free R Tutorial by Datacamp is a great way to get started. If you're are somewhat comfortable with R and are interested in going deeper into Statistics, try this Statistics with R track.

Comments are now closed for this post. Need help or want to post a correction? Please post a comment on our Facebook page and I'll do my best to help!
Orthogonality in Statistics was last modified: December 26th, 2017 by Stephanie