Alternate Forms Reliability

Reliability and Validity > Alternate Forms Reliability

Alternate forms reliability is a measure of reliability between two different forms of the same test. Two equivalent (but different) tests are administered, scores are correlated, and a reliability coefficient is calculated. A test would be deemed reliable if differences in one test’s observed scores correlate with differences in an equivalent test’s scores.

Parallel forms are very similar, but with one major difference: the observed score has the same mean and variance. This isn’t a requirement for alternate forms reliability, which just uses different versions of the same test. That said, you can only interpret correlation between tests in a meaningful way if the alternate forms are also parallel. Proving that two tests are parallel is practically impossible (Furr & Bacharach, 2008); Although interpreting correlations is theoretically possible, it isn’t usually a feasible “real life” option. In addition, although two tests might seem equivalent, a different question here and there might results in the test measuring completely different constructs.

Eliminating Issues with Alternate Forms Reliability

As noted above, it’s extremely challenging to interpret reliability with parallel forms. However, you can take several steps to ensure that your reliability estimate is as good as possible:

1. The two tests should have identical instructions, numbers of items and other core elements. The only difference should be different items.
2. Tests should be administered close together.
3. Practice and transfer effects can be eliminated if half the subjects take test A followed by test B, and half the subjects take test B followed by test A. Note that although this seems a little strange (what’s the point in subjects taking two different tests instead of one?), remember that you’re assessing reliability here, not subject performance. Once you’ve determined that the tests are reliable, you can administer test A or test B to a subject, with the knowledge that the two tests are equivalent in every way.

References

Domino, G. & Domino, M. Psychological Testing, An Introduction. 2006.
Furr, M. & Bacharach, V. Psychometrics: An Introduction, 2008
Reliability for Teachers Activity: What are the three types of reliability? Retrieved June 29, 2020 from: https://www.k-state.edu/ksde/alp/activities/Activity2-3.pdf