Consequential Validity: Definition, Examples

Design of Experiments > Consequential Validity

What is Consequential Validity?

Consequential validity refers to the positive or negative social consequences of a particular test. For example, the consequential validity of standardized tests include many positive attributes, including: improved student learning and motivation and ensuring that all students have access to equal classroom content. However, standardized tests also have several negative consequences as well. They include inappropriate use of the tests to re-allocate state funds, and teaching students to pass tests (instead of actually understanding the material). This type of validity nearly always refers to some type of educational testing, although theoretically it could be expanded to other areas.

Consequential validity was first proposed by Samuel Messick, a psychologist working for the Educational Testing Service. He defined two parts to the procedure:

Making sure tests are labeled correctly and thoughtfully. Labels should accurately describe what they are testing. For example, a mathematics test should not be called an “intelligence test”.
Identifying “…potential and actual social consequences of applied testing”.

However, it was Lorrie Shepard took Messick’s thoughts a step further and specifically argued for investigating the social consequences of tests. This offers challenges to anyone developing a test, as many consequences are only seen once the test is in use. It can be impossible to predict all consequences, even major ones. Plus, even if some effect is seen (i.e. negative attitudes by students), it’s not always possible to identify a certain cause and effect link.

For the reasons above, consequential validity, while considered a type of validity by some authors, isn’t at the time of writing considered part of the validity process by many authors.

References:
Messick, S. (1989). Validity. In R.L. Linn (Ed.), Educational measurement (3rd ed., pp. 13-103). New York: Macmillan.
Shepard, L.A. (1993). “Evaluating Test Validity.” In L. Darling-Hammon (Ed.), Review of Research in Education, Vol. 19. Washington, DC: AERA.