Statistics How To

Classical Test Theory: Definition

Statistics Definitions > Classical Test Theory

What is Classical Test Theory?

Classical Test Theory (CTT), sometimes called the true score model, is the mathematics behind creating and answering tests and measurement scales. The goal of CTT is to improve tests, particularly the reliability and validity of tests.

Reliability implies consistency: if you take the ACT five times, you should get roughly the same results every time. A test is valid if it measures what it’s supposed to.

It’s called “classic” because Item Response Theory is a more modern framework.

True Scores

Classical Test Theory assumes that each person has an innate true score. It can be summed up with an equation:

X = T + E,

Where:

  • X is an observed score,
  • T is the true score,
  • E is random error.

For example, let’s assume you know exactly 70% of all the material covered in a statistics course. This is your true score (T); A perfect end-of-semester test (which doesn’t exist) should ideally reflect this true score. In reality, you’re likely to score around 65% to 75%. The 5% discrepancy from your true score is the error (E).

The errors are assumed to be normally distributed with a mean of zero; Hypothetically, if you took the test an infinite number of times, your observed score should equal your true score.

Statistics Used in Classical Test Theory

classical test theory

Is your test measuring what it’s supposed to? Image: Wichita State U.



Classical test theory is a collection of many statistics, including the average score, item difficulty, and the test’s reliability.

1.Correlation

Correlation: shows how two variables X and Y are related to each other. Different measures are used for different test types. For example, a dichotomously scored test (e.g. yes/no answers) would be correlated with point-biserial correlation while a polytomously scored test (one with multiple answers) would be scored with the Pearson Correlation Coefficient.

2. Covariance

Covariance is a measure of how much two random variables vary together. It’s similar to variance, but where variance tells you how a single variable varies, co variance tells you how two variables vary together.

3. Discrimination Index

Discrimination Index: the ability of the test to discriminate between different levels of learning or other concept of interest. A high discrimination index indicates the test is able to differentiate between levels.

4. Item difficulty

Item difficulty: a measure of individual test question difficulty. It is the proportion of test takers who answered correctly out of the total number of test takers. For example, an item difficulty score of 89/100 means that out of 100 people, 89 answered correctly.

5. Reliability Coefficient

Reliability coefficient — a measure of how well the test measures achievement. Several methods exist for calculating the coefficient include test-retest, parallel or alternate-form and internal analysis. Rules of thumb for preferred levels of the coefficient:
For high stakes tests (e.g. college admissions), > 0.85.
For low stakes tests (e.g. classroom assessment), > 0.70.

6. Sample Variance / Standard Deviation

The sample variance and sample standard deviation are measures of how spread out the scores are.

7. Standard Error of Measurement

Standard Error of Measurement (SEm): a measure of how much measured test scores are spread around a “true” score.

References:
Crocker, L., & Algina, J. (1986). Introduction to classical &
modern test theory. Orlando, FL: Holt, Rinehart and Winston
McDonald, R. P. (1999). Test theory: A unified treatment. Mahwah, NJ: Lawrence Erlbaum Associates.

------------------------------------------------------------------------------

If you prefer an online interactive environment to learn R and statistics, this free R Tutorial by Datacamp is a great way to get started. If you're are somewhat comfortable with R and are interested in going deeper into Statistics, try this Statistics with R track.

Comments are now closed for this post. Need help or want to post a correction? Please post a comment on our Facebook page and I'll do my best to help!
Classical Test Theory: Definition was last modified: November 13th, 2017 by Stephanie Glen