Two-Sample T-Test: When to Use it

What is a Two-Sample T-Test?

A two-sample t-test is used when you want to compare two independent groups to see if their means are different.

When to use a Two-Sample T-Test vs. a Paired T Test

“Independent” implies that the two samples must have come from two completely different populations. In other words, one population can’t have any bearing on the other. If you have independent samples, you can use the two-sample t-test (also called, appropriately, the independent samples t-test). On the other hand, if your samples are connected in some way, run a paired samples t-test. “Connected” means that you are collecting data twice from the same group, person, item or thing.

Examples of when to run a paired t-test:

Testing two production lines to see if their outputs are different. One line feeds into a second line, so the second line depends on the first for at least part of the production.
Comparing test scores for the same group of students before an intensive study session and after the session. You’re testing the same people twice, so a paired test is needed.
You are subjecting two different model cars to crashworthiness, using the same equipment. Although you’re testing different items, they are being subjected to the same conditions and so are paired.

Need help with a homework question? Check out our tutoring page!
Important note: paired t tests control for variability between different sets of people or things. This usually results in the need for a much smaller sample size than with a “regular” two-sample t-test. If it’s possible to design your experiment so that you have paired samples, it will drastically reduce the number of participants you need to get a statistically significant result.

Testing for Normality and Nonparametric Tests

The samples for the two-sample t-test should come from a distribution that’s close to normal. This condition is called the assumption of normality. Signs that your data does not come from a normal distribution include skewness or unusually fat tails.

You can test for normality in a variety of ways, including visual checks (like box plots or Q-Q-plots) and statistical tests like the Shapiro-Wilk test. If you feel comfortable with graphs, a quick look at a graph may be all you need to check for normality. If you aren’t comfortable with graphs, run a statistical tests for normality. Most software packages will run these tests for you. For example, you can find instructions for running normality tests in SPSS here.

If your samples don’t appear to be normally distributed, you can still compare your data with a non parametric test like the Mann-Whitney test.

Next: Running the Independent Samples T-Test

References

Dodge, Y. (2008). The Concise Encyclopedia of Statistics. Springer.
Gonick, L. (1993). The Cartoon Guide to Statistics. HarperPerennial.