Design of Experiments > Order Effects
What are Order Effects?
- Meditation,
- Exercise,
- Medication.
The treatment plans could be ordered 123, 132, 231, 213, 312, or 321. Let’s say the percentage of people reporting that treatment 1 (meditation) worked for each treatment plan are:
- 123(25%)
- 132(25%)
- 231 (75%)
- 213 (50%)
- 312 (50%)
- 321 (75%)
When treatment 1 comes first, it has a smaller effect: 25% of patients reported meditation helped when placed first, compared to 75% when it was placed last. Clearly, the order in which treatments are given matter.
What Causes Order Effects?
Order effects can be caused by practically anything and so are notoriously difficult to control for. A few possible reasons they might happen include:
- Fatigue: Participants tire after one or two tests.
- Practice: Participants may perform better as they become more familiar with the testing environment.
- Other Testing Conditions: Participants may gradually improve (or decline) due to factors in the testing environment like heating, lighting, or ergonomics.
Solutions
The obvious way to control for order effects is to give the same order of treatments every time. However, this would not control if one group of patients were more sensitive to a particular order than another group. For example, the control group could be healthy individuals who might be more inclined to enjoy an exercise routine that follows relaxing meditation. The less healthy experimental group might find it more taxing to get up and perform exercise after relaxing.
Counterbalancing, where all possible treatments are offered in all possible orders can control for order effects. A major drawback with this method is that it is expensive and time consuming to include in an experiment to to the large numbers of subjects needed for each possible group. For example, just 4 treatments means 4! = 24 different orders. Justifying such a large experiment is difficult because you won’t be able to prove that order effects exist until the experiment is finished. You could (theoretically) offer the programs in reverse order to control for this type of effect, but again, justifying a double-sized experimental group may be hard to do.
Order Effects on Ballots
While order effects usually refers to experimental research, a related effect in Political Science is ballot order effect, where the order of names on a ballot matters. Miller & Krosnick found that on average, candidates who were placed first on a ballot received 2.5 percent more votes than others listed on the ballot.
Next: Counterbalancing in Research
Reference:
Joanne Miller, Jon A. Krosnick, “The Impact of Candidate Name Order on Election Outcomes,” Public Opinion Quarterly Volume 62 No. 3, 1998, Pg. 291-330. Retrieved October 13, 2013 from here.