The difference in differences (DiD) method is a statistical technique or quasi-experimental design method, and it is used primarily in the social sciences and econometrics. In social science it is sometimes called a “controlled before-and-after” study.
General Method
The basic DiD method involves comparing results from two groups, with data from each group being recorded over two time periods. One group (the control group) is not exposed to any treatment or intervention whatsoever; the other (treatment group) is exposed to a treatment or intervention before or during one of the two time periods. The same observations are made in both groups over each time period.
The data is analyzed by first calculating the difference in first and second time periods, and then subtracting the average gain (or difference) in the control group from the average gain (or difference) in the treatment group.
Assumptions for Difference in Differences
When doing a DiD method analysis we assume that the composition of the groups being studied are stable over the time period we are concerned about. We also assume there are no spillover effects, the amount of treatment or intervention given is not determined by the outcome, and that both groups being studied have parallel trends in their outcome—i.e., if no treatment was given, the difference between the data from the two groups would have a consistent difference over time.
Strengths and Weaknesses of Difference in Differences
The difference in difference method is intuitive and fairly flexible; it will show a causal effect from observational data if the basic assumptions are met. Since it focuses on change, rather than the absolute levels, the groups being compared can start at different levels. Another key strong point to the DiD method is that it accounts for change due to factors other then the treatment or intervention being studied.
That said, the DiD method does have limitations because of its many assumptions. To use the method you need both baseline data and a non-intervention group. You also shouldn’t use it if:
- The amount of treatment is determined by the baseline outcome
- The comparison groups have different trends in their outcomes
- The composition of the groups being studied are not stable
References
Columbia University Population Health Methods: DiD Estimation. Retrieved from
https://www.mailman.columbia.edu/research/population-health-methods/difference-difference-estimation on July 21, 2018.
Zheng, Vivian. Causal inference 101: DiD. Towards Data Science Blog. Published April 24, 2018. Retrieved from https://towardsdatascience.com/causal-inference-101-difference-in-differences-1fbbb0f55e85 on July 21, 2018.
Imbens & Wooldridge. What’s New in Econometrics? Lecture Notes 10, Summer 2007. National Bureau of Economic Research. Retrieved from http://www.nber.org/WNE/lect_10_diffindiffs.pdf on July 21, 2018.