The technique is widely used in medical research; The price tag that comes with any clinical trial means that researchers are under pressure to derive the greatest possible amount of information out of any one trial.
Subgroup analysis is often divided into two types: pre-specified analysis and post hoc analysis.
- Pre-specified analysis is subgroup analysis that was planned during the initial experiment design stage, before looking at any data.
- Post hoc analysis is decided on and planned after the data has come in.
Dangers in Subgroup Analysis
Subset analysis has a much higher rate of false positives than primary research because multiple tests are performed on the same data set; A large number of ‘uninteresting results’ may be ignored in favor of one subset result which is cherry picked by the researcher. See: Multiple testing problem.
Because of this it is important that when subset analysis is done:
- The results are clearly labeled as subgroup analysis in the resulting write up.
- Appropriate significance levels are generated and stated.
- It is made clear, in the write up, whether the analysis is pre-specified or post hoc.
If, and only if, these guidelines are followed, subgroup analysis can be a very informative part of any major research project.
References
Dijkman, Kooistra, & Bhandari. How to work with a subgroup analysis. Canadian Journal of Surgery. 2009 Dec; 52(6): 515–522. Retrieved from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2792383/ on July 7, 2018
Lagakos SW (20 April 2006). The challenge of subgroup analyses–reporting without distorting. New England Journal of Medicine. 354 (16): 1667–9. doi:10.1056/NEJMp068070. PMID 16625007. Retrieved from https://www.nejm.org/doi/full/10.1056/NEJMp068070 on July 7, 2018
Ting, N. et al. (Eds.) (2020). Design and Analysis of Subgroups with Biopharmaceutical Applications (Emerging Topics in Statistics and Biostatistics) 1st ed.
Tutorial on statistical considerations on subgroup analysis in confirmatory clinical trials. Statistics in Medicine. Volume 36, Issue 8. 15 April 2017. Pages 1334-1360
Retrieved from https://onlinelibrary.wiley.com/doi/abs/10.1002/sim.7167 on July 7, 2018
Wang, Lagakos, Ware, Hunter, and Drazen. Reporting of Sub group Analyses in Clinical Trials. N Engl J Med 2007; 357:2189-2194 DOI: 10.1056/NEJMsr077003. Retrieved from https://www.nejm.org/doi/full/10.1056/NEJMsr077003 on July 7, 2018