Replication and Replicability: Simple Definition, Examples

Experimental Design > Replication and Replicability

Contents:

1. Replication
2. Replicability

What is Replication?

In experimental design, replication is where each treatment is assigned to many participants. Or, the entire experiment is repeated on a large group of subjects. The process:

• Improves the significance of your experimental results.
• Reduces variability .
• Increases confidence in your results.

If the treatment you’re studying is effective, long-term averaging will reveal the treatment’s experimental worth. If the treatment isn’t effective, averaging will reveal that as well. However, if you have a very small number of participants, one or two flukes (participants who appear to be positively affected by the treatment) might skew your results. In general, the more participants you have who are exposed to the treatment, the more likely your results will be statistically sound.

What is Replicability?

Replicability refers to whether the results from your test or experiment can be replicated if repeated exactly the same way. In order to demonstrate replicability, you must provide statistical evidence that shows your results can be used to predict outcomes in other experiments. An example of evidence that can suggest replicability is the probability value, or p-value. If your test produces a small p-value (usually less than .05 or 5 percent), it suggests your results are not due to chance. If your results can be replicated by different researchers in different experiments, then your results are called “robust.”

According to Science News, replicability is a big issue in science research. Around half of all pre-clinical research in the United States is not replicable, to a tune of $28 billion per year. In life sciences research, a major issue that causes experiments to be non-replicable is “faulty biological agents and reference materials, such as contaminated and misidentified cell lines.” The reality is, the tiniest of issues can lead to problems with replicating results.

Factors that Can Affect Replicability in Statistics

It isn’t just issues with the original experiment that can affect replicability. Other factors include:

• Single case studies and anecdotal evidence are not replicable as they are examples of a single instance of an event without corroborating evidence.
• Differences in experimental procedure in a subsequent experiment can affect replicability. For example, an inexperienced researcher or one using slightly different equipment may have trouble replicating results.
• People or companies who have a vested interest in confirming results from a test are more likely to find confirming evidence than those who do not have a vested interest.
• Results from replication attempts that are truly independent are more reliable than attempts that are not independent.

References

Experimentation.
Science News. Replicability. Retrieved June 8, 2020 from: https://www.sciencenews.org/replicability