What does it mean to replicate a research study?
|
In principle, a well-designed and well-documented experiment will accurately describe a real phenomenon and provide other researchers with everything they need to reproduce those results in a comparable study. To replicate an experiment, researchers use the same methods of data collection and apply the same analysis, even though they have to use new subjects, often in somewhat different situations. Subjects may be generally older or
younger, for example, or from a different country. While later tests can’t possibly be identical to the original, they need to be similar enough to yield comparable data. Concepts like “similar enough” and “comparable data” involve a level of subjectivity that adds to the controversy around replication. When a study fails a replication attempt, defenders often question whether the followup truly reflected the original. Researchers help to resolve such debates when they openly share
their data and methods. Recent Posts A Juno Award-winning teacher wants all his students to feel there is a place for them in music Cities Are Driving EvolutionGlobally crowdsourced study shows that white clovers are biologically adapting to city life, demonstrating the profound impact of urbanization From Strength to StrengthU of T’s new Tanenbaum Institute for Science in Sport will help athletes at all levels perform better  What is Replicable Research?Research is replicable when an independent group of researchers can copy the same process and arrive at the same results as the original study. Hence, establishing its validity. For example, if someone conducts experiment A and arrives at conclusion B and you attempt to replicate this experiment but end up with conclusion C, the experiment is not replicable. This is closely related to the idea of empirical generalizations. Empirical generalizations are results that cannot be replicated by independent researchers using valid, but different, methods. There are three key aspects to the concept of replicability: a finding being replicated, the independent group and the use of valid, but different, methods. Replication of a single finding The concept of replicability relates only to a single finding. If a study contains multiple findings, each finding must be assessed separately. While the failure to replicate one finding of a study will typically bode badly for the rest, it is not unusual to have only a subset of findings from a study be replicable. The independent group To be considered valid, the group of researchers replicating the study need to be independent of the original researchers. "Independent" means that they have no reason not to be dispassionately objective in their attempts to replicate the findings. Examples of a lack of independence include people working in the same department, people who have met socially, and situations where the original researchers have considerable influence (e.g., editors of major journals). Valid-but-different methodsThere are many potential explanations for different research findings. The most desirable explanation is typically that the research result is consistent with the theory because it is correct. However, there are many other explanations for findings: fraud, flukes, technical errors, and spurious correlations. For this reason, research is typically only regarded as being replicable when the findings have been reproduced using a methodology that is, in some important sense, different to that of the original study. Where a finding is found in multiple different ways it is said to be generalizable. Why should you care that research is replicable?In order to take a scientific study or experiment seriously, the results need to be able to be proven multiple times by independent researchers. If something is proven to be true once, why should you trust that it will always be true? Research that has been shown to be replicable affords greater confidence in the results. It is also crucial that the process of replicating research is undertaken by independent researchers. This will reduce the chances of the research being influenced by selection bias or other dodgy elements. Related conceptsTwo concepts that are closely related to the idea of replicable research are
Each of these is less stringent than replicability. Why would a researcher want to replicate a study?Replication is one of the key ways scientists build confidence in the scientific merit of results. When the result from one study is found to be consistent by another study, it is more likely to represent a reliable claim to new knowledge.
What is an example of replication in research?For example, imagine that health psychologists perform an experiment showing that hypnosis can be effective in helping middle-aged smokers kick their nicotine habit. Other researchers might want to replicate the same study with younger smokers to see if they reach the same result.
|
