Great post by Siminie Vazire on whether independent replication work should be held to a higher standard than original research. I posted a comment to her post arguing that replication studies should definitely *not* be held to higher standards because sloppy original research has much more potential to cause damage overall when you consider the far reaching negative consequences original research can have in terms of wasted time, resources, and opportunity costs when large number of labs try to extend false positive findings.
In this post, I want to expand a bit more on these thoughts.
First, I want to make it clear that I agree that research standards for replication work should be very high, because replication work is very crucial to the scientific process! However, original research should also be held to that same very high standard because of the immeasurably high amount of damage unreliable original findings can have on a scientific literature. Highly provocative original findings will typically incite a large number of labs to try and extend those findings with the hope of learning deeper insights about those empirical phenomena. If the original findings are unreliable, however, this can lead to an enormous amount of wasted time and resources by the labs trying to extend those original findings. Furthermore, we shouldn’t forget about opportunity costs. To add insult to injury, unreliable original findings also impose immense opportunity costs for the (often) very smart researchers in those other labs that could have been working on *other* research questions that would have had a higher probability of yielding cumulative knowledge.
For example, the original research by Bargh et al. (1996) and Dijksterhuis et al. (1998) led to a gargantuan wave of research by hundreds of labs around the world attempting to shed light on the mechanisms of those social priming effects. Likely thousands of studies have been executed in this spirit (most of them likely file-drawered). Only time will tell, but it might turn out to be the case that all of the time, money, and resources invested in these efforts will not have yielded much fruit in return.
Take-home message: Yes let’s raise research standards for replication work because replication work is so crucial to ensuring the self-corrective process of science. But let’s remember that research standards for original work needs to be substantially raised as well. Hence, we definitely need to raise research standards, but we need to do this for *all* empirical research in psychology.
Visual and quantitative depiction of my take-home message: