It’s difficult to disagree with the idea that psychologists (and other social scientists) need to be more open about their methods and data. Recently, a growing number of researchers have started being a lot more open about their methods and data (see the growing number of public projects on the Open Science Framework for evidence of this). I am very happy with these developments (along with many others) because of course being more transparent in how we do our research goes a long way in increasing the reliability of our empirical findings. I will not focus on these benefits here (but see this blog post for a great review of current open science initiatives and the various benefits of open science practices).
Rather, in this post I will argue that though the open science movement is a wonderful development, a “sufficiently open science” (SOS) movement perhaps would be better. I want to consider the position that at this point in time — in this very special era we are currently living in — adopting SOS practices may actually be more beneficial overall for our field in the long run.
What do I mean by “sufficiently open science” practices? From my perspective, SOS practices means that:
- you provide minimum methods disclosure,
- you share minimum materials, and
- you share the minimum dataset underlying your claims.
That’s it! Absolutely no need to be more open than this, though of course that is completely fine if you so desire.
By “minimum methods disclosure”, I mean that you disclose the four categories of methods disclosure covered by Simmons et al.’s 21-word disclosure (and now required at Psychological Science and other journals). [UPDATE: Minimum methods disclosure actually involves a modified wording of the 21-word disclosure whereby an author needs to disclose only the measures that were analyzed with respect to the target research question, rather than all measures assessed. Thanks to Lorne Campbell for pointing this out.] Consistent with PLoS ONE’s new Data Sharing policy, by “minimal dataset”, I mean sharing “the dataset used to reach the conclusions drawn in the manuscript with related metadata and methods, and any additional data required to reproduce the reported study results in their entirety.” Finally, “minimum materials” means sharing the essential materials and procedures needed for a competent independent researcher to execute a diagnostic direct replication. (And if you’re willing to share the minimum dataset and materials with one interested researcher, then why not take 5 minutes and post these publicly on Figshare.com or the OSF so that if someone else emails you about those things, your job is already done!)
I contend that it’s more profitable — at this point in time — to strive for SOS practices because it is more feasible and pragmatic for time-starved researchers, all of whom are caught in a chaotic and rapidly-changing research landscape. Also, it’s a great stepping stone toward broader openness, which can happen in a more natural and gradual manner. And because SOS practices are more feasible, it means that reaching a tipping point where the majority of researchers are being more open may happen much sooner.
Take-home message: Practice “sufficiently open science” and spread the word to any colleagues you know who may still be on the fence about the open science movement!
Oh and here’s a visual depiction of what I’m trying to say, comparing features of the status quo, sufficiently open science (SOS), and open science (OS) approaches.
**According to Wicherts, Borsboom, Kats, & Molenaar (2006).