academic incentive structure

High-powered direct replications of social psychology findings (for in press paper; out-of-date)

Uncategorized, , , Leave a comment
***IMPORTANT NOTE***: This list was compiled on October 13, 2015 solely for an in press paper at JPSP, to be referenced as additional replications of social psych findings **beyond** large-scale replication efforts such as RP:P, Social Psych special issue, ML1, and ML3 and was not meant to be disseminated widely. Hence, this list is completely out-of-date. For a more systematic effort to track replications in psychology, see Curate Science.

The table below lists successful (n=3) and unsuccessful (n=111) high-powered direct replications of social psychology findings (known to us on October 13, 2015). For simplicity, only replications with statistical power >= 80% to detect an effect size as large (or larger) than the original finding are included (citation counts according to Google Scholar, retrieved October 2015). This list was tabulated as additional evidence to support the broader position that the current incentive structure in social psychology is not conducive to generating cumulative knowledge in light of several meta-scientific investigations revealing low replicability rates of social psychology findings (e.g., Reproducibility Project: 76% replication failure rate of social psychology studies; Social Psych special issue: 70% failure rate; Many Labs 3: 88% failure rate).

 

Effect

Cited
by

Unsuccessful
replications*

Successful
replications*

Elderly
priming

3703

Pashler et al. (2009)

Cesario et al. (2007, Study 2)

Doyen et al. (2012, Study 1)

Doyen et al. (2012, Study 2)

–

Achievement
priming

1672

Harris et al. (2013, Study 1)

Harris et al. (2013, Study 2)

–

Deliberation-without-attention
effect

973

Acker (2008)

Calvillo & Penaloza
(2009)

Lassiter et al. (2009)

Newell et al. (2009)

Rey et al. (2009)

Thorsteinson & Withrow
(2009)

Nieuwenstein & van Rijn (2012, Study 1)

Nieuwenstein & van Rijn (2012, Study 2)

–

Behavioral-consequences-of
automatic-evaluation

943

Rotteveel et al. (2015, Study 1)

Rotteveel et al. (2015, Study 2)

–

Glucose-boosts-self-control
effect

853

Cesario & Corker (2010)

Astrologo et al. (2014)

–

Disgust
priming

713

Johnson et al. (2015)

Zhong et al. (2010, Study 2)

–

Physical
warmth promotes interpersonal warmth

693

Lynott et al. (2014, Study 1)

Lynott et al. (2014, Study 2)

Lynott et al. (2014, Study 3)

–

Money
priming

686

Tate (2009)

Grenier et al. (2012)

Rohrer et al. (2015, Study 1)

Rohrer et al. (2015, Study 2)

Rohrer et al. (2015, Study 3)

–

Intelligence
priming

676

Eder et al. (2001)

Shanks et al. (2013, Study 4)

Shanks et al. (2013, Study 5)

Shanks et al. (2013, Study 6)

Shanks et al. (2013, Study 8)

–

Fertility
facial-preferences effect

602

Harris (2011)

–

Macbeth
effect

578

Earp et al. (2014, Study 1)

Earp et al. (2014, Study 2)

Earp et al. (2014, Study 3)

Gamez et al. (2011, Study 2)

Gamez et al. (2011, Study 3)

Fayard et al. (2009, Study 1)

–

Pre-cognition

393

Wagenmakers et al. (2011)

Galak et al. (2012, Study 1)

Galak et al. (2012, Study 2)

Galak et al. (2012, Study 3)

Galak et al. (2012, Study 4)

Galak et al. (2012, Study 6)

Galak et al. (2012, Study 7)

Ritchie et al. (2012, Study 1)

Ritchie et al. (2012, Study 2)

Ritchie et al. (2012, Study 3)

Galak et al. (2012, Study 5)

 

Status-legitimacy
effect

344

Brandt (2013, Study 1)

Brandt (2013, Study 2)

Brandt (2013, Study 3)

–

Red-cognitive-impairment
effect

321

Steele et al. (2015, Study 1)

Steele et al. (2015, Study 2)

Steele et al. (2015, Study 3)

Steele et al. (2015, Study 4)

–

Power
posing

304

Ranehill et al. (2015)

Koch & Broughal
(2011)

–

Cleanliness
priming

278

Johnson et al. (2014a, Study 1)

Johnson et al. (2014a, Study 2)

Lee et al. (2013)

Johnson et al. (2014b)

–

Reduced
pro-sociality of high SES effect

258

Korndorfer et al. (2015, Study 1)

Korndorfer et al. (2015, Study 2)

Korndorfer et al. (2015, Study 3)

Korndorfer et al. (2015, Study 4)

Korndorfer et al. (2015, Study 5)

Korndorfer et al. (2015, Study 6)

Korndorfer et al. (2015, Study 7)

Korndorfer et al. (2015, Study 8)

Morling et al. (2014)

Social
distance priming

247

Pashler et al. (2012, Study 1)

Pashler et al. (2012, Study 2)

Johnson & Cesario
(2012, Study 1)

Johnson & Cesario
(2012, Study 2)

Sykes et al. (2012)

–

Color
on approach/avoidance

227

Steele (2013)

Steele (2014)

–

Social
warmth embodiment effect

117

Donnellan et al. (2015, Study 1)

Donnellan et al. (2015, Study 2)

Donnellan et al. (2015, Study 3)

Donnellan et al. (2015, Study 4)

Donnellan et al. (2015, Study 5)

Donnellan et al. (2015, Study 6)

Donnellan et al. (2015, Study 7)

Donnellan et al. (2015, Study 8)

Donnellan et al. (2015, Study 9)

Ferrell et al. (2014)

McDonald et al. (2015)

–

Red-boosts-attractiveness
effect

98

Banas et al. (2013)

Blech (2014)

Hesslinger et al. (2015)

–

Fertility
on voting

60

Harris & Mickes
(2014)

–

Process
model of AMP

55

Tobin & LeBel
(Study 1)

Tobin & LeBel
(Study 2)

Honesty
priming

47

Pashler et al. (2013, Study 1)

Pashler et al. (2013, Study 2)

Pashler et al. (2013, Study 3)

–

Modulation
of 1/f noise on WIT

45

Madurski & LeBel
(2015, Study 1)

Madurski & LeBel
(2015, Study 2)

–

Embodiment
of secrets

31

LeBel & Wilbur (2014, Study 1)

LeBel & Wilbur (2014, Study 2)

Perfecto, Moon, & Nelson (2012)

–

Time
is money effect

25

Connnors et al. (in press, Study 1)

Connnors et al. (in press, Study 2)

–

Heat
priming

23

McCarthy (2014, Study 1)

McCarthy (2014, Study 1)

–

Treating-prejudice-with-imagery
effect

22

McDonald et al. (2014, Study 1)

McDonald et al. (2014, Study 2)

–

Religion
priming

11

McCullough & Hone (2015)

–

Attachment-warmth
embodiment effect

10

LeBel & Campbell (2013, Study 1)

LeBel & Campbell (2013, Study 2)

–