On the Importance of Modeling the Invisible World of Underlying Effect Sizes

Abstract: The headline findings from the Open Science Collaboration (2015)―namely, that 36% of original experiments replicated at p < .05, with the overall replication effect sizes being half as large as the original effects―cannot be meaningfully interpreted without a formal model. A simple model-based approach might ask: what would the state of original science be and what would replication results show if original experiments tested true effects half the time (prior odds = 1), true effects had a medium effect size (Cohen’s δ = 0.50), and power to detect true effects was 50%? Assuming no questionable research practices, 91% of p < .05 findings in the original literature would be true positives. However, only 58% of original p < .05 findings would be expected to replicate using the Open Science Collaboration approach, and the replication effects overall would be only ~60% as large as the original effects. A minor variant of this model yields an expected replication rate of only 45%, with ov.... https://spb.psychopen.eu/index.php/spb/article/view/9981

Object Details

• Location: Deutsche Nationalbibliothek Frankfurt am Main
• Extent: Online-Ressource
• Language: Englisch

References and Relationships

• Bibliographic citation: On the Importance of Modeling the Invisible World of Underlying Effect Sizes ; volume:18 ; day:17 ; month:11 ; year:2023 ; Social psychological bulletin ; 18 (17.11.2023)

Contributors, Places and Time

• Creator: Wilson, Brent M. ; Wixted, John T.

Further information

• DOI: 10.32872/spb.9981
• URN: urn:nbn:de:101:1-2023122304101991093695
• Rights: Open Access; Der Zugriff auf das Objekt ist unbeschränkt möglich.
• Last update: 15.08.2025, 7:38 AM CEST