The Influence of Hidden Researcher Decisions in Applied Microeconomics

Arbeitspapier

Researchers make hundreds of decisions about data collection, preparation, and analysis in their research. We use a many-analysts approach to measure the extent and impact of these decisions. Two published causal empirical results are replicated by seven replicators each. We find large differences in data preparation and analysis decisions, many of which would not likely be reported in a publication. No two replicators reported the same sample size. Statistical significance varied across replications, and for one of the studies the effect's sign varied as well. The standard deviation of estimates across replications was 3-4 times the typical reported standard error.

Object Details

• Language: Englisch

References and Relationships

• Bibliographic citation: Series: IZA Discussion Papers ; No. 13233

Classification and Topics

• Classification: Wirtschaft ; Methodology for Collecting, Estimating, and Organizing Microeconomic Data; Data Access ; Econometric and Statistical Methods and Methodology: General ; Economic Methodology
• Subject: replication ; metascience ; research

Contributors, Places and Time

• Event: Geistige Schöpfung
• (who): Huntington-Klein, Nick ; Arenas, Andreu ; Beam, Emily A. ; Bertoni, Marco ; Bloem, Jeffrey R. ; Burli, Pralhad ; Chen, Naibin ; Greico, Paul ; Ekpe, Godwin ; Pugatch, Todd ; Saavedra, Martin ; Stopnitzky, Yaniv
• Event: Veröffentlichung
• (who): Institute of Labor Economics (IZA)
• (where): Bonn
• (when): 2020

Further information

• Handle: http://hdl.handle.net/10419/223675
• Last update: 10.03.2025, 11:45 AM CET