Arbeitspapier

Unpacking P-Hacking and Publication Bias

to connected objects

We use unique data from journal submissions to identify and unpack publication bias and p-hacking. We find that initial submissions display significant bunching, suggesting the distribution among published statistics cannot be fully attributed to a publication bias in peer review. Desk-rejected manuscripts display greater heaping than those sent for review i.e. marginally significant results are more likely to be desk rejected. Reviewer recommendations, in contrast, are positively associated with statistical significance. Overall, the peer review process has little effect on the distribution of test statistics. Lastly, we track rejected papers and present evidence that the prevalence of publication biases is perhaps not as prominent as feared.

Language: Englisch

Bibliographic citation: Series: I4R Discussion Paper Series ; No. 52

Classification: Wirtschaft
Role of Economics; Role of Economists; Market for Economists
Estimation: General
Econometric and Statistical Methods: Special Topics: General

Subject: publication bias
p-hacking
selective reporting

Event: Geistige Schöpfung

(who): Brodeur, Abel
Carrell, Scott
Figlio, David
Lusher, Lester

Event: Veröffentlichung

(who): Institute for Replication (I4R)

(where): s.l.

(when): 2023

Handle: http://hdl.handle.net/10419/274141

Last update: 10.03.2025, 11:44 AM CET

Data provider

This object is provided by:
ZBW - Deutsche Zentralbibliothek für Wirtschaftswissenschaften - Leibniz-Informationszentrum Wirtschaft. If you have any questions about the object, please contact the data provider.

Show original at data provider

Object type

Arbeitspapier

Associated

Brodeur, Abel
Carrell, Scott
Figlio, David
Lusher, Lester
Institute for Replication (I4R)

Time of origin

2023

Other Objects (12)

Unpacking P-Hacking and Publication Bias

Arbeitspapier

Unpacking P-Hacking and Publication Bias

Do Pre-Registration and Pre-analysis Plans Reduce p-Hacking and Publication Bias?

Arbeitspapier

Do Pre-Registration and Pre-analysis Plans Reduce p-Hacking and Publication Bias?

Do Pre-registration and Pre-analysis Plans Reduce P-Hacking and Publication Bias?

Arbeitspapier

Do Pre-registration and Pre-analysis Plans Reduce P-Hacking and Publication Bias?

Publication bias in measuring intertemporal substitution

Arbeitspapier

Publication bias in measuring intertemporal substitution

Publication bias in measuring climate sensitivity

Arbeitspapier

Publication bias in measuring climate sensitivity

We Need to Talk about Mechanical Turk: What 22,989 Hypothesis Tests Tell us about p-Hacking and Publication Bias in Online Experiments

Arbeitspapier

We Need to Talk about Mechanical Turk: What 22,989 Hypothesis Tests Tell us about p-Hacking and Publication Bias in Online Experiments

We Need to Talk about Mechanical Turk: What 22,989 Hypothesis Tests Tell us about p-Hacking and Publication Bias in Online Experiments

Arbeitspapier

We Need to Talk about Mechanical Turk: What 22,989 Hypothesis Tests Tell us about p-Hacking and Publication Bias in Online Experiments

We Need to Talk about Mechanical Turk: What 22,989 Hypothesis Tests Tell Us about Publication Bias and p-Hacking in Online Experiments

Arbeitspapier

We Need to Talk about Mechanical Turk: What 22,989 Hypothesis Tests Tell Us about Publication Bias and p-Hacking in Online Experiments

Do Pre-Registration and Pre-Analysis Plans Reduce p-Hacking and Publication Bias?: Evidence from 15,992 Test Statistics and Suggestions for Improvement

Arbeitspapier

Do Pre-Registration and Pre-Analysis Plans Reduce p-Hacking and Publication Bias?: Evidence from 15,992 Test Statistics and Suggestions for Improvement

Do Pre-Registration and Pre-Analysis Plans Reduce p-Hacking and Publication Bias? Evidence from 15,992 Test Statistics and Suggestions for Improvement

Arbeitspapier

Do Pre-Registration and Pre-Analysis Plans Reduce p-Hacking and Publication Bias? Evidence from 15,992 Test Statistics and Suggestions for Improvement

Publication Bias and Editorial Statement on Negative Findings

Arbeitspapier

Publication Bias and Editorial Statement on Negative Findings

Unpacking Name-Based Race Discrimination

Arbeitspapier

Unpacking Name-Based Race Discrimination

Unpacking P-Hacking and Publication Bias

Arbeitspapier

Unpacking P-Hacking and Publication Bias

Do Pre-Registration and Pre-analysis Plans Reduce p-Hacking and Publication Bias?

Arbeitspapier

Do Pre-Registration and Pre-analysis Plans Reduce p-Hacking and Publication Bias?

Do Pre-registration and Pre-analysis Plans Reduce P-Hacking and Publication Bias?

Arbeitspapier

Do Pre-registration and Pre-analysis Plans Reduce P-Hacking and Publication Bias?

Publication bias in measuring intertemporal substitution

Arbeitspapier

Publication bias in measuring intertemporal substitution

Publication bias in measuring climate sensitivity

Arbeitspapier

Publication bias in measuring climate sensitivity

We Need to Talk about Mechanical Turk: What 22,989 Hypothesis Tests Tell us about p-Hacking and Publication Bias in Online Experiments

Arbeitspapier

We Need to Talk about Mechanical Turk: What 22,989 Hypothesis Tests Tell us about p-Hacking and Publication Bias in Online Experiments

We Need to Talk about Mechanical Turk: What 22,989 Hypothesis Tests Tell us about p-Hacking and Publication Bias in Online Experiments

Arbeitspapier

We Need to Talk about Mechanical Turk: What 22,989 Hypothesis Tests Tell us about p-Hacking and Publication Bias in Online Experiments

We Need to Talk about Mechanical Turk: What 22,989 Hypothesis Tests Tell Us about Publication Bias and p-Hacking in Online Experiments

Arbeitspapier

We Need to Talk about Mechanical Turk: What 22,989 Hypothesis Tests Tell Us about Publication Bias and p-Hacking in Online Experiments

Do Pre-Registration and Pre-Analysis Plans Reduce p-Hacking and Publication Bias?: Evidence from 15,992 Test Statistics and Suggestions for Improvement

Arbeitspapier

Do Pre-Registration and Pre-Analysis Plans Reduce p-Hacking and Publication Bias?: Evidence from 15,992 Test Statistics and Suggestions for Improvement

Do Pre-Registration and Pre-Analysis Plans Reduce p-Hacking and Publication Bias? Evidence from 15,992 Test Statistics and Suggestions for Improvement

Arbeitspapier

Do Pre-Registration and Pre-Analysis Plans Reduce p-Hacking and Publication Bias? Evidence from 15,992 Test Statistics and Suggestions for Improvement

Publication Bias and Editorial Statement on Negative Findings

Arbeitspapier

Publication Bias and Editorial Statement on Negative Findings

Unpacking Name-Based Race Discrimination

Arbeitspapier

Unpacking Name-Based Race Discrimination

Unpacking P-Hacking and Publication Bias

Arbeitspapier

Unpacking P-Hacking and Publication Bias

Do Pre-Registration and Pre-analysis Plans Reduce p-Hacking and Publication Bias?

Arbeitspapier

Do Pre-Registration and Pre-analysis Plans Reduce p-Hacking and Publication Bias?

Do Pre-registration and Pre-analysis Plans Reduce P-Hacking and Publication Bias?

Arbeitspapier

Do Pre-registration and Pre-analysis Plans Reduce P-Hacking and Publication Bias?

Publication bias in measuring intertemporal substitution

Arbeitspapier

Publication bias in measuring intertemporal substitution

Publication bias in measuring climate sensitivity

Arbeitspapier

Publication bias in measuring climate sensitivity

We Need to Talk about Mechanical Turk: What 22,989 Hypothesis Tests Tell us about p-Hacking and Publication Bias in Online Experiments

Arbeitspapier

We Need to Talk about Mechanical Turk: What 22,989 Hypothesis Tests Tell us about p-Hacking and Publication Bias in Online Experiments

We Need to Talk about Mechanical Turk: What 22,989 Hypothesis Tests Tell us about p-Hacking and Publication Bias in Online Experiments

Arbeitspapier

We Need to Talk about Mechanical Turk: What 22,989 Hypothesis Tests Tell us about p-Hacking and Publication Bias in Online Experiments

We Need to Talk about Mechanical Turk: What 22,989 Hypothesis Tests Tell Us about Publication Bias and p-Hacking in Online Experiments

Arbeitspapier

We Need to Talk about Mechanical Turk: What 22,989 Hypothesis Tests Tell Us about Publication Bias and p-Hacking in Online Experiments

Do Pre-Registration and Pre-Analysis Plans Reduce p-Hacking and Publication Bias?: Evidence from 15,992 Test Statistics and Suggestions for Improvement

Arbeitspapier

Do Pre-Registration and Pre-Analysis Plans Reduce p-Hacking and Publication Bias?: Evidence from 15,992 Test Statistics and Suggestions for Improvement

Do Pre-Registration and Pre-Analysis Plans Reduce p-Hacking and Publication Bias? Evidence from 15,992 Test Statistics and Suggestions for Improvement

Arbeitspapier

Do Pre-Registration and Pre-Analysis Plans Reduce p-Hacking and Publication Bias? Evidence from 15,992 Test Statistics and Suggestions for Improvement

Publication Bias and Editorial Statement on Negative Findings

Arbeitspapier

Publication Bias and Editorial Statement on Negative Findings

Unpacking Name-Based Race Discrimination

Arbeitspapier

Unpacking Name-Based Race Discrimination

Cultural heritage institutions wishing to register will find more information here.

Fields marked * need to be filled in.

Username*

Please enter your username

Email*

Please enter your email address

Please do not fill this field

First name

Last name

Password*

Please enter your password

Confirm password*

Please enter the same password

I have read the terms of use and the privacy policy for the collection of personal data and accept them. *

This field is required.

I would like to subscribe to the newsletter of the Deutsche Digitale Bibliothek. See newsletter subscription info.

Account created

Your "My DDB" account has been successfully created. Before you can log in to your account, you must click the confirmation link in the message we just sent to the email address you provided.