Arbeitspapier
The Robustness Reproducibility of the American Economic Review
We estimate the robustness reproducibility of key results from 17 non-experimental AER papers published in 2013 (8 papers) and 2022/23 (9 papers). We find that many of the results are not robust, with no improvement over time. The fraction of significant robustness tests (p﹤0.05) varies between 17% and 88% across the papers with a mean of 46%. The mean relative t/z-value of the robustness tests varies between 35% and 87% with a mean of 63%, suggesting selective reporting of analytical specifications that exaggerate statistical significance. A sample of economists (n=359) overestimates robustness reproducibility, but predictions are correlated with observed reproducibility.
- Language
-
Englisch
- Bibliographic citation
-
Series: I4R Discussion Paper Series ; No. 124
- Classification
-
Wirtschaft
- Event
-
Geistige Schöpfung
- (who)
-
Campbell, Douglas
Brodeur, Abel
Dreber, Anna
Johannesson, Magnus
Kopecky, Joseph
Lusher, Lester
Tsoy, Nikita
- Event
-
Veröffentlichung
- (who)
-
Institute for Replication (I4R)
- (where)
-
s.l.
- (when)
-
2024
- Last update
-
10.03.2025, 11:41 AM CET
Data provider
This object is provided by:
ZBW - Deutsche Zentralbibliothek für Wirtschaftswissenschaften - Leibniz-Informationszentrum Wirtschaft. If you have any questions about the object, please contact the data provider.
ZBW - Deutsche Zentralbibliothek für Wirtschaftswissenschaften - Leibniz-Informationszentrum Wirtschaft. If you have any questions about the object, please contact the data provider.
Object type
- Arbeitspapier
Associated
- Campbell, Douglas
- Brodeur, Abel
- Dreber, Anna
- Johannesson, Magnus
- Kopecky, Joseph
- Lusher, Lester
- Tsoy, Nikita
- Institute for Replication (I4R)
Time of origin
- 2024
Other Objects (12)
Reproducibility and Robustness Replicability of Gsottbauer et al. (2022)
Robustness and reproducibility for AI learning in biomedical sciences: RENOIR
Techno-Economic Analysis of the Heating System Robustness
Statistical power, accuracy, reproducibility and robustness of a graph clusterability test
The Empirical Economic Growth Literature: Robustness, Significance and Size
On the robustness of gender differences in economic behavior
A Retrospective, Longitudinal External Study of the robustness and reproducibility of National Antibacterial Prescribing Survey Data
The American Economic Problem
Robustness Reproducibility of "Improving Workplace Climate in Large Corporations: A Clustered Randomized Intervention"
On the Robustness of Robustness Checks of the Environmental Kuznets Curve
The robustness of the causal and economic relationship between construction flows and economic growth: evidence from Western Europe
The robustness of the causal and economic relationship between construction flows and economic growth: evidence from Western Europe
Reproducibility and Robustness Replicability of Gsottbauer et al. (2022)
Robustness and reproducibility for AI learning in biomedical sciences: RENOIR
Techno-Economic Analysis of the Heating System Robustness
Statistical power, accuracy, reproducibility and robustness of a graph clusterability test
The Empirical Economic Growth Literature: Robustness, Significance and Size
On the robustness of gender differences in economic behavior
A Retrospective, Longitudinal External Study of the robustness and reproducibility of National Antibacterial Prescribing Survey Data
The American Economic Problem
Robustness Reproducibility of "Improving Workplace Climate in Large Corporations: A Clustered Randomized Intervention"
On the Robustness of Robustness Checks of the Environmental Kuznets Curve
The robustness of the causal and economic relationship between construction flows and economic growth: evidence from Western Europe
The robustness of the causal and economic relationship between construction flows and economic growth: evidence from Western Europe
Reproducibility and Robustness Replicability of Gsottbauer et al. (2022)
Robustness and reproducibility for AI learning in biomedical sciences: RENOIR
Techno-Economic Analysis of the Heating System Robustness
Statistical power, accuracy, reproducibility and robustness of a graph clusterability test
The Empirical Economic Growth Literature: Robustness, Significance and Size
On the robustness of gender differences in economic behavior
A Retrospective, Longitudinal External Study of the robustness and reproducibility of National Antibacterial Prescribing Survey Data
The American Economic Problem
Robustness Reproducibility of "Improving Workplace Climate in Large Corporations: A Clustered Randomized Intervention"
On the Robustness of Robustness Checks of the Environmental Kuznets Curve
The robustness of the causal and economic relationship between construction flows and economic growth: evidence from Western Europe