The fragility of meta-regression models in observational research

Arbeitspapier

Many meta-regression analyses that synthesize estimates from primary studies have now been published in economics. Meta-regression models attempt to infer the presence of genuine empirical effects even if the authors of primary studies select statistically significant and theory-confirming estimates for publication. Meta-regression models were originally developed for the synthesis of experimental research where randomization ensures unbiased and consistent estimation of the effect of interest. Most economics research is, however, observational and authors of primary studies can search across different regression specifications for statistically significant and theory-confirming estimates. Each regression specification may possibly suffer from biases such as omitted-variable biases that result in biased and inconsistent estimation of the effect of interest. We show that if the authors of primary studies search for statistically significant and theory-confirming estimates, meta-regression models tend to systematically make false-positive findings of genuine empirical effects. The ubiquity of such search processes for specific results may limit the applicability of meta-regression models in identifying genuine empirical effects in economics.

Show object

• Language: Englisch

References and Relationships

• Bibliographic citation: Series: MAGKS Joint Discussion Paper Series in Economics ; No. 03-2016

Classification and Topics

• Classification: Wirtschaft ; Hypothesis Testing: General ; Statistical Simulation Methods: General ; Econometric and Statistical Methods: Special Topics: General
• Subject: meta-regression ; meta-analysis ; p-hacking ; publication bias ; omitted-variable bias ; sampling variability ; sampling error ; Monte Carlo simulation

Contributors, Places and Time

• Event: Geistige Schöpfung
• (who): Bruns, Stephan B.
• Event: Veröffentlichung
• (who): Philipps-University Marburg, School of Business and Economics
• (where): Marburg
• (when): 2016

Further information

• Handle: http://hdl.handle.net/10419/129290