Spurious Precision in Meta-Analysis

Arbeitspapier

Meta-analysis upweights studies reporting lower standard errors and hence more precision. But in empirical practice, notably in observational research, precision is not given to the researcher. Precision must be estimated, and thus can be p-hacked to achieve statistical significance. Simulations show that a modest dose of spurious precision creates a formidable problem for inverse-variance weighting and bias-correction methods based on the funnel plot. Selection models fail to solve the problem, and the simple mean can beat sophisticated estimators. Cures to publication bias may become worse than the disease. We introduce an approach that surmounts spuriousness: the Meta-Analysis Instrumental Variable Estimator (MAIVE), which employs inverse sample size as an instrument for reported variance.

Object Details

• Language: Englisch

Classification and Topics

• Classification: Wirtschaft ; Statistical Simulation Methods: General ; Single Equation Models: Single Variables: Instrumental Variables (IV) Estimation ; Survey Methods; Sampling Methods
• Subject: Publication bias ; p-hacking ; selection models ; meta-regression ; funnel plot ; inverse-variance weighting

Contributors, Places and Time

• Event: Geistige Schöpfung
• (who): Irsova, Zuzana ; Bom, Pedro R. D. ; Havranek, Tomas ; Rachinger, Heiko
• Event: Veröffentlichung
• (who): ZBW – Leibniz Information Centre for Economics
• (where): Kiel, Hamburg
• (when): 2023

Further information

• Handle: http://hdl.handle.net/10419/268683
• Last update: 10.03.2025, 11:45 AM CET