Deakin University
Browse

File(s) under embargo

Selective and (mis)leading economics journals: Meta-research evidence

journal contribution
posted on 2024-01-04, 04:07 authored by Z Askarov, A Doucouliagos, H Doucouliagos, Tom StanleyTom Stanley
AbstractWe assess statistical power and excess statistical significance among 31 leading economics general interest and field journals using 22,281 parameter estimates from 368 distinct areas of economics research. Median statistical power in leading economics journals is very low (only 7%), and excess statistical significance is quite high (19%). Power this low and excess significance this high raise serious doubts about the credibility of economics research. We find that 26% of all reported results have undergone some process of selection for statistical significance and 56% of statistically significant results were selected to be statistically significant. Selection bias is greater at the top five journals, where 66% of statistically significant results were selected to be statistically significant. A large majority of empirical evidence reported in leading economics journals is potentially misleading. Results reported to be statistically significant are about as likely to be misleading as not (falsely positive) and statistically nonsignificant results are much more likely to be misleading (falsely negative). We also compare observational to experimental research and find that the quality of experimental economic evidence is notably higher.

History

Journal

Journal of Economic Surveys

Location

London, Eng.

ISSN

0950-0804

eISSN

1467-6419

Language

English

Publication classification

C1 Refereed article in a scholarly journal

Publisher

WILEY