File(s) under permanent embargo

Beyond Random Effects: When Small-Study Findings Are More Heterogeneous

journal contribution
posted on 24.11.2022, 03:45 authored by Tom StanleyTom Stanley, H Doucouliagos, JPA Ioannidis
New meta-regression methods are introduced that identify whether the magnitude of heterogeneity across study findings is correlated with their standard errors. Evidence from dozens of meta-analyses finds robust evidence of this correlation and that small-sample studies typically have higher heterogeneity. This correlated heterogeneity violates the random-effects (RE) model of additive and independent heterogeneity. When small studies not only have inadequate statistical power but also high heterogeneity, their scientific contribution is even more dubious. When the heterogeneity variance is correlated with the sampling-error variance to the degree we find, simulations show that RE is dominated by an alternative weighted average, the unrestricted weighted least squares (UWLS). Meta-research evidence combined with simulations establish that UWLS should replace RE as the conventional meta-analysis summary of psychological research.

History

Journal

Advances in Methods and Practices in Psychological Science

Volume

5

Article number

ARTN 25152459221120427

ISSN

2515-2459

eISSN

2515-2467

Language

English

Publication classification

C1 Refereed article in a scholarly journal

Issue

4

Publisher

SAGE PUBLICATIONS INC