Early studies reported extreme findings with large variability: A meta-epidemiologic study in the field of endocrinology

Objectives To evaluate the presence of extreme findings and fluctuation in effect size in endocrinology. Study Design and Settings We systematically identified all meta-analyses published in 2014 in the field of endocrinology. Within each meta-analysis, the effect size of the primary binary outcome was compared across studies according to their order of publication. We pooled studies using the DerSimonian and Laird random-effects method. Heterogeneity was evaluated using the I² and tau². Results Twelve percent of the included 100 meta-analyses reported the largest effect size in the very first published study. The largest effect size occurred in the first 2 earliest studies in 31% of meta-analyses. When the effect size was the largest in the first published study, it was three times larger than the final pooled effect (ratio of rates, 3.26; 95% confidence interval: 1.80, 5.90). The largest heterogeneity measured by I² was observed in 18% of the included meta-analyses when combining the first 2 studies or 17% when combing the first 3 studies. Conclusions In endocrinology, early studies reported extreme findings with large variability. This behavior of the evidence needs to be taken into account when used to formulate clinical policies.