Scoping review of COVID-19-related systematic reviews and meta-analyses: can we really have confidence in their results?

Rachel Wurth, Michelle Hajdenberg, Francisco J. Barrera, Skand Shekhar, Caroline E. Copacino, Pablo J. Moreno-Peña, Omar A.M. Gharib, Forbes Porter, Swapnil Hiremath, Janet E. Hall, Ernesto L. Schiffrin, Graeme Eisenhofer, Stefan R. Bornstein, Juan P. Brito, José Gerardo González-González, Constantine A. Stratakis, René Rodríguez-Gutiérrez, Fady Hannah-Shmouni

Research output: Contribution to journalReview articlepeer-review


Aim The aim of this study was to systematically appraise the quality of a sample of COVID-19-related systematic reviews (SRs) and discuss internal validity threats affecting the COVID-19 body of evidence. Design We conducted a scoping review of the literature. SRs with or without meta-analysis (MA) that evaluated clinical data, outcomes or treatments for patients with COVID-19 were included. Main outcome measures We extracted quality characteristics guided by A Measurement Tool to Assess Systematic Reviews-2 to calculate a qualitative score. Complementary evaluation of the most prominent published limitations affecting the COVID-19 body of evidence was performed. Results A total of 63 SRs were included. The majority were judged as a critically low methodological quality. Most of the studies were not guided by a pre-established protocol (39, 62%). More than half (39, 62%) failed to address risk of bias when interpreting their results. A comprehensive literature search strategy was reported in most SRs (54, 86%). Appropriate use of statistical methods was evident in nearly all SRs with MAs (39, 95%). Only 16 (33%) studies recognised heterogeneity in the definition of severe COVID-19 as a limitation of the study, and 15 (24%) recognised repeated patient populations as a limitation. Conclusion The methodological and reporting quality of current COVID-19 SR is far from optimal. In addition, most of the current SRs fail to address relevant threats to their internal validity, including repeated patients and heterogeneity in the definition of severe COVID-19. Adherence to proper study design and peer-review practices must remain to mitigate current limitations.

Original languageEnglish (US)
Pages (from-to)372-379
Number of pages8
JournalPostgraduate Medical Journal
Issue number1159
StatePublished - May 1 2022


  • AMSTAR-2
  • COVID-19
  • SARS-CoV-2
  • quality
  • systematic reviews

ASJC Scopus subject areas

  • Medicine(all)


Dive into the research topics of 'Scoping review of COVID-19-related systematic reviews and meta-analyses: can we really have confidence in their results?'. Together they form a unique fingerprint.

Cite this