TY - JOUR
T1 - Scoping review of COVID-19-related systematic reviews and meta-analyses
T2 - can we really have confidence in their results?
AU - Wurth, Rachel
AU - Hajdenberg, Michelle
AU - Barrera, Francisco J.
AU - Shekhar, Skand
AU - Copacino, Caroline E.
AU - Moreno-Peña, Pablo J.
AU - Gharib, Omar A.M.
AU - Porter, Forbes
AU - Hiremath, Swapnil
AU - Hall, Janet E.
AU - Schiffrin, Ernesto L.
AU - Eisenhofer, Graeme
AU - Bornstein, Stefan R.
AU - Brito, Juan P.
AU - González-González, José Gerardo
AU - Stratakis, Constantine A.
AU - Rodríguez-Gutiérrez, René
AU - Hannah-Shmouni, Fady
N1 - Publisher Copyright:
© 2022 Author(s).
PY - 2022/5/1
Y1 - 2022/5/1
N2 - Aim The aim of this study was to systematically appraise the quality of a sample of COVID-19-related systematic reviews (SRs) and discuss internal validity threats affecting the COVID-19 body of evidence. Design We conducted a scoping review of the literature. SRs with or without meta-analysis (MA) that evaluated clinical data, outcomes or treatments for patients with COVID-19 were included. Main outcome measures We extracted quality characteristics guided by A Measurement Tool to Assess Systematic Reviews-2 to calculate a qualitative score. Complementary evaluation of the most prominent published limitations affecting the COVID-19 body of evidence was performed. Results A total of 63 SRs were included. The majority were judged as a critically low methodological quality. Most of the studies were not guided by a pre-established protocol (39, 62%). More than half (39, 62%) failed to address risk of bias when interpreting their results. A comprehensive literature search strategy was reported in most SRs (54, 86%). Appropriate use of statistical methods was evident in nearly all SRs with MAs (39, 95%). Only 16 (33%) studies recognised heterogeneity in the definition of severe COVID-19 as a limitation of the study, and 15 (24%) recognised repeated patient populations as a limitation. Conclusion The methodological and reporting quality of current COVID-19 SR is far from optimal. In addition, most of the current SRs fail to address relevant threats to their internal validity, including repeated patients and heterogeneity in the definition of severe COVID-19. Adherence to proper study design and peer-review practices must remain to mitigate current limitations.
AB - Aim The aim of this study was to systematically appraise the quality of a sample of COVID-19-related systematic reviews (SRs) and discuss internal validity threats affecting the COVID-19 body of evidence. Design We conducted a scoping review of the literature. SRs with or without meta-analysis (MA) that evaluated clinical data, outcomes or treatments for patients with COVID-19 were included. Main outcome measures We extracted quality characteristics guided by A Measurement Tool to Assess Systematic Reviews-2 to calculate a qualitative score. Complementary evaluation of the most prominent published limitations affecting the COVID-19 body of evidence was performed. Results A total of 63 SRs were included. The majority were judged as a critically low methodological quality. Most of the studies were not guided by a pre-established protocol (39, 62%). More than half (39, 62%) failed to address risk of bias when interpreting their results. A comprehensive literature search strategy was reported in most SRs (54, 86%). Appropriate use of statistical methods was evident in nearly all SRs with MAs (39, 95%). Only 16 (33%) studies recognised heterogeneity in the definition of severe COVID-19 as a limitation of the study, and 15 (24%) recognised repeated patient populations as a limitation. Conclusion The methodological and reporting quality of current COVID-19 SR is far from optimal. In addition, most of the current SRs fail to address relevant threats to their internal validity, including repeated patients and heterogeneity in the definition of severe COVID-19. Adherence to proper study design and peer-review practices must remain to mitigate current limitations.
KW - AMSTAR-2
KW - COVID-19
KW - SARS-CoV-2
KW - quality
KW - systematic reviews
UR - http://www.scopus.com/inward/record.url?scp=85101716674&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85101716674&partnerID=8YFLogxK
U2 - 10.1136/postgradmedj-2020-139392
DO - 10.1136/postgradmedj-2020-139392
M3 - Review article
C2 - 33637639
AN - SCOPUS:85101716674
SN - 0032-5473
VL - 98
SP - 372
EP - 379
JO - Postgraduate Medical Journal
JF - Postgraduate Medical Journal
IS - 1159
ER -