TY - JOUR
T1 - Above the GRADE
T2 - Evaluation of Guidelines in Critical Care Medicine
AU - Sims, Charles R.
AU - Warner, Matthew A.
AU - Stelfox, Henry Thomas
AU - Hyder, Joseph A.
N1 - Publisher Copyright:
Copyright © 2018 by the Society of Critical Care Medicine and Wolters Kluwer Health, Inc. All Rights Reserved.
PY - 2019/1/1
Y1 - 2019/1/1
N2 - Objectives: We examined recommendations within critical care guidelines to describe the pairing patterns for strength of recommendation and quality of evidence. We further identified recommendations where the reported strength of recommendation was strong while the reported quality of evidence was not high/moderate and then assessed whether such pairings were within five paradigmatic situations offered by Grading of Recommendations Assessment, Development and Evaluation methodology to justify such pairings. Data Sources and Extraction: We identified all clinical critical care guidelines published online from 2011 to 2017 by the Society of Critical Care Medicine along with individual guidelines published by Surviving Sepsis Campaign, Kidney Disease Improving Global Outcomes, American Society for Parenteral and Enteral Nutrition, and the Infectious Disease Society of America/American Thoracic Society. Data Synthesis: In all, 15 documents specifying 681 eligible recommendations demonstrated variation in strength of recommendation (strong n = 215 [31.6%], weak n = 345 [50.7%], none n = 121 [17.8%]) and in quality of evidence (high n = 41 [6.0%], moderate n = 151 [22.2%], low/very low n = 298 [43.8%], and Expert Consensus/none n = 191 [28.1%]). Strength of recommendation and quality of evidence were positively correlated (ρ = 0.66; p < 0.0001). Of 215 strong recommendations, 69 (32.1%) were discordantly paired with evidence other than high/moderate. Twenty-two of 69 (31.9%) involved Strong/Expert Consensus recommendations, a category discouraged by Grading of Recommendations Assessment, Development and Evaluation methodology. Forty-seven of 69 recommendations (68.1%) were comprised of Strong/Low or Strong/Very Low variation requiring justification within five paradigmatic scenarios. Among distribution in the five paradigmatic scenarios of Strong/Low and Strong/Very Low recommendations, the most common paradigmatic scenario was life threatening situation (n = 20/47; 42.6%). Four Strong/Low or Strong/Very Low recommendations (4/47; 8.5%) were outside Grading of Recommendations Assessment, Development and Evaluation methodology. Conclusions: Among a large, diverse assembly of critical care guideline recommendations using Grading of Recommendations Assessment, Development and Evaluation methodology, the strength of evidence of a recommendation was generally associated with the quality of evidence. However, strong recommendations were not infrequently made in the absence of high/moderate quality of evidence. To improve clarity and uptake, future guideline statements may specify why such pairings were made, avoid such pairings when outside of Grading of Recommendations Assessment, Development and Evaluation criteria, and consider separate language for Expert Consensus recommendations (good practice statements).
AB - Objectives: We examined recommendations within critical care guidelines to describe the pairing patterns for strength of recommendation and quality of evidence. We further identified recommendations where the reported strength of recommendation was strong while the reported quality of evidence was not high/moderate and then assessed whether such pairings were within five paradigmatic situations offered by Grading of Recommendations Assessment, Development and Evaluation methodology to justify such pairings. Data Sources and Extraction: We identified all clinical critical care guidelines published online from 2011 to 2017 by the Society of Critical Care Medicine along with individual guidelines published by Surviving Sepsis Campaign, Kidney Disease Improving Global Outcomes, American Society for Parenteral and Enteral Nutrition, and the Infectious Disease Society of America/American Thoracic Society. Data Synthesis: In all, 15 documents specifying 681 eligible recommendations demonstrated variation in strength of recommendation (strong n = 215 [31.6%], weak n = 345 [50.7%], none n = 121 [17.8%]) and in quality of evidence (high n = 41 [6.0%], moderate n = 151 [22.2%], low/very low n = 298 [43.8%], and Expert Consensus/none n = 191 [28.1%]). Strength of recommendation and quality of evidence were positively correlated (ρ = 0.66; p < 0.0001). Of 215 strong recommendations, 69 (32.1%) were discordantly paired with evidence other than high/moderate. Twenty-two of 69 (31.9%) involved Strong/Expert Consensus recommendations, a category discouraged by Grading of Recommendations Assessment, Development and Evaluation methodology. Forty-seven of 69 recommendations (68.1%) were comprised of Strong/Low or Strong/Very Low variation requiring justification within five paradigmatic scenarios. Among distribution in the five paradigmatic scenarios of Strong/Low and Strong/Very Low recommendations, the most common paradigmatic scenario was life threatening situation (n = 20/47; 42.6%). Four Strong/Low or Strong/Very Low recommendations (4/47; 8.5%) were outside Grading of Recommendations Assessment, Development and Evaluation methodology. Conclusions: Among a large, diverse assembly of critical care guideline recommendations using Grading of Recommendations Assessment, Development and Evaluation methodology, the strength of evidence of a recommendation was generally associated with the quality of evidence. However, strong recommendations were not infrequently made in the absence of high/moderate quality of evidence. To improve clarity and uptake, future guideline statements may specify why such pairings were made, avoid such pairings when outside of Grading of Recommendations Assessment, Development and Evaluation criteria, and consider separate language for Expert Consensus recommendations (good practice statements).
KW - GRADE
KW - critical care
KW - epistemology
KW - evidence-based medicine
KW - guidelines
UR - http://www.scopus.com/inward/record.url?scp=85058877330&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85058877330&partnerID=8YFLogxK
U2 - 10.1097/CCM.0000000000003467
DO - 10.1097/CCM.0000000000003467
M3 - Review article
C2 - 30303840
AN - SCOPUS:85058877330
SN - 0090-3493
VL - 47
SP - 109
EP - 113
JO - Critical care medicine
JF - Critical care medicine
IS - 1
ER -