TY - JOUR
T1 - Formative evaluation of standardized training for CT colonographic image interpretation by novice readers
AU - Dachman, Abraham H.
AU - B.Kelly, Katherine
AU - P.Zintsmaster, Michael
AU - Rana, Rich
AU - Khankari, Shweta
AU - Novak, Joseph D.
AU - N.Ali, Arif
AU - Qalbani, Adnan
AU - Fletcher, Joel G.
PY - 2008/10
Y1 - 2008/10
N2 - Purpose: To introduce an educational intervention-specifically, a specialized training course-and perform a formative evaluation of the effect of the intervention on novice reader interpretation of computed tomography (CT) colono- graphic data. Materials and The study was institutional review board approved. Ten Methods: normal and 50 abnormal cases, those of 60 patients with 93 polyps-61 polyps 6-9 mm in diameter and 32 polyps 10 mm or larger-were selected from a previously published trial. Seven novice readers underwent initial training that consisted of a 1-day course, reading assignments, a self-study computer module (with 61 limited data sets), observation of an expert interpreting three cases, and full interpretation of 10 cases with unblinding after each case. After training, the observers independently interpreted 60 cases by means of primary two-dimensional reading with unblinding after each case. For each case, the reading time and the location and maximal diameter of the polyp(s) were recorded. A t test was used to evaluate the observers' improvements, and empirical receiver operating characteristic (ROC) curves were constructed. Results: By-patient sensitivities and specificities were determined for each observer. The lowest by-patient sensitivity at the 6 mm or larger polyp threshold was 86%, with 90% specificity. Four observers had 100% by-patient sensitivity at the 10 mm or larger polyp threshold, with 82%-97% specificity. For polyps 10 mm or larger, mean sensitivity and specificity were 98% and 92%, respectively. For the last 20 cases, the average interpretation time per case was 25 minutes. The range of areas under the ROC curve across observers was low: 0.86-0.95. Conclusion: In the described polyp-enriched cohort, novice CT colono- graphic data readers achieved high sensitivity and good specificity at formative evaluation of a comprehensive training program. Use of a similar comprehensive training method might reduce interreader variability in interpretation accuracy and be useful for reader certification.
AB - Purpose: To introduce an educational intervention-specifically, a specialized training course-and perform a formative evaluation of the effect of the intervention on novice reader interpretation of computed tomography (CT) colono- graphic data. Materials and The study was institutional review board approved. Ten Methods: normal and 50 abnormal cases, those of 60 patients with 93 polyps-61 polyps 6-9 mm in diameter and 32 polyps 10 mm or larger-were selected from a previously published trial. Seven novice readers underwent initial training that consisted of a 1-day course, reading assignments, a self-study computer module (with 61 limited data sets), observation of an expert interpreting three cases, and full interpretation of 10 cases with unblinding after each case. After training, the observers independently interpreted 60 cases by means of primary two-dimensional reading with unblinding after each case. For each case, the reading time and the location and maximal diameter of the polyp(s) were recorded. A t test was used to evaluate the observers' improvements, and empirical receiver operating characteristic (ROC) curves were constructed. Results: By-patient sensitivities and specificities were determined for each observer. The lowest by-patient sensitivity at the 6 mm or larger polyp threshold was 86%, with 90% specificity. Four observers had 100% by-patient sensitivity at the 10 mm or larger polyp threshold, with 82%-97% specificity. For polyps 10 mm or larger, mean sensitivity and specificity were 98% and 92%, respectively. For the last 20 cases, the average interpretation time per case was 25 minutes. The range of areas under the ROC curve across observers was low: 0.86-0.95. Conclusion: In the described polyp-enriched cohort, novice CT colono- graphic data readers achieved high sensitivity and good specificity at formative evaluation of a comprehensive training program. Use of a similar comprehensive training method might reduce interreader variability in interpretation accuracy and be useful for reader certification.
UR - http://www.scopus.com/inward/record.url?scp=53749101522&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=53749101522&partnerID=8YFLogxK
U2 - 10.1148/radiol.2491080059
DO - 10.1148/radiol.2491080059
M3 - Article
C2 - 18796675
AN - SCOPUS:53749101522
SN - 0033-8419
VL - 249
SP - 167
EP - 177
JO - Radiology
JF - Radiology
IS - 1
ER -