Rationale and Objectives
We aim to evaluate the long-term performance of readers who had participated in previous magnetic resonance imaging (MRI) reader training in grading Crohn disease activity.
Materials and Methods
Fourteen readers (8 women; 12 radiologists, 2 residents; mean age 40; range 31–59), who had participated in a previous MRI reader training, participated in a follow-up evaluation after a mean interval of 29 months (range 25–34 months). Follow-up evaluation comprised 25 MRI cases of suspected or known Crohn disease patients with direct feedback; cases were identical to the evaluation set used in the initial reader training (of which readers were unaware). Grading accuracy, overstaging, and understaging were compared between training and follow-up using a consensus score by two experienced abdominal radiologists as the reference standard.
Results
In the follow-up evaluation, overall grading accuracy was 73% (95% confidence interval [CI]: 62%–81%), which was comparable to reader training grading accuracy (72%, 95% CI: 61%–80%) ( P = .66). Overstaging decreased significantly from 19% (95% CI: 12%–27%) to 13% (95% CI: 8%–21%) between training and follow-up ( P = .03), whereas understaging increased significantly from 9% (95% CI: 4%–21%) to 14% (95% CI: 7%–26%) ( P < .01).
Conclusions
Readers have consistent long-term accuracy for grading Crohn disease activity after case-based reader training with direct feedback.
Introduction
Magnetic resonance imaging (MRI) is the preferred technique for assessment of disease activity in Crohn disease because of its capability to evaluate intra- and extramural disease of both small bowel and colon, without the use of ionizing radiation. A previous study showed that inexperienced radiologists and residents could be successfully trained to grade Crohn disease activity using case-based training . In that study, readers graded 100 MRI cases of patients with suspected or known Crohn disease, with the readers receiving direct feedback after each case. Significant improvement of grading accuracy was seen (66% to 75%, P = .003), whereas understaging decreased significantly (15% to 7%, P < .001).
Although case-based reader training can provide short-term improvement, to our knowledge, no studies have investigated long-term performance, as reader evaluation was always performed at the time of training. Furthermore, there is no knowledge of how daily practice impacts reader’s level of experience after training.
Get Radiology Tree app to read full this article<
Materials and Methods
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Follow-Up Evaluation
Get Radiology Tree app to read full this article<
TABLE 1
Patient and MRI Characteristics
Parameter Gender Male or female 13/12 Age at time of imaging Years (mean/SD) 37 (13) Previous surgery_n_ (%) 11 (44%) MRI technique 1.5 T 20 3.0 T 5 Preparation Enterography 16 Enteroclysis 9 Disease activity of the patients per examination (reference standard) None 7 Mild 6 Moderate 5 Severe 7
MRI, magnetic resonance imaging; SD, standard deviation.
TABLE 2
MRI Activity Grading, Score of Features and Complications Per Patient .
MRI Features Score of MRI Features 0 1 2 3 Mural thickness 1–3 mm >3–5 mm >5–7 mm >7 mm Mural T2 signal Normal bowel Minor increase Moderate increase Severe increase T1 enhancement Normal bowel Minor increase Moderate increase Severe increase Mural enhancement pattern NA Homogeneous Mucosal Layered Total length of disease 0 cm >0–5 cm >5–15 cm >15 cm Comb sign No Yes Presence of complications Infiltrate \* No Yes Abscess No Yes Fistula No Yes Severe stenosis \\ No Yes
MRI, magnetic resonance imaging; NA, not applicable.
Lesions in the small bowel and colon were assessed. In case of involvement of more than one lesion, the lesion with the highest activity score was leading in the final assessment.
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
TABLE 3
Description of Disease Activity Based on MRI Features and Complications in Table 2
Crohn Disease Activity None No signs of Crohn disease activity Mild Signs of activity; no features with score 3; no complications; total score ≤8 Moderate Score 9–13 or contains a feature with score 3; no signs of complications Severe Presence of at least one complication or a total score of ≥14
MRI, magnetic resonance imaging.
Get Radiology Tree app to read full this article<
Reference Standard
Get Radiology Tree app to read full this article<
Statistical Analysis
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Results
Readers
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Grading Accuracy, Understaging, and Overstaging
Get Radiology Tree app to read full this article<
TABLE 4
Grading Accuracy, Understaging, and Overstaging for Reader Training and Follow-Up Evaluation with Corresponding P Values and Grading Accuracies Per Category of Severity
Reader Training Follow-Up_P_ Value Grading accuracy (95% CI) 72% (95% CI: 61%–80%) 73% (95% CI: 62%–81%) .66 Understaging (95% CI)9% (95% CI: 4%–21%)14% (95% CI: 7%–26%)<.01 Overstaging (95% CI)19% (95% CI: 12%–27%)13% (95% CI: 8%–21%).03 Grading accuracy per category: None (95% CI)75% (95% CI: 59%–86%)82% (95% CI: 70%–90%).03 Mild (95% CI) 61% (95% CI: 38%–80%) 62% (95% CI: 41%–80%) .87 Moderate (95% CI) 60% (95% CI: 50%–69%) 61% (95% CI: 46%–75%) .82 Severe (95% CI) 88% (95% CI: 61%–97%) 84% (95% CI: 58%–95%) .15
CI, confidence interval. Bold values indicate statistical significant differences ( P < .05).
TABLE 5
Individual MRI Features
MRI Feature
(Numbers in Parentheses
Reported by Reference Standard) Grading Accuracy (95% CI) Mural thickness 69% (57%–78%) Mural T2 signal 59% (50%–68%) T1 enhancement 49% (39%–59%) Enhancement pattern 65% (54%–75%) Total length of disease 66% (55%–75%) Comb sign ( n = 8) 74% (61%–84%) Infiltrate ( n = 4) 93% (88%–96%) Abscess ( n = 1) 97% (92%–99%) Fistula ( n = 5) 90% (81%–95%) Severe stenosis ( n = 6) 83% (74%–90%)
CI, confidence interval; MRI, magnetic resonance imaging.
Get Radiology Tree app to read full this article<
Intra-Rater Reliability
Get Radiology Tree app to read full this article<
Interim Experience
Get Radiology Tree app to read full this article<
TABLE 6
Comparison of Grading Accuracies Between Reader Training and Follow-Up Evaluation with Readers Divided Based on Experience with MRE and Abdominal MRI
MRE Reader Training, Grading Accuracy (95% CI) Follow-Up, Grading Accuracy \* (95% CI)P Value Low experience 69% (95% CI: 56%–79%) 75% (95% CI: 64%–83%) .1 High experience 75% (95% CI: 63%–84%) 71% (95% CI: 59%–80%) .23 Abdominal MRI Low experience 70% (95% CI: 58%–79%) 69% (95% CI: 57%–79%) .88 High experience 74% (95% CI: 62%–83%) 77% (95% CI: 66%–85%) .36
CI, confidence interval; MRE, magnetic resonance enterography or enteroclysis; MRI, magnetic resonance imaging.
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Grading Confidence
Get Radiology Tree app to read full this article<
Discussion
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Funding
Get Radiology Tree app to read full this article<
Acknowledgments
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
References
1. Tielbeek J.A., Bipat S., Boellaard T.N., et. al.: Training readers to improve their accuracy in grading Crohn’s disease activity on MRI. Eur Radiol 2014; 24: pp. 1059-1067.
2. Steward M.J., Punwani S., Proctor I., et. al.: Non-perforating small bowel Crohn’s disease assessed by MRI enterography: derivation and histopathological validation of an MR-based activity index. Eur J Radiol 2012; 81: pp. 2080-2088.
3. Tielbeek J.A., Makanyanga J.C., Bipat S., et. al.: Grading Crohn disease activity with MRI: interobserver variability of MRI features, MRI scoring of severity, and correlation with Crohn disease endoscopic index of severity. AJR Am J Roentgenol 2013; 201: pp. 1220-1228.
4. Jauregui-Amezaga A., Rimola J., Ordas I., et. al.: Value of endoscopy and MRI for predicting intestinal surgery in patients with Crohn’s disease in the era of biologics. Gut 2015; 64: pp. 1397-1402.
5. Boellaard T.N., Nio C.Y., Bossuyt P.M., et. al.: Can radiographers be trained to triage CT colonography for extracolonic findings?. Eur Radiol 2012; 22: pp. 2780-2789.
6. Leeuwenburgh M.M., Wiarda B.M., Bipat S., et. al.: Acute appendicitis on abdominal MR images: training readers to improve diagnostic accuracy. Radiology 2012; 264: pp. 455-463.
7. Liedenbaum M.H., Bipat S., Bossuyt P.M., et. al.: Evaluation of a standardized CT colonography training program for novice readers. Radiology 2011; 258: pp. 477-487.
8. Rimola J., Rodriguez S., Garcia-Bosch O., et. al.: Magnetic resonance for assessment of disease activity and severity in ileocolonic Crohn’s disease. Gut 2009; 58: pp. 1113-1120.
9. Bankier A.A., Levine D., Halpern E.F., et. al.: Consensus interpretation in imaging research: is there a better way?. Radiology 2010; 257: pp. 14-17.