Computer-aided Detection of Small Pulmonary Nodules in Chest Radiographs

Rationale and Objectives

To evaluate the impact of computer-aided detection (CAD, IQQA-Chest; EDDA Technology, Princeton Junction, NJ) used as second reader on the detection of small pulmonary nodules in chest radiography (CXR).

Materials and Methods

A total of 113 patients (mean age 62 years) with CT and CXR within 6 weeks were selected. Fifty-nine patients showed 101 pulmonary nodules (diameter 5–15mm); the remaining 54 patients served as negative controls. Six readers of varying experience individually evaluated the CXR without and with CAD as second reader in two separate reading sessions. The sensitivity per lesion, figure of merit (FOM), and mean false positive per image (mFP) were calculated. Institutional review board approval was waived.

Results

With CAD, the sensitivity increased for inexperienced readers (39% vs. 45%, P < .05) and remained unchanged for experienced readers (50% vs. 51%). The mFP nonsignificantly increased for both inexperienced and experienced readers (0.27 vs. 0.34 and 0.16 vs. 0.21). The mean FOM did not significantly differ for readings without and with CAD irrespective of reader experience (0.71 vs. 0.71 and 0.84 vs. 0.87). All readers together dismissed 33% of true-positive CAD candidates. False-positive candidates by CAD provoked 40% of all false-positive marks made by the readers.

Conclusion

CAD improves the sensitivity of inexperienced readers for the detection of small nodules at the expense of loss of specificity. Overall performance by means of FOM was therefore not affected. To use CAD more beneficial, readers need to improve their ability to differentiate true from false-positive CAD candidates.

In clinical practice, small primary lung carcinomas and pulmonary metastases may be missed on two-view chest radiographs (CXR), though they were frequently visible in retrospect. Depending on the study design, miss rates between 20% and 90% have been reported for primary lung carcinomas .

Factors that contribute to detection errors include image quality, size and type of nodules, superposition of anatomical structures, the presence of accompanying abnormalities and the radiologists’ variable experience and perception capacity. The goal of computer-aided detection (CAD) software is to reduce the effects of the latter, namely to lower the number of perception errors.

Get Radiology Tree app to read full this article<

Materials and methods

Study Group

Get Radiology Tree app to read full this article<

Intrapulmonary Nodules

Get Radiology Tree app to read full this article<

Table 1

Characteristics of the 101 Nodules

Characteristics Shape M 27 T 74 Conspicuity High (1) 21 Moderate (2) 39 Low (3) 20 Very low (4) 21 Size <10 mm 42 ≥10 mm 59

M, round nodules; T, nodules with irregular margins.

Get Radiology Tree app to read full this article<

Nodule Conspicuity

Get Radiology Tree app to read full this article<

Chest Radiography

Get Radiology Tree app to read full this article<

CAD

Get Radiology Tree app to read full this article<

Image Evaluation

Get Radiology Tree app to read full this article<

Statistical Analysis

Get Radiology Tree app to read full this article<

Results

Study Groups and Nodule Characteristics

Get Radiology Tree app to read full this article<

Standalone Performance of CAD

Get Radiology Tree app to read full this article<

Reader Performance without CAD

Get Radiology Tree app to read full this article<

Table 2

Individual Reader Outcome without and with Use of CAD

FOM Sensitivity (in%) mFP Baseline CAD Baseline CAD Baseline CAD Inexperienced readers ( n = 3) 0.71 0.71 39 45 0.27 0.34 1 0.52 0.53 39 46 0.54 0.67 2 0.77 0.79 35 39 0.08 0.13 3 0.84 0.80 42 50 0.18 0.21 Experienced readers ( n = 3) 0.84 0.87 50 51 0.16 0.21 4 0.85 0.91 45 44 0.15 0.18 5 0.80 0.83 53 54 0.20 0.24 6 0.87 0.88 53 57 0.12 0.20

Baseline, reading without CAD; CAD: reading with computer-aided detection; FOM, figure of merit; mFP, mean false positives per image.

Get Radiology Tree app to read full this article<

Table 3

Sensitivity (in %) of CAD Standalone and of Readers without and with CAD for Subgroups of Nodules, Dependent on Shape, Conspicuity, Size and Presence of Anatomic Noise

Shape Conspicuity Size AN T M 1+2 3+4 <10 mm ≥10 mm 0+1 2+3 CAD standalone 48 46 70 30 40 51 51 39 All readers Baseline 38 46 76 22 25 58 52 31 ( n = 6) CAD 42 50 84 23 29 61 56 31 Inexperienced readers Baseline 30 41 69 17 19 52 45 27 ( n = 3) CAD 37 47 82 19 25 58 52 30 Experienced readers Baseline 46 51 83 27 31 63 58 34 ( n = 3) CAD 47 52 85 28 33 64 61 33

AN, anatomic noise; baseline, reading without CAD; CAD, reading with computer-aided detection as second reader; M, round nodules; T, nodules with irregular margins.

Get Radiology Tree app to read full this article<

Reader Performance with CAD as Second Reader

Get Radiology Tree app to read full this article<

Reader-CAD Interaction

Get Radiology Tree app to read full this article<

Table 4

Dismissed True-positive CAD Candidates (in Absolute Numbers and in Percentage of all CAD True Positives between Brackets) Pooled over all Readers and over Subgroups of Experienced and Inexperienced Readers

Shape Conspicuity Size AN T M 1+2 3+4 <10 mm ≥10 mm 0+1 1+2 All readers ( n = 6) 34 (44%) 60 (29%) 21 (12%) 73 (68%) 52 (51%) 42 (23%) 50 (25%) 44 (52%) Inexperienced Readers ( n = 3) 19 (49%) 32 (31%) 11 (13%) 40 (74%) 28 (55%) 23 (26%) 29 (29%) 22 (52%) Experienced Readers ( n = 3) 15 (38%) 28 (27%) 10 (11%) 33 (61%) 24 (47%) 19 (21%) 21 (21%) 22 (52%)

AN, anatomic noise; CAD, computed-aided detection; M, round nodules; T, nodules with irregular margins.

Get Radiology Tree app to read full this article<

Interobserver Agreement and Agreement with CAD

Get Radiology Tree app to read full this article<

Discussion

Get Radiology Tree app to read full this article<

References

1. Muhm J.R., Miller W.E., Fontana R.S., et. al.: Lung cancer detected during a screening program using 4-month chest radiographs. Radiology 1983; 148: pp. 609-615.
2. Heelan R.T., Flehinger B.J., Melamed M.R., et. al.: Non-small-cell lung cancer: results of the New York screening program. Radiology 1984; 151: pp. 289-293.
3. Austin J.H., Romney B.M., Goldsmith L.S.: Missed bronchogenic carcinoma: radiographic findings in 27 patients with potentially resectable lesion evident in retrospect. Radiology 1992; 182: pp. 115-122.
4. Quekel L.G., Kessels A.G., Goei R., et. al.: Miss rate of lung cancer on the chest radiograph in clinical practice. Chest 1999; 115: pp. 720-724.
5. Monnier-Cholley L., Arrivé L., Porcel A., et. al.: Characteristics of missed lung cancer on chest radiographs: a French experience. Eur Radiol 2001; 11: pp. 597-605.
6. Shah P.K., Austin J.H., White C.S., et. al.: Missed non–small cell lung cancers: radiographic findings of potentially resectable lesions evident only in retrospect. Radiology 2003; 226: pp. 235-241.
7. Wu H.M., Gotway M.B., Lee T.J., et. al.: Features of non-small cell lung carcinomas overlooked at digital chest radiography. Clinical Radiol 2008; 63: pp. 518-528.
8. Li F., Engelmann R., Metz C.E., et. al.: Lung cancers missed on chest radiographs: results obtained with a commercial computer-aided detection program. Radiology 2008; 246: pp. 273-280.
9. Van Beek E.J., Mullan B., Thompson B.: Evaluation of a real-time interactive pulmonary nodule analysis system on chest digital radiographic images: a prospective study. Acad Radiol 2008; 15: pp. 571-575.
10. Bley T.A., Baumann T., Saueressig U., et. al.: Comparison of radiologist and CAD performance in the detection of CT-confirmed subtle pulmonary nodules on digital chest radiographs. Invest Radiol 2008; 43: pp. 343-348.
11. White C.S., Flukinger T., Jeudy J., et. al.: Use of a computer-aided detection system to detect missed lung cancer at chest radiography. Radiology 2009; 252: pp. 273-281.
12. Sakai S., Soeda H., Takahashi N., et. al.: Computer-aided detection on digital chest radiography: validation test on consecutive T1 cases of resectable lung cancer. J Digit Imaging 2006; 19: pp. 376-382.
13. Hardie R.C., Rogers S.K., Wilson T., et. al.: Performance analysis of a new computer aided detection system for identifying lung nodules on chest radiographs. Med Image Anal 2008; 12: pp. 240-258.
14. Gückel C., Hansell D.M.: Imaging the ‘dirty lung’—has high resolution computed tomography cleared the smoke?. Clin Radiol 1998; 53: pp. 717-722.
15. Chakraborty D.P., Berbaum K.S.: Observer studies involving detection and localization: modeling, analysis, and validation. Med Phys 2004; 31: pp. 2313-2330.
16. Chakraborty D.P.: Analysis of location specific observer performance data: validated extensions of the jackknife free-response (JAFROC) method. Acad Radiol 2006; 13: pp. 1187-1193.
17. Vikgren J., Zachrisson S., Svalkvist A., et. al.: Comparison of chest tomosynthesis and chest radiography for the detection of pulmonary nodules: human observer study of clinical cases. Radiology 2008; 249: pp. 1034-1041.
18. Szucs-Farkas Z., Patak M.A., Yuksel-Hatz S., et. al.: Improved detection of pulmonary nodules on energy-subtracted chest radiographs with a commercial computer-aided diagnosis software: comparison with human observers. Eur Radiol 2010; 20: pp. 1289-1296.
19. Balkman J.D., Mehandru S., DuPont E., et. al.: Dual energy subtraction digital radiography improves performance of a next generation computer-aided detection program. J Thoracic Imaging 2010; 25: pp. 41-47.
20. Samulski M., Hupse R., Boetes C., et. al.: Using computer-aided detection in mammography as a decision support. Eur Radiol 2010; 20: pp. 2323-2330.
21. Song W., Fan L., Xie Y., et. al.: A study of inter-observer variations of pulmonary nodule marking and characterizing on DR images. Proc SPIE 2005; 5749: pp. 272-280.
22. Kasai S., Li F., Shiraishi J., et. al.: Usefulness of computer-aided diagnosis schemes for vertebral fractures and lung nodules on chest radiographs. AJR Am J Roentgenol 2008; 191: pp. 260-265.
23. MacMahon H., Engelmann R., Behlen F.M., et. al.: Computer-aided diagnosis of pulmonary nodules: results of a large scale observer test. Radiology 1999; 213: pp. 723-726.
24. Freedman M.T., Lo S.C.B., Osicka T., et. al.: Computer-aided detection of lung cancer on chest radiographs: effect of machine CAD false-positive locations on radiologists’ behavior. Proc SPIE 2002; 4684: pp. 1311-1319.
25. Shiraishi J., Abe H., Li F., et. al.: Computer-aided diagnosis for the detection and classification of lung cancers on chest radiographs: ROC analysis of radiologists’ performance. Acad Radiol 2006; 13: pp. 995-1003.
26. de Hoop B., De Boo D.W., Gietema H.A., et. al.: Computer-aided detection of lung cancer on chest radiographs: effect on observer performance. Radiology 2010; 257: pp. 532-540.
27. Nishikawa R. Increased CAD use prompts look at advantages, drawbacks. RSNA News February 2007. http://www.rsna.org/Publications/rsnanews/feb07/upload/RSNANews_Feb07_CAD_Usage.pdf .

Computer-aided Detection of Small Pulmonary Nodules in Chest Radiographs

Rationale and Objectives

Materials and Methods

Results

Conclusion

Materials and methods

Study Group

Intrapulmonary Nodules

Nodule Conspicuity

Chest Radiography

CAD

Image Evaluation

Statistical Analysis

Results

Study Groups and Nodule Characteristics

Standalone Performance of CAD

Reader Performance without CAD

Reader Performance with CAD as Second Reader

Reader-CAD Interaction

Interobserver Agreement and Agreement with CAD

Discussion

References

Further Reading

A Margin Sharpness Measurement for the Diagnosis of Breast Cancer from Magnetic Resonance Imaging Examinations

A Meta-analysis of 64-section Coronary CT Angiography Findings for Predicting 30-day Major Adverse Cardiac Events in Patients Presenting with Symptoms Suggestive of Acute Coronary Syndrome

A Report on the Current Status of Grand Rounds in Radiology Residency Programs in the United States