Rationale and Objectives
We tested whether satisfaction of search (SOS) effects that occur in computed tomography (CT) examination of the chest on detection of native abnormalities are produced by the addition of simulated pulmonary nodules.
Materials and Methods
Two experiments were conducted. In the first experiment, 70 CT examinations, half that demonstrated diverse, subtle abnormalities and half that demonstrated no native lesions, were read by 18 radiology residents and fellows under two experimental conditions: presented with and without pulmonary nodules. In a second experiment, many of the examinations were replaced to include more salient native abnormalities. This set was read by 14 additional radiology residents and fellows. In both experiments, detection of the natural abnormalities was studied. Receiver operating characteristic (ROC) curve areas for each reader-treatment combination were estimated using empirical and proper ROC models. Additional analyses focused on decision thresholds and visual search time on abnormality-free CT slice ranges. Institutional review board approval and informed consent from 32 participants were obtained.
Results
Observers more often missed diverse native abnormalities when pulmonary nodules were added, but also made fewer false-positive responses. There was no change in ROC area, but decision criteria grew more conservative. The SOS effect on decision thresholds was accompanied by a reduction in search time on abnormality-free CT slice ranges.
Conclusion
The SOS effect in CT examination of the chest is similar to that found in contrast examination of the abdomen, involving induced visual neglect.
Introduction
“What information consumes is rather obvious: it consumes the attention of its recipients. Hence a wealth of information creates a poverty of attention, and a need to allocate that attention efficiently among the overabundance of information sources that might consume it.” —Herbert A. Simon
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Experiment 1: materials and methods
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Experimental Conditions
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
CT Examinations
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Image Display
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Simulation of Pulmonary Nodules on CT Examinations of Patients
Lesion removal
Get Radiology Tree app to read full this article<
Nodule placement
Get Radiology Tree app to read full this article<
Observers
Get Radiology Tree app to read full this article<
Procedure
Get Radiology Tree app to read full this article<
Data Analysis
ROC analysis
Get Radiology Tree app to read full this article<
Revision of scoring based on review of computer-aided detection findings
Get Radiology Tree app to read full this article<
Results
Get Radiology Tree app to read full this article<
Discussion
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Experiment 2: materials and methods
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
CT Examinations
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Observers
Get Radiology Tree app to read full this article<
Procedure
Get Radiology Tree app to read full this article<
Data Analysis
ROC areas
Get Radiology Tree app to read full this article<
Decision thresholds
Get Radiology Tree app to read full this article<
Inspection time
Get Radiology Tree app to read full this article<
Results
Detecting Native Abnormalities
Detection accuracy
Get Radiology Tree app to read full this article<
Decision thresholds
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Inspection Time
Get Radiology Tree app to read full this article<
Discussion
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
References
1. Simon H.A.: Designing organizations for an information-rich world.Greenberger MartinComputers, Communications, and the Public Interest.1971.Johns Hopkins PressBaltimore, MD:pp. 37-52.
2. Tuddenham W.J., Calvert W.P.: Visual search patterns in roentgen diagnosis. Radiology 1961; 76: pp. 255-256.
3. Tuddenham W.J.: Problems of perception in chest roentgenology: facts and fallacies. Radiol Clin North Am 1963; 1: pp. 227-289.
4. Tuddenham W.J.: Visual search, image organization and reader error in roentgen diagnosis. Studies of the psychophysiology of roentgen image perception. Radiology 1962; 78: pp. 694-704.
5. Smith M.J.: Error and variation in diagnostic radiology.1967.Charles C ThomasSpringfield, IL p. 27
6. Renfrew D.L., Franken E.A., Berbaum K.S., et. al.: Error in radiology: classification and lessons of 182 cases presented at a problem case conference. Radiology 1992; 183: pp. 145-150.
7. Ashman C.J., Yu J.S., Wolfman D.: Satisfaction of search in osteoradiology. AJR Am J Roentgenol 2000; 175: pp. 541-544.
8. Berbaum K.S., Franken E.A., Dorfman D.D., et. al.: Satisfaction of search in diagnostic radiology. Invest Radiol 1990; 25: pp. 133-140.
9. Berbaum K.S., Franken E.A., Dorfman D.D., et. al.: Time-course of satisfaction of search. Invest Radiol 1991; 26: pp. 640-648.
10. Berbaum K.S., Franken E.A., Anderson K.L., et. al.: The influence of clinical history on visual search with single and multiple abnormalities. Invest Radiol 1993; 28: pp. 191-201.
11. Franken E.A., Berbaum K.S., Lu C.H., et. al.: Satisfaction of search in detection of plain film abnormalities in abdominal contrast examinations. Invest Radiol 1994; 29: pp. 403-409.
12. Berbaum K.S., El-Khoury G.Y., Franken E.A., et. al.: Missed fractures resulting from satisfaction of search effect. Emerg Radiol 1994; 1: pp. 242-249.
13. Samuel S., Kundel H.L., Nodine C.F., et. al.: Mechanism of satisfaction of search: Eye position recordings in the reading of chest radiographs. Radiology 1995; 94: pp. 895-902.
14. Berbaum K.S., Franken E.A., Dorfman D.D., et. al.: Cause of satisfaction of search effects in contrast studies of the abdomen. Acad Radiol 1996; 3: pp. 815-826.
15. Berbaum K.S., Franken E.A., Dorfman D.D., et. al.: The role of faulty visual search in the satisfaction of search effect in chest radiology. Acad Radiol 1998; 5: pp. 9-19.
16. Berbaum K.S., Dorfman D.D., Franken E.A., et. al.: Proper ROC analysis and joint ROC analysis of the satisfaction of search effect in chest radiography. Acad Radiol 2000; 7: pp. 945-958.
17. Berbaum K.S., Franken E.A., Dorfman D.D., et. al.: The role of faulty decision making in the satisfaction of search effect in chest radiography. Acad Radiol 2000; 7: pp. 1098-1106.
18. Berbaum K.S., Brandser E.A., Franken E.A., et. al.: Gaze dwell times on acute trauma injuries missed because of satisfaction of search. Acad Radiol 2001; 8: pp. 304-314.
19. Berbaum K.S., Franken E.A., Dorfman D.D., et. al.: Can order of report prevent satisfaction of search in abdominal contrast studies?. Acad Radiol 2005; 12: pp. 74-84.
20. Berbaum K.S., Franken E.A., Caldwell R.T., et. al.: Can a checklist reduce SOS errors in chest radiography?. Acad Radiol 2006; 13: pp. 296-304.
21. Berbaum K.S., El-Khoury G.Y., Ohashi K., et. al.: Satisfaction of search in multitrauma patients: Severity of detected fractures. Acad Radiol 2007; 14: pp. 711-722.
22. Berbaum K.S., Caldwell R.T., Schartz K.M., et. al.: Does computer-aided diagnosis for lung tumors change satisfaction of search in chest radiography?. Acad Radiol 2007; 14: pp. 1069-1076.
23. Berbaum K.S., Schartz K.M., Caldwell R.T., et. al.: Satisfaction-of-search for subtle skeletal fractures may not be induced by more serious injury. JACR 2012; 9: pp. 344-351.
24. Schartz K, Berbaum K, Madsen M, et al. Multiple diagnostic task performance in computed tomography examination of the chest. BJR. In press.
25. Berbaum K.S., Franken E.A., Caldwell R.T., et. al.: Satisfaction of search in traditional radiographic imaging.Samei E.Krupinski E.Jacobson F.The handbook of medical image perception and techniques.2010.Cambridge University PressCambridge, UK:pp. 107-138.
26. Gurney J.W.: Missed lung cancer at CT: imaging findings in nine patients. Radiology 1996; 199: pp. 117-122.
27. Kakinuma R., Ohmatsu H., Kaneko M., et. al.: Detection failures in spiral CT screening for lung cancer: analysis of CT findings. Radiology 1999; 212: pp. 61-66.
28. Davis S.D.: Through the “retrospectroscope:” a glimpse of missed lung cancer in CT. Radiology 1996; 199: pp. 23-24.
29. White C.S., Romney B.M., Mason A.C., et. al.: Primary carcinoma of the lung overlooked at CT: analysis of findings in 14 patients. Radiology 1996; 199: pp. 109-115.
30. Schartz KM, Berbaum KS, Caldwell RT, et al. WorkstationJ: workstation emulation software for medical image perception and technology evaluation research. Medical Imaging 2007: Image Perception, Observer Performance, and Technology Assessment, Yulei Jiang, Berkman Sahiner, eds. Proc SPIE 2007; 6515:65151I.
31. Schartz KM, Berbaum KS, Caldwell RT, et al. WorkstationJ as ImageJ plugin for medical image studies. Annual Meeting of the Society for Imaging Informatics in Medicine (SIIM), 9th Annual SIIM Research & Development Symposium, Charlotte, NC, June 6, 2009. Available at: http://www.siimweb.org/assets/FCBE219A-C30B-4003-9892-FACA9230AB91.pdf .
32. Krupinski E.A., Berbaum K.S., Caldwell R.T., et. al.: Long radiology workdays reduce detection and accommodation accuracy. J Am Coll Radiol 2010; 7: pp. 698-704.
33. Madsen M.R., Berbaum K.S., Caldwell R.T.: A new software tool for removing, storing and adding abnormalities to medical images for perception research studies. Acad Radiol 2006; 13: pp. 305-312.
34. Pesce L.L., Metz C.E.: Reliable and computationally efficient maximum-likelihood estimation of “proper” binormal ROC curves. Acad Radiol 2007; 14: pp. 814-829.
35. Kirk R.E.: Experimental Design.2nd ed.1982.WadsworthBelmont, CA pp. 429–548
36. Dixon W.J.: BMDP statistical software manual, volume 2 (accompanying BMDP release 7).1992.University of California PressBerkeley pp. 1239–1433
37. Berbaum K.S., Dorfman D.D., Franken E.A.: Measuring observer performance by ROC analysis: indications and complications. Invest Radiol 1989; 24: pp. 228-233.
38. Berbaum K.S., Dorfman D.D., Franken E.A., et. al.: An empirical comparison of discrete ratings and subjective probability ratings. Acad Radiol 2002; 9: pp. 756-763.
39. Swets J.A., Pickett R.M.: Evaluation of diagnostic systems: methods from signal detection theory.1982.Academic PressNew York p. 39
40. Hillman B.J.: Economic, legal, and ethical rationales for the ACRIN National Lung Screening Trial of CT screening for lung cancer. Acad Radiol 2003; 10: pp. 349-350.
41. Rothenberg B.M., Korn A.: The opportunities and challenges posed by the rapid growth of diagnostic imaging. JACR 2005; 2: pp. 407-410.
42. Andriole K.P., Morin R.L., Arenson R.L., et. al.: Addressing the coming radiology crisis—the Society for Computing Applications in Radiology Transforming the Radiological Interpretation Process (TRIP) Initiative. J Digit Imaging 2004; 17: pp. 235-243.
43. Andriole K.P., Morin R.L.: Transforming medical imaging: the first SCAR TRIPTM conference: a position paper form the SCAR TRIPTM Subcommittee of the SCAR Research and Development Committee. J Digit Imaging 2006; 19: pp. 6-16.
44. Andriole K.P., Wolfe J.M., Khorasani R., et. al.: Optimizing analysis, visualization, and navigation of large image data sets: one 5000-section CT scan can ruin your whole day. Radiology 2011; 259: pp. 346-362.