Rationale and Objectives
To assess the nature of the satisfaction of search (SOS) effect in chest radiography when observers are fatigued; determine if we could replicate recent findings that have documented the nature of the SOS effect to be due to a threshold shift rather than a change in diagnostic accuracy as in earlier film-based studies.
Materials and Methods
Nearing or at the end of a clinical workday, 20 radiologists read 64 chest images twice, once with and once without the addition of a simulated pulmonary nodule. Half of the images had different types of “test” abnormalities. Decision thresholds were analyzed using the center of the range of false-positive (FP) and true-positive (TP) fractions associated with each receiver operating characteristic (ROC) point for reporting test abnormalities. Detection accuracy was assessed with ROC technique and inspection time was recorded.
Results
The SOS effect was confirmed to be a reduction in willingness to respond (threshold shift). The center of the FP range was significantly reduced (FP = 0.10 without added nodules, FP = 0.05 with added nodules, F(1,18) = 19.85, P = 0.0003). The center of the TP range was significantly reduced (TP = 0.39 without added nodules, TP = 0.33 with added nodules, F(1,18) = 10.81, P = 0.004).
Conclusions
This study suggests that fatigue does not change the nature of the SOS effect, but rather may be additive with the SOS effect. SOS reduces both TP and FP responses, whereas fatigue reduces TPs more than FPs.
Introduction
In recent years, there has been concern over the impact of errors and fatigue on performance in medicine in general and for radiology in particular . Increased volumes of imaging studies that are more complex in nature have led to increased workloads and longer hours interpreting cases . Fatigue has both mental and physical consequences, can occur with or without feelings of sleepiness, and can be measured in a variety of subjective and objective ways (see Reference for a recent review). The effects of fatigue can vary as a function of task, level of experience, environmental conditions, and health status. For radiologists, visual fatigue and eye strain may be of particular interest given the visual nature of medical image interpretation. Cognitive fatigue may be important as well but has not been studied systematically in radiology.
Physical measures of visual strain (accommodation and dark vergence) indicate that fatigue can reduce a radiologist’s ability to focus, especially at close viewing distances . It has also been found that subjective feelings of physical, mental, and emotional strain increase in radiologists who have spent as little as 8 hours interpreting clinical images. Both physical and subjective measures indicate that residents may actually be more vulnerable to effects of fatigue than attending radiologists.
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Materials and Methods
Cases
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Readers and Image Presentation
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Procedure
Get Radiology Tree app to read full this article<
Statistical Analyses
Get Radiology Tree app to read full this article<
Detection Accuracy
Get Radiology Tree app to read full this article<
Decision Thresholds
Get Radiology Tree app to read full this article<
Decision Threshold Shifts
Get Radiology Tree app to read full this article<
Results
Get Radiology Tree app to read full this article<
Diagnostic Performance
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Decision Thresholds
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Inspection Time
Get Radiology Tree app to read full this article<
Discussion
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Conclusions
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
References
1. Kohn L.T.Corrigan J.M.Donaldson M.S.To err is human: building a safer health system.1999.National Academy PressWashington, DC:
2. Bleich S.: Medical errors: five years after the IOM report. Commonwealth Fund Publication #830; July; Available at: http://www.commonwealthfund.org/usr_doc/830_Bleich_errors.pdf Accessed July 13, 2016
3. O’Donnell E.P., Humeniuk K.M., West C.P., et. al.: The effects of fatigue and dissatisfaction on how physicians perceive their social responsibilities. Mayo Clin Proc 2015; 90: pp. 194-201.
4. Chan A.O.M., Chan Y.H., Chuang K.P., et. al.: Addressing physician quality of life: understanding the relationship between burnout, work engagement, compassion fatigue and satisfaction. J Hosp Admin 2015; 4: pp. 46-55.
5. Bruno M.A., Walker E.A., Abujudeh H.H.: Understanding and confronting our mistakes: the epidemiology of error in radiology and strategies for error reduction. Radiographics 2015; 35: pp. 1668-1676.
6. Rohatgi S., Hanna T.N., Sliker C.W., et. al.: After-hours radiology: challenges and strategies for the radiologist. AJR Am J Roentgenol 2015; 205: pp. 956-961.
7. Hoffmann J.C., Mittal S., Hoffmann C.H., et. al.: Combating the health risks of sedentary behavior in the contemporary radiology reading room. AJR Am J Roentgenol 2016; 206: pp. 1135-1140.
8. Waite S., Kolla S., Jeudy J., et. al.: Tired in the reading room: the influence of fatigue in radiology. J Am Coll Radiol 2017; 14: pp. 191-197.
9. Krupinski E.A., Berbaum K.S.: Measurement of visual strain in radiologists. Acad Radiol 2009; 16: pp. 947-950.
10. Krupinski E.A., Berbaum K.S., Caldwell R.T., et. al.: Long radiology workdays reduce detection and accommodation accuracy. J Am Coll Radiol 2010; 7: pp. 698-704.
11. Krupinski E.A., MacKinnon L., Reiner B.I.: Feasibility of using a biowatch to monitor GSR as a measure of radiologists’ stress and fatigue. Proc SPIE Med Imaging 2015; 9416: pp. 941613.
12. Krupinski E.A., Berbaum K.S., Caldwell R.T., et. al.: Do long radiology workdays affect nodule detection in dynamic CT interpretation?. J Am Coll Radiol 2012; 9: pp. 191-198.
13. Taylor-Phillips S., Elze M.C., Krupinski E.A., et. al.: Retrospective review of the drop in observer detection performance over time in lesion-enriched experimental studies. J Digit Imaging 2015; 28: pp. 32-40.
14. Berbaum K., Franken E., Caldwell R., et. al.: Satisfaction of search in traditional radiographic imaging.Samei E.Krupinski E.The handbook of medical image perception and techniques.2010.Cambridge University PressNew York:pp. 107-138.
15. Berbaum K.S., Franken E.A., Dorfman D.D., et. al.: Satisfaction of search in diagnostic radiology. Invest Radiol 1990; 25: pp. 133-140.
16. Berbaum K.S., Dorfman D.D., Franken E.A., et. al.: Proper ROC analysis and joint ROC analysis of the satisfaction of search effect in chest radiography. Acad Radiol 2000; 7: pp. 945-958.
17. Berbaum K.S., Krupinski E.A., Schartz K.M., et. al.: Satisfaction of search in chest radiography 2015. Acad Radiol 2015; 22: pp. 1457-1465.
18. Berbaum K.S., Krupinski E.A., Schartz K.M., et. al.: The influence of a vocalized checklist on detection of multiple abnormalities in chest radiography. Acad Radiol 2016; 23: pp. 413-420.
19. DICOM PS3.14 2016c—grayscale standard display function. Available at: http://dicom.nema.org/medical/dicom/current/output/pdf/part14.pdf Accessed July 25, 2016
20. Schartz K.M., Berbaum K.S., Caldwell R.T., et. al.: WorkstationJ: workstation emulation software for medical image perception and technology evaluation research. Proc SPIE Med Imaging 2007; 6515: 6515I1–6515I11
21. Jiang Y., Metz C.E., Nishikawa R.M.: A receiver operating characteristic partial area index for highly sensitive diagnostic tests. Radiology 1996; 201: pp. 745-750.
22. Dorfman D.D., Berbaum K.S.: A contaminated binormal model for ROC data—Part II. A formal model. Acad Radiol 2000; 7: pp. 427-437.
23. Dorfman D.D., Berbaum K.S., Metz C.E.: Receiver operating characteristic rating analysis: generalization to the population of readers and patients with the jackknife method. Invest Radiol 1992; 27: pp. 723-731.
24. Hillis S.L., Berbaum K.S., Metz C.E.: Recent developments in the Dorfman-Berbaum-Metz procedure for multireader ROC study analysis. Acad Radiol 2008; 15: pp. 647-661.
25. Swets J.A., Pickett R.M.: Evaluation of diagnostic systems: methods from signal detection theory.1982.Academic PressNew Yorkpp. 39.
26. Berbaum K.S., Franken E.A., Dorfman D.D., et. al.: Time course of satisfaction of search. Invest Radiol 1991; 26: pp. 640-648.