Rationale and Objectives
To analyze the independent effects of multiple variables on resident call performance.
Materials and Methods
Independent radiology resident “on call” cross-sectional imaging interpretation quality assurance (QA) data obtained during a 171-day period at a single tertiary care Level 1 trauma teaching institution was reviewed. Clinically significant resident-faculty discrepancies were compared among three different call types: traditional single-day overnight call (OC, 15 hours/night after 9 daytime hours on weekdays), 7-night nightfloat (NF, 9 hours/night), and weekend day call (WD, 10 hours/day). Logistic regression analyses were performed to evaluate associations.
Results
There were 119 (0.89%) clinically significant resident-faculty discordances from 13,424 cross-sectional interpretations: 56 (0.79%) from 7102 interpretations on 172 OC shifts, 39 (0.85%) from 4567 interpretations on 165 NF shifts, and 24 (1.4%) from 1755 interpretations on 49 WD shifts. Individual residents ( n = 20) had a mean discrepancy rate of 0.9% (0.45%–1.9%). Overall, 102 (26.2%) of the shifts had at least one discordance. The following were associated with significantly ( P < .001) increased discrepancy rates: junior vs. senior residents (odds ratio [OR] = 1.3 [1.2–1.4]), OC vs. NF (OR = 1.5 [1.3–1.6], WD vs. NF (OR = 1.4 [1.2–1.6]), weekend vs. weekday (OR = 1.3 [1.2–1.4]), and increasing cases/hour (OR = 1.6 [1.5–1.7]). Weekend OC shifts had a higher discrepancy rate (OR 1.3[1.2–1.5], P < .001) than weekday OC shifts despite a shorter workday (15 vs. 24 hours).
Conclusion
Increasing caseload, junior residents, and weekends are associated with a significantly higher discrepancy rate. OC is associated with a significantly higher discrepancy rate than NF. Measured discrepancy rates are low, regardless of call type.
The landmark study from the Institute of Medicine, To Err is Human: Building a Safer Health System, has led to a renewed focus on patient safety . One aspect of this is a reevaluation of resident activities, especially when they are functioning in an independent fashion with less immediate supervision. This attention has in part resulted in new requirements by the Accreditation Council for Graduate Medical Education, which now mandates 12 months of radiology training before radiology residents are allowed to take independent call. Additionally, global resident work hour restrictions (per day and per week) affecting all medical specialties have also been issued by the Accreditation Council for Graduate Medical Education. Many radiology residency programs in the United States have elected to convert their overnight call to a nightfloat system to help cope with these new changes as well as meet the challenges of an ever-increasing volume of radiological studies . Although one study in the literature has reported an improvement in perceived clinical judgment of radiology residents in response to changing overnight call to a nightfloat system , no published data directly compare the effects of nightfloat with overnight call on radiology resident-faculty discordance rates. Complicating this issue are numerous other factors beyond call type that may affect discordances, including continuous work hours, case load, day of the week, and experience level. The large number of confounding variables coupled with an extremely low baseline discordant rate can make it difficult to assess the effect of any one intervention on discordance rates and, by extension, patient care.
The purpose of our study was to analyze the independent effects of work hours, hourly caseload, shift type, residency class, and experience on resident call performance.
Materials and methods
Get Radiology Tree app to read full this article<
Subjects
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Call Types
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Resident Image Interpretation
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Resident-faculty Discordance
Get Radiology Tree app to read full this article<
Table 1
Verbatim Radiology Resident Preliminary Interpretation QA Grading System Employed between April 20, 2009, and July 12, 2009
QA Grade Meaning 1 Concur with interpretation 2a Difficult diagnosis; not ordinarily expected to be made 2b Non-acute finding that was not in preliminary report but will be mentioned in final report 3 Clinically significant finding or interpretation should be made MOST of the time 4 Finding or interpretation should be made ALMOST EVERY time
QA, quality assurance.
Table 2
Verbatim Radiology Resident Preliminary Interpretation QA Grading System Employed between July 13, 2009, and October 8, 2009
QA Grade Meaning 0 Concur, Great call! 1 Concur 2 Concur, with minor changes that do not alter the overall impression 3 Disagree, difficult case, change in report may be clinically significant 4 Disagree - significant disagreement/omission/misinterpretation
QA, quality assurance.
Get Radiology Tree app to read full this article<
Statistical Analysis
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Results
Get Radiology Tree app to read full this article<
Table 3
Radiology Resident Call Shift Characteristics and Discrepancy Rates
Overnight Nightfloat Weekend Day Number of shifts 172 165 49 Mean cases 41.3 27.7 33.9 Mean cases/hour 2.8 3.1 3.4 Mean discordances/shift 0.3 0.2 0.5 Shifts with ≥1 discordance 46 (27%) 37 (22%) 19 (37%) Discrepancy rate 0.79% 0.85% 1.4% Mean resident experience (months) 31.3 32.0 31.3
Table 4
Aggregate Radiology Resident Discrepancy Rates by Shift and Day of the Week
Shift Type Day of the Week Total Cases Total Discrepancies Discrepancy Rate Overnight Friday 993 13 1.31% Saturday 1013 8 0.79% Sunday 1007 11 1.09% Monday 1067 6 0.56% Tuesday 1009 7 0.69% Wednesday 1003 7 0.70% Thursday 1010 4 0.40% Nightfloat Friday 669 7 1.05% Saturday 645 7 1.09% Sunday 700 7 1.00% Monday 650 4 0.62% Tuesday 690 5 0.72% Wednesday 607 3 0.49% Thursday 606 6 0.99% Weekend Day Friday 29 0 0.00% Saturday 799 13 1.63% Sunday 827 10 1.21% Monday 100 1 1.00%
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Table 5
Univariate Logistic Regression Analyses of Multiple Variables and Their Effect on Radiology Resident Discrepancy Rates
95% Confidence Limits Variable Odds Ratio Lower Upper_P_ Value Class (junior vs. senior) 1.1 1.1 1.2 <.001 † Shift type Overnight vs. nightfloat 1.2 1.1 1.4 <.001 † Weekend day vs. overnight 1.6 1.5 1.8 <.001 † Weekend day vs. nightfloat 2.0 1.8 2.3 <.001 † 2nd vs. 1st grading system 1.2 1.2 1.3 <.001 † Weekend vs. weekday 1.5 1.4 1.6 <.001 † Cases per hour ∗ 1.5 1.5 1.6 <.001 † Weekend vs. weekday (overnight shifts only) 1.4 1.3 1.6 <.001 †
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Table 6
Multivariate Logistic Regression Analysis of Multiple Variables and Their Effect on Radiology Resident Discrepancy Rates
95% Confidence Limits Variable Odds Ratio Lower Upper_P_ Value Class (junior vs. senior) 1.3 1.2 1.4 <.001 † Shift type Overnight vs. nightfloat 1.5 1.3 1.6 <.001 † Weekend day vs. nightfloat 1.4 1.2 1.6 <.001 † Overnight vs. weekend day 1.1 0.9 1.2 .5 2nd vs. 1st grading system 1.1 1.1 1.2 .001 † Weekend vs. weekday 1.3 1.2 1.4 <.001 † Cases per hour ∗ 1.6 1.5 1.7 <.001 †
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Table 7
Direct Comparison of Overnight Weekend Shifts with Overnight Weekday Shifts, while Controlling for: Class, Grading System, and Cases per Hour
95% Confidence Limits Variable Odds Ratio Lower Upper_P_ Value Weekend ∗ vs. weekday 1.3 1.2 1.5 <.001 †
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Discussion
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
References
1. Stelfox H.T., Palmisani S., Scurlock C., et. al.: The “To err is human” report and the patient safety literature. Qual Saf Health Care 2006; 15: pp. 174-178.
2. Kohn L.T., Corrigan J.M., Donaldson M.S.: (Institute of Medicine). To err is human: building a safer health system.2000.National Academy PressWashington, DC
3. Leape L., Epstein A.M., Hamel M.B.: A series on patient safety. N Engl J Med 2002; 347: pp. 1272-1274.
4. VanWagenen F.K., Weidman E.R., Duncan J.R., et. al.: Results of the 1996 survey of the American Association of Academic Chief Residents in Radiology. Acad Radiol 1997; 4: pp. 132-137.
5. Peterson C.M., Gerstle R., Bhalla S., et. al.: Results of the 2004 survey of the American Association of Academic Chief Residents in Radiology. Acad Radiol 2005; 12: pp. 373-378.
6. Carroll T.J.: Trends in on-call workload in an academic medical center radiology department 1998–2002. Acad Radiol 2003; 10: pp. 1312-1320.
7. Reader D.W., Spigos D.G., Bennett W.F., et. al.: The graveyard shift: experience with a night float system. Emerg Radiol 2002; 9: pp. 82-87.
8. Jackson V.P., Cushing T., Abujudeh H.H., et. al.: RADPEER scoring white paper. J Am Coll Radiol 2009; 6: pp. 21-25.
9. Cervini P., Bell C.M., Roberts H.C., et. al.: Radiology resident interpretation of on-call CT pulmonary angiograms. Acad Radiol 2008; 15: pp. 556-562.
10. Rufener S.L., Patel S., Kazerooni E.A., et. al.: Comparison of on-call radiology resident and faculty interpretation of 4- and 16-row multidetector CT pulmonary angiography with indirect CT venography. Acad Radiol 2008; 15: pp. 71-76.
11. Ginsberg M.S., King V., Panicek D.M.: Comparison of interpretations of CT angiograms in the evaluation of suspected pulmonary embolism by on-call radiology fellows and subsequently by radiology faculty. AJR Am J Roentgenol 2004; 192: pp. 61-66.
12. Stevens K.J., Griffiths K.L., Rosenberg J., et. al.: Discordance rates between preliminary and final radiology reports on cross-sectional imaging studies at a level 1 trauma center. Acad Radiol 2008; 15: pp. 1217-1226.
13. Yoon L.S., Haims A.H., Brink J.A., et. al.: Evaluation of an emergency radiology quality assurance program at a level I trauma center: abdominal and pelvic CT studies. Radiology 2002; 224: pp. 42-46.
14. Le A.H., Licurse A., Catazano T.M.: Interpretation of head CT scans in the emergency department by fellows versus general staff non-neuroradiologists: a closer look at the effectiveness of a quality control program. Emerg Radiol 2007; 14: pp. 311-316.
15. Cooper V.F., Goodhartz L.A., Nemcek A.A., et. al.: Radiology resident interpretations of on-call imaging studies: the incidence of major discrepancies. Acad Radiol 2008; 15: pp. 1198-1204.
16. Branstetter I.V.B.F., Morgan M.B., Nesbit C.E., et. al.: Preliminary reports in the emergency department: is a subspecialist radiologist more accurate than a radiology resident?. Acad Radiol 2007; 14: pp. 201-206.
17. Chung J.H., Strigel R.M., Chew A.R., et. al.: Overnight resident interpretation of torso CT at a level 1 trauma center: an analysis and review of the literature. Acad Radiol 2009; 16: pp. 1155-1160.
18. Erly W.K., Berger W.G., Krupinski E., et. al.: Radiology resident evaluation of head CT scan orders in the emergency department. Am J Neuroradiol 2002; 23: pp. 103-107.
19. Ruchman R.B., Jaeger J., Wiggins E.F., et. al.: Preliminary radiology resident interpretations versus final attending radiologist interpretations and the impact of patient care in a community hospital. AJR Am J Roentgenol 2007; 189: pp. 523-526.
20. Goddard P., Leslie A., Jones A., et. al.: Error in radiology. Br J Radiol 2001; 74: pp. 949-951.
21. Goldstein M., Kim E., Widmann W., et. al.: A 360° evaluation of a night-float system for general surgery: a response to mandated work-hours reduction. Current Surg 2004; 5: pp. 445-451.
22. Whang E.E., Mello M.M., Ashley S.W., et. al.: Implementing resident work hour limitations: lessons from the New York State experience. Ann Surg 2003; 237: pp. 449-455.
23. Barden C.B., Specht M.C., McCarter M.D., et. al.: Effects of limited work hours on surgical training. J Am Coll Surg 2002; 195: pp. 531-538.
24. Pape H.C., Pfeifer R.: Restricted duty hours for surgeons and impact on residents quality of life, education, and patient care: a literature review. Patient Saf Surg 2009; 20: pp. 3.