Rationale and Objectives
To determine whether the total number of studies interpreted during radiology residency correlates with clinical performance as measured by objective criteria.
Materials and Methods
We performed a retrospective cohort study of three graduating classes of radiology residents from a single residency program between the years 2015–2017. The total number of studies interpreted by each resident during residency was tracked. Clinical performance was determined by tracking an individual resident’s major discordance rate. A major discordance was recorded when there was a difference between the preliminary resident interpretation and final attending interpretation that could immediately impact patient care. Accreditation council for graduate medical education milestones at the completion of residency, Diagnostic radiology in-training scores in the third year, and score from the American board of radiology core exam were also tabulated. Pearson correlation coefficients and polynomial regression analysis were used to identify correlations between the total number of interpreted films and clinical, test, and milestone performance.
Results
Thirty-seven residents interpreted a mean of 12,709 studies (range 8898–19,818; standard deviation [SD] 2351.9) in residency with a mean major discordance rate of 1.1% (range 0.34%–2.54%; stand dev 0.49%). There was a nonlinear correlation between total number of interpreted films and performance. As the number of interpreted films increased to approximately 16,000, clinical performance ( p = 0.004) and test performance ( p = 0.01) improved, but volumes over 16,000 correlated with worse performance.
Conclusion
The total number of studies interpreted during radiology training correlates with performance. Residencies should endeavor to find the “sweet spot”: the amount of work that maximizes clinical exposure and knowledge without overburdening trainees.
INTRODUCTION
Residency education in diagnostic radiology continues to evolve. Over the past several years, the Accreditation Council for Graduate Medical Education has made major changes to the program requirements for graduate medical education in diagnostic radiology. Some of these revisions have included increasing the amount of training that residents undergo prior to taking independent call (from 6 months to 12 months) and adding formal requirements for scholarly activity, patient safety, and quality improvement. Perhaps the most important change, however, has been the shift to education delivery centered on the six general competencies: patient care, medical knowledge, practice based learning and improvement, interpersonal and communication skills, professionalism, and systems based practice ( ). In accordance with the next accreditation system, the ACGME implemented educational milestones in July 2013 to help program directors and core faculty evaluate and track residents’ performance across these competencies ( ). Milestones are descriptions of the goals for each competency that residents are expected to demonstrate as they progress through training. While milestones provide a framework, the ACMGE cautions that they should not be used as the only set of tools for assessing competency ( ). To aid programs in assessing competency, many Residency Review Committees for proceduralbased specialties, therefore require minimum case or procedural volumes. For example, the RRC for General Surgery requires a minimum of 850 cases for graduation ( ). In a similar light, the RRC for radiology instituted minimum numbers for certain radiologic studies that each resident must interpret over the course of their residency training ( Table 1 ).
Table 1
Diagnostic Radiology Case Log Categories and Required Minimum Numbers \*
Case Log Categories Required Minimum Number Chest X-ray 1900 CT Abd/Pel 600 CTA/MRA 100 Image Guided Bx/Drainage 25 Mammography 300 MRI Body 20 MRI Brain 110 MRI Lower Extremity Joints 20 MRI Spine 60 PET 30 US Abd/Pel 350
Abd, abdominal; CT, computed tomography; CTA, computed tomography anography; MRA, magnetic resonance angiogram; MRI, magnetic resonance imaging; PET, positron emission tomography.
⁎ Reproduced from: Accreditation Council for Graduate Medical Education (ACGME). Diagnostic Radiology Case Log Categories and Required Minimum Numbers. https://www.acgme.org/Portals/0/PFAssets/ProgramResources/DR_Case_Log_Categories.pdf?ver=2018-01-09-113333-230
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
MATERIALS AND METHODS
Get Radiology Tree app to read full this article<
Table 2
Examples of Significant Discordances
Actual Significant Discordances Reported During Study Period Sub-Specialty Preliminary Interpretation Final Interpretation Neuroradiology No acute inflammatory process Right dacrocystitis associated with right preseptal soft tissue swelling. No postseptal involvement Pediatrics No inflammatory changes in RLQ Appendicitis with appendicolith. Free fluid in pelvis concerning for perforation Thoracic Patent right subclavian artery bypass with flow distal into the right brachial artery Complete occlusion of right subclavian artery bypass graft with minimal reconstitution distally in the region of the shoulder Abdominal Imaging Small bilateral hydroceles. No evidence of testicular torsion Left epididymo-orchitis with bilateral hydroceles Nuclear Medicine No convincing areas of V/Q mismatch—low probability of PE High probability of PE with diminished perfusion to the left lung Musculoskeletal Severe osteopenia, no acute fracture Acute nondisplaced fracture of the left superior pubic ramus
RLQ, right lower quadrant
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
IRB Statement
Get Radiology Tree app to read full this article<
RESULTS
Get Radiology Tree app to read full this article<
Table 3
Resident Descriptors
Resident Descriptors Gender Male 31 Female 6 Degree MD 37 MD/PhD 0 Medical School Attended US Medical School 36 International Medical School 1 AOA Status Upon Entering Residency Junior AOA 15 Senior AOA 8 Not AOA 10 AOA Not Available at Medical School 4 USMLE Step 1 Score Mean 246.4 Standard Deviation 10.4 USMLE Step 2 Score Mean 247.1 Standard Deviation 17.9 DXIT Score (Scaled Score) Mean 73.97 Standard Deviation 6.7 ABR Core Exam (Overall Score) Mean 495.7 Standard Deviation 67.1
ABR, American Board of Radiology; DXIT™, Diagnostic Radiology In-Training;
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
DISCUSSION
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
References
1. Collins J, de Christenson MR, Gray L, et. al.: General competencies in radiology residency training: definitions, skills, education and assessment. Acad Radiol 2002; 9: pp. 721-726.
2. Vydareny KH, Amis Jr, Becker GJ, et. al.: Diagnostic radiology milestones. J Grad Med Educ 2013; 5: pp. 74-78.
3. Holmboe ES, Edgar L, Hamstra S: The milestones guidebook. http://www.acgme.org/Portals/0/MilestonesGuidebook.pdf
4. [ACGME] accreditation council for graduate medical education. ACGME Program Requirements for Graduate Medical Education in General Surgery. http://www.acgme.org/Portals/0/PFAssets/ProgramRequirements/440GeneralSurgery2018.pdf?ver=2017-10-03-110315-270 .
5. Brady A, Laoide RO, McCarthy P, et. al.: Discrepancy and error in radiology: concepts, causes and consequences. Ulster Med J 2012; 81: pp. 3-9.
6. [ACGME] accreditation council for graduate medical education. Radiology-Diagnostic: Case Logs National Data Report (Main Table) 2016-2017.
7. Sokolovskaya E, Shinde T, Ruchman RB, et. al.: The effect of faster reporting speed for imaging studies on the number of misses and interpretation errors: a pilot study. J Am Coll Radiol 2015; 12: pp. 683-688.
8. Davenport MS, Ellis JH, Khalatbari SH, et. al.: Effect of work hours, caseload, shift type, and experience on resident call performance. Acad Radiol 2010; 17: pp. 921-927.
9. Wildman-Tobriner B, Cline B, Swenson C, et. al.: Evaluating resident on-call performance: does volume affect discrepancy rate?. Curr Probl Diagn Radiol 2018;
10. Branstetter BF, Morgan MB, Nesbit CE, et. al.: Preliminary reports in the emergency department: is a subspecialist radiologist more accurate than a radiology resident?. Acad Radiol 2007; 14: pp. 201-206.
11. Agarwal V, Bump GM, Heller MT, et. al.: Do residency selection factors predict radiology resident performance?. Acad Radiol 2018; 25: pp. 397-402.
12. Sistrom C, Deitte L: Factors affecting attending agreement with resident early readings of computed tomography and magnetic resonance imaging of the head, neck, and spine. Acad Radiol 2008; 15: pp. 934-941.
13. Ruutiainen AT, Scanlon MH, Itri JN: Identifying benchmarks for discrepancy rates in preliminary interpretations provided by radiology trainees at an academic institution. J Am Coll Radiol 2011; 8: pp. 644-648.
14. Wildenberg JC, Chen P, Scanlon MH, et. al.: Attending radiologist variability and its effect on radiology resident discordance rates. Acad Radiol 2017; 24: pp. 694-699.
15. Ten Cate O: The false dichotomy of quality and quantity in the discourse around assessment in competency-based education. Adv Health Sci Educ Theory Pract 2015; 20: pp. 835-838.
16. Kohn GP, Galanko JA, Overby DW, et. al.: High case volumes and surgical fellowships are associated with improved outcomes for bariatric surgery patients: a justification of current credentialing initiatives for practice and training. J Am Coll Surg 2010; 210: pp. 909-918.
17. Kohn GP, Nikfarjam M: The effect of surgical volume and the provision of residency and fellowship training on complications of major hepatic resection. J Gastrointest Surg 2010; 14: pp. 1981-1989.
18. Wildman-Tobriner B, Cline B, Swenson C, et. al.: Evaluating resident on-call performance: does volume affect discrepancy rate?. Curr Probl Diagn Radiol 2018; 17: pp. 30241-30244.
19. Weinberg BD, Richter MD, Champine JG, et. al.: Radiology resident preliminary reporting in an independent call environment: multiyear assessment of volume, timeliness, and accuracy. J Am Coll Radiol 2015; 12: pp. 95-100.