Rationale and Objectives
Whether first-year radiology residents are ready to start call after 6 or 12 months has been a subject of much debate. The purpose of this study was to establish an assessment that would evaluate the call readiness of first-year radiology residents and identify any individual areas of weakness using a comprehensive computerized format. Secondarily, we evaluated for any significant differences in performance before and after the change in precall training requirement from 6 to 12 months.
Materials and Methods
A list of >140 potential emergency radiology cases was given to first-year radiology residents at the beginning of the academic year. Over 4 years, three separate versions of a computerized examination were constructed using hyperlinked PowerPoint presentations and given to both first-year and second-year residents. No resident took the same version of the exam twice. Exam score and number of cases failed were assessed. Individual areas of weakness were identified and remediated with the residents. Statistical analysis was used to evaluate exam score and the number of cases failed, considering resident year and the three versions of the exam.
Results
Over 4 years, 17 of 19 (89%) first-year radiology residents passed the exam on first attempt. The two who failed were remediated and passed a different version of the exam 6 weeks later. Using the oral board scoring system, first-year radiology residents scored an average of 70.7 with 13 cases failed, compared to 71.1 with eight cases failed for second-year residents who scored statistically significantly higher. No significant difference was found in first-year radiology resident scoring before and after the 12-month training requirement prior to call.
Conclusions
An emergency radiology examination was established to aid in the assessment of first-year radiology residents’ competency prior to starting call, which has become a permanent part of the first-year curriculum. Over 4 years, all first-year residents were ultimately judged ready to start call. Of the variables assessed, only resident year showed a significant difference in scoring parameters. In particular, length of training prior to taking call showed no significant difference. Areas of weakness were identified for further study.
How does a residency program decide if a resident is ready to start call? Currently, there is no uniform process across the country to assess radiology residents’ preparedness prior to taking call. A minority of programs give oral exams, but most rely on evaluations from clinical rotations. These are limited and subjective at best and in particular may not evaluate a first-year resident’s knowledge regarding the common types of exams that may be seen on call or the ability to handle a large number of studies in a short time period. Feedback from time spent on “buddy call” with a senior resident is helpful but again is influenced by the case mix and the support provided by the senior resident. If there are concerns regarding a specific resident’s competence for solo call, formal documentation of his or her abilities is key, particularly if delay of call commencement is considered. Identifying individual residents’ areas of weaknesses is even more difficult but is crucial for guiding remediation and sometimes curricular changes.
In 2006, Dartmouth-Hitchcock Medical Center, a level 1 trauma center, had no specific criteria in place to assess readiness for call. We therefore developed a series of computerized emergency radiology examinations to evaluate the competency of our first-year residents prior to call and to assess for any areas of particular weakness that would require more study. We felt that the exam should not be in a multiple-choice format but in a “picture archiving and communication system (PACS)–like” format that would more closely mimic a night of call. The Accreditation Council for Graduate Medical Education requirements for residents to complete 12 months of training before solo call that came into force in 2008 also provided us with the opportunity to compare the performances of residents who had had 6 months of training prior to call with that of residents who had had 12 months. We report our 4-year experience in developing and administering this examination annually.
Materials and methods
Get Radiology Tree app to read full this article<
Resident Preparation for Exam
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Table 1
Primary Diagnoses from One of Three Versions of Our Precall Examination, Ordered by Subspecialty and Modality
ChestGenitourinaryNeuroradiology CT - Aortic dissection CT - Ureteral stone, partially obstructing CT - Acute left middle cerebral artery infarct CT - Aortic rupture US - Ectopic pregnancy CT - Carotid dissection CT - Normal chest CT US - Normal 6-week pregnancy CT - Cerebral edema CT - Pulmonary Embolus US - Ovarian torsion CT - Normal head CT NM - Low-probability V/Q scan US - Retained products of conception CT - Ruptured posterior communicating artery aneurysm XR - Aortic rupture US - Testicular torsion CT - Subdural hematoma XR - Foreign body aspiration MR - Discitis/osteomyelitis XR - Interstitial edema MR - Spinal cord compression XR - Left lower lobe pneumonia XR - Normal chest x-ray Miscellaneous XR - Right upper lobe atelectasis Management of anaphylactic intravenous contrast reactionPediatric XR - Ruptured diaphragm XR - Buckle fractures, radius and ulna XR - Supine pneumothorax XR - Distal humeral fracture XR - IntussusceptionGastrointestinal XR - Normal fifth metatarsal ossification center CT - AppendicitisMusculoskeletal XR - Posterior rib fractures in setting of child abuse CT - Bowel infarct XR - Femoral neck fracture on pelvic x-ray XR - Pseudosubluxation C2/3 CT - Diverticulitis XR - Jefferson’s fracture XR - Radial head dislocation CT - Normal abdominal CT XR - Medial tibial plateau fracture XR - Tension pneumothorax in neonate CT - Pancreatitis XR - Metacarpal fracture CT - Ruptured kidney and liver laceration XR - Posterior ligamentous injury, C3/4 FL - Esophageal perforation with leak XR - Posterior shoulder dislocation NM - Acute cholecystitis XR - Scaphoid fracture NM - Gastrointestinal bleed XR - Cecal volvulus XR - Free intraperitoneal air XR - Sigmoid volvulus
CT, computed tomography; FL, fluoroscopy; MR, magnetic resonance; NM, nuclear medicine; US, ultrasound; V/Q, ventilation/perfusion; XR, x-ray.
Get Radiology Tree app to read full this article<
Development of the Computer-based Exam
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Exam Administration and Resident Evaluation
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Statistical Analysis
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Results
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Table 2
Breakdown of First-year and Second-year Radiology Residents’ Scores and Cases Failed for Each of the Three Exam Versions
Exam Version Score Cases Failed Mean ± Standard Deviation (Range) Median Mean ± Standard Deviation (Range) Median First-year residents 1 70.7 ± 0.5 70.8 13.5 ± 7.3 11.5 2 70.5 ± 0.4 70.5 15.5 ± 2.8 15.0 3 70.9 ± 0.3 71.0 9.2 ± 4.8 8.0 All 70.7 ± 0.4 (69.8–71.3) 70.8 13 ± 6.0 (4–30) 13.0 Second-year residents 1 71.2 ± 0.2 71.2 7.3 ± 1.6 7.5 2 71.1 ± 0.3 71.2 9.0 ± 3.6 8.0 3 71.1 ± 0.4 71.0 8.0 ± 5.0 9.5 All 71.1 ± 0.3 (70.7–71.7) 71.2 8 ± 3.2 (1-14) 8.0
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Table 3
Frequently Missed or Problematic Cases Identified Over 4 Years
ChestMiscellaneousNeuroradiology 9 - Pneumothorax 6 - Appropriate management of anaphylactic contrast reaction 9 - Cerebral herniation and or edema 6 - Low-probability V/Q scan 5 - Treatment of contrast infiltration 9 - CTA and aneurysms 4 - Diaphragmatic rupture 4 - Treatment of bradycardia 8 - Dense cerebellar sign 3 - Aortic rupture 3 - Line and tube positioning 6 - Jefferson’s fracture on x-ray 3 - Interstitial edema Waffling in general 6 - Small subdural hematoma 5 - Ligamentous injuryGastrointestinalMusculoskeletal 4 - Carotid dissection 11 - Large vs small bowel hemorrhage on bleeding scan 3 - Scaphoid fracture 3 - Abnormal meningeal enhancement 7 - Sigmoid vs cecal volvulus 2 - Posterior shoulder dislocation 3 - Early signs of infarct 5 - Liver laceration 3 - Spinal cord edema/injury 4 - Bowel ischemia 4 - Overcall of normal appendixPediatric 3 - Appendicitis 8 - Nonaccidental trauma 8 - Overcalling spinal pseudosubluxationGenitourinary 6 - Normal foot ossification centers 2 - Estimating early gestational age 5 - Aspirated foreign body 1 - Ectopic pregnancy 4 - Distal humeral fracture 1 - Ovarian torsion 1 - Testicular torsion
CTA, computed tomographic angiography; V/Q, ventilation/perfusion.
Get Radiology Tree app to read full this article<
Discussion
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Conclusions
Get Radiology Tree app to read full this article<
References
1. Ganguli S., Pedrosa I., Yam C.-S., et. al.: Part 1: preparing first-year radiology residents and assessing their readiness for on-call responsibilities. Acad Radiol 2006; 13: pp. 764-769.
2. Ganguli S., Camacho M., Yam C.-S., et. al.: Preparing first-year radiology residents and assessing their readiness for on-call responsibilities: results over 5 years. AJR Am J Roentgenol 2009; 192: pp. 539-544.
3. Monsky W.L., Levine D., Mehta T.S., et. al.: Using a sonographic simulator to assess residents before overnight call. AJR Am J Roentgenol 2002; 178: pp. 35-39.
4. Gaca A.M., Frush D.P., Hohenhaus S.M., et. al.: Enhancing pediatric safety: using simulation to assess radiology resident preparedness for anaphylaxis from intravenous contrast media. Radiology 2007; 245: pp. 236-244.
5. Yam C.-S., Kruskal J., Pedrosa I., et. al.: Part II: preparing and assessing first-year radiology resident on-call readiness. Acad Radiol 2006; 13: pp. 770-773.
6. Accreditation Council for Graduate Medical Education. ACGME program requirements for graduate medical education in diagnostic radiology. Available at: http://www.acgme.org/acWebsite/downloads/RRC_progReq/420_diagnostic_radiology_07012008.pdf . Accessed January 13, 2012.
7. Larson D.B.: Major changes in radiology residency program requirements are coming. AJR Am J Roentgenol 2007; 188: pp. 3-4.
8. Lewis P, Khan R, Benodin L. Radiology Resident Pre-Call Evaluation Quiz. MedEdPORTAL; 2010. Available at: http://services.aamc.org/30/mededportal/servlet/s/segment/mededportal/?subid=7842 .