Rationale and Objectives
The aim of this report is to describe preliminary experience with an educational tool implemented to improve radiology education and documentation, in anticipation of upcoming changes from the American Board of Radiology and the Accreditation Council for Graduate Medical Education.
Materials and Methods
Monthly online end-of-rotation examinations were developed for junior radiology residents. Examinations were administered over a period of 6 months, after which satisfaction surveys were administered to residents and faculty members.
Results
Satisfaction with end-of-rotation examinations was high for residents and faculty members, but the level of resident satisfaction was not statistically different when compared to that of residents who had taken fewer examinations.
Conclusions
End-of-rotation examinations are relatively straightforward to implement and facilitate recurring, structured, and meaningful resident evaluation and feedback. Future studies will explore whether examinations such as these result in improved clinical competency.
Because of recent and upcoming changes from both the Accreditation Council for Graduate Medical Education (ACGME) and the American Board of Radiology (ABR), radiology residency education will change dramatically over the next few years. By 2011, ACGME has mandated that all “programs are making data-driven improvements” when it comes to core competencies and specialty-specific requirements . In 2013, the first group of radiology residents will take the new ABR computerized core examination, with the first computerized certifying examination to follow in 2015 . The certifying examination will be pushed back into the second year after residency, which allows for new possibilities in the fourth year of radiology residency . To stay ahead of the curve with all of these changes, radiology residency programs need to consider the most efficient means of teaching and evaluating residents, as well as documenting the process . In this article, we discuss one possible solution for these growing program demands.
Beginning in July of the 2010–2011 academic year, the diagnostic radiology residents at our training program started taking monthly online end-of-rotation (EOR) examinations, which were specific to each radiology subspecialty. The EOR examination provided a structured framework for online examination and one-on-one evaluation between residents and attending radiologists.
Get Radiology Tree app to read full this article<
Methods
Platform
Get Radiology Tree app to read full this article<
Design
Get Radiology Tree app to read full this article<
Administration
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Rollout
Get Radiology Tree app to read full this article<
Long-term Maintenance
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Blinded Resident Satisfaction Survey
Get Radiology Tree app to read full this article<
Nonblinded Resident and Faculty Member Satisfaction Survey
Get Radiology Tree app to read full this article<
Statistical Analysis
Get Radiology Tree app to read full this article<
Results
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Table 1
Satisfaction Versus Frequency of Participation in the EOR Examination
Satisfaction In-person Feedback Performance Feedback One-on-one Teaching Board Preparation 0-2 EOR 3-6 EOR Total 0-2 EOR 3-6 EOR Total 0-2 EOR 3-6 EOR Total 0-2 EOR 3-6 EOR Total Extremely or very satisfied 12 15 27 12 19 31 16 21 37 23 21 44 Somewhat, slightly, or not at all satisfied 11 9 20 11 5 16 7 3 10 0 3 3 Total 23 24 47 23 24 47 23 24 47 23 24 47 Ratio 52% 63%P = .56 52% 79%P = .07 70% 88%P = .17 100% 88%P = .23
EOR, end-of-rotation; 0-2 EOR, residents who took zero to two examinations; 3-6 EOR, residents who took three to six examinations.
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Table 2
Frequency of Participation in the EOR Examination Versus Resident Training Level
R1 R2 R3 R4 0-2 EOR 0 5 10 8 3-6 EOR 12 4 3 5
EOR, end-of-rotation; 0-2 EOR, residents who took zero to two examinations; 3-6 EOR, residents who took three to six examinations.
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Table 3
Nonblinded Satisfaction Survey of Residents and Faculty Members
Statement Agree Neutral Disagree Attending radiologists ( n = 14) I believe that these end-of-rotation exams are a positive educational experience for residents.86% 14% 0% I believe that these end-of-rotation exams are a good use of my time as an attending.86% 14% 0% It was easy to find time to meet with the resident.64% 7% 29% I would like to continue having residents take these exams after each rotation.64% 29% 7% Residents ( n = 31) The end-of-rotation exams do not take a long time to complete.68% 18% 14% It was easy to find time to meet with an attending to review my responses.55% 18% 27% During the review session with the attending, I was also able to receive feedback regarding my performance on that rotation.64% 23% 14% During the review session with the attending, I was also able to provide feedback regarding my rotation experience. 41% 27% 32% I find these exams to be a positive educational experience.86% 5% 9% I find these exams to provide me with better feedback from faculty when compared to our current online feedback system.68% 27% 5% I think these exams are a good use of my time as a resident.82% 14% 5% I would like to continue taking these exams after each rotation.73% 9% 18%
Numbers in boldface type indicate a majority response (>50%).
Get Radiology Tree app to read full this article<
Discussion
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Conclusions
Get Radiology Tree app to read full this article<
References
1. Accreditation Council for Graduate Medical Education. ACGME timeline and working guidelines. Available at: http://www.acgme.org/outcome/project/timeline/TIMELINE_index_frame.htm . Accessed May 1, 2011.
2. Accreditation Council for Graduate Medical Education. ACGME program requirements for graduate medical education in diagnostic radiology. Available at: http://www.acgme.org/acWebsite/downloads/RRC_progReq/420_diagnostic_radiology_07012008.pdf . Accessed May 1, 2011.
3. Accreditation Council for Graduate Medical Education. ACGME common program requirements. Available at: http://www.acgme.org/outcome/comp/GeneralCompetenciesStandards21307.pdf . Accessed May 1, 2011.
4. American Board of Radiology. ABR diagnostic radiology CORE Examination study guide. Available at: http://www.theabr.org/forms/CORE%20Exam%20Study%20Guide_FINAL.pdf . Accessed May 1, 2011.
5. Nicholson B.T., Sizemore A.W., Gay S.B.: The new radiology residency fourth year under the new curriculum: a survey of current residents’ thoughts and suggestions about its design and purpose. J Am Coll Radiol 2010; 7: pp. 294-298.
6. Halsted M.J., Perry L., Racadio J.M., et. al.: Changing radiology resident education to meet today’s and tomorrow’s needs. J Am Coll Radiol 2004; 1: pp. 671-678.
7. Agarwal R.: Role models in academic radiology: current status and pathways to improvement. J Am Coll Radiol 2010; 7: pp. 50-55.
8. Donovan A.: Views of radiology program directors on the role of mentorship in the training of radiology residents. AJR Am J Roentgenol 2010; 194: pp. 704-708.
9. Itri J.N., Redfern R.O., Scanlon M.H.: Using a web-based application to enhance resident training and improve performance on-call. Acad Radiol 2010; 17: pp. 917-920.
10. Van Herck P., De Smedt D., Annemans L., et. al.: Systematic review: effects, design choices, and context of pay-for-performance in health care. BMC Health Serv Res 2010; 10: pp. 247.