Rationale and Objectives
The objective of this study was to determine whether noon conference attendance by diagnostic radiology residents is predictive of measurable performance.
Methods
This single-center retrospective Health Insurance and Portability and Accountability Act (HIPAA)-compliant cross-sectional study was considered “not regulated” by the institutional review board. All diagnostic radiology residents who began residency training from 2008 to 2012 were included (N = 54). Metrics of clinical performance and knowledge were collected, including junior and senior precall test results, American Board of Radiology scores ( z -score transformed), American College of Radiology in-training scores (years 1–3), on-call “great call” and minor and major discrepancy rates, on-call and daytime case volumes, and training rotation scores. Multivariate regression models were constructed to determine if conference attendance, match rank order, or starting year could predict these outcomes. Pearson bivariate correlations were calculated.
Results
Senior precall test results were moderately correlated with American Board of Radiology ( r = 0.41) and American College of Radiology ( r = 0.38–0.48) test results and mean rotation scores ( r = 0.41), indicating moderate internal validity. However, conference attendance, match rank order, and year of training did not correlate with ( r = −0.16–0.16) or predict ( P > .05) measurable resident knowledge. On multivariate analysis, neither match rank order ( P = .14–.96) nor conference attendance ( P = .10–.88) predicted measurable clinical efficiency or accuracy. Year started training predicted greater cross-sectional case volume ( P < .0001, β = 0.361–0.516) and less faculty-to-resident feedback ( P < 0.0001, β = [−0.628]–[−0.733]).
Conclusions
Residents with lower conference attendance are indistinguishable from those who attend more frequently in a wide range of clinical and knowledge-based performance assessments, suggesting that required attendance may not be necessary to gain certain measurable core competencies.
Introduction
The Accreditation Council for Graduate Medical Education (ACGME) mandates that faculty at residency training programs “regularly participate in organized clinical discussions, rounds, journal clubs, and conferences” (rule II.B.5.a ). ACGME further instructs that diagnostic radiology programs “document resident conference attendance for all 48 months of the education program” (rule IV.A.3.d ), “provide at least five hours per week of lectures and conferences” (rule IV.A.3.a ), and provide residents “protected time to attend all lectures and conferences scheduled by the program” (rule IV.A.3.b ). In other words, faculty are required to give conferences, and residents are expected to attend them. The implication is that conference attendance is valuable, and not having or not attending conference results in a suboptimal educational outcome.
However, at our institution, even when conferences are provided and protected time is given, residents do not always attend. This finding may be due to residents believing that the value proposition is too low—attendance takes time and does not guarantee performance improvement . For conferences to be effective, they need to be focused and to engage the learners . However, because radiology faculty generally have not been trained to be educators—they were, generally speaking, trained to be clinical radiologists—there is inevitably a wide range of didactic teaching prowess. Residents understand these parameters and make personal choices about when, and if, conference attendance would be beneficial. Those who do not attend instead may choose to learn the required material from books, journal articles, online resources, or other nontraditional methods. This can be upsetting to the faculty, who may perceive a lack of attendance as unprofessional and disrespectful, out of compliance with required ACGME regulations, or destined to result in physicians with lesser knowledge and poorer communication skills.
Get Radiology Tree app to read full this article<
Methods
Get Radiology Tree app to read full this article<
Study Population
Get Radiology Tree app to read full this article<
Precall Testing
Get Radiology Tree app to read full this article<
Multiple-choice Testing
Get Radiology Tree app to read full this article<
On-call Cross-sectional Imaging Performance
Get Radiology Tree app to read full this article<
On-call and Daytime Case Volumes
Get Radiology Tree app to read full this article<
TABLE 1
Study Population Details
Characteristic Data Subjects 54 Noon conference teaching scores by year (mean, ±) 2008 ( n = 2,624 evaluations, maximum score: 5) 4.30 (±0.8) 2009 ( n = 2,389 evaluations, maximum score: 5) 4.29 (±0.9) 2010 ( n = 1,315 evaluations, maximum score: 5) 4.36 (±0.8) 2011 ( n = 3,964 evaluations, maximum score: 5) 4.38 (±0.8) 2012 ( n = 3,234 evaluations, maximum score: 5) 4.41 (±0.8) Year started training (% [ n ]) 2008 20% (11) 2009 20% (11) 2010 20% (11) 2011 19% (10) 2012 20% (11) Rank order (mean, range) 32 (1–74) Conference attendance (mean, range) 72% (35%–90%) Precall testing Junior call test, failed or conditioned (% [ n ]) 4% (2) Junior call test, composite score (mean, range) +11 (+6 to +14) Senior call test, failed or conditioned (% [ n ]) 11% (6) Senior call test, composite score (mean, range) +14 (+10 to +17) ABR MCQ examination z -score (range) −3.03 to 1.75 ACR in-training examination percentile score (mean, range) Year 1 (postgraduate year 2) 64 (2–97) Year 2 (postgraduate year 3) 65 (15–99) Year 3 (postgraduate year 4) 63 (5–99) On-call performance (mean, range) “Great call” rate 0.8% (0.0%–2.1%) Minor discrepancy rate 0.8% (0.0%–2.5%) Major discrepancy rate 0.5% (0.0%–1.8%) Average volume of cross-sectional studies per shifts 32 (19–42) Total number of senior call shifts 103 (80–116) Total number of cross-sectional studies read on call 3,293 (1,846–4,157) Daytime performance (mean, range) Total number of chest radiographs read 5,839 (3,156–12,308) Total number of mammograms read 517 (207–937) Total number of CT or MR angiograms read 328 (126–649) Total number of CT abdomen or pelvis read 843 (528–1,226) Total number of ultrasound abdomen or pelvis read 1,226 (687–1,953) Total number of lower extremity MRI examinations read 97 (28–211) Total number of MRI body examinations read 267 (106–647) Total number of MRI spine examinations read 235 (74–518) Total number of MRI brain examinations read 287 (142–618) Total number of PET examinations read 233 (29–840) Composite clinical rotation evaluation score 4.0 (2.5–4.5)
ABR, American Board of Radiology; ACR, American College of Radiology; CT, computed tomography; MCQ, multiple-choice question; MR, magnetic resonance; MRI, magnetic resonance imaging; PET, positron emission tomography.
± indicates standard deviation.
Get Radiology Tree app to read full this article<
Evaluation Scores
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Match Rank Order
Get Radiology Tree app to read full this article<
Noon Conference Attendance
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Data Analysis
Get Radiology Tree app to read full this article<
Results
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
TABLE 2
Correlation Matrix of Clinical, Programmatic, and Social Outcome Measures of Resident Performance (N = 54 residents)
Senior Call Testing MCQ Board Examination ACR Examination 1 ACR Examination 2 ACR Examination 3 “Great Call” Rate Minor Error Rate Major Error Rate Call Case Volume Day X-ray Volume Day X-sect Volume Daytime Clinical Evaluations Junior call testing 0.36* 0.40* 0.28 0.44* 0.58* 0.11 0.17 −0.22 0.28 0.21 0.22 0.20 Senior call testing — 0.41* 0.48* 0.38* 0.43* 0.18 −0.01 −0.28 0.19 0.003 0.07 0.41* MCQ board examination — — 0.37* 0.55* 0.70* 0.21 0.04 −0.24 0.16 0.02 0.25 0.23 ACR examination 1 — — — 0.58* 0.61* 0.15 −0.04 −0.24 0.33 −0.07 0.22 0.17 ACR examination 2 — — — — 0.85* 0.09 0.03 −0.24 0.30 0.10 0.24 0.03 ACR examination 3 — — — — — 0.12 0.09 −0.32 0.33 0.11 0.32 0.08 “Great call” rate — — — — — — 0.56* 0.44* −0.36* 0.56* −0.41* 0.20 Minor error rate — — — — — — — 0.63* −0.45* 0.85* −0.11 0.05 Major error rate — — — — — — — — −0.43* 0.56* −0.38* −0.05 Call case volume — — — — — — — — — -0.31 0.58* −0.01 Day x-ray volume — — — — — — — — — — −0.27 0.05 Day x-sect volume — — — — — — — — — — — −0.11
ACR, American College of Radiology; MCQ, multiple-choice question.
Data are Pearson r . Asterisks indicate correlations with P < .01. X-ray includes chest radiography and mammography. X-sect includes cross-sectional imaging with computed tomography, magnetic resonance imaging, positron emission tomography, and ultrasound.
TABLE 3
Correlation Matrix Relating Year Started Residency (2008–2012), Match Rank Order, and Conference Attendance on Clinical, Programmatic, and Social Outcome Measures of Resident Performance (N = 54 residents)
Outcome Year Started r Rank Order r Conference Attendance r Junior call pretest Fail or conditioned −0.14 0.06 −0.13 Composite score −0.07 0.06 0.18 Senior call pretest Fail or conditioned −0.12 −0.06 0.02 Composite score −0.07 0.16 0.23 Written, physics, or core examination_z_ -Score transformation 0.07 0.09 0.11 Top quintile performance 0.07 −0.01 0.01 ACR in-training examination Year 1 0.08 0.15 0.03 Top quintile performance 0.09 0.07 0.10 Year 2 −0.002 −0.02 0.11 Top quintile performance −0.03 −0.02 0.03 Year 3 0.08 0.08 0.16 Top quintile performance 0.08 −0.13 −0.16 Senior on-call performance “Great call” rate −0.63 0.01 0.29 Minor discrepancy rate −0.71 0.09 0.27 Major discrepancy rate −0.62 0.10 0.20 Cases read per shift 0.43 −0.10 −0.32 Top quintile performance 0.33 −0.17 −0.26 Daytime performance Chest x-ray and mammography volume −0.87 0.14 0.33 Top quintile performance −0.67 0.19 0.23 CT, MRI, US, or PET volume 0.56 −0.07 −0.34 Top quintile performance 0.56 −0.13 −0.39 Mean evaluation score −0.10 0.26 0.18 Top quintile performance −0.21 0.29 0.16
ACR, American College of Radiology; CT, computed tomography; MRI, magnetic resonance imaging; PET, positron emission tomography; US, ultrasound.
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
TABLE 4
Regression Analyses Evaluating the Predictive Effect of Year Started Residency (2008–2012), Match Rank Order, and Conference Attendance on Clinical, Programmatic, and Social Outcome Measures of Resident Performance (N = 54 residents)
Outcome Year Started Rank Order Conference Attendance Year Started Year Started β Rank Order Conference Attendance Univariate P Values Multivariate P Values and β Junior call pretest Fail or conditioned .35 .69 .38 - - - - Composite score .61 .66 .21 - - - - Senior call pretest Fail or conditioned .39 .69 .88 - - - - Composite score .58 .24 .10 - - - - Written, physics, or core examination_z_ -Score transformation .59 .53 .46 - - - - ACR in-training examination Year 1 .57 .32 .85 - - - - Year 2 .99 .90 .49 - - - - Year 3 .58 .61 .28 - - - - Senior on-call performance “Great call” rate <.0001* .96 .039* <.0001*β : −0.628 - .98 Minor discrepancy rate <.0001* .52 .047* <.0001*β : −0.733 - .62 Major discrepancy rate <.0001* .48 .14 <.0001*β : −0.661 - .46 Cases read per shift .001* .51 .020* .01*β : 0.361 - .27 Daytime performance Chest x-ray and mammography volume <.0001* .33 .017* <.0001*β : −0.908 - .30 CT, MRI, US, or PET volume <.0001* .61 .012* <.0001*β : 0.516 - .39 Mean evaluation score .39 .08 .16 .91β : −0.628 .14 .45
ACR, American College of Radiology; CT, computed tomography; MRI, magnetic resonance imaging; PET, positron emission tomography; US, ultrasound.
Univariate statistics with P < .10 were included in the multivariate analyses. “-” indicates noninclusion in multivariate model because univariate P ≥ .10. β indicates standardized regression coefficient for significant variables in the multivariate analyses. Asterisks indicate P < 0.05 (statistically significant).
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Discussion
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
References
1. Accreditation Council for Graduate Medical Education (ACGME) : ACGME common program requirements. Last updated: February; effective: July 1, 2017; Available at: http://www.acgme.org/Portals/0/PFAssets/ProgramRequirements/CPRs_2017-07-01.pdf
2. Accreditation Council for Graduate Medical Education (ACGME) : ACGME program requirements for graduate medical education in diagnostic radiology. Last updated: September 30; effective: July 1, 2016; Available at: https://www.acgme.org/Portals/0/420_diagnostic_radiology_PRs_RC.pdf
3. Gene Hern H., Wills C., Alter H., et. al.: Conference attendance does not correlate with emergency medicine residency in-training examination scores. Acad Emerg Med 2009; 16: pp. S63-S66.
4. Sawatsky A.P., Zickmund S.L., Berlacher K., et. al.: Understanding resident learning preferences within an internal medicine noon conference lecture series: a qualitative study. J Grad Med Educ 2014; 6: pp. 32-38.
5. Sawatsky A.P., Berlacher K., Granieri R.: Using an ACTIVE teaching format versus a standard lecture format for increasing resident interaction and knowledge achievement during noon conference: a prospective, controlled study. BMC Med Educ 2014; 14: pp. 129.
6. Sawatsky A.P., Zickmund S.L., Berlacher K., et. al.: Understanding the challenges to facilitating active learning in the resident conferences: a qualitative study of internal medicine faculty and resident perspectives. Med Educ Online 2015; 2:
7. Cacamese S.M., Eubank K.J., Hebert R.S., et. al.: Conference attendance and performance on the in-training examination in internal medicine. Med Teach 2004; 26: pp. 640-644.
8. FitzGerald J.D., Wenger N.S.: Didactic teaching conferences for IM residents: who attends, and is attendance related to medical certifying examination scores?. Acad Med 2003; 78: pp. 84-89.
9. Shetler P.L.: Observations on the American Board of Surgery in-training examination, board results, and conference attendance. Am J Surg 1982; 144: pp. 292-294.
10. McDonald F.S., Zeger S.L., Kolars J.C.: Associations of conference attendance with internal medicine in-training exam scores. Mayo Clin Proc 2008; 83: pp. 449-453.
11. Adusumilli S., Cohan R.H., Marshall K.W., et. al.: How well does applicant rank order predict subsequent performance during radiology residency?. Acad Radiol 2000; 7: pp. 635-640.
12. Gunderman R.B., Jackson V.P.: Are NBME examination scores useful in selecting radiology residency candidates?. Acad Radiol 2000; 7: pp. 603-606.
13. Metro D.G., Talarico J.F., Patel R.M., et. al.: The resident application process and its correlation to future performance as a resident. Anesth Analg 2005; 100: pp. 502-505.
14. Van Meter M., Williams M., Banuelos R., et. al.: Does the national resident match program rank list predict success in emergency medicine residency programs?. J Emerg Med 2017; 52: pp. 77-82.