Rationale and Objectives
Workplace-based assessments gauge the highest tier of clinical competence. Chart-stimulated recall (CSR) is a workplace-based assessment method that complements chart audit with an interview based on the residents’ notes. It allows evaluation of the residents’ knowledge and heuristics while providing opportunities for feedback and self-reflection. We evaluated the utility of CSR for improving the radiology residents’ reporting skills.
Materials and Methods
Residents in each year of training were randomly assigned to an intervention group ( n = 12) or a control group ( n = 13). Five pre-intervention and five post-intervention reports of each resident were independently evaluated by three blinded reviewers using a modified Bristol Radiology Report Assessment Tool. The study intervention comprised a CSR interview tailored to each individual resident’s learning needs based on the pre-intervention assessment. The CSR process focused on the clinical relevance of the radiology reports. Student’s t test ( P < .05) was used to compare pre- and post-intervention scores of each group.
Results
A total of 125 pre-intervention and 125 post-intervention reports were evaluated (total 750 assessments). The Cronbach’s alpha for the study tool was 0.865. A significant improvement was seen in the cumulative 19-item score (66% versus 73%, P < .001) and the global rating score (59% versus 72%, P < .001) of the intervention group after the CSR. The reports of the control group did not demonstrate any significant improvement.
Conclusion
CSR is a feasible workplace-based assessment method for improving reporting skills of the radiology residents.
Introduction
Over the last two decades, there has been a progressive shift toward outcome-orientated medical education . Assessment plays an essential role in identifying the residents’ learning needs and guiding their learning efforts . The type and the frequency of assessment should match the objectives of the training program. Workplace-based assessment (WPBA) gauges the real-life practices of the residents, which represent the highest tier of clinical competence . A number of WPBA methods have been developed including Mini-Clinical Evaluation Exercise, Direct Observation of Procedural Skills, chart audits, and chart-stimulated recall (CSR) .
Chart audits have been recommended by the Accreditation Council for Graduate Medical Education as part of the practice-based learning to improve the patients’ care . The patient’s chart is an excellent source of information about the residents’ clinical practices. However, the residents’ heuristics have to be deduced during a chart audit. According to one estimate, chart audits are only 70% specific when compared to the quality of care assessments by the standardized patients .
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Materials and Methods
Get Radiology Tree app to read full this article<
Study Tool
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Study Participants
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Radiology Reports
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Intervention
Get Radiology Tree app to read full this article<
Faculty: The clinical history is shortness of breath. What should the clinician understand if your conclusion is “hilar vascular congestion?” What steps should the clinician take based on your conclusion?**Resident** : I was implying that the patient has inflammation, possibly infection.Faculty : Let’s discuss the findings on a chest radiograph associated with infection and how can we clearly communicate these findings to the referring physician.
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Data Entry and Analysis
Get Radiology Tree app to read full this article<
Results
Get Radiology Tree app to read full this article<
TABLE 1
Distribution of Residents in the Intervention and Control Groups According to the Year of Training
Year Intervention Group Control Group Total R1 3 2 5 R2 3 3 6 R3 1 3 4 R4 4 4 8 R5 1 1 2 Total 12 13 25
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
TABLE 2
Comparison of Pre- and Post-intervention Scores of the Intervention Group and the Control Group †
Intervention Group Control Group Pre-intervention Post-intervention_P_ Pre-intervention Post-intervention_P_ Mean (SD) Mean (SD) Mean (SD) Mean (SD) Item 12 \* 66.8 (22.19) 78.9 (13.98) .000 65.2 (18.70) 61.8 (20.43) .089 Item 17 \* 67.8 (18.72) 80.7 (12.13) .000 64.9 (15.87) 60.3 (17.87) .007 Technical aspects 68.0 (8.54) 72.2 (6.20) .000 65.1 (6.90) 64.2 (6.65) .214 Clarity and structure 49.8 (8.00) 54.6 (6.68) .000 49.0 (8.03) 46.8 (7.87) .006 Conclusion 62.3 (10.01) 68.3 (8.35) .000 61.3 (10.04) 58.5 (9.83) .006 Consideration of clinical implications 61.4 (6.63) 64.9 (4.18) .000 59.6 (6.18) 58.1 (6.11) .016 Total score (items 1–19) 66.2 (8.10) 72.6 (5.07) .000 63.4 (7.18) 62.2 (6.75) .096 Global rating score 59.3 (14.22) 71.7 (11.23) .000 56.8 (14.01) 54.3 (13.38) .080
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Discussion
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Conclusion
Get Radiology Tree app to read full this article<
References
1. Augustine K., McCoubrie P., Wilkinson J.R., et. al.: Workplace-based assessment in radiology-where to now?. Clin Radiol 2010; 65: pp. 325-332.
2. Epstein R.M.: Assessment in medical education. N Engl J Med 2007; 356: pp. 387-396.
3. General Medical Council : United Kingdom. Workplace Based Assessment: A guide for implementation. Available at: http://www.gmc-uk.org/Workplace_Based_Assessment___A_guide_for_implementation_0410.pdf_48905168.pdf Accessed December 15, 2016
4. Miller G.E.: The assessment of clinical skills/competence/performance. Acad Med 1990; 65: pp. S63-S67.
5. Norcini J., Burch V.: Workplace-based assessment as an educational tool: AMEE Guide No. 31. Med Teach 2007; 29: pp. 855-871.
6. Holmboe E.S., Edgar L., Hamstra S.: The Milestones Guidebook. Accreditation Council for Graduate Medical Education. Available at: http://www.acgme.org/Portals/0/MilestonesGuidebook.pdf?ver=2016-05-31-113245-103 Accessed December 30
7. Luck J., Peabody J.W., Dresselhaus T.R., et. al.: How well does chart abstraction measure quality? A prospective comparison of standardized patients with the medical record. Am J Med 2000; 108: pp. 642-649.
8. Jennett P.A., Scott S.M., Atkinson M.A., et. al.: Patient charts and physician office management decisions: chart audit and chart stimulated recall. J Contin Educ Health Prof 1995; 15: pp. 31-39.
9. Jyothirmayi R.: Case-based discussion: assessment tool or teaching aid?. Clin Oncol (R Coll Radiol) 2012; 24: pp. 649-653.
10. Sistrom C., Lanier L., Mancuso A.: Reporting instruction for radiology residents. Acad Radiol 2004; 11: pp. 76-84.
11. Wallis A., Edey A., Prothero D., et. al.: The Bristol Radiology Report Assessment Tool (BRRAT): developing a workplace-based assessment tool for radiology reporting skills. Clin Radiol 2013; 68: pp. 1146-1154.
12. Barrett A., Galvin R., Steinert Y., et. al.: A BEME (Best Evidence in Medical Education) systematic review of the use of workplace-based assessment in identifying and remediating poor performance among postgraduate medical trainees. Syst Rev 2015; 4: pp. 65.
13. Colman A.M., Norris C.E., Preston C.C.: Comparing rating scales of different lengths: equivalence of scores from 5-point and 7-point scales. Psychol Rep 1997; 80: pp. 355-362.
14. Williamson K.B., Gunderman R.B., Cohen M.D., et. al.: Learning theory in radiology education. Radiology 2004; 233: pp. 15-18.
15. Woodfield C.A., Mainiero M.B.: Radiology resident dictation instruction: effectiveness of the didactic lecture. J Am Coll Radiol 2008; 5: pp. 842-846.
16. Johnson G., Booth J., Crossley J., et. al.: Assessing trainees in the workplace: results of a pilot study. Clin Med (Lond) 2011; 11: pp. 48-53.
17. Jolly B.C., Ayers B., Macdonald M.M., et. al.: The reproducibility of assessing radiological reporting: studies from the development of the General Medical Council’s Performance Procedures. Med Educ 2001; 35: pp. 36-44.
18. Williamson K.B., Steele J.L., Gunderman R.B., et. al.: Assessing radiology resident reporting skills. Radiology 2002; 225: pp. 719-722.
19. Veloski J., Boex J.R., Grasberger M.J., et. al.: Systematic review of the literature on assessment, feedback and physicians’ clinical performance: BEME Guide No. 7. Med Teach 2006; 28: pp. 117-128.
20. Nicol D.J., Macfarlane-Dick D.: Formative assessment and self-regulated learning: a model and seven principles of good feedback practice. Stud High Educ 2006; 31: pp. 199-218.
21. Boyd C.M., Darer J., Boult C., et. al.: Clinical practice guidelines and quality of care for older patients with multiple comorbid diseases: implications for pay for performance. JAMA 2005; 294: pp. 716-724.
22. Jennett P., Affleck L.: Chart audit and chart stimulated recall as methods of needs assessment in continuing professional health education. J Contin Educ Health Prof 1998; 18: pp. 163-171.
23. Collard M.D., Tellier J., Chowdhury A.S., et. al.: Improvement in reporting skills of radiology residents with a structured reporting curriculum. Acad Radiol 2014; 21: pp. 126-133.