Rationale and Objectives
Educating medical students on appropriate imaging utilization has been increasingly recognized as important for patient care. The American College of Radiology Appropriateness Criteria (ACR-AC) is designed to support evidence-based imaging examination selection. We sought to assess whether medical students order imaging studies independently, what resources they use for guidance, and whether they use the ACR-AC in clinical practice. A secondary aim was to determine whether increasing familiarity with the ACR-AC could impact student usage.
Materials and Methods
We surveyed third year medical students at a single institution on their imaging practices, familiarity with the ACR-AC, and preferences among available resources to guide proper examination selection. The survey was performed in person before a lecture. We also designed a brief intervention to improve familiarity with the ACR-AC and then reassessed students to determine any effect on utilization.
Results
The response rate for the initial survey was 103 of 109 (94%) and the response rate for the second survey was 99 of 109 (91%). Our initial survey found students initiated imaging orders independently (74 of 100, 74.8%) and consulted resources to assist in examination selection (50 of 74, 67.6%). Students expressed a preference for non-ACR-AC resources, notably UptoDate via its online mobile application. Few students (8 of 71, 11.3%) were familiar with the ACR-AC. After an intervention to increase familiarity with the ACR-AC, student awareness of the ACR-AC increased to 61 of 74 (82.4%). However, usage among those familiar with the resource remained low, 13 of 61 (21.3%) versus 3 of 8 (37.5%).
Conclusions
Use of the ACR-AC was low among third year medical students. After increasing students’ familiarity with the ACR-AC, their usage in a clinical setting did not increase. The largest barrier to use may be the lack of a quick, easy to use online mobile application–based interface.
Medical imaging is a critical and growing component of modern medical diagnosis and practice. Over the past 4 decades, advances in diagnostic imaging have contributed greatly to patient care, improving our ability to detect disease, guide procedures, and deliver treatments . Most medical specialties now regularly use medical imaging, which has led to a substantial increase in the number of diagnostic imaging examinations performed in recent years . It is estimated that imaging services have grown at about twice the rate of other health care technologies over the past decade .
The increased use of medical imaging comprised examinations that are beneficial to patients’ welfare and examinations that could be considered inappropriate to use . Several publications have documented that as many as 25%–50% of advanced imaging studies fail to improve patient welfare and may be unnecessary . This not only contributes to escalating health care costs in the United States but also exposes patients to unnecessary risks including radiation, contrast-related complications , and unnecessary interventions for incidentalomas .
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Materials and methods
Get Radiology Tree app to read full this article<
Survey Design and Administration
Get Radiology Tree app to read full this article<
Table 1
Third Year Medical Student Responses to “Yes/No,” “Multiple Choice,” and Likert-Scale Survey Questions
Question Answer Choices Preintervention Responses, n (%) Postintervention Responses_P_ Value Have you ever placed an order for an imaging exam (including orders which needed to be co-signed) before? Yes 70 (68) 74 (74.8) .35 No 33 (32) 25 (25.2) How often do you initiate ordering an imaging study on your own (as opposed to being told that ordering an imaging study is needed)? 5 Always 0 (0) 0 (0) .025 4 Often 4 (5.6) 7 (9.5) 3 Sometimes 18 (25.4) 31 (41.9) 2 Rarely 34 (47.9) 31 (41.9) 1 Never 15 (21.15) 5 (6.8) Do you typically consult any resource in deciding what study to order? Yes 46 (64.8) 50 (67.6) .72 No 25 (35.2) 24 (32.4) If so, which resource do you primarily use? UpToDate 42 ( ∗ ) 55 ( ∗ ) .001 Medscape 4 ( ∗ ) 3 ( ∗ ) ACR-AC 6 ( ∗ ) 6 ( ∗ ) Google 11 ( ∗ ) 7 ( ∗ ) Access Medicine 1 ( ∗ ) 0 ( ∗ ) Other 15 ( ∗ ) 2 ( ∗ ) Have you ever heard of the ACR Appropriateness Criteria? Yes 8 (11.3) 61 (82.4) <.001 No 63 (88.7) 13 (17.6) If you have heard of it have you ever used the ACR Appropriateness Criteria in deciding what to order? Yes 3 (37.5) 13 (21.3) .37 No 5 (62.5) 48 (78.7) Did learning about the ACR Appropriateness Criteria change how you approach ordering imaging studies? † Yes N/A 12 (20.1) N/A No 46 (79.3) If you have heard of the ACR Appropriateness Criteria, how likely are you to use it in your future practice? † 5 Very likely N/A 18 (23.1) N/A 4 Likely 28 (35.9) 3 Possibly 29 (37.2) 2 Unlikely 2 (2.6) 1 Not at all 1 (1.3) How often do you have input on what imaging study is ordered in collaboration with the team? 5 Always 5 (4.9) 9 (9.1) .24 4 Often 14 (13.7) 19 (19.2) 3 Sometimes 37 (36.3) 41 (41.4) 2 Rarely 40 (39.2) 27 (27.3) 1 Never 6 (5.9) 3 (3.0) When orders are placed for an imaging study, how strong is your understanding of the rationale for the choice of imaging (ie, CT vs. MRI vs. US)? 5 Very strong 2 (1.9) 4 (4.0) .38 4 Strong 27 (26.2) 32 (32.3) 3 Moderate 65 (63.1) 59 (59.6) 2 Low 9 (8.7) 4 (4.0) 1 None 0 (0) 0 (0) If a resource were designed to help you learn about and select appropriate radiology studies, what format would you prefer? Mobile application 66 ( ∗ ) 69 ( ∗ ) .68 Book 4 ( ∗ ) 8 ( ∗ ) Internet site 43 ( ∗ ) 36 ( ∗ ) Electronic medical record based 19 ( ∗ ) 19 ( ∗ ) Other 2 ( ∗ ) 1 ( ∗ )
ACR-AC, American College of Radiology Appropriateness Criteria; CT, computed tomography; MRI, magnetic resonance imaging; N/A, not applicable; US, ultrasound.
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Table 2
Responses to the Free Text Question: “Why Do You Prefer This Resource?” Coded Categories for Those Respondents Choosing the Most Common Option: UptoDate
Coded Responses for Students Choosing UptoDate as Preferred Resource Number of Respondents Easy to use 17 Habit/familiarity 10 Trust/reliability 7 Availability of other information (medical management, and so forth) 7 Used by other members of the team 4 Links to literature 4 Organization/format 3 Fast/efficient 2 Comprehensive 2 Other 1 Total responses 57 ∗
Get Radiology Tree app to read full this article<
Table 3
Coded and Categorized Responses to the Free Text Question: “If You Have Heard of the ACR Appropriateness Criteria but Have Not Used it in Practice, Why Not?”
Coded Responses Number of Respondents Forgot 21 Attending/resident decided what to order 7 Habits/familiarity 6 Did not need resource 4 Accessibility/usability 3 Did not place order 3 Other 3 Not used by team 2 Total responses 47 ∗
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Increasing Student Familiarity with the ACR-AC
Get Radiology Tree app to read full this article<
Follow-up Survey Design and Administration
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Data Coding
Get Radiology Tree app to read full this article<
Statistical Analyses
Get Radiology Tree app to read full this article<
Results
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Discussion
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
References
1. Straus C.M., Webb E.M., Kondo K.L., et. al.: Medical student radiology education: summary and recommendations from a national survey of medical school and radiology department leadership. J Am Coll Radiol 2014; 11: pp. 606-610.
2. Gunderman R.B., Siddiqui A.R., Heitkamp D.E., et. al.: The vital role of radiology in the medical school curriculum. AJR Am J Roentgenol 2003; 180: pp. 1239-1242.
3. Kondo K.L., Swerdlow M.: Medical student radiology curriculum: what skills do residency program directors believe are essential for medical students to attain?. Acad Radiol 2013; 20: pp. 263-271.
4. Hendee W.R., Becker G.J., Borgstede J.P., et. al.: Addressing overutilization in medical imaging. Radiology 2010; 257: pp. 240-245.
5. Picano E.: Sustainability of medical imaging. BMJ 2004; 328: pp. 578-580.
6. America’s Health Insurance Plans: Ensuring quality through appropriate use of diagnostic imaging.2008.American’s Health Insurance PlansWashington, DC
7. Brenner D.J., Hall E.J.: Computed tomography–an increasing source of radiation exposure. N Engl J Med 2007; 357: pp. 2277-2284.
8. Iglehart J.K.: The new era of medical imaging–progress and pitfalls. N Engl J Med 2006; 354: pp. 2822-2828.
9. Rao V.M., Levin D.C.: The overuse of diagnostic imaging and the choosing wisely initiative. Ann Intern Med 2012; 157: pp. 574-576.
10. Mettler F.A., Bhargavan M., Faulkner K., et. al.: Radiologic and nuclear medicine studies in the united states and worldwide: frequency, radiation dose, and comparison with other radiation sources–1950-2007. Radiology 2009; 253: pp. 520-531.
11. Berrington de Gonzalez A., Mahesh M., Kim K.P., et. al.: Projected cancer risks from computed tomographic scans performed in the United States in 2007. Arch Intern Med 2009; 169: pp. 2071-2077.
12. 2009. http://www.ncrponline.org/Publications/Press_Releases/160press.html Accessed May 9, 2015
13. Spring D.B., Bettmann M.A., Barkan H.E.: Deaths related to iodinated contrast media reported spontaneously to the U.S. food and drug administration, 1978-1994: effect of the availability of low-osmolality contrast media. Radiology 1997; 204: pp. 333-337.
14. Spring D.B., Bettmann M.A., Barkan H.E.: Nonfatal adverse reactions to iodinated contrast media: Spontaneous reporting to the U.S. food and drug administration, 1978-1994. Radiology 1997; 204: pp. 325-332.
15. Sistrom C.L.: The ACR appropriateness criteria: translation to practice and research. J Am Coll Radiol 2005; 2: pp. 61-67.
16. Ip I.K., Schneider L.I., Hanson R., et. al.: Adoption and meaningful use of computerized physician order entry with an integrated clinical decision support system for radiology: ten-year analysis in an urban teaching hospital. J Am Coll Radiol 2012; 9: pp. 129-136.
17. Lehnert B.E., Bree R.L.: Analysis of appropriateness of outpatient CT and MRI referred from primary care clinics at an academic medical center: how critical is the need for improved decision support?. J Am Coll Radiol 2010; 7: pp. 192-197.
18. Taragin B.H., Feng L., Ruzal-Shapiro C.: Online radiology appropriateness survey: results and conclusions from an academic internal medicine residency. Acad Radiol 2003; 10: pp. 781-785.
19. Rice H.E., Frush D.P., Harker M.J., et. al., APSA Education Committee: Peer assessment of pediatric surgeons for potential risks of radiation exposure from computed tomography scans. J Pediatr Surg 2007; 42: pp. 1157-1164.
20. Krille L., Hammer G.P., Merzenich H., et. al.: Systematic review on physician’s knowledge about radiation doses and radiation risks of computed tomography. Eur J Radiol 2010; 76: pp. 36-41.
21. Merzenich H., Krille L., Hammer G., et. al.: Paediatric CT scan usage and referrals of children to computed tomography in Germany–a cross-sectional survey of medical practice and awareness of radiation related health risks among physicians. BMC Health Serv Res 2012; 12: pp. 47. 6963-12-47
22. Wong C.S., Huang B., Sin H.K., et. al.: A questionnaire study assessing local physicians, radiologists and interns’ knowledge and practice pertaining to radiation exposure related to radiological imaging. Eur J Radiol 2012; 81: pp. e264-e268.
23. Zhou G.Z., Wong D.D., Nguyen L.K., et. al.: Student and intern awareness of ionising radiation exposure from common diagnostic imaging procedures. J Med Imaging Radiat Oncol 2010; 54: pp. 17-23.
24. Sadigh G., Khan R., Kassin M.T., et. al.: Radiation safety knowledge and perceptions among residents: a potential improvement opportunity for graduate medical education in the United States. Acad Radiol 2014; 21: pp. 869-878.
25. Leschied J.R., Knoepp U.S., Hoff C.N., et. al.: Emergency radiology elective improves second-year medical students’ perceived confidence and knowledge of appropriate imaging utilization. Acad Radiol 2013; 20: pp. 1168-1176.
26. Chiunda A.B., Mohammed T.L.: Knowledge of ACR thoracic imaging appropriateness criteria(R) among trainees: one institution’s experience. Acad Radiol 2012; 19: pp. 635-639.
27. Bautista A.B., Burgos A., Nickel B.J., et. al.: Do clinicians use the American College of Radiology appropriateness criteria in the management of their patients?. AJR Am J Roentgenol 2009; 192: pp. 1581-1585.
28. Prezzia C., Vorona G., Greenspan R.: Fourth-year medical student opinions and basic knowledge regarding the field of radiology. Acad Radiol 2013; 20: pp. 272-283.
29. Dillon J.E., Slanetz P.J.: Teaching evidence-based imaging in the radiology clerkship using the ACR appropriateness criteria. Acad Radiol 2010; 17: pp. 912-916.
30. Choosing wisely. http://www.choosingwisely.org . Accessed May 9, 2015.
31. Image gently. http://www.imagegently.org . Accessed May 9, 2015.
32. Sistrom C.L., Dang P.A., Weilburg J.B., et. al.: Effect of computerized order entry with integrated decision support on the growth of outpatient procedure volumes: seven-year time series analysis. Radiology 2009; 251: pp. 147-155.
33. ACR appropriateness criteria. http://www.acr.org/Quality-Safety/Appropriateness-Criteria . Accessed May 9, 2015.
34. Logie C.I., Smith S.E., Nagy P.: Evaluation of resident familiarity and utilization of the ACR musculoskeletal study appropriateness criteria in the context of medical decision support. Acad Radiol 2010; 17: pp. 251-254.
35. Saha A., Roland R.A., Hartman M.S., et. al.: Radiology medical student education: an outcome-based survey of PGY-1 residents. Acad Radiol 2013; 20: pp. 284-289.
36. Naeger D.M., Phelps A., Kohi M., et. al.: Reading room electives: say goodbye to the “radi-holiday”. J Am Coll Radiol 2013; 10: pp. 442-448.
37. Webb E.M., Vella M., Straus C.M., et. al.: Interpretive versus noninterpretive content in top-selling radiology textbooks: what are we teaching medical students?. Acad Radiol 2015; 22: pp. 520-526.
38. ACR select. http://www.acrselect.org/ . Accessed May 19, 2015.
39. Blackmore C.C., Medina L.S.: Evidence-based radiology and the ACR Appropriateness Criteria. J Am Coll Radiol 2006 Jul; 3: pp. 505-509.