Rationale and Objectives
To examine whether an intervention strategy consisting of a tailored web-based intervention, which provides individualized audit data with peer comparisons and other data that can affect recall, can assist radiologists in setting goals for reducing unnecessary recall.
Materials and Methods
In a multisite randomized controlled study, we used a tailored web-based intervention to assess radiologists’ ability to set goals to improve interpretive performance. The intervention provided peer comparison audit data, profiled breast cancer risk in each radiologist’s respective patient populations, and evaluated the possible impact of medical malpractice concerns. We calculated the percentage of radiologists who would consider changing their recall rates, and examined the specific goals they set to reduce recall rates. We describe characteristics of radiologists who developed realistic goals to reduce their recall rates, and their reactions to the importance of patient risk factors and medical malpractice concerns.
Results
Forty-one of 46 radiologists (89.1%) who started the intervention completed it. Thirty-one (72.1%) indicated they would like to change their recall rates and 30 (69.8%) entered a text response about changing their rates. Sixteen of the 30 (53.3%) radiologists who included a text response set realistic goals that would likely result in reducing unnecessary recall. The actual recall rates of those who set realistic goals were not statistically different from those who did not (13.8% vs. 15.1%, respectively). The majority of selected goals involved re-reviewing cases initially interpreted as Breast Imaging Reporting and Data System category 0. More than half of radiologists who commented on the influence of patient risk (56.3%) indicated that radiologists planned to pay more attention to risk factors, and 100% of participants commented on concerns radiologists have about malpractice with the primary concern (37.5%) being fear of lawsuits.
Conclusions
Interventions designed to reduce unnecessary recall can succeed in assisting radiologists to develop goals that may ultimately reduce unnecessary recall.
The full potential of mortality reduction by mammography may be limited by the current variability in radiologists’ interpretive performance . Sources of variability affecting performance have included fellowship training , which is associated with improved performance. Other studies have shown that, for some radiologists, inflated perceptions of breast cancer risk in their patient populations and inflated perceptions of their own malpractice risk are associated with higher than recommended recall rates, which cause harms such as patient anxiety, and excessive false-positive biopsies. The potential harms of mammography were a consideration when the US Preventive Services Task Force revised its recommendation for mammography frequency among average risk women age 50 and older .
Prior research suggests that educational interventions may have value, but intervention research that includes rigorous study designs have limitations. The UK National Health Program evaluated a 2-week multidisciplinary course with specialist training at high-volume screening sites, which included three sessions per week of interpreting screening mammograms. Radiologists additionally attended routine breast disease−related meetings and received personal and group audit reports that included data on cancer detection rate, recall rate, and positive predictive value of biopsy recommendation . Performance indices showed a reduction in the recall rate from 7% to 4%, and an increase in the small invasive cancer detection rate from 1.6/1000 to 2.5/1000. Unfortunately, mechanisms to track such indices in the United States do not typically exist. In another study, performed within a major US health maintenance organization , 21 radiologists were provided with personal and group audit reports, participated in a self-assessment program, attended case review sessions, and were required to interpret 8000 mammograms per year per radiologist. Sensitivity improved from 70% to 80%, with a mean cancer detection rate of 7.5/1000 and a mean recall rate of 7%.
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Methods
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Radiologist Survey
Get Radiology Tree app to read full this article<
Web-based Tailored Educational Intervention Data System
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Data Analyses
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Results
Get Radiology Tree app to read full this article<
Table 1
Radiologist Characteristics ∗ According to Whether or Not Realistic Goals Were Set to Reduce Unnecessary Recall
Characteristics All Radiologists Combined
( n = 41) Radiologists Who Set Realistic Goals ( n = 16) Radiologists Who Did Not Set Realistic Goals ( n = 25)P Value Demographics Sex Male 21 (52.5) 8 (50.0) 13 (54.2) .796 Female 19 (47.5) 8 (50.0) 11 (45.8) Practice type Primary affiliation with academic medical center No 34 (85.0) 13 (81.3) 21 (87.5) 1.00 Adjunct 2 (5.0) 1 (6.3) 1 (4.2) Primary 4 (10.0) 2 (12.5) 2 (8.3) Breast imaging experience Fellowship training No 39 (97.5) 15 (93.8) 24 (100.0) .400 Yes 1 (2.5) 1 (6.3) 0 (0.0) Years of mammography interpretation <10 8 (20.0) 4 (25.0) 4 (16.7) .837 10–19 16 (40.0) 6 (37.5) 10 (41.7) ≥20 16 (40.0) 6 (37.5) 10 (41.7) Percent of time spent in breast imaging <20 10 (25.0) 2 (12.5) 8 (33.3) .214 20–39 15 (37.5) 5 (31.3) 10 (41.7) 40–79 9 (22.5) 6 (37.5) 3 (12.5) 80–100 6 (15.0) 3 (18.8) 3 (12.5) Actual recall rate Mean 14.6 13.8% 15.1% .627 95% Confidence intervals 12.0–17.2 10.8–16.9 11.2–19.1 Range 4.0–50.0 7.3–27.4 4.0–50.0
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Table 2
Radiologists’ Recall Rates and Realistic Goals Set as Part of the Auditing Component of the Tailored Web-based Intervention
Radiologists Recall Rate (%) Goals Set to Improve Recall Rates Interpretive Themes Set goal to increase recall 7.3 Recall more of the probably benign rather than following at six months. Change interpretive threshold ∗ Set goal to decrease unnecessary recall 9.2 I’d like it to be closer to 6%–7% and could re-review some cases. Re-review 9.3 Raise threshold of concern for asymmetry. Change interpretive threshold 10.2 Double-check each possible recall before calling patient back. Re-review 10.7 By reviewing the results/images of most/all of the cases that I have placed into a category 0. Re-review 11.2 Less call back for lesions that are highly suggestive of a benign process cyst or LN (lymph node) that I would call back only because of double read and another radiologist would insist on calling back. Change interpretive threshold 12.1 Decrease call-backs for densities that I am reasonably certain are summation densities. change interpretive threshold 12.3 Try to determine benign findings at the time of screening exam. Re-review 13.3 Be sure to review more prior mammograms than are currently on the alternator. Change interpretive practices 14.1 Change more zero codes to early follow-up at 1 year. Change interpretive practices 15.2 Trust my judgment, initially. Look at the mammogram less, so as not to second guess. Ask for second opinions from colleagues less often, as they often push my rate higher. Change interpretive practices 17.2 Giving extra thought before call-backs, although my recall rate has been higher the last 2 years due to adjustment to a digital system. Change interpretive threshold 19.8 I need to increase my sensitivity for small cancers and ask for 2nd read more often Seek second independent review 22.2 On uncertain cases, get second opinion so decision-making is optimized. Seek second independent review 24.6 Try to limit unnecessary recalls with re-review Re-review 27.4 High recall rate is related to our previous methodology of final reads. We didn’t read out BI-RADS 0. I was reading many other radiologists callbacks. We have changed. Change interpretive practices
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Table 3
Responses for How Use of Risk Factors May Change Recall Rates As a Result of the Intervention
Radiologist’s Recall Rate (%) Physicians Responses Regarding the Use of Risk Factors—Is there anything you would do differently as a result of what you have learned? Interpretive Themes Radiologist Set Realistic Goals 4.0 Pay more attention to age and race/ethnicity Pay more attention to risk No 5.0 Look at BMI. Pay more attention to risk No 5.7 Try not to be swayed by risk factors when dealing with equivocal findings Pay less attention to risk Yes 8.6 I currently review risk factors on all patients. No Change in Practice No 9.2 I generally review risk factors and would continue No Change in Practice No 9.4 Review risk factors more often Pay more attention to risk No 10.2 Yes, consider more select factors depending on pre- vs. postmenopausal status Pay more attention to risk Yes 13.0 Not consider menarche and consider race Pay less attention to risk No 13.2 Awareness only No change in practice No 13.3 Closer look at obese dense breasts Pay more attention to risk Yes 18.7 Worry a bit more about questionable findings in dense breasts (be bit more aggressive); and same with older women Consider risk in certain situations No 18.8 Try not to overestimate a woman’s risk for breast cancer, especially for premenopausal. Would also remember that breast density is a risk factor which is predictive of cancer in 1 year. Pay more attention to some risk factors and less to others No 19.8 Put more weight on past negative benign biopsies and breast density as risk factors. Pay more attention to risk Yes 22.2 Only marginally. If I was unsure of the need for a possible callback, risk factors could possibly influence my decision. Pay more attention to risk Yes 24.6 Be more certain of checking for past procedures Pay more attention to risk Yes
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Table 4
Responses to the Medical Malpractice Component of the Tailored Web-based Intervention According to Whether or Not Realistic Goals Were Set to Reduce Unnecessary Recall
Physicians Responses Regarding Medical Malpractice Radiologists Who Set Realistic Goals ( n = 16) Radiologists Who Did Not Set Realistic Goals ( n = 25)P Value Preintervention Mean (SD) †
Scale 0–10 ∗ Mean (SD) †
Scale 0–10 ∗ To what extent do you think your medical malpractice concerns have influenced your recall rate? 3.8 (2.2) 3.5 (2.4) .70 To what extent do you think your medical malpractice concerns have influenced your recommendation for breast biopsies? 3.9 (2.6) 3.4 (2.2) .62 Postintervention To what extent do you think your medical malpractice concerns will influence your recall rate? 2.0 (1.4) 1.6 (1.5) .45 To what extent do you think your medical malpractice concerns will influence your recommendation for breast biopsies? 1.9 (1.8) 1.6 (1.4) .52 What is the probable risk of a mammography related malpractice suit occurring in the next five years among radiologists working full-time in breast imaging? Less than 10% 16 (100.0) 25 (100.0) NA How many radiologists are concerned that fear about medical malpractice affects how they interpret mammograms? 25% 1 (6.3) 3 (12.0) .56 50% 4 (25.0) 9 (36.0) 75% 11 (68.8) 13 (52.0)
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Table 5
Responses for How Risk of Medical Malpractice Change Recall Rates As a Result of the Intervention
Radiologist’s Recall Rate Reactions to Malpractice Content: Why do you think so many radiologists over estimate their own malpractice risk? Interpretive Themes Radiologist Set Realistic Goals 4.0 This seems to be much more prevalent in young, recently trained docs. NA No 5.0 They hear about the sentinel cases of large awards and they extrapolate. Impact of hearsay No 5.7 Because it is in the news and the dollar amount is usually very high. We interpret a lot of mammograms and it is so subjective compared to other exams. Media sensation No 6.7 Anecdotal cases leave deep impression on radiologists. Impact of hearsay No 7.2 Fear. And the lack of accurate data to counteract lawsuits. Fear of lawsuit No 7.3 Publicity regarding large settlements. Media sensation Yes 8.6 Personal fear of being sued, high publicity in media of suits, work environment (e.g., being tired, being distracted while interpreting mammograms). Fear of lawsuit and media exposure No 8.7 Breast cancer is so frequently in the media. Patients all have “a friend in whom the breast cancer was missed…” Media sensation No 9.2 Because you never know when it will be your turn, and you assume all misses will result in a lawsuit. Fear of lawsuit No 9.3 The press. Media unspecified Yes 9.3 Fear of being sued, hearsay and gossip about suits. Fear of lawsuit No 9.4 Reports in the literature of lawsuits. Fear of lawsuit No 9.9 We are told that the most common reason for malpractice suit against a radiologist is failure to detect cancer on a mammogram. Fear of lawsuit No 10.2 Because of the monetary, emotional, and time cost of a malpractice suit. Fear of lawsuit Yes 10.7 Do not know, perhaps because of hearsay information which is not based on fact. Impact of hearsay Yes 11.2 Because of newspaper articles and nonverified data. Media sensation Yes 11.9 It is everywhere in the media…also a recent study does show increasing malpractice suits in radiology. Media sensation No 12.0 Probably cause media reports big cases and settlements, and malpractice premiums are higher if we read mammography. Media sensation No 12.1 Fear—know a colleague who has been involved in a suit, do not wish to risk negative media exposure. Media exposure Yes 13.0 The suits we do hear about are frightening and the awards are growing ever higher. Impact of hearsay No 13.2 Fear, and horror stories. Fear unspecified No 13.3 First because of the difficulty in mammography interpretation. Second, because of the very litigious society. Fear of lawsuit Yes 13.3 Fear of litigious patients/lawyers. Fear of lawsuit Yes 13.4 We don’t understand how the world works. We are surprised to find out that the legal system protects physicians, attorneys, and insurers rather than patients. Fear of lawsuit No 14.1 Fear Fear unspecified No 14.1 Because risk is for general numbers…a lawsuit for them is 100%. Fear of lawsuit Yes 15.2 Fear of being responsible for a woman’s untimely death. Fear for patient Yes 17.2 Fear of a trial is overwhelming and potentially devastating and therefore the risk is overestimated. Fear of lawsuit Yes 18.7 Media attention. Traumatic effect of even one such case per career. Media exposure No 18.8 Newspapers/television articles. Media unspecified No 19.4 Fear of the consequences Fear unspecified No 19.4 Fear Fear unspecified No 19.5 News media Media unspecified No 21.5 Pervasive fear, attitudes of patients, lawsuits we have witnessed. Fear of Lawsuit No 22.2 You state that less than 10% of radiologists are sued… statistics are only statistics. Fear of lawsuit 24.4 Fear Fear unspecified Yes 24.6 General publicity of malpractice cases. Perception of potential large monetary damages. Media sensation Yes 26.3 Fear of lawsuits. Mammography is known as one of the highest classes of risk to radiologists in our country. Misdiagnosis of cancer, and misdiagnosis of fractures have been the highest. Fear of lawsuit No 27.4 Every false-negative case is surrounded by much concern and handwringing on part of referring physicians. We wonder what is said to patients behind closed doors. Fear of professional Yes 50.0 Lawsuits are a nuisance. Fear of lawsuit No
Get Radiology Tree app to read full this article<
Discussion
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
References
1. Elmore J.G., Wells C.K., Lee C.H., et. al.: Variability in radiologists’ interpretations of mammograms. N Engl J Med 1994; 331: pp. 1493-1499.
2. Beam C.A., Layde P.M., Sullivan D.C.: Variability in the Interpretation of screening mammograms by US radiologists. Arch Intern Med 1996; 156: pp. 209-213.
3. Elmore J.G., Miglioretti D.L., Reisch L.M., et. al.: Screening mammograms by community radiologists: variability in false-positive rates. J Natl Cancer Inst 2002; 94: pp. 1373-1380.
4. Smith-Bindman R., Chu P., Miglioretti D.L., et. al.: Physician predictors of mammographic accuracy. J Natl Cancer Inst 2005; 97: pp. 358-367.
5. Carney P.A., Elmore J.G., Abraham L.A., et. al.: Radiologist uncertainty and the interpretation of screening mammography. Medical Decision Making 2004; 24: pp. 255-264.
6. Elmore J.G., Taplin S., Barlow W.E., et. al.: Does litigation influence medical practice? The influence of community radiologists’ medical malpractice perceptions and experience on screening mammography. Radiology 2005; 236: pp. 37-46.
7. Egger J.R., Cutter G.R., Carney P.A., et. al.: Mammographers’ perception of women’s breast cancer risk. Medical Decision Making 2005; 25: pp. 283-289.
8. Fenton J.J., Egger J., Carney P.A., et. al.: Reality check: perceived versus actual performance of community mammographers. AJR Am J Roentgenol 2006; 187: pp. 42-46.
9. Miglioretti D.L., Smith-Bindman R., Abraham L.A., et. al.: Radiologist characteristics associated with interpretive performance of diagnostic mammography. J Natl Cancer Inst 2007; 99: pp. 1854-1863.
10. Taplin S.H., Abraham L.A., Barlow W.E., et. al.: Mammography facility characteristics associated with screening mammography interpretive accuracy. J Natl Cancer Inst 2008; 100: pp. 876-887.
11. Geller B.M., Aiello Bowles E., Sohng H.Y., et. al.: Radiologists’ performance and their enjoyment of interpreting screening mammography. AJR Am J Roentgenol 2009; 192: pp. 361-369.
12. Miglioretti D.L., Gard C.C., Carney P.A., et. al.: When radiologists perform best: The learning curve in screening mammography interpretation. Radiology 2009; 253: pp. 632-640.
13. Smith-Bindman R., Chu P., Miglioretti D.L., et. al.: Physician predictors of mammographic accuracy. J Natl Cancer Inst 2005; 97: pp. 358-367.
14. Barlow W.E., Chi Chen, Carney P.A., et. al.: Accuracy of screening mammography interpretation by characteristics of radiologists. J Natl Cancer Inst 2004; 96: pp. 1840-1850.
15. Elmore J.G., Jackson S.L., Abraham L., et. al.: Variability in interpretive performance of screening mammography and radiologist characteristics associated with accuracy. Radiology 2009; 253: pp. 641-651.
16. http://www.uspreventiveservicestaskforce.org/uspstf/uspsbrca.htm . Accessed October 27, 2010.
17. Perry N.M.: Breast cancer screening—the European experience. Int J Fertility Women’s Med 2004; 49: pp. 228-230.
18. Adcock K.A.: Initiative to improve mammogram interpretation. Permanente J 2004; 8: Available at: http://xnet.kp.org/permanentejournal/spring04/vohswin.html .
19. American College of Radiology, Breast Imaging Reporting and Data System (BI-RADS), Copyright 2004.
20. Carney P.A., Geller B.M., Moffett H., et. al.: Current medico-legal and confidentiality issues in large multi-center research programs. Am J Epidemiol 2000; 152: pp. 371-378.
21. Carney PA, Geller BM, Abraham L, et al. Feasibility and satisfaction associated with using a tailored web-based intervention for recalibrating radiologists thresholds for conducting additional work-up. In press.
22. Berelson B.R.: Content analysis.Lindsey G.Handbook of social psychology.1954.Addison-WellesleyReading, MA:
23. Carney P.A., Sickles E., Monsees B., et. al.: Identifying minimally acceptable interpretive performance criteria for screening mammography. Radiology 2010; 255: pp. 354-361.
24. Rosenberg R.D., Yankaskas B.C., Abraham L., et. al.: Performance benchmarks for screening mammography. Radiology 2006; 241: pp. 55-66.
25. Wakefield J., Herbert C.P., Maclure M., et. al.: Commitment to change statements can predict actual change in practice. J Conti Educ Health Prof 2003; 23: pp. 81-92.
26. Smith-Bindman R., Chu P.W., Miglioretti D.L., et. al.: Comparison of screening mammography in the United States and the United Kingdom. JAMA 2003; 290: pp. 2129-2137.
27. Hofvind S., Vacek P.M., Skelly J., et. al.: Comparing screening mammography for early breast cancer detection in Vermont and Norway. J Natl Cancer Inst 2008; 100: pp. 1082-1091.
28. Carney PA, Cook AJ, Miglioretti DL, et al. Radiologists’ Use of Clinical History and its Impact on Interpretive Performance (In Review).