Rationale and Objectives
The study aimed to determine the inter-observer agreement among academic breast radiologists when using the Breast Imaging Reporting and Data System (BI-RADS) lesion descriptors for suspicious findings on diagnostic mammography.
Materials and Methods
Ten experienced academic breast radiologists across five medical centers independently reviewed 250 de-identified diagnostic mammographic cases that were previously assessed as BI-RADS 4 or 5 with subsequent pathologic diagnosis by percutaneous or surgical biopsy. Each radiologist assessed the presence of the following suspicious mammographic findings: mass, asymmetry (one view), focal asymmetry (two views), architectural distortion, and calcifications. For any identified calcifications, the radiologist also described the morphology and distribution. Inter-observer agreement was determined with Fleiss kappa statistic. Agreement was also calculated by years of experience.
Results
Of the 250 lesions, 156 (62%) were benign and 94 (38%) were malignant. Agreement among the 10 readers was strongest for recognizing the presence of calcifications (k = 0.82). There was substantial agreement among the readers for the identification of a mass (k = 0.67), whereas agreement was fair for the presence of a focal asymmetry (k = 0.21) or architectural distortion (k = 0.28). Agreement for asymmetries (one view) was slight (k = 0.09). Among the categories of calcification morphology and distribution, reader agreement was moderate (k = 0.51 and k = 0.60, respectively). Readers with more experience (10 or more years in clinical practice) did not demonstrate higher levels of agreement compared to those with less experience.
Conclusions
Strength of agreement varies widely for different types of mammographic findings, even among dedicated academic breast radiologists. More subtle findings such as asymmetries and architectural distortion demonstrated the weakest agreement. Studies that seek to evaluate the predictive value of certain mammographic features for malignancy should take into consideration the inherent interpretive variability for these findings.
Introduction
The mammography lexicon of the American College of Radiology Breast Imaging Reporting and Data System (ACR BI-RADS) was developed to standardize reporting and encourage uniform use of terms among radiologists . This lexicon has since become central to the practice of mammography. Utilization of a common language allows for clearer communication of mammographic findings to providers across practices. In addition, standardized BI-RADS terminology has proven invaluable for collecting and analyzing data on mammographic features, and for determining the predictive value of lesion descriptors for malignancy.
Several studies have reported the positive predictive value of the BI-RADS mammographic features . For example, studies have demonstrated that BI-RADS descriptors for calcification morphology and distribution can help determine the risk of malignancy , with round or punctate calcifications demonstrating the lowest probability of malignancy and fine linear branching calcifications having the highest probability. Similarly, the distribution of calcifications from least to most likely to represent malignancy is diffuse, regional, grouped, linear, and segmental. Positive predictive values for malignancy have also been described for non-calcified mammographic lesions, including masses, asymmetries, and architectural distortion .
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Materials and Methods
Study Participants
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Image Database and Interpretation
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Table 1
BI-RADS Descriptors for Suspicious Lesions
Descriptor Definition Mass Seen on two projections; convex outward borders; appears denser in the center than the periphery Focal asymmetry Seen on two projections; lacks the convex outward borders and conspicuity of a mass; may be interspersed with fat Asymmetry Potential mass or focal asymmetry but seen on only one mammographic projection Architectural distortion Distorted parenchyma: thin straight lines or spiculations radiating from a point; focal retraction, distortion or straightening at the edge of parenchyma Calcifications Morphology Round/punctate, amorphous, coarse heterogeneous, fine pleomorphic, or fine linear branching Distribution Diffuse, regional, grouped, linear, or segmental
BI-RADS, Breast Imaging Reporting and Data System.
Descriptors for lesions deemed suspicious (BI-RADS 4/5) were collected. Characteristically benign findings were not described.
Get Radiology Tree app to read full this article<
Case Details
Get Radiology Tree app to read full this article<
Table 2
Pathologic Diagnosis by Percutaneous or Surgical Biopsy
Pathology_N_ Benign (includes acellular amorphous debris, atypical ductal hyperplasia, atypical lobular hyperplasia, apocrine metaplasia, benign microcalcifications, cyst, duct ectasia, fibroadenoma, fibroadipose tissue, fibrocystic changes, fibrosis, lobular hyperplasia, lobular carcinoma in situ, proteinaceous debris and neutrophils, sclerosing adenosis, stromal hyperplasia, and usual ductal hyperplasia) 156(62) Malignant 94(38) Invasive carcinoma (includes ductal, lobular, mixed ductal and lobular, tubular, and mucinous) 66 Ductal carcinoma in situ 26 Adenocarcinoma NOS 2
NOS, not otherwise specified.
Data in parentheses are percentages of total ( N = 250) examinations. Benign pathology was confirmed with at least 1 year of imaging follow-up.
Get Radiology Tree app to read full this article<
Statistical Analysis
Get Radiology Tree app to read full this article<
Results
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Table 3
Agreement Between 10 Readers
Mammographic Finding k 95% CI Architectural distortion 0.28 0.26–0.30 Mass 0.67 0.65–0.69 Focal asymmetry (two-view) 0.21 0.19–0.23 Asymmetry (one-view) 0.09 0.08–0.11 Microcalcifications 0.82 0.80–0.84 Distribution 0.60 0.59–0.61 Morphology 0.51 0.50–0.52
CI, confidence interval; k, kappa coefficient.
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Discussion
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Conclusions
Get Radiology Tree app to read full this article<
Acknowledgment
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
References
1. D’Orsi C.J.: The American College of Radiology mammography lexicon: an initial attempt to standardize terminology. AJR Am J Roentgenol 1996; 166: pp. 779-780.
2. American College of Radiology : Breast Imaging Reporting and Data System (BI-RADS).1993.American College of RadiologyReston, VA
3. Liberman L., Abramson A.F., Squires F.B., et. al.: The Breast Imaging Reporting and Data System: positive predictive value of mammographic features and final assessment categories. AJR Am J Roentgenol 1998; 171: pp. 35-40.
4. Leung J.W., Sickles E.A.: Developing asymmetry identified on mammography: correlation with imaging outcome and pathologic findings. AJR Am J Roentgenol 2007; 188: pp. 667-675.
5. Venkatesan A., Chu P., Kerlikowske K., et. al.: Positive predictive value of specific mammographic findings according to reader and patient variables. Radiology 2009; 250: pp. 648-657.
6. Bent C.K., Bassett L.W., D’Orsi C.J., et. al.: The positive predictive value of BI-RADS microcalcification descriptors and final assessment categories. AJR Am J Roentgenol 2010; 194: pp. 1378-1383.
7. Burnside E.S., Ochsner J.E., Fowler K.J., et. al.: Use of microcalcification descriptors in BI-RADS 4th edition to stratify risk of malignancy. Radiology 2007; 242: pp. 388-395.
8. Baker J.A., Kornguth P.J., Floyd C.E.: Breast Imaging Reporting and Data System standardized mammography lexicon: observer variability in lesion description. AJR Am J Roentgenol 1996; 166: pp. 773-778.
9. Berg W.A., Campassi C., Langenberg P., et. al.: Breast Imaging Reporting and Data System: inter- and intraobserver variability in feature analysis and final assessment. AJR Am J Roentgenol 2000; 174: pp. 1769-1777.
10. Gulsun M., Demirkazik F.B., Ariyurek M.: Evaluation of breast microcalcifications according to Breast Imaging Reporting and Data System criteria and Le Gal’s classification. Eur J Radiol 2003; 47: pp. 227-231.
11. Kerlikowske K., Grady D., Barclay J., et. al.: Variability and accuracy in mammographic interpretation using the American College of Radiology Breast Imaging Reporting and Data System. J Natl Cancer Inst 1998; 90: pp. 1801-1809.
12. Elmore J.G., Wells C.K., Lee C.H., et. al.: Variability in radiologists’ interpretations of mammograms. N Engl J Med 1994; 331: pp. 1493-1499.
13. Kopans D.B.: Accuracy of mammographic interpretation. N Engl J Med 1994; 331: pp. 1521-1522.
14. American College of Radiology : Breast Imaging Reporting and Data System (BI-RADS).2003.American College of RadiologyReston, VA
15. American College of Radiology : Breast Imaging Reporting and Data System (BI-RADS).2013.American College of RadiologyReston, VA
16. Elmore J.G., Jackson S.L., Abraham L., et. al.: Variability in interpretive performance at screening mammography and radiologists’ characteristics associated with accuracy. Radiology 2009; 253: pp. 641-651.
17. Esserman L., Cowley H., Eberle C., et. al.: Improving the accuracy of mammography: volume and outcome relationships. J Natl Cancer Inst 2002; 94: pp. 369-375.
18. Smith-Bindman R., Chu P., Miglioretti D.L., et. al.: Physician predictors of mammographic accuracy. J Natl Cancer Inst 2005; 97: pp. 358-367.
19. Berg W.A., D’Orsi C.J., Jackson V.P., et. al.: Does training in the Breast Imaging Reporting and Data System (BI-RADS) improve biopsy recommendations or feature analysis agreement with experienced breast imagers at mammography?. Radiology 2002; 224: pp. 871-880.
20. Onega T., Anderson M.L., Miglioretti D.L., et. al.: Establishing a gold standard for test sets: variation in interpretive agreement of expert mammographers. Acad Radiol 2013; 20: pp. 731-739.
21. Onega T., Smith M., Miglioretti D.L., et. al.: Radiologist agreement for mammographic recall by case difficulty and finding type. J Am Coll Radiol 2012; 9: pp. 788-794.
22. Lazarus E., Mainiero M.B., Schepps B., et. al.: BI-RADS lexicon for US and mammography: interobserver variability and positive predictive value. Radiology 2006; 239: pp. 385-391.
23. Elson S.L., Hiatt R.A., Anton-Culver H., et. al.: The Athena Breast Health Network: developing a rapid learning system in breast cancer prevention, screening, treatment, and care. Breast Cancer Res Treat 2013; 140: pp. 417-425.
24. Calas M.J., Almeida R.M., Gutfilen B., et. al.: Intraobserver interpretation of breast ultrasonography following the BI-RADS classification. Eur J Radiol 2010; 74: pp. 525-528.
25. Lee H.J., Kim E.K., Kim M.J., et. al.: Observer variability of Breast Imaging Reporting and Data System (BI-RADS) for breast ultrasound. Eur J Radiol 2008; 65: pp. 293-298.
26. Youk J.H., Jung I., Yoon J.H., et. al.: Comparison of inter-observer variability and diagnostic performance of the fifth edition of BI-RADS for breast ultrasound of static versus video images. Ultrasound Med Biol 2016; 42: pp. 2083-2088.
27. Park C.S., Kim S.H., Jung N.Y., et. al.: Interobserver variability of ultrasound elastography and the ultrasound BI-RADS lexicon of breast lesions. Breast Cancer 2015; 22: pp. 153-160.
28. Ekpo E.U., Ujong U.P., Mello-Thoms C., et. al.: Assessment of interradiologist agreement regarding mammographic breast density classification using the fifth edition of the BI-RADS Atlas. AJR Am J Roentgenol 2016; 206: pp. 1119-1123.
29. Gard C.C., Aiello Bowles E.J., Miglioretti D.L., et. al.: Misclassification of Breast Imaging Reporting and Data System (BI-RADS) mammographic density and implications for breast density reporting legislation. Breast J 2015; 21: pp. 481-489.
30. Spayne M.C., Gard C.C., Skelly J., et. al.: Reproducibility of BI-RADS breast density measures among community radiologists: a prospective cohort study. Breast J 2012; 18: pp. 326-333.
31. Sartor H., Lang K., Rosso A., et. al.: Measuring mammographic density: comparing a fully automated volumetric assessment versus European radiologists’ qualitative classification. Eur Radiol 2016;
32. Landis J.R., Koch G.G.: The measurement of observer agreement for categorical data. Biometrics 1977; 33: pp. 159-174.
33. Knutzen A.M., Gisvold J.J.: Likelihood of malignant disease for various categories of mammographically detected, nonpalpable breast lesions. Mayo Clin Proc 1993; 68: pp. 454-460.
34. Timmers J.M., van Doorne-Nagtegaal H.J., Verbeek A.L., et. al.: A dedicated BI-RADS training programme: effect on the inter-observer variation among screening radiologists. Eur J Radiol 2012; pp. 2184-2188.
35. National Cancer Policy Forum; Board on Health Care Services , Institute of Medicine , The National Academies of Sciences, Engineering, and Medicine : Assessing and Improving the Interpretation of Breast Images: Workshop Summary.2015.National Academies Press (US)Washington (DC) Available from: https://www.ncbi.nlm.nih.gov/books/NBK327232/
36. Beam C.A., Conant E.F., Sickles E.A.: Association of volume and volume-independent factors with accuracy in screening mammogram interpretation. J Natl Cancer Inst 2003; 95: pp. 282-290.