Purpose
Evaluate the reliability and validity of a standardized reporting system designed to improve communication between the clinician and radiologist regarding likelihood of hepatocellular carcinoma (HCC).
Materials and Methods
The system assigns liver lesions into 1 of 5 categories of estimated likelihood of HCC: 1, <5%; 2, 5%–20%; 3, 21%–70%; 4, 71%–95%; 5, >95%. Six American Board of Radiology–certified radiologists reviewed 100 abdominal MRI studies (performed between September 2009 and June 2010 for HCC surveillance) blinded to the official reports and clinical information. Each reader recorded the highest category (1–5) assigned to any lesion per study. Reliability between readers was calculated by the Shrout-Fliess random sets intraclass correlation (ICC). To examine validity, original pretransplant reports from January 2009 to December 2010 were compared to pathology reports on liver explants. Sensitivities, specificities, predictive values, and receiver operating characteristic (ROC) curves were then produced.
Results
The ICC for retrospective readings was 0.80, indicating very good reliability. Of 45 pathologically proven cases, 16 category 1 or 2 cases were all free of HCC (negative predictive value 100%). Five of nine category 3 cases contained HCC. Six of eight category 4 cases contained HCC (PPV 75%). All 12 category 5 cases contained HCC (positive predictive value 100%). The area underneath the ROC curve was 0.949. If categories 1 and 2 are considered negative and categories 3–5 considered positive, this achieves 100% sensitivity with 73% specificity.
Conclusion
This standardized system for reporting likelihood of HCC, which is a forerunner of the recently introduced Liver Imaging Reporting and Data System, produces strong reliability and validity, while aiming to improve the clarity of clinical magnetic resonance imaging reports.
Hepatocellular carcinoma (HCC) is the third leading cause of cancer mortality worldwide. The incidence of HCC in the United States has tripled from 1975 to 2005 . Certain high-risk populations for HCC typically undergo imaging surveillance at annual or biannual intervals, including patients with chronic viral hepatitis (hepatitis B and C), alcoholic hepatitis, autoimmune hepatitis, hemochromatosis, and nonalcoholic steatohepatitis. No universally accepted imaging surveillance algorithm exists, but some studies suggest that magnetic resonance imaging (MRI) is superior to both computed tomography (CT) and ultrasound in both sensitivity and specificity for the detection of HCC . The size and number of HCCs determines treatment decisions, including eligibility for liver transplant .
Imaging is recognized as a valid alternative to biopsy for diagnosing HCC . When imaging is considered diagnostic, biopsy is usually not performed because of reports of seeding the biopsy tract with tumor and other occasional complications . However, when imaging findings are inconclusive, biopsy or short-interval follow-up may be recommended. Therefore, it is important to accurately communicate the radiologist’s confidence in a diagnosis of possible or definite HCC. Ambiguous terms in clinical MRI reports such as “suspicious of,” “concerning for,” and “possibility of” convey no consistent information to physicians interpreting the report, at best leading to requests for further clarification, which in turn slow down workflow and productivity. At worst, this misinterpretation can lead to mismanagement of a potentially treatable and/or curable disease. In fact, physician-to-physician communication errors are frequently cited as one of the most common causes of medical errors and it is one of the top five indications for medical malpractice in radiology .
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Open full size image
Figure 1
Our institution’s HCC categorization system. The findings in an abdominal MRI report are assigned a category based on the features of the highest category lesion that is present. HCC, hepatocellular carcinoma; MRI, magnetic resonance imaging; US, ultrasound.
Get Radiology Tree app to read full this article<
Materials and methods
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Reliability Study
Get Radiology Tree app to read full this article<
Validity Study
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Results
Reliability Study
Get Radiology Tree app to read full this article<
Table 1
Weighted Kappa Statistics between Reader Pairs
Reader 1 Reader 2 Reader 3 Reader 4 Reader 5 Reader 6 Reader 1 — 0.76 (0.69–0.84) 0.73 (0.66–0.80) 0.67 (0.59–0.75) 0.58 (0.48–0.69) 0.73 (0.65–0.81) Reader 2 — — 0.74 (0.67–0.81) 0.68 (0.60–0.77) 0.58 (0.47–0.68) 0.80 (0.73–0.87) Reader 3 — — — 0.71 (0.61–0.80) 0.58 (0.47–0.68) 0.70 (0.61–0.79) Reader 4 — — — — 0.55 (0.44–0.67) 0.72 (0.64–0.80) Reader 5 — — — — — 0.52 (0.40–0.63)
95% confidence intervals are noted in parentheses.
Get Radiology Tree app to read full this article<
Validity Study
Get Radiology Tree app to read full this article<
Table 2
Comparison of MRI Liver Lesion Category Versus the Presence of HCC on Pathologic Examination after Liver Transplant
+ HCC − HCC + MRI (category 3–5) 23 6 − MRI (category 1–2) 0 16 Total = 45 Sensitivity = 100% (85.2%–100%) Specificity = 73% (50.0%–89.5%) PPV = 79% NPV = 100%
+ HCC, hepatocellular carcinoma present on pathology in similar location; − HCC, hepatocellular carcinoma absent on pathology in similar location; MRI, magnetic resonance imaging; NPV, negative predictive value; PPV, positive predictive value.
95% confidence intervals are indicated in parentheses.
Table 3
Comparison of MRI Liver Lesion Category (Excluding Category 3) Versus the Presence of HCC on Pathologic Examination after Liver Transplant
+ HCC − HCC + MRI (category 4–5) 18 2 − MRI (category 1–2) 0 16 Total = 36 Sensitivity = 100% (81.5%–100%) Specificity = 88.9% (68.4%–98.2%) PPV = 90% NPV = 100%
+ HCC, hepatocellular carcinoma present on pathology in similar location; − HCC, hepatocellular carcinoma absent on pathology in similar location; MRI, magnetic resonance imaging; NPV, negative predictive value; PPV, positive predictive value.
95% confidence intervals are indicated in parentheses.
Get Radiology Tree app to read full this article<
Discussion
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
References
1. Altekruse S., McGlynn K., Reichman M.: Hepatocellular carcinoma incidence, mortality, and survival trends in the United States from 1975 to 2005. J Clin Oncol 2009; 27: pp. 1485-1491.
2. Choi S., Lee J., Yu N., et. al.: Hepatocellular carcinoma in liver transplantation candidates: detection with gadobenate dimeglumine-enhanced MRI. AJR Am J Roentgenol 2008; 191: pp. 529-536.
3. Yu N., Chaudhari V., Raman S., et. al.: CT and MRI improve detection of hepatocellular carcinoma, compared with ultrasound alone, in patients with cirrhosis. Clin Gastroenterol Hepatol 2011; 9: pp. 161-167.
4. Livraghi T., Solbiati L., Meloni M., et. al.: Treatment of focal liver tumors with percutaneous radio-frequency ablation: complications encountered in a multicenter study. Radiology 2003; 226: pp. 441-451.
5. Mazzaferro V., Regalia E., Doci R., et. al.: Liver transplantation for the treatment of small hepatocellular carcinomas in patients with cirrhosis. N Engl J Med 1996; 334: pp. 693-699.
6. Choi D., Kim S.H., Lim J.H., et. al.: Detection of hepatocellular carcinoma: combined T2-weighted and dynamic gadolinium-enhanced MRI versus combined CT during arterial portography and CT hepatic arteriography. J Comput Assist Tomogr 2001; 25: pp. 777-785.
7. Llovet J., Burroughs A., Bruix J.: Hepatocellular carcinoma. Lancet 2008; 362: pp. 1907-1917.
8. Stigliano R., Marelli L., Yu D., et. al.: Seeding following percutaneous diagnostic and therapeutic approaches for hepatocellular carcinoma. What is the risk and the outcome? Seeding risk for percutaneous approach of HCC. Cancer Treat Rev 2007; 33: pp. 437-447.
9. Kahn C.E., Langlotz C.P., Burnside E.S., et. al.: Toward best practices in radiology reporting. Radiology 2009; 252: pp. 852-856.
10. Siegel E.L., Goodbye Mr: Cox: time for automated closure of the radiology communication loop. Appl Radiol 2005; 34: pp. 312.
11. Physician Insurers Association of America and American College of Radiology: Practice standards claims survey.1997.Physician Insurers Association of AmericaRockville (MD)
12. Kushner D.C., Lucey L.L., American College of Radiology: Diagnostic radiology reporting and communication: the ACR guideline. J Am Coll Radiol 2005; 2: pp. 15-21.
13. Flanders A.E., Lakhani P.: Radiology reporting and communications: a look forward. Neuroimaging Clin N Am 2012; 22: pp. 477-496.
14. Schwartz L., Panicek D., Berk A., et. al.: Improving communication of diagnostic radiology findings through structured reporting. Radiology 2011; 260: pp. 174-181.
15. D’Orsi C., Kopans D.: Mammography interpretation: the BI-RADS method. Am Fam Physician 1997; 55: pp. 1548-1552.
16. American College of Radiology: Breast imaging reporting and data system atlas.4th ed.2003.American College of RadiologyReston, VA
17. Langlotz C., Meininge L.: Enhancing the expressiveness and usability of structured image reporting systems. Proc AMIA Symp 2000; pp. 467-471.
18. Khorasani R., Bates D.W., Teeger S., et. al.: Is terminology used effectively to convey diagnostic certainty in radiology reports?. Acad Radiol 2003; 10: pp. 685-688.
19. American College of Radiology. LI-RADS (Liver Imaging Reporting and Data System). http://www.acr.org/SecondaryMainMenuCategories/quality_safety/LI-RADS.aspx . 2011.
20. Santillan C, Cruite I, Yeh B, et al. Liver Imaging Reporting and Data System (LI-RADS): an ACR-supported System for Classification of Hepatic Lesions on CT and MR Imaging in Patients with Cirrhosis. Abstract Presentation RSNA Annual Meeting, November 2011.
21. Shrout P.E., Fleiss J.L.: Intraclass correlations: uses in assessing rater reliability. Psychol Bull 1979; 86: pp. 420-428.
22. Fliess J.L., Cohen J.: The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educ Psychol Measure 1973; 33: pp. 613-619.
23. Matsui O.: Detection and characterization of hepatocellular carcinoma by imaging. Clin Gastroenterol Hepatol 2005; 3: pp. S136-S140.