This editorial refers to the article “Potential effect of different radiologist reporting methods on studies showing benefit of CAD” by Horsch et al in the present issue of Academic Radiology ( ).
There are several communities or “cultures” that will find common ground in this article. One community includes the public and private funding sponsors and developers of new technologies for medical imaging—in particular, the technology of computer algorithms that assist radiologists in their decision making. Another community therefore is that of the radiologists themselves who will use these technologies. Then there is the community developing study designs and statistical assessment methodologies for evaluating the new technologies. Downstream we encounter the community of public policy makers who approve and clear these technologies for entrance to the marketplace for their specified task and the third-party payers who determine when a combination of a technology and a diagnostic task meet their criteria for reimbursement.
The pace of advance of the technology community is fairly obvious, particularly at the many professional society meetings throughout the year. A prime example of an advance in the radiology profession—especially in the particular setting of the present article—has been the evolution of consensus development by the Breast Imaging Reporting and Data Systems (BIRADS) committee of the American College of Radiology ( ). The community of experts devoted to study design and statistical assessment methods in the more general field of diagnostic medicine has moved forward into its fifth decade of maturity, with increasingly challenging tasks. And public policy makers continue to depend critically on data and wisdom from all of these communities to fulfill their ongoing missions.
In the last decade, at least two additional communities have arisen that have a vital interest in the present work. One is the young but rapidly growing community of developers of standardized databases of images and associated annotations for use by developers and users of imaging modalities ( ). And now we are witnessing in real time the emergence of technologies developed to expand the number of diagnostic biomarkers by several orders of magnitude using microarrays (“DNA chips”) that measure, for example, the level of expression of a large number of genes in blood or other tissue samples ( ). These technologies will drive all diagnostic modalities to attempt to become more refined as quantitative sciences so that their results may be optimally merged to yield a more informative diagnosis.
Now comes this seemingly unprepossessing article ( ) that focuses our attention on an essential ingredient needed by all of these cultures—namely, how the user of the image reports his or her final impression of the diagnostic information in the image to the medical informatics system for patient management as well as to the methodologist scorekeepers for assessment of the technology. The authors address not the question of whether computer-aided diagnosis (CAD) is effective in improving the diagnostic accuracy of mammographers, but rather whether the quantitative method used by mammographers for providing their final report affects the quantitative analysis and conclusion regarding the effectiveness of CAD.
A central conclusion of the authors is that, in this study, a global measure of the improvement in diagnostic information (without versus with CAD) yielded by receiver operating characteristic (ROC) analysis was self-consistent whether the readers of the images used a quasicontinuous rating of their subjective probability of malignancy (PM o ) or, alternatively, the effectively many fewer categories of the BIRADS reporting scale. However, if a single-point (sensitivity, specificity) measure of diagnostic accuracy is used based on the binary decision of whether to recommend biopsy or not (“biopsy decision” or BD), inferences regarding a possible improvement may not be consistent with the ROC-based assessment. This can be a source of bias or additional noise in the measurement.
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
References
1. Horsch K., Giger M.L., Metz C.E.: Potential effect of different radiologist reporting methods on studies showing benefit of CAD. Acad Radiol 2008; pp. 15.
2. American College of Radiology: Breast Imaging Reporting and Data System (BIRADS®).Fourth Edition1993.American College of RadiologyReston, VA
3. Armato S.G., McLennan G., NcNitt-Gray M.F., et. al.: Lung Image Database Consortium: developing a resource for the medical imaging research community. Radiology 2004; 232: pp. 739-748.
4. Wagner R.F.: From medical imaging to multiple-biomarker microarrays. Med Phys Vision 2020 series 2007; 34: pp. 4944-4951.
5. Wagner R.F., Beam C.A., Beiden S.V.: Reader variability in mammography and its implications for expected utility over the population of readers and cases. Med Decis Making 2004; 24: pp. 561-572.