Dear editor:
The article by Gur et al presents interesting data for those who perform reader studies of radiological devices. The article reports differences between two methods of estimating a change in the probability of correct discrimination, or area under the receiver operating characteristic (ROC) curve (AUC). It uses data from one particular study in which the breast cancer detection of full-field digital mammography (FFDM) was compared to that of FFDM plus the new investigational use technology of digital breast tomosynthesis (DBT). Both estimates use a nonparametric empirical method, but one estimate uses multicategory or semicontinuous rating data, whereas the other uses two-category or binary data.
We want to highlight three points relevant to this study and other controlled studies undertaken before a technology is in wide use in the clinical setting.
1. Binary decision data should be a subset of the rating or ranking data collected in a typical ROC experiment.
2. The binary decision thresholds of physicians in such studies may differ from their thresholds in future clinical practice for many reasons.
3. A limited number or range of binary operating points, or sensitivity and specificity pairs, can lead to ambiguous ROC curves, and therefore ambiguous estimates of AUC.
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Point 1
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Point 2
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Point 3
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
References
1. Gur D., Bandos A.I., Rockette H.E., et. al.: Is an ROC-type response truly always better than a binary response in observer performance studies?. Acad Radiol 2010; 17: pp. 639-645.
2. Gur D., Abrams G.S., Chough D.M., et. al.: Digital breast tomosynthesis: observer performance study. AJR Am J Roentgenol 2009; 193: pp. 586-591.
3. Gur D., Rockette H.E., Bandos A.I.: “Binary” and “non-binary” detection tasks: are current performance measures optimal?. Acad Radiol 2007; 14: pp. 871-876.
4. Gur D., Bandos A.I., Rockette H.E.: Comparing areas under receiver operating characteristic curves: potential impact of the “last” experimentally measured operating point. Radiology 2008; 247: pp. 12-15.
5. Dorfman D.D., Berbaum K.S.: A contaminated binormal model for ROC data, Part II, A formal Model. Acad Radiol 2000; 7: pp. 427-437.