Background
Disease prevalence alters the number of true positives (TP), true negatives (TN), false negatives (FN), and false positives (FP), even if the sensitivity and specificity of a test stays the same.
Methods and Materials
We illustrate this using data for the detection of suspected acute pulmonary embolism (PE) from the Prospective Investigation of Pulmonary Embolism Diagnosis II (PIOPED II). We chose PE because of the clinical significance of the disease, the low prevalence of PE in the patient population being tested with CTPA with the widespread adoption of CTPA, and the serious clinical consequences of anticoagulation therapy in FP patients.
Results
Based on PIOPED II data (sensitivity 83%, specificity 96%), at a disease prevalence of approximately 5%, the number of FP patients is greater than the number of TP patients. Scaled to the US population, at a disease prevalence of 5%, there would be 139,800 FPs and 3,356,200 TNs. Assuming a mortality rate of 0.5% and a 3.0% rate of major bleeding secondary to anticoagulation therapy for well-controlled patients, if all FP patients received anticoagulation, there would be 699 deaths and 4194 major bleeding complications.
Conclusions
At a prevalence of approximately 5% for PE, the number of FPs approaches or is greater than the number of TPs for CTPA for the detection of suspected acute PE. Patients with FP results may receive unnecessary, potentially harmful treatment with anticoagulation therapy. Population prevalence of disease needs to be taken into account along with the diagnostic accuracy of a test, because this may significantly affect downstream patient outcomes.
The proportion of patients with the disease and the proportion of patients without the disease changes as population prevalence changes. This alters the number of true positives, true negatives, false negatives, and false positives, even if the sensitivity and specificity of a test stays the same. At a low prevalence, the number of false positives can approach or be greater than the number of true positives even for diagnostic tests of high sensitivity and specificity. At a prevalence of approximately 5% for pulmonary embolism (PE), the number of false positives approaches or is greater than the number of true positives for computed tomography pulmonary angiography (CTPA) or CTPA with CT venography for the detection of suspected acute PE. Patients with false-positive results may receive unnecessary, potentially harmful treatment with anticoagulation therapy and potential adverse events (bleeding). Population prevalence of disease needs to be taken into account along with the diagnostic accuracy of a test, because this may significantly affect downstream patient outcomes.
When considering diagnostic tests including imaging, it is important to assess the test’s diagnostic accuracy. However, diagnostic accuracy is only a single aspect of the overall test characteristic. Other impacts of test performance need to be considered as well as diagnostic accuracy, including diagnostic thinking accuracy, physician decision making, patient outcomes and cost-effectiveness. Even within diagnostic accuracy, which is often expressed as sensitivity and specificity, positive and negative predictive value and accuracy (as measured by the area of the receiver operating characteristic curve), the picture is incomplete. As well as sensitivity (true positive [TP] rate), and specificity (true negative [TN] rate), the false positive (FP) rate (1, specificity), and false negative (FN) rate (1-sensitivity) should also be considered. However, different prevalences of disease change the proportion of people with the disease (TP and FP) and also the proportion of those without the disease (TN and FP) even if the test’s sensitivity and specificity remain the same. The effect of different prevalences of disease is most evident in the changes of the positive and negative predictive value even if the test’s sensitivity and specificity remain the same.
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Contingency Table
Get Radiology Tree app to read full this article<
Table 1
2 × 2 Contingency Table with True-positive, True-negative, False-negative, and False-positive Results
Disease True False Test outcome Positive True positive
a False positive
b → Positive predictive value = TP/(TP+FP) = a/(a+b) Negative False negative
c True negative
d → Negative predictive value = TN/(TN + FN) = d/(c+d) ↓
Sensitivity = True-positive rate TP/(TP+FN) = a/(a+c)
↓
1-sensitivity = False-negative rate = FN/(TP+FN) = c/(a+c) ↓
Specificity = True-negative rate TN/(FP+TN) = d/(b+d)
↓
1-specificity = False-positive rate = FP/(FP+TN) = b/(b+) → Accuracy = (TP+TN)/(TP+FP+FN+TN) = (a+d)/(a+b+c+d)
→ Prevalence = (TP+FN)/(TP+FP+FN+TN) = (a+c)/(a+b+c+d)
Sensitivity (true-positive rate), specificity (true-negative rate), false-negative rate (1-sensitivity), false-positive rate (1-specificity), accuracy, and prevalence calculated.
FN, false negative = c; FP, false positive = b; TN, true negative = d; TP, true positive = a.
Get Radiology Tree app to read full this article<
Diagnostic Accuracy
Get Radiology Tree app to read full this article<
Sensitivity and Specificity
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
FN Rate/FP Rate
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Positive and Negative Predictive Values
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Prevalence
Get Radiology Tree app to read full this article<
Pulmonary embolism
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
PIOPED II
Get Radiology Tree app to read full this article<
Scenarios
Get Radiology Tree app to read full this article<
Table 2A
Numbers of True-positive, False-positive, False-negative, and True-negative, and Positive and Negative Predictive Values of a Cohort of 2500 Patients Undergoing CTPA for Suspected Acute PE Based on the PIOPED II Data (Sensitivity 83%, Specificity 96%) for Different Disease Prevalences
Scenario Prevalence True Positives False Positive False Negative True Negative Positive Predictive Value Negative Predictive Value 1. Prevalence of PE in the PIOPED I study 33% 685 68 140 1607 91% 92% 2. Prevalence of PE in the PIOPED II study 23.4% 485 78 100 1837 86% 94.8% 3. Prevalence of PE suspected acute PE 10% 207 90 43 2160 70% 98.1% 4. Prevalence of PE in a cohort of a study at our institution 5% 105 95 20 2280 52.5% 99.1% 5. Prevalence of PE in pregnant patients 3.7% 77 95 15 2313 45% 99.4%
Table 2B
Numbers of True-positive, False-positive, False-negative, and True-negative, and Positive and Negative Predictive Values of a Cohort of 2500 Patients Undergoing CTPA with CTV Based on the PIOPED II Data (Sensitivity 90%, Specificity 95%) for Different Disease Prevalences
Scenario Prevalence True Positives False Positive False Negative True Negative Positive Predictive Value Negative Predictive Value 1. Prevalence of PE in the PIOPED I study 33% 742 83 83 1592 90% 95% 2. Prevalence of PE in the PIOPED II study 23.4% 527 95 58 1820 84.7% 96.9% 3. Prevalence of PE suspected acute PE 10% 225 113 25 2137 66.7% 98.8% 4. Prevalence of PE in a cohort of a study at our institution 5% 112 118 13 2257 48.9% 99.4% 5. Prevalence of PE in pregnant patients 3.7% 82 120 10 2288 40.7% 99.6%
CTPA, computed tomography pulmonary angiography; CTV, computed tomography venography; PE, pulmonary embolism; PIOPED, Prospective Investigation of Pulmonary Embolism Diagnosis.
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Scaled to the US population
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Table 3A
Numbers of True-positive, False-positive, False-negative, and True-negative, and Positive and Negative Predictive Values of a Cohort of Patients within the US Population Needing to Undergoing CTPA for Suspected Acute PE Based on the PIOPED II Data (Sensitivity 83%, Specificity 96%) for Different Disease Prevalences (23.4%, 10% and 5%) to Identify the Estimated 184,000 Cases Annually of Symptomatic PE in the United States
Scenario Prevalence True Positives False Positive False Negative True Negative Total Scanned Positive Predictive Value Negative Predictive Value 1. Prevalence of PE in the PIOPED II study 23.4% 152,700 24,100 31,300 578,200 786,300 84.7% 96.9% 2. Prevalence of PE suspected acute PE 10% 152,700 66,200 31,300 1,589,800 1,840,000 66.7% 98.8% 3. Prevalence of PE in a cohort of a study at our institution 5% 152,700 139,800 31,300 3,356,200 3,680,000 48.9% 99.4%
Table 3B
Numbers of True-positive, False-positive, False-negative, and True-negative, and Positive and Negative Predictive Values of a Cohort of Patients within the US Population Needing to Undergoing CTPA and CTV for Suspected Acute PE Based on the PIOPED II Data (Sensitivity 90% and Specificity 95%) for Different Disease Prevalences (23.4%, 10% and 5%) to Identify the Estimated 184,000 Cases Annually of Symptomatic PE in the United States
Scenario Prevalence True Positives False Positive False Negative True Negative Total Scanned Positive Predictive Value Negative Predictive Value 1. Prevalence of PE in the PIOPED II study 23.4% 165,600 30,100 18,400 572,200 786,300 84.7% 96.9% 2. Prevalence of PE suspected acute PE 10% 165,600 82,800 18,400 1,573,200 1,840,000 66.7% 98.8% 3. Prevalence of PE in a cohort of a study at our institution 5% 165,600 174,800 18,400 3,321,200 3,680,000 48.9% 99.4%
CTPA, computed tomography pulmonary angiography; CTV, computed tomography venography; PE, pulmonary embolism; PIOPED, Prospective Investigation of Pulmonary Embolism Diagnosis.
Get Radiology Tree app to read full this article<
Discussion
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Conclusion
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
References
1. Maceneaney P.M., Malone D.E.: The meaning of diagnostic test results: a spreadsheet for swift data analysis. Clin Radiol 2000; 55: pp. 227-235.
2. Brismar J., Jacobsson B.: Definition of terms used to judge the efficacy of diagnostic tests: a graphic approach. AJR Am J Roentgenol 1990; 155: pp. 621-623.
3. Rosenquist C.J.: Pitfalls in the use of diagnostic tests. Clin Radiol 1989; 40: pp. 448-450.
4. Black W.C.: How to evaluate the radiology literature. AJR Am J Roentgenol 1990; 154: pp. 17-22.
5. Phillips W.C., Scott J.A., Blasczcynski G.: Statistics for diagnostic procedures. I. How sensitive is “sensitivity”; how specific is “specificity”?. AJR Am J Roentgenol 1983; 140: pp. 1265-1270.
6. Montori V.M., Wyer P., Newman T.B., et. al.: Tips for learners of evidence-based medicine: 5. The effect of spectrum of disease on the performance of diagnostic tests. CMAJ 2005; 173: pp. 385-390.
7. Torbicki A., Perrier A., Konstantinides S., et. al.: Guidelines on the diagnosis and management of acute pulmonary embolism: the Task Force for the Diagnosis and Management of Acute Pulmonary Embolism of the European Society of Cardiology (ESC). Eur Heart J 2008; 29: pp. 2276-2315.
8. Nikolaou K., Thieme S., Sommer W., et. al.: Diagnosing pulmonary embolism: new computed tomography applications. J Thorac Imaging 2010; 25: pp. 151-160.
9. Kelly A.M., Patel S., Carlos R.C., et. al.: Multidetector row CT pulmonary angiography and indirect venography for the diagnosis of venous thromboembolic disease in intensive care unit patients. Acad Radiol 2006; 13: pp. 486-495.
10. British Thoracic Society guidelines for the management of suspected acute pulmonary embolism. Thorax 2003; 58: pp. 470-483.
11. Levine M.N., Raskob G., Beyth R.J., et. al.: Hemorrhagic complications of anticoagulant treatment: the Seventh ACCP Conference on Antithrombotic and Thrombolytic Therapy. Chest 2004; 126: pp. 287S-310S.
12. Stein P.D., Fowler S.E., Goodman L.R., et. al.: Multidetector computed tomography for acute pulmonary embolism. N Engl J Med 2006; 354: pp. 2317-2327.
13. Value of the ventilation/perfusion scan in acute pulmonary embolism. Results of the prospective investigation of pulmonary embolism diagnosis (PIOPED). The PIOPED Investigators. JAMA 1990; 263: 2753–2739
14. Stein P.D., Matta F.: Acute pulmonary embolism. Curr Probl Cardiol 2010; 35: pp. 314-376.
15. Mamlouk M.D., vanSonnenberg E., Gosalia R., et. al.: Pulmonary embolism at CT angiography: implications for appropriateness, cost, and radiation exposure in 2003 patients. Radiology 2010; 256: pp. 625-632.
16. The Cochrane Collaboration. http://www.cochrane.org . Accessed May 3, 2010.
17. Shahir K., Goodman L.R., Tali A., et. al.: Pulmonary embolism in pregnancy: CT pulmonary angiography versus perfusion scanning. AJR Am J Roentgenol 2010; 195: pp. W214-220.
18. Torbicki A., Perrier A., Konstantinides S., et. al.: Guidelines on the diagnosis and management of acute pulmonary embolism: the Task Force for the Diagnosis and Management of Acute Pulmonary Embolism of the European Society of Cardiology (ESC). Eur Heart J 2008; 29: pp. 2276-2315.
19. Ceriani E., Combescure C., Le Gal G., et. al.: Clinical prediction rules for pulmonary embolism: a systematic review and meta-analysis. J Thromb Haemost 2010; 8: pp. 957-970.
20. Stein P.D., Chenevert T.L., Fowler S.E., et. al.: Gadolinium-enhanced magnetic resonance angiography for pulmonary embolism: a multicenter prospective study (PIOPED III). Ann Intern Med 2010; 152: pp. 434-443.
21. Thompson I.M., Goodman P.J., Tangen C.M., et. al.: The influence of finasteride on the development of prostate cancer. N Engl J Med 2003; 349: pp. 215-224.
22. Swensen S.J., Jett J.R., Hartman T.E., et. al.: CT screening for lung cancer: five-year prospective experience. Radiology 2005; 235: pp. 259-265.
23. Heitman S.J., Ronksley P.E., Hilsden R.J., et. al.: Prevalence of adenomas and colorectal cancer in average risk individuals: a systematic review and meta-analysis. Clin Gastroenterol Hepatol 2009; 7: pp. 1272-1278.