Computer Disease Simulation Models

Computer disease simulation models are increasingly being used to evaluate and inform health care decisions across medical disciplines. The aim of researchers who develop these models is to integrate and synthesize short-term outcomes and results from multiple sources to predict the long-term clinical outcomes and costs of different health care strategies. Policy makers, in turn, can use the predictions generated by disease models together with other evidence to make decisions related to health care practices and resource utilization. Models are particularly useful when the existing evidence does not yield obvious answers or does not provide answers to the questions of greatest interest, such as questions about the relative cost-effectiveness of different practices. This review focuses on models used to inform decisions about imaging technology, discussing the role of disease models for health policy development and providing a foundation for understanding the basic principles of disease modeling. This manuscript draws from the collective computed tomographic colonography modeling experience, reviewing 10 published investigations of the clinical effectiveness and cost-effectiveness of computed tomographic colonography relative to colonoscopy. The discussion focuses on implications of different modeling assumptions and difficulties that may be encountered when evaluating the quality of models. This underscores the importance of forging stronger collaborations between researchers who develop disease models and radiologists, to ensure that policy-level models accurately represent the experience of everyday clinical practices.

Computer disease simulation models are increasingly being used to integrate and synthesize short-term evidence about diagnostic tests and procedures to project the long-term clinical and cost outcomes that are typically of interest to policy makers. A recent example of the use of modeling to inform policy decisions is the high-profile evaluation of mammographic screening intervals for the 2009 US Preventive Services Task Force breast cancer screening recommendations . Models incorporate evidence on a variety of elements that affect the effectiveness of a particular technology or procedure, including disease incidence and mortality, operating characteristics of screening and diagnostic tests, patient acceptance of the tests, and treatment effectiveness.

For example, the Centers for Medicare and Medicaid Services recently considered predictions from colorectal cancer models together with an independent review of the evidence on colorectal cancer screening that was commissioned by the Agency for Healthcare Research and Quality when deciding if computed tomographic (CT) colonography (CTC) should be a reimbursable method of colorectal cancer screening among Medicare enrollees. The modeling study, conducted by an international team of public health researchers, cancer experts, and biostatisticians, used three independently developed models to predict health outcomes and lifetime cancer-related costs associated with a program of CTC screening compared to colonoscopy screening, sigmoidoscopy screening, fecal occult blood testing, and no colorectal cancer screening . Each of the three models demonstrated that the predicted life-years gained (vs no colorectal cancer screening) from CTC screening among Medicare enrollees were only slightly less than the life-years gained from colonoscopy screening. They found that CTC screening could be a cost-effective option for colorectal cancer screening of the Medicare population if the cost of a CTC was substantially lower than a colonoscopy or if the availability of CTC screening would attract people who would otherwise not undergo colorectal cancer screening. This analysis of CTC motivates our discussion about the uses of disease models, the types of disease models, and how they are developed and applied in comparative effectiveness and cost-effectiveness research.

Role of disease simulation models in health policy for radiologic imaging

Researchers seeking information about the impact of imaging technologies on clinical outcomes encounter a variety of types of clinical evidence. However, each has its limitations. These limitations can often be overcome by integrating data with a computer disease simulation model. These models are particularly useful when some short-term outcomes are available about a technology but evidence of the impact on long-term health outcomes is lacking.

Most studies of imaging tests focus on test accuracy, that is, how well a test can identify disease. Although accuracy studies are key to understanding one-time test performance, they provide no information about clinical outcomes . Randomized trials are the gold-standard study design for assessing outcomes. However, they are usually not a feasible method for addressing health policy questions because of their costs and long time horizon for completion and because of the rapidly evolving nature of health care technologies. This is especially true for imaging technologies. In addition, randomized trials may be ethically difficult, or even impossible, to carry out once an imaging test is widely accepted as the standard of care, as is the case with mammography, or when the benefits are widely perceive to outweigh the risks, as in the case of coronary CT angiography . Finally, long-term randomized trials are often not possible because of funding limitations. As a result, randomized trials generally provide information about shorter term outcomes such as the severity of disease detected (eg, early or late stage cancer) or 1-year to 5-year survival outcomes. Trials that focus on long-term disease outcomes following imaging are rare. One example is the Prostate, Lung, Colorectal, and Ovarian Cancer Screening Trial, which randomized 38,349 men and 39,115 women to receive either routine medical care or active screening. In addition to colorectal and prostate cancer screening, the active screening arm included lung cancer screening with a baseline chest x-ray followed by annual chest x-rays for 3 years and, among women, ovarian cancer screening with transvaginal ultrasound and cancer antigen 125 testing . Despite the high value of randomized trials such as the Prostate, Lung, Colorectal, and Ovarian Cancer Screening Trial, the results are not available for guiding policy now: the trial began in 1993 and complete mortality results are anticipated in 2015.

Observational studies are a common first approach for estimating the effect of imaging tests on disease outcomes because they use existing data and can be carried out at relatively low cost. Observational studies often include a broader patient population than randomized trials, providing important additional information, even when results from randomized trials are available . Observational studies can also provide information about longer term outcomes following imaging and information about practices that are “off protocol.” However, observational studies are prone to multiple biases . Although analyses can adjust for these biases, they can account only for known potential biases that were measured in the study. Accordingly, the results from observational studies alone may not provide sufficient evidence to inform health policy decision makers of the effectiveness of a technology.

Get Radiology Tree app to read full this article<

Basics of disease models

Get Radiology Tree app to read full this article<

Example: models used to evaluate computed tomographic colonography versus colonoscopy

Get Radiology Tree app to read full this article<

Table 1

Ten Studies That Used Computer Simulation Models to Compare CTC and Colonoscopy

Study Model Type Study Population and Time Horizon De Novo Cancers Location Multiple Adenomas Malignant Potential Heitman et al Decision tree Canadian 50-year-old men to age 53 years No No No Adenoma size: 6–9 vs ≥10 mm Sonnenberg et al Markov cohort US 50-year-olds to death No No No Not modeled Ladabaum et al || Markov cohort US 50-year-olds to death Yes No No Adenoma size: <10 vs ≥10 mm Vijan et al Markov cohort US 50-year-olds to death No No No Low risk: 1–5 or 6–9 mm without high-risk histology § High risk: ≥10 or <10 mm (1–5 or 6–9 mm) with high-risk histology § Hassan et al ¶ Markov cohort Italian 50-year-olds to age 80 years Yes No No Adenoma size: 1–5, 6–9, ≥10 mm Lee et al Markov cohort British 50-year-olds to death No Yes ∗ Yes ‡ Adenoma size: 6–9, ≥10 mm Telford et al Markov cohort Canadian 50-year-olds to death No No No Low risk: <9 mm without high-risk histology High risk: ≥9 or <9 mm with high-risk histology § Knudsen et al MISCAN # Microsimulation US 65-year-olds to death No Yes † Yes Adenoma size: 1–5, 6–9, ≥10 mm CRC-SPIN Microsimulation US 65-year-olds to death No Yes † Yes Adenoma size: continuous size in millimeters SimCRC Microsimulation US 65-year-olds to death No Yes † Yes Adenoma size: 1–5, 6–9, ≥10 mm

CRC-SPIN, ColoRectal Cancer Simulated Population; CTC, computed tomographic colonography; MISCAN, Microsimulation Screening Analysis; SimCRC, Simulation Model of Colorectal Cancer.

Get Radiology Tree app to read full this article<

Predicted Effectiveness of CTC Relative to Colonoscopy

Get Radiology Tree app to read full this article<

Table 2

Conclusions About Relative Costs and of CTC and Colonoscopy and Assumptions About Test Sensitivity, Specificity, and Costs for 10 Disease Simulation Models

Study Sensitivity for Adenoma by Size/Risk Cost per Exam Least Costly Screening Program 1–5 mm/Low Risk 6–9 mm/Low Risk ≥10 mm/High Risk Cancer Specificity CTC COL CTC COL CTC COL CTC COL CTC COL COL COL with Polypectomy CTC Heitman et al ∗ ∗ 0.61 0.94 0.71 0.96 † † 0.84 1.0 $547 |||| $668 |||| $445 |||| COL Sonnenberg et al 0.80 0.95 0.80 0.95 0.80 0.95 0.80 0.95 0.94 1.0 $728 $1,139 $478 CTC § Ladabaum et al 0.70 0.85 0.70 0.85 0.75 0.90 0.95 0.95 0.85 1.0 $820 $1,200 $820 COL Vijan et al || 0.46 0.85 0.83 0.85 0.91 0.95 ‡ ‡ 0.91 1.0 $653 $831 $559 COL Hassan et al 0.48 0.80 0.70 0.85 0.85 0.90 0.95 0.95 0.86 0.90 €148.2 €228.6 €100.9 CTC Lee et al ∗ ∗ 0.653 0.924 ¶ ; 0.950 # 0.899 0.859 ¶ ; 0.883 # 0.899 0.859 ¶ ; 0.883 # 0.88 1.0 ₤488 ‡‡ ₤488 ‡‡ ₤128 CTC Telford et al 0.50 ∗∗ 0.92 ∗∗ 0.50 ∗∗ 0.92 ∗∗ 0.78 †† 0.97 †† 0.89 0.93 0.91 1.0 $424 |||| $609 |||| $590 |||| COL Knudsen et al ¶¶ 0 ## 0.75 0.84 0.85 0.92 0.95 0.92 0.95 0.80 0.90 $498 §§ $649 §§ $488 §§ COL

COL, colonoscopy; CTC, computed tomographic colonography.

Unless noted, sensitivity estimates are per person and costs are in US dollars.

Get Radiology Tree app to read full this article<

Predicted Costs and Cost-effectiveness of CTC Relative to Colonoscopy

Get Radiology Tree app to read full this article<

Model evaluation

Get Radiology Tree app to read full this article<

Discussion

Get Radiology Tree app to read full this article<

References

1. Berry D.A., Cronin K.A., Plevritis S.K., et. al.: Effect of screening and adjuvant therapy on mortality from breast cancer. N Engl J Med 2005; 353: pp. 1784-1792.
2. Stout N.K., Rosenberg M.A., Trentham-Dietz A., et. al.: Retrospective cost-effectiveness analysis of screening mammography. J Natl Cancer Inst 2006; 98: pp. 774-782.
3. Tosteson A.N., Stout N.K., Fryback D.G., et. al.: Cost-effectiveness of digital mammography breast cancer screening. Ann Intern Med 2008; 148: pp. 1-10.
4. Mandelblatt J.S., Cronin K.A., Bailey S., et. al.: Effects of mammography screening under different screening schedules: model estimates of potential benefits and harms. Ann Intern Med 2009; 151: pp. 738-747.
5. Lee J.M., McMahon P.M., Kong C.Y., et. al.: Cost-effectiveness of breast MR imaging and screen-film mammography for screening BRCA1 gene mutation carriers. Radiology 2010; 254: pp. 793-800.
6. Whitlock E.P., Lin J.S., Liles E., et. al.: Screening for colorectal cancer: a targeted, updated systematic review for the U.S. Preventive Services Task Force. Ann Intern Med 2008; 149: pp. 638-658.
7. Knudsen A.B., Lansdorp-Vogelaar I., Rutter C.M., et. al.: Cost-effectiveness of computed tomographic colonography screening for colorectal cancer in the Medicare population. J Natl Cancer Inst 2010; 102: pp. 1238-1252.
8. Gatsonis C.: The promise and realities of comparative effectiveness research. Stat Med 2010; 29: pp. 1977-1981.
9. Douglas P.S., Budoff M., Tunis S., et. al.: A new era for cardiovascular imaging? Implications of the revoked national coverage decision for CT angiography on future imaging reimbursement. JACC Cardiovasc Imaging 2008; 1: pp. 398-403.
10. Partridge E., Kreimer A.R., Greenlee R.T., et. al.: Results from four rounds of ovarian cancer screening in a randomized trial. Obstet Gynecol 2009; 113: pp. 775-782.
11. Hocking W.G., Hu P., Oken M.M., et. al.: Lung cancer screening in the randomized Prostate, Lung, Colorectal, and Ovarian (PLCO) Cancer Screening Trial. J Natl Cancer Inst 2010; 102: pp. 722-731.
12. Fleurence R.L., Naci H., Jansen J.P.: The critical role of observational evidence in comparative effectiveness research. Health Aff (Millwood) 2010; 29: pp. 1826-1833.
13. Berger M.L., Mamdani M., Atkins D., et. al.: Good research practices for comparative effectiveness research: defining, reporting and interpreting nonrandomized studies of treatment effects using secondary data sources: the ISPOR Good Research Practices for Retrospective Database Analysis Task Force report—part I. Value Health 2009; 12: pp. 1044-1052.
14. Cox E., Martin B.C., Van Staa T., et. al.: Good research practices for comparative effectiveness research: approaches to mitigate bias and confounding in the design of nonrandomized studies of treatment effects using secondary data sources: the International Society for Pharmacoeconomics and Outcomes Research Good Research Practices for Retrospective Database Analysis Task Force report—part II. Value Health 2009; 12: pp. 1053-1061.
15. Johnson M.L., Crown W., Martin B.C., et. al.: Good research practices for comparative effectiveness research: analytic methods to improve causal inference from nonrandomized studies of treatment effects using secondary data sources: the ISPOR Good Research Practices for Retrospective Database Analysis Task Force report—part III. Value Health 2009; 12: pp. 1062-1073.
16. Pisano E.D., Gatsonis C., Hendrick E., et. al.: Diagnostic performance of digital versus film mammography for breast-cancer screening. N Engl J Med 2005; 353: pp. 1773-1783.
17. McMahon P.M., Kong C.Y., Johnson B.E., et. al.: Estimating long-term effectiveness of lung cancer screening in the Mayo CT Screening Study. Radiology 2008; 248: pp. 278-287.
18. Claxton K., Sculpher M.: Using value of information analysis to prioritise health research: some lessons from recent UK experience. Pharmacoeconomics 2006; 24: pp. 1055-1068.
19. Hassan C., Di Giulio E., Hunink M.G.M., et. al.: Value-of-information analysis to guide future research in colorectal cancer screening. Radiology 2009; 253: pp. 745-752.
20. Gold M.R.: Cost-effectiveness in Health and Medicine.1996.Oxford University PressNew York
21. Weinstein M.C., Stason W.B.: Foundations of cost-effectiveness analysis for health and medical practices. N Engl J Med 1977; 296: pp. 716-721.
22. Elmore J.G., Jackson S.L., Miglioretti D.L., et. al.: Variability in interpretive performance at screening mammography and radiologists’ characteristics associated with accuracy. Radiology 2009; 253: pp. 641-651.
23. Johnson C.D., Chen M.H., Toledano A.Y., et. al.: Accuracy of CT colonography for detection of large adenomas and cancers. N Engl J Med 2008; 359: pp. 1207-1217.
24. Drummond M.F.: Methods for the Economic Evaluation of Health Care Programmes.3rd ed.2005.Oxford University PressOxford, UK
25. Detsky A.S., Naglie G., Krahn M.D., et. al.: Primer on medical decision analysis: part 1—Getting started. Med Decis Making 1997; 17: pp. 123-125.
26. Beck J., Pauker S.: The Markov process in medical prognosis. Med Decis Making 1983; 3: pp. 419-458.
27. Alagoz O., Hsu H., Schaefer A., et. al.: Markov decision processes: a tool for sequential decision making under uncertainty. Med Decis Making 2010; 30: pp. 474-483.
28. Rutter CM, Zaslavsky AM, Feuer EJ. Dynamic microsimulation models for health outcomes: a review. Med Decis Making. In press.
29. Ladabaum U., Song K.: Projected national impact of colorectal cancer screening on clinical and economic outcomes and health services demand. Gastroenterology 2005; 129: pp. 1151-1162.
30. Pickhardt P.J., Hassan C., Laghi A., et. al.: Cost-effectiveness of colorectal cancer screening with computed tomography colonography: the impact of not reporting diminutive lesions. Cancer 2007; 109: pp. 2213-2221.
31. Hassan C., Pickhardt P.J., Laghi A., et. al.: Computed tomographic colonography to screen for colorectal cancer, extracolonic cancer, and aortic aneurysm: model simulation with cost-effectiveness analysis. Arch Intern Med 2008; 168: pp. 696-705.
32. Pickhardt P.J., Hassan C., Laghi A., et. al.: CT colonography to screen for colorectal cancer and aortic aneurysm in the Medicare population: cost-effectiveness analysis. AJR Gastrointest Imaging 2009; 192: pp. 1-9.
33. Regge D., Hassan C., Pickhardt P.J., et. al.: Impact of computer-aided detection on the cost-effectiveness of CT colonography. Radiology 2009; 250: pp. 488-497.
34. Hassan C., Laghi A., Pickhardt P.J., et. al.: Projected impact of colorectal cancer screening with computerized tomographic colonography on current radiological capacity in Europe. Aliment Pharmacol Ther 2008; 27: pp. 366-374.
35. Lansdorp-Vogelaar I., Ballegooijen M.V., Zauber A.G., et. al.: At what costs will screening with CT colonography be competitive? A cost-effectiveness approach. Int J Cancer 2009; 124: pp. 1161-1168.
36. Heitman S.J., Manns B.J., Hilsden R.J., et. al.: Cost-effectiveness of computerized tomographic colonography versus colonoscopy for colorectal cancer screening. CMAJ 2005; 173: pp. 877-881.
37. Sonnenberg A., Delco F., Bauerfeind P.: Is virtual colonoscopy a cost-effective option to screen for colorectal cancer?. Am J Gastroenterol 1999; 94: pp. 2268-2274.
38. Ladabaum U., Song K., Fendrick A.M.: Colorectal neoplasia screening with virtual colonoscopy: when, at what cost, and with what national impact?. Clin Gastroenterol Hepatol 2004; 2: pp. 554-563.
39. Vijan S., Hwang I., Inadomi J., et. al.: The cost-effectiveness of CT colonography in screening for colorectal neoplasia. Am J Gastroenterol 2007; 102: pp. 380-390.
40. Hassan C., Zullo A., Laghi A., et. al.: Colon cancer prevention in Italy: cost-effectiveness analysis with CT colonography and endoscopy. Dig Liver Dis 2007; 39: pp. 242-250.
41. Lee D., Muston D., Sweet A., et. al.: Cost effectiveness of CT colonography for UK NHS colorectal cancer screening of asymptomatic adults aged 60-69 years. Appl Health Econ Health Pol 2010; 8: pp. 141-154.
42. Telford J.J., Levy A.R., Sambrook J.C., et. al.: The cost-effectiveness of screening for colorectal cancer. CMAJ 2010; 182: pp. 1307-1313.
43. Muto T., Bussey H.J., Morson B.C.: The evolution of cancer of the colon and rectum. Cancer 1975; 36: pp. 2251-2270.
44. Leslie A., Carey F.A., Pratt N.R., et. al.: The colorectal adenoma-carcinoma sequence. Br J Surg 2002; 89: pp. 845-860.
45. O’Brien M.J.: Hyperplastic and serrated polyps of the colorectum. Gastroenterol Clin N Am 2007; 36: pp. 947-968.
46. Sonnenberg A., Delco F., Inadomi J.M.: Cost-effectiveness of colonoscopy in screening for colorectal cancer. Ann Intern Med 2000; 133: pp. 573-584.
47. Rutter C.M., Yu O., Miglioretti D.L.: A hierarchical non-homogenous Poisson model for meta-analysis of adenoma counts. Stat Med 2007; 26: pp. 98-109.
48. Jemal A., Siegel R., Xu J., et. al.: Cancer statistics, 2010. CA Cancer J Clin 2010; 60: pp. 277-300.
49. Weinstein M.C., Siegel J.E., Gold M.R., et. al.: Recommendations of the Panel on Cost-effectiveness in Health and Medicine. JAMA 1996; 276: pp. 1253-1258.
50. Rutter C.M., Savarino J.E.: An evidence-based microsimulation model for colorectal cancer. Cancer Epidemiol Biomarkers Prev 2010; 19: pp. 1992-2002.
51. Lansdorp-Vogelaar I., Kuntz K.M., Knudsen A.B., et. al.: Stool DNA testing to screen for colorectal cancer in the Medicare population. A cost-effectiveness analysis. Ann Intern Med 2010; 153: pp. 368-377.
52. Zauber A.G., Lansdorp-Vogelaar I., Knudsen A.B., et. al.: Evaluating test strategies for colorectal cancer screening: a decision analysis for the U.S. Preventive Services Task Force. Ann Intern Med 2008; 149: pp. 659-669.
53. Draisma G., Etzioni R., Tsodikov A., et. al.: Lead time and overdiagnosis in prostate-specific antigen screening: importance of methods and context. J Natl Cancer Inst 2009; 101: pp. 374-383.
54. Etzioni R., Tsodikov A., Mariotto A., et. al.: Quantifying the role of PSA screening in the U.S. prostate cancer mortality decline. Cancer Causes Contr 2008; 19: pp. 175-181.
55. Normand S.L., McNeil B.J.: What is evidence?. Stat Med 2010; 29: pp. 1985-1988.
56. U.S. Department of Health and Human Services. MedCAC meeting, computed tomography colonography (CTC). Available at: http://www.cms.gov/mcd/viewmcac.asp?from2=viewmcac.asp&where=index&mid=45& . Accessed November 2, 2010.

Computer Disease Simulation Models

Role of disease simulation models in health policy for radiologic imaging

Basics of disease models

Example: models used to evaluate computed tomographic colonography versus colonoscopy

Predicted Effectiveness of CTC Relative to Colonoscopy

Predicted Costs and Cost-effectiveness of CTC Relative to Colonoscopy

Model evaluation

Discussion

References

Further Reading

235 Minus 55 Equals 180

A Pulmonary Nodule View System for the Lung Image Database Consortium (LIDC)

Accuracy, Reproducibility and Repeatability of Ultrasonography in the Assessment of Abdominal Adiposity