Rationale and Objectives
The purpose of our study is to perform an internal validation of a new reference standard for vasospasm diagnosis in aneurysmal subarachnoid hemorrhage (A-SAH) patients.
Materials and Methods
A retrospective study was performed on A-SAH patients between January 2002 and May 2009. All patients were applied to this new reference standard using a multistage hierarchical approach incorporating clinical and imaging criteria. An internal validation method was performed in two phases to compare the new reference standard with digital subtraction angiography (DSA) and to assess accuracy. In Phase I, the diagnostic outcomes from DSA at the primary level were compared with the secondary/tertiary levels in the reference standard. In Phase II, the new reference standard was compared with chart diagnosis. Accuracy test characteristics, agreement rates, kappa values, and bias indices were calculated.
Results
In Phase I ( n = 85), there was 87% agreement rate, 0.674 kappa, and 0.12 bias index. However, there was 100% agreement in patients diagnosed with vasospasm by DSA. Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were 100%, 61%, 83%, and 100% respectively. In Phase II ( n = 137), there was 91% agreement rate, 0.824 kappa, and 0.04 bias index. Sensitivity, specificity, PPV, and NPV were 88%, 95%, 96%, and 87%, respectively.
Conclusion
Performing validation methods for a new reference standard is an evolving and ongoing process because limitations and bias in the reference standard are identified. Based on the results of this internal validation, a modification in the new reference standard is made at the primary level, resulting in improvement in its accuracy and classification of A-SAH patients.
Acute subarachnoid hemorrhage as a result of a ruptured aneurysm is a devastating condition, with an incidence of approximately 10 per 100,000 persons annually , associated with as great as 67% patient fatality . Delayed cerebral vasospasm is a serious complication of aneurysmal subarachnoid hemorrhage (A-SAH), typically developing 4–9 days after the hemorrhagic event. Symptomatic vasospasm has been reported in 22%–40% of patients and is a significant cause of morbidity and mortality in this patient population . Poor clinical outcome is associated with vasospasm leading to permanent neurologic deficits, stroke, and death.
Today, developing new technology is being studied to improve early and more accurate diagnosis of vasospasm. Identification of these patients with vasospasm is important for initiation of prompt treatment to prevent stroke and death. In addition, accurate classification of patients without vasospasm is also needed to limit the neurologic and systemic adverse effects associated with treatment of vasospasm. Serious complications can occur in patients incorrectly classified as either false positive or false negative in this patient population. Thereby, critical assessment of the classification scheme and reference standard for vasospasm in A-SAH patients has become a primary focus of our research.
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Materials and methods
Study Population
Get Radiology Tree app to read full this article<
Study Design
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Step 1
Get Radiology Tree app to read full this article<
Step 2
Get Radiology Tree app to read full this article<
Step 3
Get Radiology Tree app to read full this article<
Validation Process
Get Radiology Tree app to read full this article<
Statistical Analysis
Get Radiology Tree app to read full this article<
Results
Patients
Get Radiology Tree app to read full this article<
Table 1
Demographic Data
All ( n = 137) Vasospasm ( n = 70) No Vasospasm ( n = 67) Age (y) Median 51.5 49 52 Range 24–88 28–88 24–83 Gender Male 38 (28%) 21 (30%) 17 (25%) Female 99 (72%) 49 (70%) 50 (75%) Aneurysm location Anterior 94 (69%) 52 (74%) 42 (63%) Posterior 43 (31%) 18 (26%) 25 (37%) Treatment type Surgical clipping 75 (55%) 37 (53%) 37 (55%) Coil embolization 59 (43%) 30 (43%) 29 (43%) Untreated 3 (2%) 3 (4%) 1 (1%) Hunt Hess Grade Low grade (1–2) 69 (50%) 28 (40%) 42 (59%) High grade (3–5) 68 (50 %) 42 (60%) 25 (37%)
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Table 2
Comparison of Secondary/Tertiary Levels with Primary Level, Digital Subtraction Angiography as Reference Standard
Primary Level (Digital Subtraction Angiography) Secondary/Tertiary
Levels Vasospasm No Vasospasm Total Vasospasm 57 11 68 No vasospasm 0 17 17 Total 57 28 85
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Table 3
Comparison of Primary/Secondary/Tertiary Levels with Chart Review and Discharge Summary as Reference Standard
Primary Level (Digital Subtraction Angiography) Multistage Reference Standard Vasospasm No Vasospasm Total Vasospasm 67 3 70 No Vasospasm 9 58 67 Total 76 61 137
Get Radiology Tree app to read full this article<
Discussion
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
References
1. Ingal I.T.J., Whisnant J.P.: Epidemiology of subarachnoid hemorrhage.Yanagihara T.Pepegras D.C.Atkinson J.L.D.Subarachnoid hemorrhage: medical and surgical management.1998.Marcel DekkerNew York, NY:pp. 194-206.
2. Hop J.W., Rinkel G.J., Algra A., et. al.: Case fatality rates and functional outcome after subarachnoid hemorrhage: a systematic review. Stroke 1997; 28: pp. 660-664.
3. Suarez J., Qureshi A., Abutaher Y., et. al.: Symptomatic vasospasm diagnosis after subarachnoid hemorrhage: evaluation of transcranial Doppler ultrasound and cerebral angiography as related to compromised vascular distribution. Neurol Crit Care 2002; pp. 1348-1355.
4. Janardhan V., Biondi A., Riina H., et. al.: Vasospasm in aneurysmal subarachnoid hemorrhage: diagnosis, prevention, and management. Neuroimaging Clin N Am 2006; 16: pp. 483-496.
5. Macdonald R.: Management of cerebral edema. Neurosurg Rev 1998; 29: pp. 179-193.
6. Reichman M., Greenberg E., Gold R., et. al.: Developing patient-centered outcome measures for evaluating vasospasm in aneurysmal subarachnoid hemorrhage. Acad Radiol 2009; 16: pp. 541-545.
7. Zubkov A., Rabinstein A.: Medical management of cerebral vasospasm: present and future. Neurol Res 2009; 31: pp. 626-631.
8. Alonzo T., Pepe M.: Assessing the accuracy of a new diagnostic test when a gold standard does not exist. UW Biostatistics Working Paper Series 1998; pp. 3-32.
9. Byrt T., Bishop J., Carlin J.B.: Bias, prevalence and kappa. J Clin Epidemiol 1993; 46: pp. 423-429.
10. Landis R., Koch G.: The measurement of observer agreement for categorical data. Biometrics 1977; 33: pp. 159-174.
11. Pfeiffer R., Castle P.: With or without a gold standard. Epidemiology 2005; 16: pp. 595-597.
12. Altman D.G., Royston P.: What do we mean by validating a prognostic model?. Stat Med 2000; 19: pp. 453-473.
13. Rabinstein A., Friedman J., Weigand S., et. al.: Predictors of cerebral infarction in aneurysmal SAH. Stroke 2004; 35: pp. 1862-1866.