Rationale and Objectives
Chest radiographic findings are important for diagnosis and management of tuberculosis. The reliability of these findings is therefore of interest. We sought to describe interobserver reliability of chest radiographic findings in pulmonary tuberculosis, and to understand how the reliability of these findings might affect the utility of radiographic findings in predicting tuberculosis relapse.
Materials and Methods
Three blinded readers independently reviewed chest radiographs from a randomly selected group of 10% of HIV-seronegative subjects participating in a tuberculosis treatment trial. The three readers then arrived at a fourth, consensus radiographic interpretation.
Results
A total of 241 films obtained from 99 patients were reviewed. Agreement among the independent readers was very good for the findings of bilateral disease (κ = 0.71–0.86 among readers) and cavitation (κ = 0.66–0.73). The original interpretation was reasonably sensitive and specific (compared to the consensus interpretation) for bilateral disease, but the sensitivity for cavity decreased from 81% for the 2-month film to 47% at end of treatment ( P = 0.013). Substituting the consensus interpretation for the original interpretation increased the odds ratio for the association between cavitation on early chest radiograph and subsequent tuberculosis relapse from 4.97 to 8.97.
Conclusion
Radiographic findings were reasonably reliable between independent reviewers and the original interpretations. The original investigators, who knew the patient’s clinical course, were less likely to identify cavitation on the end of treatment chest radiograph. Improving the reliability of these findings could improve the utility of chest radiographs for predicting tuberculosis relapse.
Chest radiographic findings have traditionally been important for diagnosis and management of tuberculosis (TB). The current American Thoracic Society/Centers for Disease Control and Prevention/Infectious Diseases Society of America guidelines for TB treatment recommend extending treatment from 6 to 9 months for patients with both a cavity on early (taken after 2 months or less of treatment) chest radiograph (CXR) and a positive sputum culture after 2 months of antituberculous therapy . This recommendation was based to a large extent on the findings of the Tuberculosis Trials Consortium Study 22. Study 22 was a multicenter, randomized, controlled trial that compared a standard twice-weekly TB treatment regimen during the continuation phase of therapy with a once-weekly regimen substituting a new drug, rifapentine, for the standard agent, rifampin. The final study cohort of 1004 subjects was restricted to HIV-seronegative individuals . Five characteristics were found to be independently associated with increased risk for disease relapse: positive sputum cultures for Mycobacterium tuberculosis after 2 months of antituberculous therapy; the presence of a cavity on a CXR taken during the first 2 months of treatment, being underweight, having bilateral pulmonary involvement, and being a non-Hispanic white person. The risk for TB relapse was 23% in the group with both culture-positive sputum after 2 months of therapy plus a cavity on an early CXR, and less than 2% in the group with neither a cavity on chest film nor positive sputum cultures after 2 months of antituberculous therapy .
There are several reasons why it is important to better understand the reliability of CXR interpretations. First, there are now clinical ramifications for recognizing or missing the presence of a cavity on an early (taken either at initiation of treatment or at 2 months after initiation of treatment) CXR. Second, in response to the findings of Study 22, clinical trials examining new therapeutic options for treatment of TB are basing enrollment strategies and analyses on subjects’ radiographic pattern. Unfortunately, several studies have described only moderate agreement among independent observers in evaluating radiographs for manifestations of TB . Furthermore, agreement may vary depending on the observer’s specialty and level of experience . When the individual interpreting the chest radiograph is familiar with the patient’s clinical condition, the interpretation of the chest radiograph may also be biased by the clinical data. Bias and interreader variability may both affect the utility of radiographic data in predicting TB outcomes. We therefore conducted a substudy to validate the radiographic findings of Study 22 and to assess how interreader reliability and bias might have affected the association between radiographic findings and TB relapse.
Materials and methods
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Statistical Methods
Get Radiology Tree app to read full this article<
OR(sensitivity,specificity)=Relapse/(1−Relapse)No relapse/(1−No relapse), O
R
(
s
e
n
s
i
t
i
v
i
t
y
,
s
p
e
c
i
f
i
c
i
t
y
)
=
Relapse
/
(
1
−
Relapse
)
No relapse
/
(
1
−
No relapse
)
,
where
Relapse=TrueRelapse×sensitivity+(1−TrueRelapse)×(1−specificity) Relapse
=
Relapse
True
×
s
e
n
s
i
t
i
v
i
t
y
+
(
1
−
Relapse
True
)
×
(
1
−
s
p
e
c
i
f
i
c
i
t
y
)
and
No relapse=TrueNo relapse×sensitivity+(1−TrueNo relapse)×(1−specificity) No relapse
=
No relapse
True
×
s
e
n
s
i
t
i
v
i
t
y
+
(
1
−
No relapse
True
)
×
(
1
−
s
p
e
c
i
f
i
c
i
t
y
)
Relapse Relapse represents the proportion of subjects with a cavity, as determined by an observer with sensitivity/specificity as noted, who relapse after TB treatment. TrueRelapse Relapse
True represents the proportion of subjects who “truly” have a cavity, as determined by a theoretical perfect observer, and relapse after TB treatment. No relapse No relapse represents the proportion of subjects with a cavity, as determined by an observer with sensitivity/specificity as noted, who do not relapse after TB treatment. TrueNo relapse No relapse
True represents the proportion of subjects who “truly” have a cavity, as determined by a theoretical perfect observer, and do not relapse after TB treatment. Sensitivity was defined as the proportion of radiographs for which a given observer detected a cavity when a cavity was “truly” present. Specificity was defined as the proportion of radiographs for which a given observer did not detect a cavity when a cavity was not “truly” present.
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Results
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Table 1
Agreement among Three Independent, Blinded Readers with Regard to Chest Radiograph Findings in a Subset of Participants in Tuberculosis Trials Consortium Study 22 (99 Subjects, 241 Radiographs)
% Agreement (kappa value) Prevalence of finding on consensus interpretation ( n = 241) Finding Reader1-Reader2 Reader1-Reader3 Reader2-Reader3 Normal film 93 (0.35) 95 (0.45) 95 (0.45) 94% normal Bilateral 85 (0.71) 93 (0.86) 87 (0.74) 56% bilateral Cavitary 84 (0.66) 85 (0.67) 88 (0.73) 34% cavitary Extension of old lesion(s) 85 (0.70) 87 (0.75) 85 (0.70) 46% with extension New lesion 82 (0.28) 90 (0.47) 87 (0.41) 12% with new lesions
Readers were blinded to all clinical data (including relapse/failure status) as well as to the chronologic order of the radiographs.
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Table 2
Sensitivity and Specificity of Blinded Readers for the Finding of Cavity on Chest Radiographs Obtained after 2 Months of Tuberculosis Treatment, as Estimated by Comparison to the Consensus Interpretation and as Estimated by Latent Class Modeling
Sensitivity for cavity (95% confidence interval) Specificity for cavity (95% confidence interval) Reader Consensus Latent class Consensus Latent class Reader 1 0.85 (0.66–0.96) 0.88 (0.66–0.96) 0.90 (0.79–0.97) 0.92 (0.79–0.97) Reader 2 0.89 (0.71–0.98) 0.92 (0.69–0.98) 0.90 (0.79–0.97) 0.92 (0.79–0.97) Reader 3 0.96 (0.81–1.0) 0.92 (0.67–0.99) 0.98 (0.90–1.0) 0.96 (0.82–0.99)
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Table 3
Sensitivity and Specificity of the Original Site Interpretations of Study 22 Chest Radiographs as Compared to the Consensus Interpretation of Three Blinded Readers
Initial radiograph 2-month radiograph End of treatment radiograph Finding Sensitivity
(95% CI) Specificity
(95% CI) Sensitivity
(95% CI) Specificity
(95% CI) Sensitivity
(95% CI) Specificity
(95% CI) Abnormal 1.00 (0.95–1.0) 0.50 (0.01–0.99) 1.00 (0.96–1.0) 0.75 (0.19–0.99) 0.94 (0.88–0.98) 0.75 (0.35–0.97) Bilateral 0.81 (0.64–0.92) 0.93 (0.76–0.99) 0.89 (0.76–0.96) 0.91 (0.76–0.98) 0.74 (0.60–0.85) 0.98 (0.88–1.0) Cavitary 0.70 (0.47–0.87) 0.86 (0.71–0.95) 0.81 (0.61–0.93) 0.82 (0.68–0.91) 0.47 ∗ (0.28–0.66) 0.93 (0.84–0.98)
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Discussion
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
References
1. Blumberg H.M., Burman W.J., Chaisson R.E., et. al.: American Thoracic Society/Centers for Disease Control and Prevention/Infectious Diseases Society of America: treatment of tuberculosis. Am J Respir Crit Care Med 2003; 167: pp. 603-662.
2. Benator D., Bhattacharya M., Bozeman L., et. al.: Rifapentine and isoniazid once a week versus rifampicin and isoniazid twice a week for treatment of drug-susceptible pulmonary tuberculosis in HIV-negative patients: a randomised clinical trial. Lancet 2002; 360: pp. 528-534.
3. Nyboe J.: Results of the international study on x-ray classification. Bull Int Union Tuberc 1968; 41: pp. 115-124.
4. Zellweger J.P., Heinzer R., Touray M., et. al.: Intra-observer and overall agreement in the radiological assessment of tuberculosis. Int J Tuberc Lung Dis 2006; 10: pp. 1123-1126.
5. Kwong J.S., Carignan S., Kang E.Y., et. al.: Miliary tuberculosis. Diagnostic accuracy of chest radiography. Chest 1996; 110: pp. 339-342.
6. Graham S., Das G.K., Hidvegi R.J., et. al.: Chest radiograph abnormalities associated with tuberculosis: reproducibility and yield of active cases. Int J Tuberc Lung Dis 2002; 6: pp. 137-142.
7. Yerushalmy J., Garland L.H., Harkness J.T., et. al.: An evaluation of the role of serial chest roentgenograms in estimating the progress of disease in patients with pulmonary tuberculosis. Am Rev Tuberc 1951; 64: pp. 225-248.
8. Balabanova Y., Coker R., Fedorin I., et. al.: Variability in interpretation of chest radiographs among Russian clinicians and implications for screening programmes: observational study. BMJ 2005; 331: pp. 379-382.
9. Dawson-Saunders B., Trapp R.: Estimating and comparing proportions. Basic and clinical biostatistics.1994.Appleton and LangeEnglewood Cliffs, NJ
10. Benjamin D.K., DeLong E., Steinbach W.J.: Latent class analysis: an illustrative application for education in the assessment of resident otoscopic skills. Ambul Pediatr 2004; 4: pp. 13-17.
11. Walter S.D., Irwig L.M.: Estimation of test error rates, disease prevalence and relative risk from misclassified data: a review. J Clin Epidemiol 1988; 41: pp. 923-937.
12. Kosinski A.S., Barnhart H.X.: Accounting for nonignorable verification bias in assessment of diagnostic tests. Biometrics 2003; 59: pp. 163-171.
13. Little R.J.A., Rubin D.B.: Statistical analysis with missing data.1987.WileyNew York
14. Landis J.R., Koch G.G.: The measurement of observer agreement for categorical data. Biometrics 1977; 33: pp. 159-174.
15. Schmidt P.G.: [Open negative cavities.]. Tuberkulosearzt 1963; 17: pp. 137-151.
16. Oyama T., Tatsuoka M.: Prediction of relapse in pulmonary tuberculosis; an application of discriminant analysis. Am Rev Tuberc 1956; 73: pp. 472-484.
17. Tripathy S.P.: Relapse in tuberculosis. Ind J Tuberc 1981; 28: pp. 45-57.
18. Aber V.R., Nunn A.J.: [Short term chemotherapy of tuberculosis. Factors affecting relapse following short term chemotherapy]. Bull Int Union Tuberc 1978; 53: pp. 260-264.
19. Johnson J.L., Okwera A., Vjecha M.J., et. al.: Risk factors for relapse in human immunodeficiency virus type 1 infected adults with pulmonary tuberculosis. Int J Tuberc Lung Dis 1997; 1: pp. 446-453.
20. Waaler H.T.: Description of the study material and organization of the study. Bull Int Union Tuberc 1968; 41: pp. 110-114.
21. Den Boon S., Bateman E.D., Enarson D.A., et. al.: Development and evaluation of a new chest radiograph reading and recording system for epidemiological surveys of tuberculosis and lung disease. Int J Tuberc Lung Dis 2005; 9: pp. 1088-1096.
22. Albaum M.N., Hill L.C., Murphy M., et. al.: Interobserver reliability of the chest radiograph in community-acquired pneumonia. PORT Investigators. Chest 1996; 110: pp. 343-350.
23. Spearman C.: The proof and measurement of association between two things. Am J Psychol 1904; 15: pp. 72-101.
24. Thomas D., Stram D., Dwyer J.: Exposure measurement error: influence on exposure-disease. Relationships and methods of correction. Annu Rev Public Health 1993; 14: pp. 69-93.
25. Koppaka R., Bock N., World Health Organization: How reliable is chest radiography?.Toman K.Frieden T.Toman’s tuberculosis case detection, treatment, and monitoring: questions and answers.2004.World Health OrganizationGeneva:pp. 51-60.