Home Quantitative Measurements Versus Receiver Operating Characteristics and Visual Grading Regression in CT Images Reconstructed with Iterative Reconstruction
Post
Cancel

Quantitative Measurements Versus Receiver Operating Characteristics and Visual Grading Regression in CT Images Reconstructed with Iterative Reconstruction

Rationale and Objectives

This study aimed to evaluate the correlation of quantitative measurements with visual grading regression (VGR) and receiver operating characteristics (ROC) analysis in computed tomography (CT) images reconstructed with iterative reconstruction.

Materials and Methods

CT scans on a liver phantom were performed on CT scanners from GE, Philips, and Toshiba at three dose levels. Images were reconstructed with filtered back projection (FBP) and hybrid iterative techniques (ASiR, iDose, and AIDR 3D of different strengths). Images were visually assessed by five readers using a four- and five-grade ordinal scale for liver low contrast lesions and for 10 image quality criteria. The results were analyzed with ROC and VGR. Standard deviation, signal-to-noise ratios, and contrast-to-noise ratios were measured in the images.

Results

All data were compared to FBP. The results of the quantitative measurements were improved for all algorithms. ROC analysis showed improved lesion detection with ASiR and AIDR and decreased lesion detection with iDose. VGR found improved noise properties for all algorithms, increased sharpness with iDose and AIDR, and decreased artifacts from the spine with AIDR, whereas iDose increased the artifacts from the spine. The contrast in the spine decreased with ASiR and iDose.

Conclusions

Improved quantitative measurements in images reconstructed with iterative reconstruction compared to FBP are not equivalent to improved diagnostic image accuracy.

Introduction

Iterative reconstruction algorithms decrease image noise in computed tomography (CT) images compared to filtered back projection (FBP) . In FBP, the image noise is inversely proportional to the square of the radiation dose, but with iterative reconstruction, this relationship is changed. Some iterative algorithms change the image texture, which is shown by the different shape of the noise power spectrum . The shape of the noise power spectrum can be dose-dependent , and thereby influence the relationship between noise and low contrast resolution. Studies have shown that regardless of vendors’ claims of dose reduction because of use of iterative reconstruction, low contrast resolution does not benefit from the same improvement as noise .

In addition to noise, spatial resolution may influence the visibility of small low-contrast objects. Some iterative reconstruction algorithms improve spatial resolution , but there are also studies that show that the spatial resolution can be degraded . Iterative reconstruction can reduce artifacts such as metal artifacts, beam hardening artifacts, and scattering artifacts ; however, they can also introduce phenomena perceived by viewers as artifacts, like an artificial or blotchy appearance .

Get Radiology Tree app to read full this article<

Get Radiology Tree app to read full this article<

Get Radiology Tree app to read full this article<

Get Radiology Tree app to read full this article<

Get Radiology Tree app to read full this article<

Get Radiology Tree app to read full this article<

Materials and Methods

Phantom

Get Radiology Tree app to read full this article<

Figure 1, Upper abdominal phantom used in the study ( left ). Four liver inserts with lesions in 16 of 32 sectors ( right ).

Get Radiology Tree app to read full this article<

Image Acquisition and Reconstruction

Get Radiology Tree app to read full this article<

TABLE 1

Scan Parameters

CT Model Reconstruction Method Tube Voltage (kV) mAs Rotation Time (s) Pitch Collimation Reconstructed Slice Thickness (mm) Reconstruction Filter Reconstructed Field of View (mm) CTDI (mGy) GE

LightSpeed VCT FBP 120 70, 140, and 210 0.7 0.98 64 × 0.625 3 Std 350 5.1, 10.2, and 15.2 ASiR 30 % ASiR 40 % ASiR 50 % Philips

Brilliance 64 FBP 120 89, 163, and 180 0.74/0.94/0.74 1 64 × 0.625 3 B/C/B 350 5.8, 10.6, and 11.8 iDose 1 iDose 3 iDose 6 Toshiba

Aquilion One FBP 120 45, 85, and 130 0.5 0.81 80 × 0.5 3 FC19 350 5.2, 9.9, and 15.2 AIDR 3D mild AIDR 3D std AIDR 3D strong

Get Radiology Tree app to read full this article<

Quantitative Measurements

Get Radiology Tree app to read full this article<

Figure 2, Image of the phantom. The six largest lesions in the liver are shown with arrows.

Get Radiology Tree app to read full this article<

VGR Analysis

Get Radiology Tree app to read full this article<

ROC Analysis

Get Radiology Tree app to read full this article<

Statistical Analysis

Get Radiology Tree app to read full this article<

Get Radiology Tree app to read full this article<

Get Radiology Tree app to read full this article<

Results

Get Radiology Tree app to read full this article<

Figure 3, Phantom images at 10 mGy reconstructed with filtered back projection (FBP) and iterative reconstruction. First row: GE LightSpeed VCT—FBP, ASiR 40% and ASiR 50%. Second row: Philips Brilliance 64—FBP, iDose 3 and iDose 6. Third row: Toshiba Aquilion One—FBP, AIDR 3D standard and AIDR 3D strong.

Get Radiology Tree app to read full this article<

Get Radiology Tree app to read full this article<

Figure 4, Standard deviation, signal-to-noise ratio, and contrast-to-noise ratio at 5 mGy, 10 mGy, and 15 mGy. Left column: filtered back projection with GE LightSpeed VCT, ASiR 30%, ASiR 40%, and ASIR 50%. Middle column: filtered back projection with Philips Brilliance 64, iDose 1, iDose 3, and iDose 6. Right column: filtered back projection with Toshiba Aquilion One, AIDR 3D mild, AIDR 3D standard, and AIDR 3D strong. Range is shown with vertical lines. Stars indicate significant results compared to filtered back projection at each dose level with a significance level of 0.05.

Get Radiology Tree app to read full this article<

Get Radiology Tree app to read full this article<

TABLE 2

VGR Results

GE LightSpeed VCT Philips Brilliance 64 Toshiba Aquilion One Algorithm Strength (%) Coefficient (Conf. Int.)P Value R McF 2 Algorithm Strength Coefficient (Conf. Int.)P Value R McF 2 Algorithm Strength Coefficient (Conf. Int.)P Value R McF 2 Noise ASiR 30 −0.5 (−1.6 to 0.7) .42 0.51 iDose 1 − 1.6 (2.5 to0.7)<.01 0.37 AIDR 3D Mild − 3.0 (4.5 to1.5)<.01 0.51 40 −0.5 (−1.6 to 0.7) .43 3 − 2.4 (3.4 to1.5)<.01 Std − 2.1 (3.5 to0.6)<.01 50 − 1.8 (3.0 to0.6)<.01 6 − 4.8 (6.0 to3.5)<.01 Strong − 5.2 (6.9 to3.4)<.01 Contrast resolution liver 30 1.0 (−1.1 to 3.0) .36 0.55 1 −1.9 (−3.9 to 0.1) .07 0.46 Mild −1.3 (−3.5 to 1.0) .28 0.49 40 1.0 (−1.1 to 3.0) .35 3 −1.9 (−3.9 to 0.1) .06 Std −1.7 (−4.0 to 0.7) .18 50 −1.3 (−3.2 to 0.7) .20 6 − 3.0 (5.2 to0.9)<.01 Strong −1.3 (−3.5 to 1.0) .28 Sharpness 30 0.3 (−0.5 to 1.1) .44 0.40 1 − 1.4 (2.2 to0.6)<.01 0.25 Mild − 1.6 (2.4 to0.8)<.01 0.31 401.0 (0.1 to 1.8).03 3 − 0.8 (1.6 to 0.0).05 Std − 1.0 (1.8 to0.1).02 50 0.6 (−0.2 to 1.4) .14 6 0.6 (−0.2 to 1.4) .16 Strong − 1.0 (1.8 to0.1).03 Artifact spine 30 0.1 (−1.3 to 1.5) .88 0.03 1 −0.8 (−2.3 to 0.8) .31 0.13 Mild − 3.8 (6.0 to1.6)<.01 0.25 40 0.1 (−1.2 to 1.5) .88 3 0.3 (−1.2 to 1.7) .72 Std − 4.2 (6.4 to1.9)<.01 50 −0.9 (−2.3 to 0.5) .23 61.7 (0.1–3.3).04 Strong − 3.9 (6.1 to1.7)<.01 Contrast spine 302.2 (0.2 to 4.1).03 0.32 1 −0.5 (−2.0 to 0.9) .47 0.20 Mild −0.8 (−2.2 to 0.6) .25 0.11 40 1.6 (−0.4 to 3.5) .12 3 −0.1 (−1.5 to 1.4) .92 Std 0.3 (−1.1 to 1.6) .71 50 1.4 (−0.5 to 3.4) .14 61.6 (0.0–3.2).05 Strong −0.4 (−1.8 to 1.1) .61

Regression coefficient estimates (95% confidence intervals) for effect of each reconstruction algorithm, P Value and McFadden coefficient of determination, R McF 2 for ASiR, iDose, and AIDR 3D for the criteria evaluated: noise, low contrast resolution in the liver, sharpness, artifacts from spine, and contrast in spine.

Boldface denotes significant ( P < .05) differences from FBP. Because higher image quality was coded with lower numbers, a negative coefficient denotes an improvement in image quality.

Get Radiology Tree app to read full this article<

Get Radiology Tree app to read full this article<

Figure 5, Phantom images from Philips Brilliance 64 at 5 mGy. Filtered back projection, iDose 3 and iDose 6. Image reconstructed with iDose 6 shows bright “shadows” from the spine.

Get Radiology Tree app to read full this article<

Get Radiology Tree app to read full this article<

TABLE 3

The Area Under the Curve (AUC) and the Difference (Diff) Between Iterative Reconstruction Algorithm and FBP with 95 % Confidence Intervals for Filtered Back Projection (FBP), ASiR, iDose, and AIDR 3D at 5 mGy, 10 mGy, and 15 mGy

GE LightSpeed VCT Philips Brilliance 64 Toshiba Aquilion One AUC 95% CI ± Diff. 95% CI ± AUC 95% CI ± Diff. 95% CI ± AUC 95% CI ± Diff. 95% CI ± 5 mGy ASiR FBP 0.74 0.06 iDose FBP 0.82 0.04 AIDR 3D FBP 0.67 0.04 30% 0.70 0.03 −0.037 0.058 1 0.79 0.07 −0.035 0.046 Mild 0.68 0.10 0.012 0.100 40% 0.73 0.06 −0.007 0.046 3 0.80 0.03 −0.021 0.028 Std 0.71 0.08 0.042 0.082 50% 0.72 0.06 −0.023 0.084 60.760.030.0680.032 Strong0.750.080.0830.076 10 mGy FBP 0.79 0.03 FBP 0.83 0.02 FBP 0.75 0.08 30%0.840.030.0560.051 1 0.82 0.01 −0.006 0.011 Mild0.810.030.0650.063 40% 0.83 0.03 0.041 0.048 3 0.82 0.02 −0.007 0.022 Std0.830.020.0790.063 50%0.830.020.0430.030 6 0.82 0.02 −0.006 0.021 Strong0.820.020.0750.074 15 mGy FBP 0.92 0.03 FBP 0.92 0.04 FBP 0.75 0.09 30% 0.90 0.03 −0.020 0.047 1 0.89 0.02 −0.015 0.019 Mild0.810.080.0630.033 40% 0.92 0.01 −0.003 0.034 30.850.010.0570.041 Std0.840.100.0960.015 50% 0.91 0.04 −0.004 0.037 60.870.040.0350.022 Strong0.840.040.0880.062

Boldface denotes significant differences from FBP (confidence intervals of difference not containing zero).

Figure 6, Receiver operating characteristics (ROC) curves for filtered back projection and iterative reconstruction algorithms. First column: GE LightSpeed VCT, second column: Philips Brilliance 64, and third column: Toshiba Aquilion One. First row: 5 mGy, second row: 10 mGy, and third row: 15 mGy.

Get Radiology Tree app to read full this article<

Get Radiology Tree app to read full this article<

TABLE 4

Significant Improvements in Standard Deviation (SD), Signal-to-Noise Ratio (SNR) and Contrast-to-Noise Ratio (CNR), Visual Grading Score from VGR, or Area Under the Curve (AUC) from ROC Analysis with Iterative Reconstruction Compared to Filtered Back Projection

Measured Increase VGR ROC SD SNR CNR 5 mGy 10 mGy 15 mGy 5 mGy 10 mGy 15 mGy 5 mGy 10 mGy 15 mGy Noise Contrast Res. Liver Sharpness Artifact Spine Contrast Spine 5 mGy 10 mGy 15 mGy GE LightSpeed VCT ASiR 30 % \* \* [ \* ] \* ASiR 40 % \* † \* † [ \* ] ASiR 50 % \* ‡ † † † \* Philips Brilliance 64 iDose 1 † \* † ‡ iDose 3 ‡ \* \* \* ‡ \* [ \* ] iDose 6 ‡ † \* \* † † ‡ ‡ † [ \* ] [ \* ] [ \* ] [ \* ] Toshiba Aquilion One AIDR 3D mild ‡ \* \* \* ‡ ‡ † \* \* AIDR 3D standard ‡ \* \* \* \* † \* ‡ \* \* AIDR 3D strong ‡ \* \* \* † ‡ \* † ‡ \* \* \*

Asterisks within [] denote results that are significantly worse than filtered back projection.

Get Radiology Tree app to read full this article<

Get Radiology Tree app to read full this article<

Get Radiology Tree app to read full this article<

Get Radiology Tree app to read full this article<

Get Radiology Tree app to read full this article<

Figure 7, Area under the curve (AUC) versus standard deviation (SD), signal-to-noise ratio (SNR),and contrast-to-noise ratio (CNR). o ASiR, x iDose, * AIDR 3D. The Spearman correlation coefficient, ρ, is shown in each plot.

Get Radiology Tree app to read full this article<

Discussion

Get Radiology Tree app to read full this article<

Get Radiology Tree app to read full this article<

Get Radiology Tree app to read full this article<

Get Radiology Tree app to read full this article<

Get Radiology Tree app to read full this article<

Get Radiology Tree app to read full this article<

Get Radiology Tree app to read full this article<

Get Radiology Tree app to read full this article<

Conclusion

Get Radiology Tree app to read full this article<

References

  • 1. Löve Á., Olsson M.-L., Siemund R., et. al.: Six iterative reconstruction algorithms in brain CT: a phantom study on image quality at different radiation dose levels. Br J Radiol 2013; 86: pp. 1-11.

  • 2. Schaller F., Sedlmair M., Raupach R., et. al.: Noise reduction in abdominal computed tomography applying iterative reconstruction (ADMIRE). Acad Radiol 2016; 23: pp. 1230-1238.

  • 3. Lee S., Kwoon H., Cho J.: The detection of focal liver lesions using abdominal CT: a comparison of image quality between Adaptive Statistical Iterative Reconstruction V and Adaptive Statistical Iterative Reconstruction. Acad Radiol 2016; 23: pp. 1532-1538.

  • 4. Chen B., Richard S., Samei E.: Relevance of MTF and NPS in quantitative CT: towards developing a predictable model of quantitative performance. Proc SPIE 8313, Medical Imaging 2012: Physics of Medical Imaging, 83132O (February 23, 2012)2012.

  • 5. Li K., Tang J., Chen G.-H.: Statistical model based iterative reconstruction (MBIR) in clinical CT systems: experimental assessment of noise performance. Med Phys 2014; 41: pp. 041906. 1-15

  • 6. Dobeli K., Lewis S., Meikle S., et. al.: Noise-reducing algorithms do not necessarily provide superior dose optimisation for hepatic lesion detection with multidetector CT. Br J Radiol 2013; 86:

  • 7. McCollough C.H., Yu L., Kofler J.M., et. al.: Degradation of CT low-contrast spatial resolution due to the use of iterative reconstruction and reduced dose levels. Radiology 2015; 276: pp. 499-506.

  • 8. Schindera S., Odedra D., Raza S.: Iterative reconstruction algorithm for CT: can radiation dose be decreased while low-contrast detetability is preserved?. Radiology 2013; 269: pp. 511-518.

  • 9. Euler A., Stieltjes B., Szucs-Farkas Z., et. al.: Impact of model-based iterative reconstruction on low-contrast lesion detection and image quality in abdominal CT: a 12-reader-based comparative phantom study with filtered back projection at different tube voltages. Eur Radiol 2017; [Epub ahead of print]

  • 10. Goenka A.H., Herts B.R., Dong F., et. al.: Image noise, CNR, and detectability of low-contrast, low-attenuation liver lesions in a phantom: effects of radiation exposure, phantom size, integrated circuit detector and iterative reconstruction. Radiology 2016; 280: pp. 475-482.

  • 11. Omotayo A., Elbakri I.: Objective performance assessment of five computed tomography iterative reconstruction algorithms. J Xray Sci Technol 2016; 24: pp. 913-930. Preprint:1–18

  • 12. Nuyts J., De Man B., Fessler J.A., et. al.: Modelling the physics in iterative reconstruction for transmission computed tomography. Phys Med Biol 2013; 58: pp. R63-R96.

  • 13. Beister M., Kolditz D., Kalender W.: Iterative reconstruction methods in x-ray CT. Phys Med 2012; 28: pp. 94-108.

  • 14. Boas E.F., Fleischmann D.: CT artifacts: causes and reduction techniques. Imaging Med 2012; 4: pp. 229-240.

  • 15. Kotsenas A., Michalak G., DeLone D., et. al.: CT metal artifact reduction in the spine: can an iterative reconstruction technique improve visualization?. AJNR Am J Neuroradiol 2015; 36: pp. 2184-2190.

  • 16. Miéville F.A., Gudinchet F., Rizzo E., et. al.: Paediatric cardiac CT examinations: impact of the iterative reconstruction method ASIR on image quality—preliminary findings. Pediatr Radiol 2011; 41: pp. 1154-1164.

  • 17. Singh S., Kalra M., Hsieh J., et. al.: Abdominal CT: comparison of adaptive statistical iterative and filtered back projection reconstruction techniques. Radiology 2010; 257: pp. 373-383.

  • 18. Smedby Ö., Fredrikson M.: Visual grading regression: analysing data from visual grading experiments with regression models. Br J Radiol 2009; 83: pp. 767-775.

  • 19. Keeble C.B., Baxter P.D., Gislason-Lee A.J., et. al.: Methods for the analysis of ordinal response data in medical image quality assessment. Br J Radiol 2016; 89: pp. 1-11.

  • 20. Båth M., Månsson L.: Visual grading characteristics (VGC) analysis: a non-parametric rank-invariant statistical method for image qualtiy evaluation. Br J Radiol 2007; 80: pp. 169-176.

  • 21. Metz C.: Basic principles of ROC analysis. Semin Nucl Med 1978; 8: pp. 283-298.

  • 22. Sandborg M., Tingberg A., Dance D., et. al.: Demonstration of correlations between clinical and physical image quality measures in chest and lumbar spine screen-film radiography. Br J Radiol 2001; 74: pp. 520-528.

  • 23. Moore C., Wood T., Beavis A., et. al.: Correlation of the clinical and physical image quality in chest radiography for average adults with a computed radiography imaging system. Br J Radiol 2013; 86:

  • 24. De Crop A., Smeets P., Van Hoof T., et. al.: Correlation of clinical and physical-technical image quality in chest CT: a human cadaver study applied on iterative reconstruction. BMC Med Imaging 2015; 15:

  • 25. Lanca L., Andersen E., Carvalho G., et. al.: Are physical measures good indicators of image quality at low dose levels? A pilot study. Optimax 2015, Multicultural team-based research in radiography, a holistic educational approach2015.pp. 128-142.

  • 26. Olerud H.M., Olse J.B., Skretting A.: An anthropomorphic phantom for receiver operating characteristic studies in CT imaging of liver lesions. Br J Radiol 1999; 72: pp. 35-43.

  • 27. Smedby Ö., Fredrikson M., De Geer J., et. al.: Visual grading regression with random effects. Proc SPIE 8318, Medical Imaging 2012: Image Perception, Observer Performance, and Technology Assessment, 831805-12012.

  • 28. DeLong E., DeLong D., Clarke-Pearson D.: Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 1988; 44: pp. 837-845.

  • 29. Saffari E.S., Löve Á., Fredrikson M., et. al.: Regression models for analysing radiological visual grading studies—an empirical comparison. BMC Med Imaging 2015; 15: pp. 1-10.

  • 30. Vaishnav J.Y., Jung W.C., Popescu L.M., et. al.: Objective assessment of image quality and dose reduction in CT iterative reconstruction. Med Phys 2014; 41:

  • 31. Philips Healthcare : Iterative Model Reconstrucion—Reconstruction Technology. Philips Healthcare2016.

  • 32. Pooler B.D., Lubner M.G., Kim D.H., et. al.: Prospective evaluation of reduced dose computed tomography for the detection of low-contrast liver lesions: direct comparison with concurrent standard dose imaging. Eur Radiol 2017; 27: pp. 2055-2066.

  • 33. Winslow J., Zhang Y., Samei E.: A method for characterizing and matching CT image quality across CT scanners from different manufacturers. Med Phys 2017; [Epub ahead of print]

  • 34. Smedby Ö., Fredrikson M., De Geer J., et. al.: Quantifying the potential for dose reduction with visual grading regression. Br J Radiol 2012; 86:

  • 35. Metz C.E.: Fundamental ROC analysis.Jacob Beutel H.L.K.Van Metter R.L.Handbook of medical imaging volume 1 physics and psychophysics.2000.SPIE PressBellingham, WA:pp. 751-771.

This post is licensed under CC BY 4.0 by the author.