Scientific rigor should be consistently applied to quality improvement (QI) research to ensure that healthcare interventions improve quality and patient safety before widespread implementation. This article provides an overview of the various study designs that can be used for QI research depending on the stage of investigation, scope of the QI intervention, constraints on the researchers and intervention being studied, and evidence needed to support widespread implementation. The most commonly used designs in QI studies are quasi-experimental designs. Randomized controlled trials and cluster randomized trials are typically reserved for large-scale research projects evaluating the effectiveness of QI interventions that may be implemented broadly, have more than a minimal impact on patients, or are costly. Systematic reviews of QI studies will play an important role in providing overviews of evidence supporting particular QI interventions or methods of achieving change. We also review the general requirements for developing quality measures for reimbursement, public reporting, and pay-for-performance initiatives. A critical part of the testing process for quality measures includes assessment of feasibility, reliability, validity, and unintended consequences. Finally, publication and critical appraisal of QI work is discussed as an essential component to generating evidence supporting QI initiatives in radiology.
Introduction
Fundamental differences between quality improvement (QI) research and traditional scientific research can be best understood by articulating the goals of these two distinct activities. QI research is designed to achieve positive change in a specific process or service that tends to be highly dependent on the local environment. These projects usually focus on a well-defined problem, build in feedback on immediate outcomes to allow for the adjustment of interventions, and are generally not dependent on a deep understanding of the mechanisms of change involved in the interventions . Traditional scientific research is designed to explicitly test a hypothesis, eliminate or minimize bias, and yield new generalizable knowledge with a focus on the mechanisms of cause and effect. Some features of QI and traditional research overlap. Hypothesis-driven testing constitutes a critical part of QI, although QI includes other systematic activities distinct from scientific research such as goal-setting, performance measurement and feedback, standardization, and education . Scientific rigor should be consistently applied to QI research to ensure that patients do not suffer harm in the name of QI.
Considerable debate exists regarding the use of health-care interventions designed to improve quality and patient safety without clear evidence that benefits outweigh costs and harms. Proponents of implementing large-scale health-care interventions without compelling evidence note that in many instances, it may be costly, difficult, or impossible to generate evidence. This difficulty may relate to problems blinding investigators or participants, inability to establish causality, or issues related to interventions that are highly dependent on local context and culture . Opponents argue that if the evidence is not compelling, well-intentioned interventions may fail to improve health or may even cause harm, while costing dearly . One example of a QI initiative in radiology that has been successfully implemented in many different local environments is the Image Gently campaign, which also serves as one of the National Quality Forum’s (NQF) Safe Practice guidelines to reduce unnecessary exposure to ionizing radiation for children .
The goals of this review are to (1) provide a review of QI research methodologies and study designs that can generate high-quality evidence regarding QI interventions in radiology, (2) review the ethics of QI research, (3) discuss the rationale and process for developing quality measures, and (4) provide guidance on the preparation and review of manuscripts regarding QI initiatives. The Association of University Radiologists Radiology Research Alliance convened a task force to explore this topic with the results presented in this review.
Quality Improvement Study Designs
Many QI study designs draw from health services research, social science, and other disciplines, each with advantages and disadvantages in dealing with the complexities of clinical practice . These designs attempt to establish a causal relationship between an intervention and a change in outcome. The choice of design depends on both the purpose of the study and the degree of control the investigators have over delivery of the intervention . Campbell et al suggest that the evaluation of complex interventions should follow a sequential approach that provides increasing evidence :
Explore relevant theory to ensure the best choice of intervention and hypothesis and to predict major confounders and strategic design issues;
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Qualitative Methods
Get Radiology Tree app to read full this article<
Interview-based Qualitative Methods
Get Radiology Tree app to read full this article<
Observational-based Qualitative Methods
Get Radiology Tree app to read full this article<
Quality Improvement Projects
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Table 1
Common Tests for Special Cause Variation
SD, standard deviation.
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Quasi-experimental Trials
Get Radiology Tree app to read full this article<
Uncontrolled before and after Studies
Get Radiology Tree app to read full this article<
Controlled before and after Studies
Get Radiology Tree app to read full this article<
Time-series Designs
Get Radiology Tree app to read full this article<
Randomized Controlled Trials
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Cluster Randomized Trial
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Systematic Reviews
Get Radiology Tree app to read full this article<
Ethics of QI Research
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Developing Measures for Quality Improvement
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Development of Quality Measures
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Preparation and Critical Evaluation of QI Manuscripts
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Conclusion
Get Radiology Tree app to read full this article<
References
1. Portela M.C., Pronovost P.J., Woodcock T., et. al.: How to study improvement interventions: a brief overview of possible study types. BMJ Qual Saf 2015; 24: pp. 325-336.
2. Crump B.: Should we use large scale healthcare interventions without clear evidence that benefits outweigh costs and harms? Yes. BMJ 2008; 336: pp. 1276.
3. Larson D.B., Duncan J.R., Nagy P.G., et. al.: Guide to effective quality improvement reporting in radiology. Radiology 2014; 271: pp. 561-573.
4. Landefeld C.S., Shojania K.G., Auerbach A.D.: Should we use large scale healthcare interventions without clear evidence that benefits outweigh costs and harms? No. BMJ 2008; 336: pp. 1277.
5. Roth C.J.: The National Quality Forum: how common sense becomes policy. J Am Coll Radiol 2010; 7: pp. 309-311.
6. 2013.National Quality Forum Available from: http://www.hfap.org/pdf/patient_safety.pdf
7. Eccles M., Grimshaw J., Campbell M., et. al.: Research designs for studies evaluating the effectiveness of change and improvement strategies. Qual Saf Health Care 2003; 12: pp. 47-52.
8. Campbell M., Fitzpatrick R., Haines A., et. al.: Framework for design and evaluation of complex interventions to improve health. BMJ 2000; 321: pp. 694-696.
9. Pope C., van Royen P., Baker R.: Qualitative methods in research on healthcare quality. Qual Saf Health Care 2002; 11: pp. 148-152.
10. 1997.Diane Publishing Company 132 pp; Available from: http://www.nsf.gov/pubs/1997/nsf97153/chap_3.htm
11. Chassin M.R., Loeb J.M.: The ongoing quality improvement journey: next stop, high reliability. Health Aff (Millwood) 2011; 30: pp. 559-568.
12. Kruskal J.B., Eisenberg R., Sosna J., et. al.: Quality initiatives: quality improvement in radiology: basic principles and tools required to achieve success. Radiographics 2011; 31: pp. 1499-1509.
13. Rawson J.V., Kannan A., Furman M.: Use of process improvement tools in radiology. Curr Probl Diagn Radiol 2016; 45: pp. 94-100.
14. Itri J.N., Bakow E., Woods J.: Creating an outpatient center of excellence in CT. J Am Coll Radiol 2014; 11: pp. 1137-1143.
15. Benneyan J.C., Lloyd R.C., Plsek P.E.: Statistical process control as a tool for research and healthcare improvement. Qual Saf Health Care 2003; 12: pp. 458-464.
16. 2016.Minitab Inc. Available at: http://support.minitab.com/en-us/minitab/17/topic-library/quality-tools/control-charts/basics/using-tests-for-special-causes/
17. 1999.SAS Institute Inc. Available at: http://www.okstate.edu/sas/v8/saspdf/qc/chap48.pdf
18. Provost L.P., Murray S.K.: The health care data guide: learning from data for improvement.1st ed.2011.Jossey-BassSan Francisco, CA xxviii, 445 pp
19. Fan E., Laupacis A., Pronovost P.J., et. al.: How to use an article about quality improvement. JAMA 2010; 304: pp. 2279-2287.
20. Benning A., Ghaleb M., Suokas A., et. al.: Large scale organisational intervention to improve patient safety in four UK hospitals: mixed method evaluation. BMJ 2011; 342: pp. d195.
21. Asarnow J.R., Jaycox L.H., Duan N., et. al.: Effectiveness of a quality improvement intervention for adolescent depression in primary care clinics: a randomized controlled trial. JAMA 2005; 293: pp. 311-319.
22. Glynn R.J., Brookhart M.A., Stedman M., et. al.: Design of cluster-randomized trials of quality improvement interventions aimed at medical care providers. Med Care 2007; 45: pp. S38-S43.
23. Neuhauser D., Diaz M.: Quality improvement research: are randomised trials necessary?. Qual Saf Health Care 2007; 16: pp. 77-80.
24. Hemming K., Lilford R., Girling A.J.: Stepped-wedge cluster randomised controlled trials: a generic framework including parallel and multiple-level designs. Stat Med 2015; 34: pp. 181-196.
25. Brown C.A., Lilford R.J.: The stepped wedge trial design: a systematic review. BMC Med Res Methodol 2006; 6: pp. 54.
26. Grimshaw J., McAuley L.M., Bero L.A., et. al.: Systematic reviews of the effectiveness of quality improvement strategies and programmes. Qual Saf Health Care 2003; 12: pp. 298-303.
27. Lynn J., Baily M.A., Bottrell M., et. al.: The ethics of using quality improvement methods in health care. Ann Intern Med 2007; 146: pp. 666-673.
28. Shojania K.G., Showstack J., Wachter R.M.: Assessing hospital quality: a review for clinicians. Eff Clin Pract 2001; 4: pp. 82-90.
29. Mendiratta-Lala M., Eisenberg R.L., Steele J.R., et. al.: Quality initiatives: measuring and managing the procedural competency of radiologists. Radiographics 2011; 31: pp. 1477-1488.
30. Dunnick N.R., Applegate K.E., Arenson R.L.: Quality—a radiology imperative: report of the 2006 Intersociety Conference. J Am Coll Radiol 2007; 4: pp. 156-161.
31. Mainz J.: Defining and classifying clinical indicators for quality improvement. Int J Qual Health Care 2003; 15: pp. 523-530.
32. 2013.The American College of Radiology Available at: http://www.acr.org/~/media/ACR/Documents/PDF/QualitySafety/NRDR/GRID/MeasuresGRID
33. McGlynn E.A., Asch S.M.: Developing a clinical performance measure. Am J Prev Med 1998; 14: pp. 14-21.
34. You-Ling Shyu J., Burleson J., Tallant C., et. al.: Performance measures in radiology. J Am Coll Radiol 2014; 11: pp. 456-463.
35. 2010.American Medical Association Available at: http://www.ama-assn.org/resources/doc/cqi/pcpi-testing-protocol.pdf
36. Gur D., Wallace L.P., Klym A.H., et. al.: Trends in recall, biopsy, and positive biopsy rates for screening mammography in an academic practice. Radiology 2005; 235: pp. 396-401.
37. 2016.Centers for Medicare and Medicaid Services Available at: http://www.cms.gov/apps/ama/license.asp?file=/PQRS/Downloads/PQRS_2016_Measure_List_01072016.xlsx
38. Davidoff F., Batalden P.: Toward stronger evidence on quality improvement. Draft publication guidelines: the beginning of a consensus project. Qual Saf Health Care 2005; 14: pp. 319-325.
39. SQUIRE : Writing is part of quality and safety work. Available at: http://squire.citysoft.org/index.cfm?fuseaction=page.viewpage&pageid=488 Accessed Feburary 14, 2016
40. Cronin P., Rawson J.V., Heilbrun M.E., et. al.: How to critically appraise the clinical literature. Acad Radiol 2014; 21: pp. 1117-1128.
41. Cronin P., Rawson J.V., Heilbrun M.E., et. al.: How to report a research study. Acad Radiol 2014; 21: pp. 1088-1116.
42. Radiologic Society of North America : Radiology publication information for authors: checklists. Available at: http://pubs.rsna.org/page/radiology/pia/checklists Accessed February 14, 2016
43. SQUIRE : SQUIRE promoting excellence in healthcare improvement reporting. Available at: http://www.squire-statement.org/ Accessed February 13, 2016
44. Ogrinc G., Davies L., Goodman D., et. al.: SQUIRE 2.0 (Standards for Quality Improvement Reporting Excellence): revised publication guidelines from a detailed consensus process. BMJ Qual Saf 2015;
45. Davies L., Batalden P., Davidoff F., et. al.: The SQUIRE Guidelines: an evaluation from the field, 5 years post release. BMJ Qual Saf 2015; 24: pp. 769-775.
46. Hempel S., Shekelle P.G., Liu J.L., et. al.: Development of the Quality Improvement Minimum Quality Criteria Set (QI-MQCS): a tool for critical appraisal of quality improvement intervention publications. BMJ Qual Saf 2015; 24: pp. 796-804.