Survey Research

Survey research is appealing to many clinical researchers, including radiologists. Emerging interest in patient preferences and patient-centered outcomes related to imaging likely will stimulate additional use of questionnaires in our field. However, like other quantitative methods, survey-based research requires meticulous planning, execution, and analysis to generate reliable results and support meaningful conclusions. The purpose of this review is to provide a guideline for radiologists embarking on this type of research, with attention to questionnaire design, sampling, survey administration, and analysis.

Introduction

Survey research appeals to many academic radiologists. On the surface, these projects seem quick, inexpensive, and relatively simple. In the age of e-mail, social media, and smartphones, the broad potential reach and clickable immediacy of a questionnaire are attractive. However, the perception that “anyone can do it” and the rush to push “send” may contribute to flawed data and inappropriate conclusions .

Survey research should be approached with the same rigor and planning as other quantitative research methods . In fact, survey research is itself an entire academic discipline within social science. Although abundant throughout the medical literature, most medical professionals lack formal training in this area .

A point about terminology: a survey, in its pure sense, is a sampling of any value from a larger population, whereas a questionnaire is a tool often used in survey research that consists of a series of written or verbal questions. Therefore, although “survey” and “questionnaire” often are interchangeably used, they are not synonymous. Examples of survey research that do not include a questionnaire might include samples of blood pressure values, heights, or test scores. For the purposes of this manuscript, we will be focusing on questionnaire-based survey research.

Reliable survey research requires substantial forethought to refine the scientific question, design the survey instrument, select and engage the appropriate population, and analyze the data. This review outlines the steps in designing and implementing a questionnaire-based study. The process is broken down into its key components in Figure 1 . The purpose of this review is to provide a stepwise guideline for radiologists embarking on this type of research.

A Stepwise Approach to Questionnaire-based Research

The “Survey Construct”: What Do I Want to Measure, or Why Am I Doing This?

Get Radiology Tree app to read full this article<

Questionnaire Design: What Questions Am I Asking and Are These the Right Ones?

Get Radiology Tree app to read full this article<

TABLE 1

Pearls and Pitfalls for Questionnaire-based Studies Conducted by Radiologists

Phase of Study Pearls Pitfalls Developing the construct Establish a clear vision of the purpose of the project Proceed to questionnaire design without developing a research question Define important outcomes in advance Questionnaire design Engage statistician early in the process Overlapping answer options Use validated instruments when available Noncontiguous answer options Build redundant items around important concepts to confirm internal validity “Stacked” questions within one stem Pilot the questionnaire Double negatives Difficult hypothetical questions Keep questions at an 8th grade reading level or below Intimate questions without regard for respondent privacy Sampling Identify target population Use convenience sample without considering implications Define method of sampling, considering all options Know your population and obstacles to survey completion Compare sample demographics to target population Data collection Consider implications of mode of survey administration Use the quickest and cheapest method without regard for population characteristics Use institutional resources to assist in data collection Assume clerks, technicians, or clinical colleagues will be motivated to consent patients and collect questionnaires Pay modest incentives to boost response rate Use reminders to boost response rate Publish questionnaire without pilot-testing the method of administration Data analysis Carefully oversee data entry and coding steps Make conclusions before comparing sample to target population Spend time on descriptive analysis and “get to know” your data Ignore missing data and haphazardly include or exclude patients Assess for missing data, create a formal analysis plan, and report it Perform multiple hypothesis tests without statistical significance correction Inference Present results in the context of prior literature Make bold claims without reference to relevant prior research or considering limitations Provide appropriate confidence intervals for results Consider potential bias Ignore issues of sampling and representation Create road map for future research Ignore potential bias

Get Radiology Tree app to read full this article<

Study Population and Sampling: Who Am I Asking?

Get Radiology Tree app to read full this article<

Data Collection: How Am I Getting My Responses?

Get Radiology Tree app to read full this article<

Data Analysis: What Are My Responses?

Get Radiology Tree app to read full this article<

Inference: What Does This All Mean?

Get Radiology Tree app to read full this article<

Conclusions

Get Radiology Tree app to read full this article<

References

1. Hillman B.J., Carlos R.C.: How hard could it be?. J Am Coll Radiol: JACR 2016; 13: pp. 889.
2. Jones D., Story D., Clavisi O., et. al.: An introductory guide to survey research in anaesthesia. Anaesth Intensive Care 2006; 34: pp. 245-253.
3. Jones T.L., Baxter M.A., Khanduja V.: A quick guide to survey research. Ann R Coll Surg Engl 2013; 95: pp. 5-7.
4. Kelley K., Clark B., Brown V., et. al.: Good practice in the conduct and reporting of survey research. Int J Qual Health Care 2003; 15: pp. 261-266.
5. Bennett C., Khangura S., Brehaut J.C., et. al.: Reporting guidelines for survey research: an analysis of published guidance and reporting practices. PLoS Med 2010; 8: pp. e1001069.
6. Cronbach L.J., Meehl P.E.: Construct validity in psychological tests. Psychol Bull 1955; 52: pp. 281-302.
7. Messick S.: Validity of psychological assessment: validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. Am Psychol 1995; 50: pp. 741-749.
8. Office of Behavioral and Social Science Research. : Objective measurement of subjective phenomenon. Available at http://www.esourceresearch.org/tabid/36/Default.aspx
9. Ware J., Kosinski M., Keller S.D.: A 12-Item Short-Form Health Survey: construction of scales and preliminary tests of reliability and validity. Med Care 1996; 34: pp. 220-233.
10. Swan J.S., Hur C., Lee P., et. al.: Responsiveness of the testing morbidities index in colonoscopy. Value Health 2013; 16: pp. 1046-1053.
11. Woolen S.A., Maturen K.E., Nettles A., et. al.: Patient-centered assessment of the value of oral contrast material. J Am Coll Radiol: JACR 2017; 14: pp. 1626-1631.
12. Rosenkrantz A.B., Flagg E.R.: Survey-based assessment of patients’ understanding of their own imaging examinations. J Am Coll Radiol: JACR 2015; 12: pp. 549-555.
13. Gaetke-Udager K., Knoepp U., Maturen K., et. al.: A women in radiology group fosters career development for faculty and trainees. AJR Am J Roentgenol 2018; In press
14. Shankar P.R., Woodard J., Heller M.T., et. al.: Implementation and outcomes of a resident-led physics curriculum with integration of the RSNA/AAPM physics modules. Acad Radiol 2013; 20: pp. 1306-1310.
15. Magee C., Rickards G., A Byars L., et. al.: Tracing the steps of survey design: a graduate medical education research example. J Grad Med Educ 2013; 5: pp. 1-5.
16. Edwards P.: Questionnaires in clinical trials: guidelines for optimal design and administration. Trials 2010; 11: pp. 2.
17. Peter J.P.: Construct validity: a review of basic issues and marketing practices. J Mark Res 1981; 18: pp. 133-145.
18. Fowler F.J.: Improving survey questions: design and evaluation.1995.Sage PublicationsThousand Oaks, CA
19. Sakala M.D., Carlos R.C., Mendiratta-Lala M., et. al.: Understanding patient preference in female pelvic imaging: transvaginal ultrasound and MRI. Acad Radiol 2017;
20. Etchegaray J.M., Fischer W.G.: Understanding evidence-based research methods: pilot testing surveys. Herd 2011; 4: pp. 143-147.
21. Groves R., Fowler F., Couper M., et. al.: Survey methodology.2009.WileyHoboken, NJ
22. Brick J.M., Kalton G.: Handling missing data in survey research. Stat Methods Med Res 1996; 5: pp. 215-238.
23. Rogelberg S.G., Stanton J.M.: Introduction: understanding and dealing with organizational survey nonresponse. Organ Res Methods 2007; 10: pp. 195-209.
24. Turner C.F., Ku L., Rogers S.M., et. al.: Adolescent sexual behavior, drug use, and violence: increased reporting with computer survey technology. Science 1998; 280: pp. 867.
25. Gaetke-Udager K., Yablon C.M.: Medical education research for radiologists: a road map for developing a project. AJR Am J Roentgenol 2015; 204: pp. 692-697.
26. Beckman T.J., Cook D.A.: Developing scholarly projects in education: a primer for medical teachers. Med Teach 2007; 29: pp. 210-218.
27. Sitzia J., Wood N.: Response rate in patient satisfaction research: an analysis of 210 published studies. Int J Qual Health Care 1998; 10: pp. 311-317.
28. Link M.W., Mokdad A.H.: Effects of survey mode on self-reports of adult alcohol consumption: a comparison of mail, web and telephone approaches. J Stud Alcohol 2005; 66: pp. 239-245.
29. Link M.W., Mokdad A.H.: Alternative modes for health surveillance surveys: an experiment with web, mail, and telephone. Epidemiology 2005; 16: pp. 701-704.
30. VanGeest J.B., Johnson T.P., Welch V.L.: Methodologies for improving response rates in surveys of physicians: a systematic review. Eval Health Prof 2007; 30: pp. 303-321.
31. Everett S.A., Price J.H., Bedell A.W., et. al.: The effect of a monetary incentive in increasing the return rate of a survey to family physicians. Eval Health Prof 1997; 20: pp. 207-214.
32. Mizes J.S., Fleece E.L., Roos C.: Incentives for increasing return rates: magnitude levels, response bias, and format. Public Opin Q 1984; 48: pp. 794-800.
33. Michaels B., Harris K., Gupta N., et. al.: Clinical predictors of sexual function in gynecologic cancer survivors. Society of Gynecologic Oncology annual meeting. San Diego, CA2016.
34. Seaman S.R., White I.R.: Review of inverse probability weighting for dealing with missing data. Stat Methods Med Res 2013; 22: pp. 278-295.
35. Peyre H., Leplege A., Coste J.: Missing data methods for dealing with missing items in quality of life questionnaires. A comparison by simulation of personal mean score, full information maximum likelihood, multiple imputation, and hot deck techniques applied to the SF-36 in the French 2003 decennial health survey. Qual Life Res 2011; 20: pp. 287-300.
36. Jakobsen J.C., Gluud C., Wetterslev J., et. al.: When and how should multiple imputation be used for handling missing data in randomised clinical trials—a practical guide with flowcharts. BMC Med Res Methodol 2017; 17: pp. 162.
37. Rombach I., Rivero-Arias O., Gray A.M., et. al.: The current practice of handling and reporting missing outcome data in eight widely used PROMs in RCT publications: a review of the current literature. Qual Life Res 2016; 25: pp. 1613-1623.

Survey Research

Introduction

A Stepwise Approach to Questionnaire-based Research

The “Survey Construct”: What Do I Want to Measure, or Why Am I Doing This?

Questionnaire Design: What Questions Am I Asking and Are These the Right Ones?

Study Population and Sampling: Who Am I Asking?

Data Collection: How Am I Getting My Responses?

Data Analysis: What Are My Responses?

Inference: What Does This All Mean?

Conclusions

References

Further Reading

(Lack of) Measurable Clinical or Knowledge Gains From Resident Participation in Noon Conference

Attitudes of Radiology Program Directors Toward MD-PhD Trainees, Resident Research Productivity, and Dedicated Research Time

Best Practices From the APDR