Rationale and Objectives
To compare the utility of a search engine, Google, with other medical and non-medical, web-based resources for identifying specific medical information.
Materials and Methods
This institutional review board–approved case cross-over study randomly assigned 89 medical student volunteers to use either Google or any other web-based resource (excluding Google) to research 10 advanced medical questions in a multiple choice exam. Primary outcome measures were resource efficiency (inversely related to number of links used to identify the correct answer for each question) and correctness (number of correct answers/total number of questions answered). For Google searches, the sites providing the information in question were also evaluated.
Results
The most frequently selected non-Google resources were Yahoo ( n = 531), Ask.com ( n = 110), and the interactive encyclopedia Wikipedia.com ( n = 74). Google was more efficient than all other resources (1.50 vs. 1.94 mean links, P < .0001), with no significant difference in correctness (97% [756/780] vs. 96% [747/780], P = .16). After a Google search, the four most common categories of sites that provided the correct answer were dictionary/encyclopedia sites, medical websites, National Library of Medicine resources, or journal websites. Yahoo was less efficient than Google (1.90 vs. 1.54 mean links, P < .0001). However, non-Google search engines were more efficient than web sites (eg, Wikipedia, medical websites) and PubMed (1.87 vs. 2.54 mean links, P = .0004).
Conclusion
Google is an efficient web resource for identifying specific medical information, by guiding users to an array of medical resources.
Data reported in 2006 revealed that 99% of practicing US physicians who participated in a survey use the Internet, and 95% use search engines to identify medical information ( ). With respect to preferred web resources, most physicians (92%) in a recent Missouri survey reported that they favor a specific website (including research databases) over commercial search engines to “gather medical information” ( ). Nearly one-third (32%) preferred sites that present “edited data,” including UpToDate ( www.uptodate.com ), Medscape ( www.medscape.com ), WebMD ( www.webmd.com ), MD Consult ( www.mdconsult.com ), and eMedicine ( www.emedicine.com ), whereas 27% indicated that they favor research databases, such as PubMed ( www.pubmed.org ) ( ). Limitations of generalized search engines include physicians’ concerns about the accuracy of the information identified ( ) and evidence that medical search engines identify websites of higher quality than generalized search engines ( ).
Despite these limitations, generalized search engines are used increasingly to direct web users to the medical literature. Google was the most frequent source of search engine referrals (including PubMed) to articles in the journals hosted by HighWire in 2006 ( ) and to the British Medical Journal website in 2005 ( ). However, commercial search engines access a broad assortment of non-medical and medical web resources beyond journals. Although the disadvantage of a search engine is a concern over the quality of the content presented, medical websites have limitations as well when searching for medical information; the resource selected may not contain information on the topic of interest. With PubMed, navigating through a chronologic list of results can be time-consuming, access to the full text of an article may require a fee, and the article may not ultimately provide the specific information needed.
Get Radiology Tree app to read full this article<
Materials and methods
Study Population
Get Radiology Tree app to read full this article<
Medical Questions
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Study Design
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Data Collection
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Statistical Analysis
Get Radiology Tree app to read full this article<
Results
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Table 1
Results According to Initiating Search Source for the non-Google Arm
Initiating Source Total No. of Searches Failed Searches Data Missing Incorrect Answers Percent Correct Answers ⁎ Mean Links to Correct Answer † Search engines Yahoo 531 9 3 20 96% (499/519) 1.92 Ask 110 6 1 6 94% (97/103) 1.70 Altavista 45 5 1 2 95% (37/39) 2.19 Microsoft ‡ 33 2 NA 1 97% (30/31) 1.45 Dogpile 11 1 NA 1 90% (9/10) 1.56 Lycos 4 0 1 0 100% (3/3) 2.00 Onelook 3 0 NA 0 100% (3/3) 1.00 Webcrawler 2 0 NA 0 100% (2/2) 1.00 Metacrawler 2 0 NA 0 100% (2/2) 2.00 Other resources Wikipedia 74 28 NA 2 96% (44/46) 2.13 eMedicine 8 5 NA 0 100% (3/3) 2.29 MDConsult 7 3 NA 0 100% (4/4) 4.29 Medline Plus 4 2 NA 0 100% (2/2) 1.50 UptoDate 2 2 NA NA NA 4.50 NCBI 2 0 NA 0 100% (2/2) 8.00 PubMed 10 2 NA 1 88% (7/8) 2.71
Excluded: Six questions that did not list the source, six sources used only once, and all questions from participants who potentially performed a Google-powered search (using www.answers.com or www.aol.com ) for any question ( n = 30).
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Table 2
Demographics of Medical Student Participants
Google First Non-Google First Total No. of Participants 42 (47%) 47 (53%) 89 (100%) Gender Female 20 31 51 (57%) Male 22 16 38 (43%) Medical Education Level 1 11 9 20 (22%) 2 11 11 22 (25%) 3 10 11 21 (24%) 4 10 16 26 (29%) Age 20–25 y 30 26 56 (63%) 25–30 y 10 18 28 (32%) >30 y 1 3 4 (5%) Blank 1 0 1 (1%) Internet use: Multiple times per day 25 27 52 (58%) Daily 4 10 14 (16%) Weekly 10 8 18 (20%) Monthly 2 2 4 (5%) Blank 1 0 1 (1%)
Table 3
Most Common Online Resources Reported in Response to the Question “My Preferred Website for Medical Information is”
Web Resource No. of Students ( n = 89) High–frequency ⁎ Web Users ( n = 52) UpToDate 37 31 Google 33 18 eMedicine 25 17 Wikipedia 17 9 PubMed 15 9
Some students listed more than one resource.
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Correctness
Get Radiology Tree app to read full this article<
Efficiency
Get Radiology Tree app to read full this article<
Table 4
Comparison of Google, All non-Google Search Engines, and Websites/PubMed
Resource Total No. of Searches Failed Searches Correct/Total Questions Answered ⁎ Mean Links to Correct Answer † Google 890 0% (0/884) 97% (855/884) 1.50 Non-Google search engines 742 3% (24/733) 96% (682/712) 1.87 Websites, PubMed 112 41% (45/111) 96% (64/67) 2.54
Among the alternatives, search engines were more efficient than websites and PubMed ( P = .0004).
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
End Sites Following a Google Search
Get Radiology Tree app to read full this article<
Table 5
End Sites After Searches Initiated with Google
Web-based Resource Category or Name No. Mean Links to Correct Answer for Common End Sites Encyclopedia or dictionary total 126 Wikipedia 62 1.66 Who Named It 30 1.44 Dorland’s Medical Dictionary 18 Medterms 10 The Free Dictionary 4 Other 2 Medical website total 106 Emedicine 49 1.45 Disease-specific website 18 Merck Source 14 Medscape 3 WebMD 2 Medical Society Site 1 Hospital website 1 Other 18 National Library of Medicine Resources total 59 PubMed 30 1.93 Medline 2 Medline Plus 16 1.65 Other 11 Journal website ( n = 36) or citation of article ( n = 1) 37 1.90 State Department of Health website 14 College/university website 2 Other 8
Students recorded the site that ultimately provided the correct answer for 352 of the 855 questions researched accurately with Google. The difference between mean links for eMedicine and journals or PubMed was significant ( P = .04).
Get Radiology Tree app to read full this article<
Discussion
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
References
1. American Medical News: Doctors are going digital. http://www.ama-assn.org/amednews/2006/05/22/bicb0522.htm Accessed April 5, 2007.
2. De Leo G., LeRouge C., Ceriani C., et. al.: Websites most frequently used by physician for gathering medical information. AMIA Annu Symp Proc 2006; 902:
3. Maloney S., Ilic D., Green S.: Accessibility, nature and quality of health information on the internet: a survey on osteoarthritis. Rheumatology 2005; 44: pp. 382-385.
4. Steinbrook R.: Searching for the right search—reaching the medical literature. N Engl J Med 2006; 354: pp. 4-7.
5. Lindberg D.A.B., Sack J., Steinbrook R.: Searching the medical literature. N Engl J Med 2006; 354: pp. 2393.
6. Giustini D.: How Google is changing medicine. BMJ 2005; 331: pp. 1487-1488.
7. Levine M.S., Halvorsen R.A.: Esophageal cancer.Gore R.M.Levine M.S.Laufer I.Textbook of gastrointestinal radiology.1994.WB Saunders CompanyPhiladelphia:pp. 447-448.
8. Sartoris D.J., Resnick D.: Plain film radiography: routine techniques.Resnick D.Bone and joint imaging.1989.WB Saunders CompanyPhiladelphia:pp. 50.
9. Kadir S.: Venous system.Diagnostic angiography.1986.WB Saunders CompanyPhiladelphia:pp. 574.
10. Kirks D.R., Caron K.H.: Gastrointestinal tract.Kirks D.R.Practical pediatric imaging: diagnostic radiology of infants and children.1991.Little Brown and CompanyBoston:pp. 869.
11. Warren Weissman B.N.: Imaging after bone surgery.Resnick D.Bone and joint imaging.1989.WB Saunders CompanyPhiladelphia:pp. 238.
12. Smith C.: Postoperative stomach and duodenum.Gore R.M.Levine M.S.Laufer I.Textbook of gastrointestinal radiology.1994.WB Saunders CompanyPhiladelphia:pp. 755.
13. Chin D.H.: Ultrasound evaluation of hydrops fetalis.Callen P.W.Ultrasonography in obstetrics and gynecology.1994.WB Saunders CompanyPhiladelphia:pp. 430.
14. Osborn A.G.: Inherited metabolic, white matter and degenerative diseases of the brain.Diagnostic neuroradiology.1994.Mosby–Year BookSt Louis:pp. 737-739.
15. Fraser R.S., Pare J.A.P., Fraser R.G., et. al.: Pulmonary disease caused by drugs, poisons, and inhaled toxic gases and aerosols.Synopsis of diseases of the chest.1994.WB Saunders CompanyPhiladelphia:pp. 775.
16. Connor D.H., Gibson D.W.: Infectious and parasitic diseases.Rubin E.Farber J.L.Pathology.1988.J.B. Lippincott CompanyPhiladelphia:pp. 350.
17. Alper B.S., White D.S., Ge B.: Physicians answer more clinical questions and change clinical decisions more often with synthesized evidence: a randomized trial in primary care. Ann Fam Med 2005; 3: pp. 507-513.
18. McKibbon K.A., Fridsma D.B.: Effectiveness of clinician-selected electronic information resources for answering primary care physicians’ information needs. J Am Med Inform Assoc 2006; 13: pp. 653-659.
19. Ely J.W., Osheroff J.A., Chambliss M.L., et. al.: Answering physicians’ clinical questions: obstacles and potential solutions. J Am Med Inform Assoc 2005; 12: pp. 217-224.
20. Yu H., Kaufman D.: A cognitive evaluation of four online search engines for answering definitional questions posed by physicians. Pac Symp Biocomput 2007; 12: pp. 328-339.
21. Ripple A.S.: Expert Googling: best practices and advanced strategies for using Google in health sciences libraries. Med Ref Serv Q 2006; 25: pp. 97-107.
22. Google Corporate Information: Google milestones. http://www.Google.com/corporate/history.html Accessed February 15, 2007.
23. Google Corporate Information: company overview. http://www.Google.com/corporate/index.html Accessed February 15, 2007.
24. Our search: Google technology. http://www.Google.com/technology/ Accessed April 17, 2007.
25. Bennett N.L., Casebeer L.L., Kristofco R.E., et. al.: Physicians’ internet information-seeking behaviors. J Contin Educ Health Prof 2004; 24: pp. 31-38.
26. Bennett N.L., Casebeer L.L., Zheng S., et. al.: Information-seeking behaviors and reflective practice. J Contin Educ Health Prof 2006; 26: pp. 120-127.
27. Jadad A.R., Gagliardi A.: Rating health information on the internet: navigating to knowledge or to Babel?. JAMA 1998; 279: pp. 611-614.
28. HONcode: Principles—quality and trustworthy health information. http://www.hon.ch/HONcode/Conduct.html Accessed January 16, 2008.
29. Putting clinical information into practice. http://www.uptodate.com/ Accessed April 17, 2007.
30. Patel M.R., Schardt C.M., Sanders L.L., et. al.: Randomized trial for answers to clinical questions: evaluating a pre-appraised versus a MEDLINE search protocol. J Med Libr Assoc 2006; 94: pp. 382-386.
31. Ilic D., Bessell T.L., Silagy C.A., et. al.: Specialized medical search-engines are no better than general search-engines in sourcing consumer information about androgen deficiency. Hum Reprod 2003; 18: pp. 557-561.
32. Tang H., Ng J.H.K.: Googling for diagnosis—the use of Google as a diagnostic aid: internet based study. BMJ 2006; 333: pp. 1143-1145.
33. Griffiths K.M., Christensen H.: Website quality indicators for consumers. J Med Internet Res 2005; 7: pp. e55.