Rationale and Objectives
Whereas data support the h index (reflecting both publications and citations) as an indicator of academic productivity, other advanced bibliometric indices aiming to address shortcomings of the h index remain poorly studied. Our objective was to compare the associations between bibliometric indices and total National Institutes of Health (NIH) grant funding among investigators within U.S. academic radiology departments.
Materials and Methods
NIH grant funding amounts for 400 NIH-funded investigators within radiology departments were obtained from Blue Ridge Institute for Medical Research. Investigators’ publications and associated citations were identified using Scopus. Indices computed for each investigator included: publication count, citation count, h index, i-10 index, h c index (h index adjusted for recency of publications), m quotient (h index adjusted for career duration), and e index and g index (both account for highly cited articles). Spearman correlations were performed between indices and funding. Multivariable linear regression was performed to identify significant independent predictors of funding.
Results
For MD investigators: the indices exhibited no-to-weak correlations with funding (r = 0.173–0.387); m quotient exhibited the largest correlation and was the only significant (albeit weak) independent predictor of funding ( P = 0.011). For PhD investigators: correlation with funding was weak for m quotient (r = 0.323), although moderate for other indices (r = 0.518–0.568); publication count exhibited highest correlation; publication count ( P < 0.001) and h c index ( P = 0.024) were significant independent predictors of funding.
Conclusions
Bibliometric indices were more strongly associated with grant funding for PhD than for MD radiology investigators, with publication count exhibiting the strongest association in the latter group. Time-weighted adjustments, as reflected by the m quotient and h c index, may improve efforts to predict funding using bibliometrics.
Introduction
An optimal approach for objectively measuring the research productivity of faculty working within an academic medical center is lacking. A straightforward and historically applied method is to simply perform a count of an individual investigator’s publications. However, this scheme fails to account for wide variability in quality and impact among publications. In 2005, Hirsch described the h index , a measure impacted not only by the number of publications by an investigator, but also by the citations to these articles. According to Hirsch’s original description, an investigator “has index h if h of his or her N p papers have at least h citations each and the other (N p − h ) papers have ≤ h citations each” . Accordingly, an investigator with an h index of 10 has 10 articles with at least 10 citations, along with any number of additional articles that all have less than (or equal to) 10 citations. Since its initial description, the h index has rapidly gained interest and acceptance in academic medicine, becoming applied as a measure of academic performance . Substantial associations were identified between the h index and other measures of academic achievement in numerous biomedical disciplines . Within radiology, the h index was observed to be predictive of academic rank as well as attainment of at least one grant . Currently, the h index is used by departments for assessing faculty productivity, academic medical centers for tenure and promotions decisions, and funding agencies in grant considerations .
Despite the recent enthusiasm for the h index, the measure has also received criticism. The h index favors more senior investigators, increasing as one gains in number of publications and associated citations over the course of a career . In this regard, it fails to consider the recency of one’s publications or the total time span over which they are published. In addition, all of an investigator’s publications with more citations than the h index contribute equally to the index, such that the h index fails to account for highly cited publications that have an even larger number of citations . Given these limitations, other advanced bibliometrics indices have been proposed. These include the i-10 index, h c index, m quotient, e index, and g index, which seek to complement or improve upon the h index through adjustments for the timing of publications or very highly cited articles .
Get Radiology Tree app to read full this article<
Methods
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Table 1
Summary of Bibliometric Indices Evaluated in This Study
Index Definition Direct indices Publication count Total number of publications by investigator Citation count Total number of citations to investigator’s publications Composite indices h index The number of publications by an investigator having at least h citations, whereas all remaining publications have no more than h citations i-10 index The number of publications by an investigator that have been cited at least 10 times Time-adjusted composite indices h c index A “contemporary” version of the h index in which each publication’s citation count is multiplied by four and then divided by the number of years since publication, thereby giving greater weight to more recently published articles m quotient H index divided by the number of years since an investigator’s first publication \* Composite indices adjusted for highly cited articles e index Average number of citations beyond the h index for those articles included in the h index; provides a measure of measure of excess citations not considered by the h index \* g index Maximal number of publications that have received an average of g citations; accounts for all citations included in both the h index and the e index
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Results
Get Radiology Tree app to read full this article<
Table 2
Summary of Total NIH Funding and Bibliometric Indices (Expressed as Mean ± Standard Deviation) for Radiological Investigators, Stratified by Graduate Degree
Measure MD PhD \* P__n 44 356 — Total NIH funding $628,509 ± $620,184 $627,563 ± $649,049 0.287 Bibliometric indices: Publication count 148 ± 108 119 ± 107 0.025 Citation count 6795 ± 10,515 3713 ± 5243 0.010 h index 34 ± 22 27 ± 17 0.015 i-10 index 81 ± 67 61 ± 63 0.014 h c index 22 ± 13 17 ± 9 0.006 m quotient 1.41 ± 0.65 1.19 ± 0.55 0.017 e index 49 ± 38 35 ± 22 0.013 g index 64 ± 46 48 ± 30 0.015
NIH, National Institutes of Health.
Get Radiology Tree app to read full this article<
Table 3
Correlations Between Radiology Investigator’s Bibliometric Indices and National Institutes of Health Grant Funding
Index MD PhD \* Publication count 0.224 0.568 Citation count 0.179 0.537 h index 0.173 0.541 i-10 index 0.207 0.554 h c index 0.233 0.518 m quotient 0.387 0.323 e index 0.183 0.496 g index 0.207 0.534
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Discussion
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
References
1. Hirsch J.E.: An index to quantify an individual’s scientific research output. Proc Natl Acad Sci U S A 2005; 102: pp. 16569-16572.
2. Ball P.: Achievement index climbs the ranks. Nature 2007; 448: pp. 737.
3. Carpenter C.R., Cone D.C., Sarli C.C.: Using publication metrics to highlight academic productivity and research impact. Acad Emerg Med 2014; 21: pp. 1160-1172.
4. Benway B.M., Kalidas P., Cabello J.M., et. al.: Does citation analysis reveal association between h index and academic rank in urology?. Urology 2009; 74: pp. 30-33.
5. Pagel P.S., Hudetz J.A.: H index is a sensitive indicator of academic activity in highly productive anaesthesiologists: results of a bibliometric analysis. Acta Anaesthesiol Scand 2011; 55: pp. 1085-1089.
6. Poynard T., Thabut D., Jabre P., et. al.: Ranking hepatologists: which Hirsch’s h index to prevent the “e-crise de foi-e”?. Clin Res Hepatol Gastroenterol 2011; 35: pp. 375-386.
7. Quigley M.R., Holliday E.B., Fuller C.D., et. al.: Distribution of the h index in radiation oncology conforms to a variation of power law: implications for assessing academic productivity. J Cancer Edu 2012; 27: pp. 463-466.
8. Svider P.F., Choudhry Z.A., Choudhry O.J., et. al.: The use of the h index in academic otolaryngology. Laryngoscope 2013; 123: pp. 103-106.
9. Rad A.E., Brinjikji W., Cloft H.J., et. al.: The h index in academic radiology. Acad Radiol 2010; 17: pp. 817-821.
10. Rezek I., McDonald R.J., Kallmes D.F.: Is the h index predictive of greater NIH funding success among academic radiologists?. Acad Radiol 2011; 18: pp. 1337-1340.
11. Durieux V., Gevenois P.A.: Bibliometric indicators: quality measurements of scientific publication. Radiology 2010; 255: pp. 342-351.
12. Choudhri A.F., Siddiqui A., Khan N.R., et. al.: Understanding bibliometric parameters and analysis. Radiographics 2015; 35: pp. 736-746.
13. Rad A.E., Shahgholi L., Kallmes D.: Impact of self-citation on the H index in the field of academic radiology. Acad Radiol 2012; 19: pp. 455-457.
14. Blue Ridge Institute for Medical Research : Ranking tables of NIH funding to US medical schools in 2014. Radiology PI; Available at: http://www.brimr.org/NIH_Awards/2014/RadiologyPI_2014.xls Accessed May 11, 2015
15. Blue Ridge Institute for Medical Research : From the Blue Ridge Institute for Medical Research (BRIMR.ORG). Available at: http://www.brimr.org/NIH_Awards/NIH_Awards.htm Accessed December 22, 2015
16. Steinberg M., McBride W.H., Vlashi E., et. al.: National Institutes of Health funding in radiation oncology: a snapshot. Int J Radiat Oncol Biol Phys 2013; 86: pp. 234-240.
17. Elsevier B.V.: Scopus. Available at: http://www.scopus.com Accessed on June 5, 2015
18. Rezek I., McDonald R.J., Kallmes D.F.: Pre-residency publication rate strongly predicts future academic radiology potential. Acad Radiol 2012; 19: pp. 632-634.
19. Vieira E.S., Gomes J.A.: A comparison of Scopus and Web of Science for a typical university. Scientometrics 2009; 81: pp. 587-600.