Rationale and Objectives
This study aimed to evaluate associations between traditional and advanced bibliometric indices with academic rank for radiologists in the United States.
Methods
Faculty web pages were searched to classify 538 members of the Association of University Radiologists as assistant ( n = 212), associate ( n = 128), or full ( n = 198) professors. Radiologists’ publication and citation records were extracted from Scopus to compute the following indices: publication count, citation count, h-index, i-10 index, h c -index, m-quotient, e-index, and g-index. Analysis of variance, multivariable logistic regression, and receiver operating characteristic curve analysis were performed.
Results
All indices were significantly different among the three groups ( P ≤.001), progressively increasing with increasing rank (eg, mean publication count of 17, 41, and 128 among assistant, associate, and full professors, respectively; mean citation count of 205, 687, and 3622, respectively; mean h-index of 5, 11, and 27, respectively). At multivariable analysis, the h-index (reflecting publications and citations) was a strong significant independent positive predictor of associate (β=+0.32, P <.001) or full professor (β=+0.26, P <.001) status, whereas the m-quotient (adjusted h-index that is greater for more rapid publication) was a strong significant independent negative predictor of associate (β=−1.87, P = .009) or full professor (β =−4.97, P <.001) status. The models exhibited moderate goodness-of-fit (r 2 = 0.534–0.655; P <.001). The model for predicting at least associate professor achieved area under the curve 0.876 (sensitivity 74.6%, specificity 88.8%). The model for predicting full professor achieved area under the curve 0.925 (sensitivity 85.5%, specificity 86.1%).
Conclusion
When controlling for the h-index, more rapid publication, as indicated by the m-quotient, was negatively associated with radiologists’ academic rank, indicating the additional influence of career duration in promotions decisions.
Introduction
Decisions by academic faculty regarding when to seek promotion, as well as decisions by institutional committees concerning when to award such promotions, are challenging. Faculty’s peer-reviewed publication record is widely applied for guiding such processes. However, efforts to objectively assess one’s publication record can be difficult as well given a paucity of data indicating the expected publication performance associated with various academic ranks in a given discipline. Such data could be useful for suggesting approximate benchmarks that could be applied as a guide when evaluating this one component of an academic physician’s portfolio. A prior investigation of research output among academic radiologists identified both the number of publications as well as the h-index to be significantly predictive of academic rank . The h-index, first described by Hirsch in 2005 , is intended to improve upon deficiencies of the simple publication count by also reflecting citations to one’s publications as a measure of impact. Specifically, the h-index is defined as the maximal number of publications that each has at least h citations , such that an investigator with an h-index of 15 has 15 publications with at least 15 citations, in addition to any number of additional publications with 15 or fewer citations. The h-index has similarly been demonstrated as a useful measure of academic performance in other medical disciplines and indeed has become recognized as a metric considered by promotions committees .
Although the h-index has performed favorably as a measure of academic performance, the h-index also has a number of its own limitations. Of note, it does not reflect a number of aspects of the timing of one’s publications, such as the duration over which one produced publications or the recency of such publications . Also, the h-index does not reflect the number of publications having fewer citations than the h-index for the given individual, or the number of citations occurring beyond the individual’s h-index. To address such concerns, an array of advanced bibliometric indices has been introduced more recently, including the i-10 index, h c -index, m-quotient, e-index, and g-index. These advanced indices have received far less formal investigation than traditional metrics such as the publication count or h-index. However, one study suggested added value of some of these indices for predicting National Institutes of Health funding among radiological investigators . It is possible that these advanced indices could also serve as better predictors of academic rank and complement the traditional metrics in providing benchmarks associated with various academic ranks. Thus, the purpose of this study was to evaluate associations among traditional and advanced bibliometric indices with academic rank for radiologists in the United States.
Methods
Study Sample
Get Radiology Tree app to read full this article<
Data Collection
Get Radiology Tree app to read full this article<
Table 1
Summary of Bibliometric Indices \*
Index Definition Direct indices Publication count Total number of publications by investigator Citation count Total number of citations to investigator’s publications Composite indices h-index The number of publications by an investigator having at least h citations, while all remaining publications have no more than h citations i-10 index The number of publications by an investigator that have been cited at least 10 times Time-adjusted composite indices h c -index A “contemporary” version of the h-index in which each publication’s citation count is multiplied by four and then divided by the number of years since publication, thereby giving greater weight to more recently published articles m-quotient h-index divided by the number of years since an investigator’s first publication † Composite indices adjusted for highly cited articles e-index Average number of citations beyond the h-index for those articles included in the h-index; provides a measure of measure of excess citations not considered by the h-index † g-index Maximal number of publications that have received an average of g citations; accounts for all citations included in both the h-index and the e-index
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Statistical Assessment
Get Radiology Tree app to read full this article<
Results
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Table 2
Summary of Bibliometric Indices for Academic Radiologists, Stratified by Academic Rank (Expressed as Mean ± Standard Deviation)
Measure Assistant Professor Associate Professor Full Professor_P__n_ 212 128 198 Bibliometric indices Publication count 17 ± 23 41 ± 40 128 ± 137 <.001 Citation count 205 ± 587 687 ± 1070 3622 ± 4929 <.001 h-index 5 ± 5 11 ± 8 27 ± 17 <.001 i-10 index 5 ± 11 14 ± 16 65 ± 78 <.001 h c -index 4 ± 3 7 ± 5 13 ± 8 <.001 m-quotient 0.41 ± 0.27 0.59 ± 0.39 0.86 ± 0.51 <.001 e-index 8 ± 8 16 ± 12 34 ± 20 <.001 g-index 9 ± 9 19 ± 15 47 ± 28 <.001
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Table 3
Summary of Multivariable Logistic Regressions and ROC Curve Analyses
Rank Independent Predictors \* r 2 AUC Sensitivity Specificity Associate or full professor h-index (β=+0.32, P <.001)
publication count (β=+0.02, P = .036)
i-10 index (β=−0.05, P = .043)
m-quotient (β=−1.87, P = .009) 0.534 0.884 77.0% 88.2% Full professor h-index (β =+0.26, P <.001)
m quotient (β =−4.97, P <.001) 0.655 0.928 86.4% 85.9%
AUC, area under the curve; ROC, receiver operating characteristic.
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Discussion
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
References
1. Rad A.E., Brinjikji W., Cloft H.J., et. al.: The H-index in academic radiology. Acad Radiol 2010; 17: pp. 817-821.
2. Hirsch J.E.: An index to quantify an individual’s scientific research output. Proc Natl Acad Sci USA 2005; 102: pp. 16569-16572.
3. Ball P.: Achievement index climbs the ranks. Nature 2007; 448: pp. 737.
4. Carpenter C.R., Cone D.C., Sarli C.C.: Using publication metrics to highlight academic productivity and research impact. Acad Emerg Med 2014; 21: pp. 1160-1172.
5. Benway B.M., Kalidas P., Cabello J.M., et. al.: Does citation analysis reveal association between h-index and academic rank in urology?. Urology 2009; 74: pp. 30-33.
6. Pagel P.S., Hudetz J.A.: H-index is a sensitive indicator of academic activity in highly productive anaesthesiologists: results of a bibliometric analysis. Acta Anaesthesiol Scand 2011; 55: pp. 1085-1089.
7. Poynard T., Thabut D., Jabre P., et. al.: Ranking hepatologists: which Hirsch’s h-index to prevent the “e-crise de foi-e”?. Clin Res Hepatol Gastroenterol 2011; 35: pp. 375-386.
8. Quigley M.R., Holliday E.B., Fuller C.D., et. al.: Distribution of the h-index in radiation oncology conforms to a variation of power law: implications for assessing academic productivity. J Cancer Educ 2012; 27: pp. 463-466.
9. Svider P.F., Choudhry Z.A., Choudhry O.J., et. al.: The use of the h-index in academic otolaryngology. Laryngoscope 2013; 123: pp. 103-106.
10. Durieux V., Gevenois P.A.: Bibliometric indicators: quality measurements of scientific publication. Radiology 2010; 255: pp. 342-351.
11. Rosenkrantz A.B., Jiang A.: Associations Between NIH Funding and Advanced Bibliometric Indices Among Radiological Investigators. Acad Radiol 2016; 23: pp. 669-674.
12. Association of University Radiologists : Full membership requirements. Available at: https://www.aur.org/Secondary.aspx?id=421 Accessed August 4, 2016
13. Elsevier : Scopus. Available at: http://www.scopus.com Accessed August 4, 2015
14. Choudhri A.F., Siddiqui A., Khan N.R., et. al.: Understanding bibliometric parameters and analysis. Radiographics 2015; 35: pp. 736-746.
15. Rezek I., McDonald R.J., Kallmes D.F.: Is the h-index predictive of greater NIH funding success among academic radiologists?. Acad Radiol 2011; 18: pp. 1337-1340.
16. Rad A.E., Shahgholi L., Kallmes D.: Impact of self-citation on the H index in the field of academic radiology. Acad Radiol 2012; 19: pp. 455-457.
17. Walker B., Alavifard S., Roberts S., et. al.: Inter-rater reliability of h-index scores calculated by Web of Science and Scopus for clinical epidemiology scientists. Health Info Libr J 2016; 33: pp. 140-149.
18. Silvestre J., Agarwal D., Lee D.I.: Analysis of National Institutes of Health funding to departments of urology. Urology 2016; 91: pp. 6-11.