Metrics of research impact in astronomy: Predicting later impact from metrics measured 10-15 years after the PhD
This paper calibrates how metrics derivable from the SAO/NASA Astrophysics
Data System can be used to estimate the future impact of astronomy research
careers and thereby to inform decisions on resource allocation such as job
hires and tenure decisions. Three metrics are used, citations of refereed
papers, citations of all publications normalized by the numbers of co-authors,
and citations of all first-author papers. Each is individually calibrated as an
impact predictor in the book Kormendy (2020), "Metrics of Research Impact in
Astronomy" (Publ Astron Soc Pac, San Francisco). How this is done is reviewed
in the first half of this paper. Then, I show that averaging results from three
metrics produces more accurate predictions. Average prediction machines are
constructed for different cohorts of 1990-2007 PhDs and used to postdict 2017
impact from metrics measured 10, 12, and 15 years after the PhD. The time span
over which prediction is made ranges from 0 years for 2007 PhDs to 17 years for
1990 PhDs using metrics measured 10 years after the PhD. Calibration is based
on perceived 2017 impact as voted by 22 experienced astronomers for 510 faculty
members at 17 highly-ranked university astronomy departments world-wide.
Prediction machinery reproduces voted impact estimates with an RMS uncertainty
of 1/8 of the dynamic range for people in the study sample. The aim of this
work is to lend some of the rigor that is normally used in scientific research
to the difficult and subjective job of judging people's careers.