NIH peer review percentile scores are poorly predictive of grant productivity

The impacts of studies ranked in the 3rd to 20th percentile are more or less statistically indistinguishable. With current funding lines as low as 10th percentile, this means that many unfunded proposals are better than funded studies.

NIH peer review percentile scores are poorly predictive of grant productivity
DOI: 10.7554/eLife.13323.001

Peer review is widely used to assess grant applications so that the highest ranked applications can be funded. A number of studies have questioned the ability of peer review panels to predict the productivity of applications, but a recent analysis of grants funded by the National Institutes of Health (NIH) in the US found that the percentile scores awarded by peer review panels correlated with productivity as measured by citations of grant-supported publications. Here, based on a re-analysis of these data for the 102,740 funded grants with percentile scores of 20 or better, we report that these percentile scores are a poor discriminator of productivity. This underscores the limitations of peer review as a means of assessing grant applications in an era when typical success rates are often as low as about 10%.