This page is historical material reflecting the Feedback Loop Blog as it existed on September 22, 2009. This page is no longer updated and links to external websites and some internal pages may not work.
September 22, 2009

Archived: The New Scoring System

At the recent meeting of the National Advisory General Medical Sciences Council, our Council members had their first opportunity to examine summary statements using the new peer review scoring system. Many aspects of the new scoring system are unfamiliar, including the use of overall impact scores as integers from 10 (best) to 90 (worst). A summary of the new scoring system is well described in a scoring system and procedure document, and an earlier version of this was shared widely with reviewers. As background, I compiled some data for approximately 300 NIGMS R01 applications reviewed under the new system.

This plot shows the distribution of overall impact scores along with the corresponding percentiles.

This plot shows the distribution of overall impact scores along with the corresponding percentiles. Note the relative spread of percentile scores at a given impact score. This spread is due to the fact that percentiles are determined independently for each study section that considered 25 or more R01 applications. Otherwise, percentiles are determined across the overall pool of R01 applications reviewed by the Center for Scientific Review. For comparison, here is a plot of a similar number of NIGMS R01 applications reviewed using the old scoring system.

A plot of a similar number of NIGMS R01 applications reviewed using the old scoring system.

Note the similar spread of percentiles at a given score due to study section-specific percentiling. I would like to mention another major change as a result of the NIH Enhancing Peer Review effort. You must use restructured application forms and instructions, including a 12-page length limit for R01s, for applications due on or after January 25, 2010. For details, see the recent NIH Guide notice. We plan to post updates about these changes as key dates approach.

About the Author

Jeremy Berg

As former NIGMS director, Jeremy oversaw the Institute’s programs to fund biomedical research and to train the next generation of scientists. He was a leader in many NIH-wide activities and also found time to study a variety of molecular recognition processes in his NIH lab.