This page is historical material reflecting the Feedback Loop Blog as it existed on
July 21, 2010. This page is no longer updated and links to external websites
and some internal pages may not work.
July 21, 2010
Archived: Even More on Criterion Scores: Full Regression and Principal Component Analyses
After reading yesterday’s post, a Feedback Loop reader asked for a full regression analysis of the overall impact score based on all five criterion scores. With the caveat that one should be cautious in over-interpreting such analyses, here it is:
As one might expect, the various parameters are substantially correlated.
A principal component analysis reveals that a single principal component accounts for 71% of the variance in the overall impact scores. This principal component includes substantial contributions from all five criterion scores, with weights of 0.57 for approach, 0.48 for innovation, 0.44 for significance, 0.36 for investigator and 0.35 for environment.
Here are more results of the full principal component analysis:
The second component accounts for an additional 9% of the variance and has a substantial contribution from approach, with significant contributions of the opposite sign for investigator and environment. The third component accounts for an additional 8% of the variance and appears to be primarily related to innovation. The fourth component accounts for an additional 7% of the variance and is primarily related to significance. The final component accounts for the remaining 5% of the variance and has contributions from investigator and environment of the opposite sign.

Pearson correlation coefficients of overall impact score and five criterion scores in a sample of 360 NIGMS R01 applications reviewed during the October 2009 Council round.

Principal component analysis of overall impact score based on the five criterion scores in a sample of 360 NIGMS R01 applications reviewed during the October 2009 Council round.