At the National Science Foundation, program managers decide whom to support, and for this they get specialist advice. But reviewers who understand a scientist's work are often competitors in the same field. Their advice ranges from honest to artfully misleading. The project manager tries to filter out the disinformation.
NIH lets its "study sections," panels of reviewers, choose which projects to fund, and trusts that panel members will keep each other honest. Each application is described and evaluated by study section members skilled in its field. The panel discusses it, then votes to accept or reject. If it accepts, as it most often does, it votes a priority score that largely determines if the project will be funded. Subsequent stages of review are either principally symbolic or concerned with whether proposed projects fit NIH's overall plan for research. No one at NIH has suggested to me that later reviews second-guess ...
