Criteria for the Evaluation of Structure Determination Proposals

Release Date:
2/10/1999
Contact:
N/A

These "criteria" are meant as guidelines to aid reviewer discussion and do not replace the five official NIH rating criteria: Significance, Approach, Innovation, Investigator, and Environment. Rather, they are intended to help reviewers to evaluate and weight appropriately those criteria in the context of structure determination.

Background

An evaluation of project feasibility is a crucial component in the review of any research grant proposal. The preliminary studies reported by an applicant naturally provide important information for this evaluation. In principle, each proposal should carefully be examined on its own specific merits, but in practice there is a tendency to define de facto criteria for commonly encountered classes of experiments. This led, in the case of crystallography, to the dictum "no crystal, no grant" and a related clich� for NMR structure determination. While such criteria may have a certain utility, they are inappropriate if applied indiscriminately. Allowance has to be made for thefact that structural projects can progress extremely rapidly, especially when recombinant DNA methods for producing tractable molecules are in the hands of structural biologists. Requirements that are too stringent tend to reward risk-free research at the expense of innovation.

Proposed Criteria

In the absence of specific rules, what criteria should be used to assess the merits of structural biology projects that are still in their formative stages? The most important questions that need to be addressed are: how interesting the biology is, and what new insights into biological function might result. Preliminary evidence supporting the plausibility of the proposed research should be taken into account, even if it does not assure ultimate success. Much will also depend on the nature of the proposed research project.

For example, a proposal in which NMR studies are fully integrated into a much broader project that will probably yield results of biological significance even if the NMR structure cannot be completed, should be viewed very differently from a proposal that would produce no results of significance in the absence of a structure. To give a specific example, a proposal in which molecular biological and biochemical experiments are deemed likely, on the basis of preliminary data, to lead to valuable insights into the domain structure of some important biomolecule and to allow mapping of key binding sites might well be viewed favorably, even if the ultimate goal is NMR structure determination of the domains and no preliminary spectra are available to establish feasibility. Of course, the requirements of plausibility must still be met, e.g., there must be reason to expect that the domains will be of a size suitable for NMR.

Consideration should also be given to the track record of the investigators involved. The previously demonstrated expertise of an X-ray crystallographer or NMR spectroscopist in carrying out successful structure analyses in other systems is no guarantee of future success, but is clearly a factor that should be taken into account. In the case of a starting independent investigator, reviewers could consider whether the investigator's experience as a student or postdoc clearly qualifies him/her to carry out the proposed research.

In X-ray crystallography, one can anticipate proposals that will fall into distinct categories. Consideration of some characteristic examples may be helpful. First, there are instances when preliminary results, such as well-diffracting crystals, might be required. For example, if a structural biologist receives a protein from a collaborator but lacks a facility for biochemical and molecular biological manipulation, crystals might be needed for assurance that the structure determination is actually plausible.

For different reasons, a project in structure-based drug design would likely require appropriate crystals, if not a solved high-resolution structure and a lead compound as well, to give confidence in feasibility.

On the other hand, a well planned, comprehensive crystallographic study on a system defined by cloned gene products and some limited biochemical characterization, but no crystals, could nevertheless be seen as plausible if expression systems and purification procedures, including assays of activity, have been established and there is evidence that the investigator has the capability to modify the proteins enzymatically and to change expression constructs. It is then plausible that crystals of meaningful fragments will be forthcoming.

There are other cases where the status of a given field may be such that preliminary work is itself a defendable research project even while the ultimate goal is structure determination. The crystallization of membrane proteins either in two-dimensional sheets for electron crystallography or in three-dimensional crystals are such examples.

Finally, it is useful to distinguish a proposals' primary aims from its longer-term, secondary aims. Thus, an aim of crystal structure analysis in a biochemistry proposal or a proposal for crystallographic analyses of an NMR sample could be a plausible extension of the work, and should not discredit the proposal as a whole.

Summary

Proposed studies in structural biology need to be considered within the broad context of both the proposal and the underlying science. It is important not to discriminate against promising proposals that seem likely to yield results of biological significance just because they contain an X-ray or NMR component for which success is not assured. In these days of intense competition for funding, it is too easy to apply the clich� "no crystal, no grant" or "no NMR spectrum, no support" to help differentiate among the many outstanding proposals. However, such a policy tends to assure that, in many cases, funding is not available when it is needed most, namely in a project's formative phases, which ultimately will set the stage for the successful determination of a given structure and associated biological insights.