Skip to Content


Center for Scientific Review Announcement of Requirements and Registration for Strategies To Strengthen Fairness and Impartiality in Peer Review

Document Details

Information about this document as published in the Federal Register.

Document Statistics
Document page views are updated periodically throughout the day and are cumulative counts for this document including its time on Public Inspection. Counts are subject to sampling, reprocessing and revision (up or down) throughout the day.
Published Document

This document has been published in the Federal Register. Use the PDF linked in the document sidebar for the official electronic format.

Start Preamble Start Authority

Authority: 15 U.S.C. 3719.

End Authority


The National Institutes of Health (NIH) Center for Scientific Review (CSR) is issuing a challenge titled “Strategies to Strengthen Fairness and Impartiality in Peer Review.” This notice provides information regarding requirements and registration for the challenge.


Submission Period: May 5, 2014 through 11:59 p.m. Eastern Time, June 30, 2014.

Judging Period: July 16, 2014 through August 29, 2014.

Winners Announced: September 2, 2014.


Details on the NIH/CSR Peer Review process and current reviewer training materials can be found on the Reviewer Resources tab at (See NIH Peer Review Process Revealed and Resources for Reviewers). For questions about this challenge, please contact or call at 301-300-3839.

Start Further Info


Monica Basco, Center for Scientific Review, phone: 301-300-3839 or email at

End Further Info End Preamble Start Supplemental Information


The mission of the NIH is to seek fundamental knowledge about the nature and behavior of living systems and to apply that knowledge to enhance health, lengthen life, and reduce the burdens of illness and disability. NIH has a longstanding and time tested system of peer review to identify the most promising biomedical research. The core values of NIH Peer Review are (1) expert assessment, (2) transparency, (3) impartiality, (4) fairness, (5) confidentiality, (6) integrity, and (7) efficiency. These values drive NIH to seek the highest level of ethical standards and form the foundation for the laws, regulations, and policies that govern the NIH peer review process.

The NIH's Center for Scientific Review is issuing a challenge titled “Strategies to Strengthen Fairness and Impartiality in Peer Review,” under and consistent with sections 492 and 492A of the Public Health Service Act and federal regulations governing “Scientific Peer Review of Research Grant Applicants and Research and Development Contract Projects” (42 CFR Part 52h). The goal of this challenge is to seek ideas for strengthening reviewer training practices to enhance impartiality and fairness in peer review of grant applications. Research findings (Ginther et al, 2011; 2012) suggest a discrepancy in success rates for NIH R01 grant funding between White applicants and Black applicants, suggesting possible bias in the peer review process. This challenge aims to address that discrepancy by soliciting ideas for reviewer training methods to enhance fairness and impartiality in peer review. It directly supports the mission of CSR to ensure that the best and brightest minds have an equal opportunity to contribute to the realization of our national research goals.

Subject of Challenge: The subject of this challenge is to seek ideas for reviewer training methods to enhance fairness and impartiality in peer review.

The NIH Peer Review process is a dual peer review system used by NIH to award research funds. Under this system, each application must undergo two levels of NIH Peer Review. The first level of review is carried out by a Scientific Review Group (SRG) composed primarily of non-federal scientists who have expertise in relevant scientific disciplines and current research areas. The second level of review is performed by Institute and Center National Advisory Councils or Boards that make recommendations on priority areas of research, pending policy, and funding of particular applications. Councils are composed of both scientific and public representatives chosen for their expertise, interest, or activity in matters related to health and disease. Only applications that are recommended for approval by both the SRG and the Council may be recommended for funding. Final funding decisions are made by the director of the relevant NIH Institute or Center.

NIH recognizes a unique and compelling need to promote diversity in the NIH-funded biomedical research workforce. The NIH expects efforts that diversify the workforce to lead to the recruitment of the most talented researchers from all groups, improve the quality of the training environment, balance and broaden the perspective in setting research priorities, and improve the Nation's capacity to address and eliminate health disparities. Yet, despite longstanding efforts from the NIH and other entities across the biomedical and behavioral research landscape to enhance the diversity of workforce, more work remains to be done. Recent studies (Ginther et al., 2011; 2012) have shown that African American researchers are less likely than White researchers to receive NIH R01 grant funding. These findings have raised concerns regarding the degree to which reviewers are demonstrating the core values of impartiality and fairness.

This challenge seeks ideas for reviewer training methods aimed at enhancing fairness and impartiality in peer review. Submissions need not include fully developed training materials (See complete submission requirements below). However, ideas should be provided in sufficient detail to assess their ability to address and promote fairness and impartiality in the peer review of grant applications with regards to: gender, race/ethnicity, institutional affiliation, area of science, and amount of research experience of the applicant.

Eligibility Rules for Participating in the Challenge: The challenge is open to any individual, group of individuals, or entity (each referred to in this notice as a “participant”) who meets the eligibility criteria below. There is no limit to the number of entries a participant can submit.Start Printed Page 25607

To be eligible to win a prize under this challenge:

(1) The participant shall have registered to participate in the competition under the rules promulgated by CSR as described in this notice.

(2) The participant (including each individual participating as a member of group participant) shall have complied with all the requirements under this section.

(3) In the case of a private entity, the entity shall be incorporated in and maintain a primary place of business in the United States, and in the case of an individual, whether participating singly or in a group, each shall be a citizen or permanent resident of the United States.

(4) Individuals (whether competing alone or as part of a group) who are younger than 18 must have their parent or legal guardian complete the Parental Consent Form. The form can be found on the Challenge Web page at

(5) The participant may not be a Federal entity or Federal employee acting within the scope of his or her employment.

(6) The participant shall not be an HHS employee working on their applications or submissions during assigned duty hours.

(7) The participant shall not be an employee of the National Institutes of Health, the Center for Scientific Review, a member of the Subcommittee on Peer Review or any other party involved with the design, production, execution, or distribution of the Challenge or their immediate family (spouse, parents or step-parents, siblings and step-siblings and children and step-children).

(8) Federal grantees may not use Federal funds to develop COMPETES Act challenge applications unless consistent with the purpose of their grant award.

(9) Federal contractors may not use Federal funds from a contract to develop COMPETES Act challenge applications or to fund efforts in support of a COMPETES Act challenge submission.

(10) CSR reserves the right to cancel, suspend, modify the challenge and/or not award a prize if no submissions are deemed worthy.

(11) CSR will claim no rights to intellectual property. By participating in this challenge, participant grants to CSR an irrevocable, paid-up, royalty-free, nonexclusive worldwide license to post, link to, share, and display publicly the submission on the Web, newsletters or pamphlets, and other information products such as a future Funding Opportunity Announcement or other study to develop the methodology. In addition, CSR may incorporate proposed ideas into a future Request for Applications (RFA), Request for Proposals (RFP) or an implemented study to develop the methodology, but an award of a prize does not guarantee the proposed idea will be implemented.

(12) By participating in this challenge, participant agrees that the submission is participant's original work and that all proposed ideas are participant's original effort. It is the responsibility of the participant to obtain any rights necessary to use, disclose, or reproduce any intellectual property owned by third parties and incorporated in the entry for all anticipated uses of the submission. Submissions must not violate or infringe upon any copyright or any other rights of other parties, including, but not limited to, privacy, publicity or intellectual property rights, or material that constitutes copyright or license infringement.

(13) By participating in this challenge, each participant (including each individual making up a group participant) agrees to assume any and all risks and waive claims against the Federal Government and its related entities, except in the case of willful misconduct, for any injury, death, damage, or loss of property, revenue, or profits, whether direct, indirect, or consequential, arising from participation in this prize challenge, whether the injury, death, damage, or loss arises through negligence or otherwise.

(14) Based on the subject matter of the challenge, the type of work that it will possibly require, as well as an analysis of the likelihood of any claims for death, bodily injury, or property damage, or loss potentially resulting from challenge participation, participants are not required to obtain liability insurance or demonstrate financial responsibility in order to participate in this challenge.

(15) By participating in this challenge, each participant agrees to indemnify the Federal Government against third party claims for damages arising from or related to challenge activities.

(16) An individual shall not be deemed ineligible because the individual used Federal facilities or consulted with Federal employees during this challenge if the facilities and employees are made available to all individuals participating in the challenge on an equitable basis.

(17) In the case of groups, a single, individual group member will submit the submission on behalf of the group and certify that the submission meets all challenge rules.

(18) The decision of the award approving official is final and cannot be contested. The award approving official is the Director of the Center for Scientific Review.

Submission Process for Participants: Participants should submit all entry materials to

Amount of the Prize: CSR may award up to two prizes. A First Prize in the amount of $10,000 and a Second Prize in the amount of $5,000 may be given. Each submission is eligible for only one prize (i.e., a single submission cannot win more than one prize for this challenge).

Prizes awarded under this challenge will be paid by electronic funds transfer and may be subject to Federal income taxes. HHS will comply with the Internal Revenue Service withholding and reporting requirements, where applicable. If a group or entity is selected as a winner, CSR will pay the prize to an individual representative of the group designated in the cover letter required as part of the submission. To the extent applicable, it is this individual's responsibility to distribute the prize to group (or entity) members.

Basis Upon Which Submissions Will be Evaluated: After CSR receives and de-identifies the submissions, the submissions will be evaluated according to a two-stage process: (1) Technical merit will be evaluated for potential to enhance fairness and impartiality in peer review (High, Medium, Low impact) by a panel of experts in fields relevant to peer review, evaluation and training methods, and bias in assessment, and (2) High Impact submissions will be evaluated and rank ordered based on the judging criteria (see judging criteria below) by a panel of judges comprised of federal employees who will recommend the winning entries.

The final awards will be approved by the Director of the Center for Scientific Review; provided, however, that CSR reserves the right to cancel, suspend, modify the challenge and/or not award a prize if no submissions are deemed worthy.

The judging criteria for this challenge are as follows.

  • Demonstrates general knowledge of peer review practices.
  • Grounded in the empirical literature.
  • Feasible for implementation with reviewers in the NIH Peer Review system.
  • The proposed methods could be delivered to reviewers in a variety of delivery formats, including an electronic format.Start Printed Page 25608
  • Demonstrates an understanding of the literature on principles of learning/training.
  • Effectively moves theory to practice.
  • Provides evidence that supports the effectiveness of the approach in promoting fair and unbiased peer review.

Submission Requirements: This challenge is for the solicitation of ideas for reviewer training methods to strengthen reviewer fairness and impartiality in NIH Peer Review. Submissions, therefore, need not include fully developed training materials. The following materials must be emailed to or sent in hardcopy to the Office of the Director, Attention: Denise McGarrell, Center for Scientific Review, 6701 Rockledge Drive, Suite 3030, Bethesda, Maryland 20892 by the deadline. Incomplete submissions will not be considered. All submissions must be written in English.

  • Cover sheet with title of the submission and the participant's name or names of group members and contact information. In the case of groups (and entities), indicate one group member responsible for corresponding with CSR. Also indicate which group member will be responsible for receiving the prize for distribution, as applicable, among group members.
  • Challenge submission documents. Note: The 2-page challenge idea should be anonymous (i.e., not include identifying information of the participant). Submissions shall not exceed 2 single-spaced pages (not to include cover page, references or parental consent document, if applicable) and shall be constrained to no less than one inch margins and 11 pt. Ariel font. All submissions must be submitted in .docx (Word) format. Submissions should include the following sections:

Aims: Describe the goals for your proposed approach for reviewer training to enhance fairness and impartiality in peer review and the anticipated outcomes.

Approach: Provide a detailed description of your proposed methods and procedures. Describe how you might measure the effectiveness of your plan in accomplishing your proposed aims.

Implementation: Explain how your methods might be implemented as part of reviewer training. Include how your proposed method might be tested and, if effective, how it might be disseminated across the NIH.

  • As applicable, the signed Parental Consent Document.
  • Submissions not conforming to these specifications will be disqualified.


Ginther DK et al. (2011). Race, ethnicity, and NIH research awards. Science, 333 (1015-1019).

Ginther DK, Haak LL, Schaffer WT, & Kington R. (2012). Are race, ethnicity, and medical school affiliation associated with NIH R01 type 1 award probability for physician investigators? Academic Medicine, 87 (11), 1516-1524.

Start Signature

Dated: April 29, 2014.

Richard Nakamura,

Director, Center for Scientific Review, National Institutes of Health.

End Signature End Supplemental Information

[FR Doc. 2014-10203 Filed 5-2-14; 8:45 am]