Skip Over Navigation Links

Clearinghouse for Training Modules to Enhance Data Reproducibility

In January 2014, NIH launched a series of initiatives to enhance rigor and reproducibility in research. As a part of this initiative, NIGMS, along with nine other NIH institutes and centers, issued the funding opportunity announcement RFA-GM-15-006 to develop, pilot and disseminate training modules to enhance data reproducibility. Graduate students, postdoctoral fellows and early stage investigators are the primary audiences for these training modules.

For the benefit of the scientific community, we will be posting the products of these grants on this Web site as they become available in the future.

In addition, we are sharing here a series of four training modules developed by NIH. These modules focus on integral aspects of rigor and reproducibility in the research endeavor, such as bias, blinding and exclusion criteria. The modules are not meant to be comprehensive, but rather are intended as a foundation to build on and a way to stimulate conversations, which may be facilitated by the use of the accompanying discussion materials. Currently, the modules are being integrated into NIH intramural training activities.

NIH Rigor and Reproducibility Training Modules

Introduction to the Modules [PDF, 110KB]

Module 1: Lack of TransparencyModule 1: Lack of Transparency
In order to reproduce someone else’s findings adequately, the experimental methods, rationale and other pertinent information must be accessible and understandable. This module highlights the need to include all relevant details in publications to ensure that other studies are able to build upon the research appropriately and accurately.
Lack of Transparency Discussion Material [PDF, 97.2KB]

Module 2: Blinding and RandomizationModule 2: Blinding and Randomization
Sample blinding and randomization are key elements in reducing selection and other biases as well as in permitting reliable statistical testing. This module presents the importance of blinding and randomization, as well as the impact of issues that may introduce bias, such as pressure to publish.
Blinding and Randomization Discussion Material [PDF, 104KB]

Module 3: Biological and Technical ReplicatesModule 3: Biological and Technical Replicates
Including replicates in the experimental process is essential to ensuring the most rigorous research approach. In this module, reviewers discuss a figure included in a grant application and the potential significance of the finding, which leads to a brief conversation about the differences between biological and technical replicates.
Biological and Technical Replicates Discussion Material [PDF, 98.7KB]

Module 4: Sample Size, Outliers, and Exclusion CriteriaModule 4: Sample Size, Outliers, and Exclusion Criteria
Calculating the appropriate sample size and characterizing what is “normal” for a specific experiment are important factors in identifying outliers and determining exclusion criteria prior to initiating an experiment. This module illustrates the value of making these determinations by providing a possible example of the unintended consequences of overlooking this key element of experimental design.
Sample Size, Outliers, and Exclusion Criteria Discussion Material [PDF, 107KB]

Society for Neuroscience Rigor and Reproducibility Training Webinars

Promoting Awareness and Knowledge to Enhance Scientific Rigor in Neuroscience Link to External website

Webinar 1: Improving Experimental Rigor and Enhancing Data Reproducibility in Neuroscience Link to External website
Post-Webinar Discussion Questions Link to External website
The topics of scientific rigor and data reproducibility have been increasingly covered in the scientific and mainstream media, and they are being addressed by publishers, professional organizations and funding agencies. This webinar addresses topics of scientific rigor as they pertain to preclinical neuroscience research.

Webinar 2: Minimizing Bias in Experimental Design and Execution Link to External website
Post-Webinar Discussion Questions Link to External website
Investigations into the lack of reproducibility in preclinical research often identify unintended biases in experimental planning and execution. This webinar covers random sampling, blinding and balancing experiments to avoid sources of bias.

Webinar 3: Best Practices in Post-Experimental Data Analysis Link to External website
Post-Webinar Discussion Questions Link to External website
Proper data handling standards, including appropriate use of statistical tests, are integral to rigorous and reproducible neuroscience research. Training in quantitative neuroscience is a specific area of emphasis for the BRAIN Initiative, and rigorous statistical analysis methods are included in the recent Proposed Principals and Guidelines for Reporting Preclinical Research [PDF, 69KB]. This webinar covers best practices in post-experimental data analysis.

Webinar 4: Best Practices in Data Management and Reporting Link to External website
Post-Webinar Discussion Questions Link to External website
Efforts to enhance scientific rigor, reproducibility and robustness critically depend on archiving and retrieving experimental records, protocols, primary data and subsequent analyses. In this webinar, presenters discuss best practices and challenges for data management and reporting, particularly when dealing with information security and sensitive material; archiving and disclosure of pre- and post-hoc data analytics; and data management on multidisciplinary teams that include collaborators around the globe.

Webinar 5: Statistical Applications in Neuroscience Link to External website
Post-Webinar Discussion Questions Link to External website
How can neuroscientists improve their “statistical thinking” and make full and effective use of their data? This webinar covers common applications of statistics in neuroscience, including the types of research questions statistics are best positioned to address, modeling paradigms and exploratory data analysis. The presenters also share examples and case studies from their research.

Webinar 6: Experimental Design to Minimize Systemic Biases: Lessons from Rodent Behavioral Assays and Electrophysiology Studies Link to External website
Post-Webinar Discussion Questions Link to External website
Common sources of bias in animal behavior and electrophysiology experiments can be minimized or avoided by following best practices of unbiased experimental design and data analysis and interpretation. In this webinar, presenters discuss experimental design and hypothesis testing for mouse behavioral assays, as well as sampling, interpretational bias and referencing in in vitro and in vivo electrophysiology recording studies.

Workshop: Tackling Challenges in Scientific Rigor: The (Sometimes) Messy Reality of Science Link to External website
This webinar explores practical examples of the challenges and solutions in conducting rigorous science from neuroscientists at various career stages. It focuses on development of the interpersonal, scientific and technical skills needed to address various issues in scientific rigor, such as what to do when you can't replicate a published result, how to get support from a mentor and how to cope with various career pressures that might affect the quality of your science.

This page last reviewed on May 24, 2017