Research Curriculum Development

2016

Listed below are the details of the projects funded under PA-16-142.

A Novel Statistics-Based Course for Life Science and Engineering Students
Principal Investigator: Martin L. Yarmush, Ph.D., Rutgers, The State University of New Jersey
This project is to develop a novel statistics-based course specifically tailored to the needs of Life Science and Engineering students. The course activities will include: use of enquiry-based learning to strengthen understanding of statistical analysis; emphasis on practical rather than theoretical aspects of statistical analysis methods; analysis of real data generated by the student pool; teaching the importance of considering data analysis as a pre-requisite to experimental design; and training students how to use open-source, easy-to-use statistical analysis tools such as R, as well as commercially available graphing and statistics packages, such as GraphPad Prism. The evaluation plan for the course includes an anonymous short online survey by students early in the course to provide feedback to the instructors to make alterations early in the semester followed by a Likert scale course evaluation at the end.
Accomplishments: A new course was created, entitled: "Interdisciplinary Biostatistics Research Training for Molecular and Cellular Sciences: Enhancing Rigor and Reproducibility", and has been registered with the Graduate School and assigned as course #16:125:578. It was advertised and included in the Spring graduate course schedule. 25 students remain registered and 4 individuals are auditing. In addition to attending lectures, students participate in labs where they generate and analyze statistical data sets using z-tests, t-tests, ANOVA, regression, etc. The course is supplemented with instructional videos, and on-line resources such as practice problems. At the end of the semester, students will complete an anonymous Likert-scale course evaluation to inform further course design. The long-term plan is to use this course as the quantitative course requirement for all biotech training fellows.

Advanced Training in Quantitative Skills, Research Methodology and Team Science
Principal Investigator: Myles Huge Akabas, M.D., Ph.D., Albert Einstein College of Medicine
This project is to incorporate additional training in quantitative skills, research methodology and team science into the MSTP and other predoctoral programs. The activities will include integration of new methodological content to existing workshops and courses; developing new courses, workshops, online tutorials and other resources to teach different aspects of scientific rigor and reproducibility and the tools to implement principles in practice; and organize activities and working groups that promote rigor and reproducibility and team science. The effectiveness of the activities will be assessed by using a combination of course evaluations and feedback from academic advisors and mentor pairs and collection of pre-/post-training activity surveys from students.
Accomplishments: The Rigor and Reproducibility Supplement supported the creation of a week-long summer bootcamp "On Becoming a Scientist" for incoming Ph.D. students. It focuses on 1) R and Programming Workshop, and 2) Quantitative Skills Workshop. This has become part of the two week long Ph.D. student Orientation. For the entering MSTP students, a series of lunchtime meetings was developed focused on reading and discussion of papers addressing issues of rigor and reproducibility in the use of antibodies, cell lines and mouse models. In addition, a new 3 week course on experimental rigor, reproducibility and quantitative skills was developed and is now a requirement for all first year Ph.D. and M.D.-Ph.D. students. Finally, a one day data hackathon was organized where students competed to analyze large data sets related to aging. They developed apps to interpret the data. Forty-five students on nine teams participated in an event that is featured on the Einstein website http://www.einstein.yu.edu/features/stories/1261/einstein-tackles-aging-at-first-ever-hackathon/ The winning team, BioAge, created a website that predicts a user's biological age based on information from the RNA in their blood. The runner-up, team WDUCS, created "CareCo," an app to help seniors better coordinate medical appointments and strengthen their social networks to improve their quality of life. Team AARB won third place with a map of New York City's zip codes showing how variables like income, transportation or air quality affect how long a person lives.

Characterization of Antibodies or Antibody Constructs
Principal Investigator: William M Atkins, Ph.D., University of Washington
This project will develop a laboratory course focused on relevant characterization of antibodies or antibody constructs, with hands-on experience. This will complement an existing lecture course on Biophysical Enzymology and Protein Therapeutics that covers many biophysical methods. The course will be 3 weeks of intensive lab time. Data collection will take place during the 3-week period and students will have an additional week to analyze data and write a report. The course will be assessed by student evaluations, the Corporate Advisory Board of the School of Pharmacy at the University of Washington, and input from the outside lecturers that give talks.
Accomplishments: The T32 supplement was to design a lab class for grad students to learn specialized experimental skills and technologies relevant to the development of protein therapeutics. In spring 2017, thirteen students from 4 graduate programs completed the course. Students worked in 2 or 3 person teams to complete the experiments and write lab reports. In addition to the laboratory experience, lectures provided background information about the relevant methods and how they are applied in the development of therapeutic proteins. Course evaluations were uniformly positive and expressed the value in hands on experience with experimental methods that are not frequently taught in academic settings, but are used routinely in biotechnology. The exposure to the basics of antibody structure and function in the context of therapeutic endpoints was noted as a particular strength. The course will be offered again in spring 2018 and then switch to alternate (even) years.

Development and Evaluation of a Business Strategy Skills Course for Biomedical Graduate Students
Principal Investigator: Karl Mark Ansel, Ph.D., University of California, San Francisco
This project is to develop and evaluate a business strategy skills course for biomedical graduate students. Course topics will include fundamentals of business strategy, tools for strategic analysis and strategic collaborations. Course format includes a blended classroom with interactive in-class discussions following review of online videos and podcasts. The success of the course will be assessed via mandatory course evaluations. Course evaluations will be administered via online surveys at two time-points: prior to the first workshop to provide a point of comparison to see what students learned and after completion of each weekly topic. Evaluation of students' understanding of course concepts will be determined by asking facilitators to return assessments of the in-class discussions.
Accomplishments: We developed and launched "Strategy4Scientists", a course that teaches predoctoral research students basic business strategy skills that can be applied in both academic and non-academic settings. The course was tailored to life sciences trainees, and utilized a flipped classroom model in which students participated in interactive, small group discussions after first watching online videos and listening to podcasts. The podcast (Strategy4Scientists on iTunes) included interviews with faculty and Ph.D. professionals in industry and consulting. In-class discussions centered on real cases provided by academic faculty and UCSF alumni in industry and consulting. This program provided exposure to the business strategy language and tools used by faculty and industry professionals to analyze their respective competitive landscapes and engage in strategic collaborations. Assessment data indicate that the content was applicable for trainees seeking science careers in multiple career areas. Participation in the course increased knowledge of basic business concepts, and increased the ability to apply these concepts to practical business cases. Furthermore, students reported they felt more competent to use this new knowledge in networking and interview settings, which shows impact in their professional development. Several peer institutions have implemented portions of this course. In summary, this project advanced the goals of NIGMS by creating and implementing a novel course that helps students build the professional skills and confidence necessary to make structured, strategic decisions that will impact their research and career success.

Develop Curriculum and Assessment Tools for Experimental Design, Statistical Analysis and Interpretation of Data
Principal Investigator: Richard Carthew, Ph.D., Northwestern University
This project is to develop curriculum and assessment tools for a new 10-week course for training in scientific rigor and reproducibility that will be embedded in the graduate curriculum in the Interdisciplinary Biological Sciences graduate program at Northwestern University. The course activities include experimental design and data analysis, making data structures that humans and computers can each read and process, a basic introduction to the reproducibility software package Git, and how to use GitHub scales reproducible code and analyses for individuals or large groups. The impact of the course will be assessed by testing students' knowledge of the fundamentals of experimental design, statistical analysis and interpretation of data, at the beginning of the course in comparison with test results obtained after instruction.
Accomplishments: Two new graduate courses in scientific rigor and reproducibility (R&R) were developed . Both courses were launched in the Spring Quarter (March-June) of 2017. One course is called Rigor and Reproducibility in Research. Experimental design and data analysis is discussed through analysis of case studies on the topics of rigorous statistical analysis, transparency in reporting, data and material verification, and sharing. The course also establishes best practice guidelines for image based data and description of biological materials to uniquely identify the reagents (in particular antibodies, cell lines and animal models). Students demonstrate knowledge and use of the techniques discussed by presenting experimental design and data analysis of their own doctoral research. The other course, is called Practical Training in Chemical Biology Methods and Experimental Design. It features two weeks of classroom-and lab-based instruction on experimental design and analysis, supplemented by R&R training modules using case studies. For both courses, the curriculum and assessment tools were developed in consultation with NIGMS training program directors across Northwestern. Both courses were piloted with a small number of trainees this spring using team instruction approaches. Student reviews indicated that they found this training to be a valuable addition to traditional coursework.

Development of a Course in Quantitative Measurement and Analysis
Principal Investigator: Timothy J. Mitchison, Ph.D., Harvard Medical School
This project is to develop a course, Quantitative Measurement and Analysis, which will be a compulsory course of the Systems Biology Graduate Program first year students and open to all Harvard graduate students. The course will begin with a series of foundation lectures followed by two main modules. These modules will cover instruction on building an assay, characterizing its performance and obtaining reproducible and statistically significant results. The impact of the proposed curriculum will be assessed by confidential student evaluations, success of the student's qualifying exams and evaluation by co-directors of the program.
Accomplishments: The course will be delivered as a halflength pilot in Spring 2017 with the goal of full deployment in Spring 2018. The aim is to establish a fullydocumented curriculum that is self-contained for teaching in future years, and could be distributed for teaching beyond Harvard. All modules for the full course are being developed this year and are being documented in some detail. Program hired a senior teaching fellow to work with them on writing the entire course, and have recruited five addition short-term teaching fellows with deep knowledge on particular measurement techniques to work on developing subject modules. So far have developed the detailed synopsis and written the first few lectures.

Training in Computational Techniques and Application of Reproducible Research Methodologies
Principal Investigator: David J. Christini, Ph.D., Weill Medical College of Cornell University
This project proposes to expand access to a series of well-regarded small-group workshops that teach computational techniques essential for the practical and effective application of reproducible research methodologies. This will be achieved by leveraging emerging tools for online learning so that an unlimited number of students may acquire the basic concepts and mechanics independently and at their own pace. These courses will be made freely available to the public. The courses will be supplemented by shorter, more frequently held direct engagement group lab sessions where the skills learned are consolidated and put into practical use in a flipped classroom setting using content developed specific to the needs of Weill Cornell's students' research activities. The impact of the proposed project will be self-assessed by students of their abilities and understanding before and after each course.
Accomplishments: Two self-paced, online courses that teach R and git (core tools that enable reproducible research) have been created, and are now publicly available to the whole biomedical research community:

The courses have been implemented using the Datacamp platform. One of the main advantages of this platform is the ability to require the student to solve a computational task and finely assess the result, giving feedback tailored to specific errors (i.e., not just inform the learner if he/she is correct or not). These courses are now being introduced into our graduate program curricula; the R course will be a useful adjunct to our Quantitative Understanding in Biology class starting Fall 2017.

New Course in Experimental Design and Data Handling, Analysis, Interpretation and Presentation
Principal Investigator: Lori L. Isom, Ph.D., University of Michigan
This project proposes to develop a credit-bearing graduate level course in practical statistics and experimental design to provide basic working knowledge and best practices that can be directly applied to trainees' everyday laboratory research activities. The course topics will include: descriptive statistics, measures of data spread, hypothesis testing and comparisons, Student's t-test, analysis of variance, power analysis, nonlinear regression, dose-response analysis, data transformation and normalization, and effective data presentation through graphs and tables. The impact of course will be assessed by evaluating instructional delivery, course planning and assessment of student learning.
Accomplishments: We have developed a graduate course entitled: Data processing and analysis: A statistics primer for Pharmacologists and Biomedical Scientists". In this 2 credit course, the students gain proficiency in approaches to experimental design, data capture, handling, transformation, modeling, analysis, and interpretation. It is designed to provide best practices and skills that can be directly applied to the student's laboratory work. After an initial pilot, the University of Michigan Curriculum Committee approved the course and it is now a requirement of the Pharmacology graduate program curriculum. The performance and impact of the course are formally being monitored in conjunction with the Center for Research on Learning and Teaching at the University of Michigan. We are developing longitudinal assessment tools to study the long-term impact of the course on graduate student performance.

Theoretical, Computational and Experimental Training Modules in Molecular Biophysics
Principal Investigator: Juliette T. Lecomte, Ph.D., Johns Hopkins University
This project plans to develop 10 modules for training: five theoretical and computational modules and five experimental modules. The theoretical and computational modules include instruction in the UNIX operating system, basic fitting and modeling procedures and specialized mathematical operations, data processing skills, statistics, and data analysis and molecular dynamics. The experimental modules include diffraction and scattering methods, nuclear magnetic resonance spectroscopy, single molecule and super resolution techniques, solutions biophysics and analytical ultracentrifugation. Each module will be assessed by electronic and anonymous survey at the end of the module period, collecting feedback from thesis advisors and graduate board oral examination process.
Accomplishments: In September 2016, first-year trainees were exposed to Unix, Python, and MatLab in three week-long modules. Through these concentrated sessions, the students developed the computer skills necessary for coursework and further modules. Novel instruction began at the end of December 2016. As a prelude to the X-ray Diffraction module, first-year trainees learned how to lay crystallization trays. Students have participated in the Solutions Biophysics module and the X-ray Diffraction module. Each module includes both instruction and practical, i.e. hands-on or computational, components. Starting in May 2017, modules in NMR spectroscopy, Statistical Analysis , Single Molecule Method, and Molecular Dynamic Simulation will be presented. Initial feedback from students has been positive.

Training in Theory and Application of Statistical Methods
Principal Investigator: Lynne E. Maquat, Ph.D., University of Rochester
This project is to add additional activities focused on the theory and application of statistical methods and other best practices to complement the existing training. The new training activities will include a new annual summer mini-course to train students in effective design and communication of basic research, creation of a statistical science web portal and wiki, and monthly sessions in scientific best practices. The impact of the project will be assessed by the statistical training oversight committee and anonymous feedback from students after each session.
Accomplishments: The summer classes were run in July. Dr. Grossfield directed a presentation skills workshop attended by 16 students. Based on the favorable feedback, we will run another session this winter in addition to the originally planned session next summer. Dr. Brendan Mort supervised the big data section, with Dr. Grossfield consulting; 14 students attended (several from computational labs were excused). The statistical science web portal is up and running, as is the interactive demo version. More demos and quantitative thinking articles are in preparation.