What is required of a researcher to verify the reproducibility of a study?
In order to ensure the results of research, it is necessary that not only experiments by the researchers themselves, but also third parties be able to reproduce the same results. The U.S.
A controlled trial for reproducibility
https://www.nature.com/articles/d41586-020-00672-7
DARPA has introduced independent verification and validation (IV & V) since 2016 to verify the reproducibility of the research results of teams that have passed the review of research results. The team formed for IV & V will conduct research and experiments as independent third parties to verify reproducibility.
DARPA's IV & V team consists mainly of 3-5 researchers, with around 3-8% of the project funding dedicated to IV & V work. In order to make accurate assessments, some laboratories have installed about 200,000 dollars (approximately 20.67 million yen) for IV & V, apart from project funding, at DARPA. The cost for IV & V may seem high, but DARPA states that by installing appropriate equipment in advance, it will be able to prevent wasted money and time and advance basic research.
The IV & V team consists of impartial third-party researchers employed by DARPA. Researchers performing IV & V are required not to plagiarize ideas or conduct derailed research. Also, depending on the outcome of the IV & V, the original research team may be challenged, requiring a counter-argument and communication skills. The fact that the research team and the IV & V team discuss and cooperate with each other is a prerequisite to demonstrate the research results.
One of the main initiatives of the IV & V team is to spend more than a day on experiments and surveys in the same lab as the team that did the research. The team that conducted the research also conducts experiments and surveys in the laboratory on the IV & V side. Performing experiments while going back and forth between laboratories can better understand the research content than reading a dissertation, and can reduce differences in individual understanding. In particular, collaborative experiments are considered important because IV & V researchers differ in expertise and knowledge of prior research compared to the team that conducted the research.
Conducting experiments jointly between the research team and the IV & V team also helps to minimize obstacles in the research. For example, even if there are no researchers with specialized knowledge in the laboratory, it is possible to support research on behalf of multiple researchers who share research content closely.
With the introduction of a team dedicated to IV & V, DARPA has made research results more reliable. Because many of the technologies developed by DARPA are expected to be used in a wide variety of applications by an unspecified number of people, the IV & V team needs to evaluate not only the effectiveness of the technology but also its reliability. there is.
For example, in genetic research, instead of just investigating the reproducibility of the results, it is important to say, 'Which parts of the cells take up the new genetic material?' It is necessary to investigate whether it functions in the cell, and whether the function is maintained even after the cells proliferate.
Shihan said, 'IV & V offers more benefits than replicating individual studies. Verifying reproducibility can also give researchers more insight into experimental procedures and techniques. Introducing IV & V Can provide reliable and cost-effective research, and other institutions should consider investing in IV & V as well, reducing the number of irreproducible articles and reducing the quality of work of researchers. It will improve. '
Related Posts:
in Posted by darkhorse_log