What is the mechanism by which the reproducibility of scientific discoveries decreases over time?

byFlorian F. (Flowtography)

Even if it is a researcher everyone thinks "I want to make a big discovery that remains in history", even scientific discoveries that should have demonstrated experiment data in good faith, even with time Reproducibility declines, The New Yorker reports.

The Truth Wears Off | The New Yorker
https://www.newyorker.com/magazine/2010/12/13/the-truth-wears-off

On September 18, 2007, dozens of neuroscientists, psychiatrists and pharmaceutical company employees gathered in the hotel conference room in Brussels, and they were informed of the shocking news. How, starting sales from the early 1990's used for treatment of schizophrenia "Second generation antipsychotics"There is data that the curative ability of drugs against patients with schizophrenia has fallen to less than half of the effect reported in the study in the early 1990s". Originally, second generation antipsychotic drugs were sold on the basis of clinical results "dramatically improving the symptoms of schizophrenic patients" and are most used for the treatment of patients with schizophrenia in 2007 It was an antipsychotic drug. However, the data disclosed in Brussels shows that "second generation antipsychotic drugs are less effective than first generation antipsychotics that have been in use since the 1950s, and sometimes they may worsen patients" It was.

Needless to say, repeatability checks are carried out to confirm the effectiveness of drugs before sales of second-generation antipsychotic drugs. If research results of a certain experiment are announced, unless researchers at multiple laboratories repeat the experiment and acknowledge that the same results as those of the presenter have been obtained, research is accepted in the scientific community There is none. The check of reproducibility is a protective device against "subjective factors enter and distort the experimental results" and it was a big fortification to distinguish science from pseudoscience. However, the fort is about to collapse.

Currently, there are many cases that many cases are found to be "not actually the case" against many scientific findings "Reproducibility was recognized at that time". Although there is no official name for the phenomenon that this reproducibility declines, it is confirmed in various wide fields from psychology to physiology, and plagues many scientists.

byNate Steiner

He was a graduate student at the University of Washington in the 1980sJonathan ScurlerHe made an interesting discovery about language and memory. That is, "If you show a human face to a subject and let me explain the face you saw later with a word or if you did not explain anything, I did not remember exactly the face that the group explained in words explained exactly It is what you say. Mr. Schooler announced a series of experimental results to the academic society,Language concealment effectI named it.

The language concealment effect is a sensational discovery, and as of 1990, Mr. Scuoler became a star of the academic society, such as already published quotations more than 400 times. Mr. Scuoler continued to extend "human face" in experiments to "wine taste", "strawberry jam", "creative puzzle solution", etc. After that, it was dramatic It is said that performance deterioration was seen.

However, Mr. Schooler posted a new research result to a prominent academic journal and gained fame while he was suffering from some troubles. In fact, the reproducibility about the experiment that remembers the first person's face "human face" gradually declined. "Although I could often confirm the effect of hiding the language, it was not as strong as the original one," Mr. Scuoler said. In the end, the group of subjects who participated in the first experiment, I thought that it was easy to accept.

byLee Summers

"One of my mentors encouraged me by saying," I want to disappoint myself, I adhere to reproducing the first experiment, but that is a mistake, "but I am convinced It was not a clear explanation of it, "he said. Mr. Scuoler later announced several new research results apart from the language concealment effect, and now he works for Santa Barbara University. However, gradual decline in the reproducibility of the language concealment effect made Mr. Scuoler extremely irritated.

"It was almost as if the first experimental result was a coincidence, it seemed that the shaking back was coming," Mr. Scuoler said, but he is not currently conducting research on language hiding effect. However, the language hiding effect still remains widely accepted and is often quoted when asserting ambiguity of human eyewitness testimony.

byJeff Meyer

In the 1930s there was also a case where a rapid decline in reproducibility was confirmed. Psychologist at Duke UniversityJoseph Banks LineHe conducts research on ESP,Zenor CardI made an experiment to predict the mark of the card in a state that the inside outward mark can not be seen by using the card with the five types of marks drawn. While most of the subjects were able to produce just 20% correct answer rate as expected, only student Adam Linzmeyer recorded an incredible rate of 50% in the first session. If happened accidentally, this is a very low probability of only 1/2 million, but Mr. Linzmeyer said that he left the same result three times after that.

According to Mr. Linz 's result of Mr. Linz, she prepared to submit a paper entitled "ESP ability exists" to the academic society, but Mr. Linzemyer's supernatural ability has been lost as a result of publication of the paper Thing. Mr. Lin repeatedly carried out the same experiment for Mr. Linzmeier, but ultimately the ability was not reproduced eventually, Mr. Rhein concluded that Mr. Linzmeier's supernatural ability was lost. Mr. Lin named the phenomenon that Mr. Linzmeier's rapidly lost supernatural power was "decline effect" and recorded that not only Mr. Linzmayer but also other subjects had declining effect.

Mr. Scuoler looked at the discovery of Mr. Rhee's "decline effect" and said he did an experiment to find "predictive ability" similar to "Mr. Rhee's discovery of ESP". Then, just as Mr. Rin found Mr. Linzmeyer, Mr. Scuoler was able to find a subject who was "a predictive person". And as the experiment was repeated, the predictive ability of the subjects disappeared as if the subjects declined like Mr. Line's case. It is easy to settle as "just returning to the average", but it is easy for some subjects who randomly extract subjects to strike outliers, and if they happen in the early stages of the experiment, It is very difficult to proceed without experimenting.

byYuri Belotserkovsky

A similar case,Andres MollerThere is also a report on the research that Mr. Fuji announced in 1991, "The possibility that the wing is symmetry individual, the possibility that the wing is more likely to find breeders than the asymmetry individual" is also reported. Mr. Mohler 's findings were also published in science magazine' s Nature, and in the natural world as a sexual choice, there was a case that liked symmetry beauty and it became a very big topic.

In the next three years when Mr. Mohler published his thesis, ten independent experimental results on sexual selection and symmetry were presented, among which nine experiments confirmed correlation between sexual selection and symmetry. However, among the 14 papers published in 1994, there were only 8 cases that recognized the correlation between sexual selection and symmetry, and gradually the correlation between sexual selection and symmetry collapsed . And in 1995, among eight papers, positive ones of correlation were reduced to 4 in half, and in 1998 only one third of the total was correlated between sexual selection and symmetry I did not approve.

He is a biologist at the University of Western AustraliaRay Simmons"I was inspired by Mr. Mouler's research results and conducted some experiments, but unfortunately we did not find a correlation between sexual choice and symmetry, but the worst The point is that it was very difficult to publish the experimental results that "the experiment was not successful, the significant data was zero". " In other words, many research results published in prominent academic journals are overwhelmingly "new discoveries" or "experiments succeeded", so there are circumstances that negative research results are difficult to be posted.

byCarlos Henrique

In addition, it is said that "selective reporting" by scientists is a problem, and the difference between obvious clinical results in Asian cultural area and European cultural area in acupuncture treatment is stated. Experiments on 47 acupuncture treatments were conducted in China, Taiwan, and Japan from 1966 to 1995 and it was concluded that "needle treatment has a significant effect", but during the same period the United States, Sweden, In the 94 experiments conducted by the UK, only 46% were able to confirm the therapeutic merit. In the selective report, as the researchers aim to "acknowledge importance" and "find a statistically significant difference", it is sometimes unknown to search for only the number that is convenient for you It is regarded as a problem.

In this way scientific experiments often contain unintended errors and researchers must pay attention to the handling of research data. However, once sensational research results are announced, research results are presented in various places, which will create bias to researchers unknowingly. This can lead to a vicious circle that announces the results of experiments to confirm new erroneous research results, reinforcing more erroneous research results.

Researchers should not forget to believe that "this theory is correct because other researchers are also adhering to it," and must remember that there is always a need to suspect.