What does it take for researchers to change their minds based on new findings? Do they place higher demands on studies that contradict their beliefs than on those that confirm them? And how does the scientific community react when someone publishes results that contradict their own previous publications? A scientific-theoretical discussion contribution deals with these questions.
Whether in physics, medicine or the reconstruction of the history of the earth and the origin of the first life: in many areas of science there are competing theories and explanations where it is not yet clear which of them are true and which are not. This also applies to psychological research. For example, there are different models to explain how people make decisions. Some researchers have spent most of their careers advocating one of these models. They have carried out countless experiments on this and have repeatedly argued for the model they favor in their publications.
Biased interpretations?
With such a background that is strongly influenced by one perspective or model, can researchers still have an unbiased view of new results? What if, in a new experiment, they suddenly discovered that the competing model fits much better? And how would a corresponding publication be perceived in the scientific community? Psychology professor Ami Eidels from the University of Newcastle in Australia raises these questions in a discussion article. “We should be concerned with how our prior beliefs and expectations affect our interpretation of scientific results, and perhaps also the amount of evidence we need to change our beliefs,” he writes.
To illustrate this, Eidels creates a scenario in which the fictional characters Alex and Bea have been researching the same phenomenon for years, with Alex representing theory A and Bea representing theory B. Both of them now independently receive the same experimental results that better fit theory A. From Eidel's perspective, it would be plausible that Bea would place higher demands on the evidence before admitting that the new results support theory A, while Alex may also feel that his opinion is confirmed by weaker evidence.
What role does the context of a publication play?
“Assuming that Alex and Bea were both to publish the new results: How would the scientific community evaluate the studies? Would Bea’s work be given more weight because she has changed the beliefs she has held for years?” asks Eidels. “If we assume that the studies in Alex and Bea's laboratories were carried out with the same care, it seems unfair to attach greater importance to the same results from one laboratory than to those from the other.” Nevertheless, Eidels believes it is It is likely that this context plays a role in the evaluation of the publication.
As an example, he cites a study from 2011 in which 100 test subjects were asked to guess which of two curtains had an erotic image behind them. While the random hit rate would have been 50 percent, the test subjects actually guessed correctly 53 percent of the time. The study's team of authors interpreted this to mean that the people were able to see into the future to some degree - a pretty bold interpretation. “Since then, there have been many attempts to replicate these results,” reports Eidels. “Here, too, the question arises to what extent the results are influenced by whether the researchers believe in supernatural phenomena.”
Collect and include presets?
The results were not confirmed in larger experiments. Studies from various laboratories, in which more than 2,000 test subjects completed almost 38,000 runs, achieved a hit rate of 49.89 percent. The result therefore only spoke for a random distribution of hits. The researchers asked all members of the research teams involved whether they believed in supernatural phenomena. “Asking directly about the default settings is an important step,” writes Eidels. “However, since this individual information can be distorted, supplementary approaches can be useful.”
For example, you could use researchers' previous publications to create a kind of score based on how convinced they are of a particular theory. According to Eidels, artificial intelligence, which can automatically evaluate hundreds of studies, could also help in the future. “If you record the default settings with the help of self-reports or a futuristic AI, new questions arise,” writes Eidels. “Should readers consider information about researchers’ prior beliefs when interpreting scientific results? And if so, how? I leave this question for future discussions.”
Source: Ami Eidels (University of Newcastle, Callaghan, New South Wales, Australia), Royal Society Open Science, doi: 10.1098/rsos.231613