2023-10-26 19:00:07
Nest of blue tits (“cyanistes caeruleus”), in Alsace, in May 2019. Among 135 studies, the majority, but not all, concluded that there is a link between the number of chicks in a clutch and their weight. But the strength of the relationship highlighted varies from one to ten. SYLVAIN CORDIER/BIOSPHOTO
Ecologists have just thrown a wrench into the pond of their discipline, in the form of a manuscript currently being published in the journal BMC Biology and posted online on October 4. More than two hundred researchers analyzed exactly the same data to arrive at… different, even sometimes opposing, conclusions.
In a batch of 135 analyses, the majority found that the more chicks there are in a brood of blue tits, the less each one weighs. But the strength of the relationship between number of pups and weight varies from one to ten. In another batch, of 81 analyses, one part found that grassing a plot harmed the growth of eucalyptus trees, while another part found the opposite. Three-quarters find that land use by grass has no effect.
Is ecological research so fragile? “For more than ten years, I have been interested in the reliability of scientific literature in ecology and evolutionary biology, explains Hannah Fraser, co-pilot of this unique experiment at the University of Melbourne (Australia). The variability of results is often attributed to the fact that environmental conditions are never exactly the same. But we wanted to know to what extent individual decisions in data analysis can also influence this variability. »
This may seem surprising, but when faced with a set of data, each researcher makes different choices based on what seems relevant to them. It can, for example, only keep certain observations. In the “tits” case, there were measurements made in the wild, but also others, where chicks were added or removed by humans to study the evolution of the litter.
Then, we must take into account the factors likely to influence the desired relationship between two variables. In the “eucalyptus” case, the distance of the plot from a mature tree likely to drop seeds obviously counts when estimating a tree’s chances of growth. Selecting these cofactors adds even more options.
“A surprising variability”
Finally, there is the choice of statistical methods. A linear relationship between two variables, or another, more subtle one? A frequentist or Bayesian point of view (the name of two statistical schools)? Which version of software to estimate the parameters? Still sources of variability.
To shed light on the situation, the authors were methodical. Two datasets, “tit” and “eucalyptus”, were proposed, with a specific question for each, never studied before. Each “analyst” then became a critical reviewer of the others’ work, assigning a score from 0 to 100.
You have 50% of this article left to read. The rest is reserved for subscribers.
#Crash #test #reproducibility #biological #research