Lab life. Peer review, or peer review, scientific studies is supposed to ensure their quality. For each article submitted to a journal, a handful of specialists are responsible for evaluating whether the manuscript is suitable for publication or not. In a study of an unprecedented scale, economics researchers from the University of Innsbruck (Austria) note, however, how much the judgments of evaluators can be biased. Available in preprint, this study was presented on Friday, September 9, in Chicago, at the International Congress on Peer Review.
Jürgen Huber, professor of finance and first author of the study, immediately recognizes that “the scientific publication process is not perfect”. “When you are a researcher, you know that there is a random element in peer review, that there are biases, and we wanted to provide new data to quantify this. » His idea: that an article written by a Nobel laureate and a junior researcher be submitted for evaluation, then see if it is evaluated in the same way if only one name or the other is mentioned.
Luckily, Vernon Smith, winner of the 2002 Nobel Prize in Economics, was able to provide them with an original article written with Sabiou Inoua, a young thesis researcher at Chapman University (California, United States). This article was submitted to Journal of Behavioral and Experimental Finance with the complicity of Stefan Palan, co-editor-in-chief of the journal, and the support of Elsevier, the publishing house. An invitation for evaluation was sent to 3,300 researchers in the field and more than 500 evaluation reports were collected. An unprecedented device: the evaluation in this journal is usually limited to two specialists.
Status bias
Each evaluator received a version of the manuscript that was either completely anonymized or cited only one of the authors. Everyone had to assess whether the article was acceptable, as it was or after corrections, to be published or whether the newspaper should refuse it.
When the manuscript bears the name of the young researcher, 65% of the evaluators reject the article, whereas only 22% make this decision when the Nobel-winning researcher is mentioned. The completely anonymized article was refused by 48% of its evaluators. Conversely, 21% of the evaluators of the article bearing the name of Vernon Smith accept it without asking for the slightest correction, against only 2% of the proofreaders who received an anonymized manuscript or in the name of Sabiou Inoua. Results “staggering” for the authors, who expected to see differences but not of such magnitude. Such discrepancies were also observed in the evaluation of the relevance of the work or the quality of organization of the manuscript. Would they have been even more marked if the second signatory had been a woman?
You have 33.71% of this article left to read. The following is for subscribers only.