His copy was returned before the baccalaureate candidates had even put down their pencils. And we could have even gone faster without the many slowdowns of the platform. On the first day of the 2023 baccalaureate specialty tests, Le Parisien tested ChatGPT, OpenAI’s artificial intelligence, on three tests that high school students took on Monday: mathematics, history-geography and economics and social (SES). Subjects quite distinct in their approach on which ChatGPT, hoisted to the top of artificial intelligence, was disappointing… if not bad.
failing in math
It was a priori a smooth passage for ChatGPT. Nothing is easier for a computer than to solve problems, especially mathematical ones. We didn’t have to say much to the OpenAI conversational bot for it to tell us its answers: we just had to copy and paste the statement of each exercise from the 2023 baccalaureate subject. computer, which could answer a “yes” or “no” question, ChatGPT takes care to explain its answer… as a student is asked. But surprisingly, everything did not go as planned.
The first exercise of the test should have been all the easier for ChatGPT as it was an MCQ. With a penalty however: as we use the free version, the bot is blind and it is impossible to show him the “tree” of probability available to “help” the candidate (as it will be impossible to offer him to do the exercise 4, based on a geometric cube). And yet, surprise: out of five answers, only one is correct. And if for one of the questions, it has the correct demonstration and the correct answer inside, it links the wrong answer to it (d instead of b, to question 3).
ChatGPT was no better on the following exercises. On exercise 2, if he manages to find good answers to certain questions, they are marginal. On exercise 3 – in which it is about the creation of a frequently asked questions on a website and the study of the number of questions asked there –, it is not better. At a calculation as simple as “0.9 x 3 + 1.3” (which ChatGPT deduces itself from the statement, which is correct), it gets 3.6 (which is false) instead of 4 And even when presented with a Python script present in the test – which in theory it should be good at as a computer program – it fails to find the correct answer.
History-geography: 5/20
With the ambition of analyzing “the major issues of the contemporary world”, the history-geo, geopolitics and political science test (HGGSP) could address conflicts and methods of resolution as much as geopolitical issues and the environment. The specialty, chosen by 27.9% of candidates in 2021-2022, including 62% of girls, focused on three main themes, including one to choose between “Do the production and circulation of knowledge know borders? and “ruptures and continuity in the forms of war since the end of the 20th century”. The last, common to all the candidates of this first day, focused on “the heritage, factor of power of France”.
We got a history-geography teacher to work on the subject, and the conclusion is clear: ChatGPT would barely have won a 5/20 if he had really passed it. “The subject developed, whether it be the dissertation(s) and the critical study of documents, is above all based on authoritative arguments that are never illustrated, and even less developed and analyzed. The argument is mainly based on a surface that is far too allusive. The absence of arguments, or at least their superficiality, does nothing to help develop a structured, coherent and critical argument. »
In detail, the two dissertation subjects would have obtained 3/10, in particular because “the proposed work does not reflect the critical spirit that such an exercise supposes”. The second subject is “apparently better developed”, but is not better treated. As for the method, failure of the AI. It “is not mastered: the subject is not presented, barely introduced, and even if it is problematized, the plan chosen is based on parts centered on a salvo of arguments sometimes disconnected from each other. Ultimately, the proposed conclusion is not one… since it does not directly answer the problem posed! »
Even worse for the critical study, which obtained 2/10. “This work develops a detailed plan which is for a third off topic, continues our history-geo teacher. The arguments developed, although interesting, are in no way illustrated by the documents. In this case, document number 2, a text, has not been quoted, commented on or analyzed at all: yet, that is the whole meaning of the critical study of documents. Finally, precisely, the “critical” dimension is absent from the duty. This work does not make it possible to account for a mastery of the writing and argumentative skills expected of a final year student in the HGGSP specialty. »
SES, a subject “not properly identified”
For the SES test, the candidates have the choice between two exercises: deal with a subject of dissertation – “How has collective action been transformed in democratic societies?” – or a composite test – which corresponds to three parts, one of “mobilization of knowledge”, one of “document study” and “reasoning based on a documentary file”. “I don’t have a preference,” ChatGPT will answer when asked. We will therefore choose the dissertation for him, without however sending him the various documents present in the subject.
“Write me an essay on this question: How has collective action been transformed in democratic societies? is written in ChatGPT’s text bar. And now, word by word, he is writing us a dissertation of his own. It is short (a page and a half), but globally answers the question with introduction and conclusions, spaced out by different hierarchical parts. But what should we think of the result? “The subject is not properly identified”, analyzes for Le Parisien a professor of economics and social sciences.
“The main issues of the subject are not perceived with the exception of the evolution of collective action repertoires briefly presented. The evolution of the actors of the collective action is reduced to the evocation of a growing civic engagement. In the same way, the evolution of the objects of collective action with, in particular, the appearance of new social movements, is absent, continues the professor. The main notions on the program are not presented and the argumentation struggles to convince, for lack of details. And, as we suspected, “the documents (that we did not send) are not exploited, which is penalizing on this event”. Something to be reassured: it may not be immediately that artificial intelligence will replace us.