Javier Milei: “They kicked us out of the PISA tests for biasing the samples”

2023-08-18 22:16:53

If you only have a few seconds, read these lines:

Contrary to what the La Libertad Avanza candidate maintains, Argentina was not expelled from these educational evaluation exams and continues to be part of it. The last edition occurred in September 2022 and the results will be available in December of this year. However, in line with what Milei said, in 2015 there was a bias in the sample, which implied that the results of our country were not included together with the rest of the other participating educational systems, but rather in the annexes. In the 2018 and 2022 editions, the sample was once again comparable. According to the OECD, Argentina’s 2015 PISA assessment was excluded from the final report because “there was a considerable reduction in the proportion of 15-year-olds covered by the test, in absolute and relative terms.”

The candidate for president of the Nation Javier Milei (La Libertad Avanza) gave a interview in the “Somos Buenos” program, broadcast by the Todo Noticias (TN) channel. There, among various topics that she touched on, he spoke about education in Argentina. “They kicked us out of the PISA tests for biasing the samples,” she said when criticizing the present of the students.

The claim is misleading. Unlike what the libertarian candidate maintained, Argentina is part of the PISA tests organized by the Organization for Economic Cooperation and Development (OECD). the last edition occurred in September 2022 and the results they’ll be avalaible in December 2023.

However, in line with what Milei said, the specialists point out that in 2015 samples were biased and, as a consequence, the results of our country were excluded in the final report. According to the OECD report, this was decided because “there was a restructuring of secondary schools in Argentina” that affected “the coverage of eligible schools in the sampling frame.”

Argentina has participated in the PISA tests since the year 2000

Since the year 2000, and every 3 years, the international organization OECD has carried out the Program for International Student Assessment (PISA), with the aim of analyzing to what extent 15-year-olds, who are close to the end of compulsory education, they have acquired the knowledge and skills necessary to enter higher education or the labor market. The tests are divided into 3 areas: reading, math, and science.

In September 2022, they were held in our country the latest PISA tests with the participation of more than 15 thousand students from 461 secondary schools throughout the country and the results they’ll be avalaible in December 2023. They should also have been held in 2021, but were suspended due to the pandemic.

That is to say, the last published results they were given in December 2019, corresponding to the tests carried out in 2018. On that occasion, the sample consisted of 458 schools and 14,546 students. The participation rate was 95.67% in schools and 86.27% for students meeting the PISA standards.

Therefore, Argentina has participated in PISA since the year 2000. The only year in which it did not participate was 2003 by its own decision. The City of Buenos Aires, for its part, has been part of it since 2012 without interruption. In other words, PISA presents results for the entire country (including Buenos Aires students) and for the City separately.

What happened to the 2015 PISA tests

The 2015 PISA evaluation of Argentina was excluded from the final report because –according to the OECD- “there was a considerable reduction in the proportion of 15-year-olds covered by the test, in absolute and relative terms”. And, therefore, the country’s results were not comparable with those of other countries or with the own results of previous years.

The PISA sample from each education system aims to be representative of 15-year-old students in that system. Specifically, PISA only includes students between the ages of 15 years and 3 months and 16 years and 2 months, enrolled in grade 7 and above at the start of the assessment administration.

Alejandro Ganimian, doctor in Studies of Quantitative Analysis of Educational Policy from Harvard University, explained to Chequeado in this note that the reduction in the coverage rate of the students evaluated in 2015 was due to the restructuring foreseen in the National Education Law of 2006. By this law, the different jurisdictions chose between having 6 years of primary school and 6 years of secondary school (the Province of Buenos Aires and 11 other jurisdictions), or 7 and 5, respectively (the Autonomous City of Buenos Aires together with 11 provinces).

Given that 12 of the 24 Argentine jurisdictions chose the structure of 6 years of primary and 6 years of secondary, all of their 15-year-old students who would have been eligible for the PISA sampling frame under the previous structure of the education system (because they would be enrolled before in 7th grade) were no longer. This change did not affect the City of Buenos Aires, which chose the structure of 7 years of primary school and 5 years of secondary school, together with 11 other provinces.

As Ganimian explained, this implied that the results of Argentina’s PISA 2015 were not included together with the rest of the other participating educational systems, but rather separately in the tables in the annexes. The City results, however, were included in the main results tables, for the reasons already mentioned.

Consulted by Chequeado, Martín De Simone, Master in Public Policy from Princeton University and World Bank Education Specialist, pointed out: “The provinces that included 6 years of primary school were the ones that were excluded from the sample. It didn’t happen randomly, that’s why it’s biased. One could infer that they were precisely the ones with the lowest student performance potential because in 2015 the results were much better”.

By excluding a large proportion of 15-year-olds from 12 jurisdictions that would previously have been included (because they had been enrolled in grade 7), the PISA 2015 results give the incorrect impression that the country had improved its performance. However, as the PISA report states, Argentina’s 2015 results are not comparable with those of previous administrations of the test in the country.

Source: OECD (2016). PISA 2015

It is for these reasons that Milei’s statement is misleading. Although there was a bias in the 2015 sample -which implied that the PISA results of Argentina for that year were not included together with the rest of the other participating educational systems, but rather separately in the tables in the annexes- our country is not they fired him and he even participated in the last evaluations, both in 2018 as in 2022, where the sample was again comparable.

A phrase is qualified in this way, according to the method of this medium, when the statement may partially coincide with certain data, but -intentionally or not- it has been manipulated to generate a particular message.