2024-02-01 23:00:00
© AndreyPopov GENEVA (ILO News) – The International Labor Organization (ILO) has published a new study on how artificial intelligence (AI) assesses the prestige and social value of occupations, shedding light on the potential and risks of the use of such methods for sociological and occupational research.
The document, A Technological Construction of Society: Comparing GPT-4 and Human Respondents for Occupational Evaluation in the UKcompares assessments of occupations made by GPT-4 (a type of AI Large Language Model (LLM) capable of recognizing and generating text) with those of a high-quality survey conducted in the United Kingdom.
Occupational assessment captures people’s perceptions of occupations in society. The researchers used the most universally applicable occupational classification, the ILO’s International Standard Classification of Occupations (ISCO-08), to organize jobs into clearly defined groups based on their tasks and functions.
First, human respondents in the United Kingdom were asked to rate the prestige and social value of a selection of occupations. GPT-4 was subsequently asked to provide a similar classification, taking the role of 100 random respondents with what it would consider an “average UK profile”. The human ratings were then compared to these algorithmic opinions, in order to understand to what extent the AI system was able to predict human opinions, and whether its way of perceiving human opinions was tailored to certain demographic groups.
The study found a high correlation between the results generated by the two different approaches. The GPT-4 demonstrated great skill in predicting UK average views of the prestige and social value of different occupations, and in using these predictions to create relative occupational rankings. This “algorithmic understanding” of general human opinions could enable the use of AI in occupational research, with advantages such as efficiency, cost-effectiveness, speed and accuracy in capturing general trends.
However, the study also revealed some problems. The AI model tended to overestimate the prestige and value of occupations associated with the digital economy or with strong marketing and sales components. It also underestimated, compared to human evaluators, the prestige and social value accorded to some illicit or traditionally stigmatized occupations. Furthermore, the researchers manipulated the AI’s algorithmic instructions, showing that it was not capable of understanding the hierarchies of prestige and social value of occupations as perceived by demographic minorities in the British context.
The article warns that current LLMs tend to primarily reflect the views of Western, Educated, Industrialized, Wealthy and Demographic (WEIRD) populations, who constitute a global demographic minority but have produced the majority of the data on which research is conducted. have trained such AI models. Therefore, while they can be a useful complementary research tool – for example, for processing large amounts of unstructured text, voice and image input – they carry the serious risk of omitting the views of demographic minorities or vulnerable groups. The researchers argue that these limitations must be carefully considered when applying AI systems to the world of work, for example when providing career advice or conducting algorithmic performance evaluations.
The paper’s co-authors are Paweł Gmyrek of the ILO, Christoph Lutz of the Norwegian Business School, and Gemma Newlands of the Oxford Internet Institute.
#ILO #study #highlights #role #assessing #prestige #jobs