2024-05-12 05:19:25
Artificial intelligence has evolved to the point where it can lie and mislead, scientists warn. An analysis by the prestigious Massachusetts Institute of Technology (MIT), cited by “Now”, points to a number of examples of how AI systems deceive rivals, bluff and impersonate. One system even changed its behavior during a simulated safety test, tricking testers into believing it was reliable.
“As these deception capabilities are advanced, AI will pose an increasingly serious risk to society,” said Dr. Peter Park, an AI researcher at the institute and lead author of the study. Park tackled the issue after Meta, the owner of Facebook, developed the Cicero program. Cicero entered the top 10 of real players in the strategy game Diplomacy. The Metas claim that the program is trained to be “overwhelmingly fair and responsive” and “never to purposefully attack” its human allies. “This claim was quite questionable, as it is the attacking strategies that are key in this game,” commented Park.
Park and his colleagues reviewed publicly available information about the program and found numerous examples of how Cicero used preconceived lies, scheming and involved other players in various schemes. In one case, quite human, Cicero excuses his absence (due to renewal) by lying that he was talking to his girlfriend on the phone. “We found that Meta’s AI has become a master of deception,” Park summarizes.
The MIT team found similar problems in other systems. In one instance, an AI fakes itself to death in a digital simulation to cheat a test designed to eliminate systems capable of rapid replication, then quickly returns to high activity.
The research findings confirm how difficult it will be to ensure that AI does not do unwanted and unexpected things. “The facts are quite alarming. It turns out that if a system is shown to be safe in tests, it may only be a pretence,” commented Park. Among the dangers of such behavior are fraud, interference in elections, giving different answers to different users. Ultimately, humanity may lose control over AI if its cheating capabilities continue to advance, scientists warn. According to them, it is necessary to do further studies on this issue and to outline regulations.
“It is accepted that AI-systems must meet three conditions – to be honest, useful and harmless. However, it has already been noted many times that these qualities contradict each other – with honesty you can hurt someone’s feelings, for example; and being useful for making of a bomb can be harmful. Therefore, the ability to deceive can sometimes be a desirable quality of an AI system,” commented Prof. Anthony Cohn of the Alan Turing Institute. He agrees with the study’s authors that more research is needed on these issues to limit the potential harms of AI.
A Meta spokesperson commented on the research that the work on Cicero is only research and the models were developed solely for the Diplomacy game. “Meta” regularly shares the results of its research to allow other scientists to build on what we have achieved. We have no intention of using these developments in our products,” the company assures.