A team of leading AI experts from technology consulting firm Accenture has teamed up to voluntarily produce an artificial intelligence-based (AI) solution for the Arolsen Institute archives in Germany, which helps extract information about victims of Nazi persecution from archival documents 40 times faster than previous efforts in the field. The Arolsen Archive preserves the world’s largest collection of documents documenting Nazi persecution – over 110 million digital documents and artefacts, some of which belong to UNESCO’s World Memorial Program, an international initiative aimed at preserving humanity’s documentary heritage against extermination. Is to make these documents accessible to all who seek to seek out what happened to Holocaust victims and survivors.
At the end of World War II, the Allies set up an international search service for Nazi victims in the German town of Bad Arolsen. Over the years the locating service has become a research institute bearing the name of the town “Arolsen”, and its main activity focuses on documenting Nazi crimes through a huge archive of over 110 million documents that belonged to the Nazis. The vast Arolsen archive contains the largest collection of information on Nazi victims, including documents on concentration camps, forced labor and displaced persons.
In an effort to make the search for information as accessible as possible, Arolsen began several years ago the process of digitizing all documents and publishing them online as part of the #everynamecounts project. Since these are old documents, some of which were uploaded in a searchable way (for example as an image) and some of which were damaged, a human eye is required for the translation, reading, transcription, cataloging and verification of these documents, which may take decades. Each document is added independently by three volunteers, and if the entries entered do not match exactly then they have been reviewed by an Arolsen archive employee who has decided on the matter. In fact, it has taken an average of four people to date to accurately index four documents in one hour.
Ian Lover, an Accenture volunteer and activist in the company’s Jewish employee group (Accenture employs more than 670,000 workers worldwide), quickly realized that artificial intelligence can accelerate this process significantly. Laver was able to recruit other volunteers at Accenture and within 10 weeks the group was able to create a dedicated AI solution for indexing the Arolsen Institute documents. The volunteers worked with the expert teams of Accenture’s AI.Solutions group and were able to use an existing Accenture AI solution that uses optical character recognition and other advanced technology that allows information to be extracted from documents that are particularly difficult to read for the human eye. These include records of prisoners and detainees containing dozens of lines as well as records of concentration camps and other administrative documents of the Nazi regime containing information about the location and fate of entire families.
Because the AI captures the information faster and increases its accuracy as the process goes on, four volunteers can now validate about 160 documents in one hour – a 40-fold doubling of the power. Since Accenture implemented the solution in December 2021, more than 160,000 names of victims of Nazi persecution have been added to the Arolsen Institute’s index, more than 18,000 documents have been extracted, and more than 60,000 documents have been grouped into similar groups to improve identification and analysis. About 1,000 Accenture people have volunteered for the project to date, with Accenture also supporting the maintenance and further development of the artificial intelligence solution.
Although artificial intelligence does most of the work, human oversight is essential to the process, not only to ensure accuracy, but also to perform learning about the functioning of artificial intelligence. By reviewing and correcting information, volunteers “teach” the solution to identify handwritten characters and abbreviations that were typical of the period. Thanks to human effort so far, artificial intelligence has gradually improved its accuracy by 10% and is now operating at a level of 99% success.
“We are proud of the efforts of our employees to help and preserve the memory of those who went through unimaginable hardships and suffering, especially at a time when anti-Semitism, racism and extreme nationalism are once again erecting their ugly heads,” said David Matnik, Director of Education and Public Sector at Accenture, USA , And whoever took the project under his wing. “We saw a problem, which embodies an opportunity to realize our values as a society and as human beings and harness our technology for the benefit of all.
“We are thrilled by the responsiveness of the volunteers as well as the willingness and professional abilities they have demonstrated when it comes to the digitization process of our archive,” said Florian Azoulay, director of the Arolsen Archive. “We are shocked by some volunteers who support the digitization of our archive,” said Florian Azoulay, director of the Arolsen Archive. “Our collaboration with Accenture’s team is exceptional in every aspect of it. It is an amazing achievement that will allow us to access the contents of tens of millions of documents containing important information about the fate of victims of Nazi persecution at record speed, and even allow many around the world to discover what happened. “Dear ones.”