These are the results of the 5A project of the Politecnico di Milano, Fondazione Sacra Famiglia and Irccs Medea which involved 27 adolescents with this disorder, caregivers and therapists
Virtual reality, augmented reality and conversational agents (chatbots) can help young people with autism spectrum disorder (ASD) to improve their independence to promote their social inclusion and improve their quality of life. the conclusion of the 5A project (Autonomie for Autism Through virtual reality, Augmented reality and conversational agents), which lasted two years and was carried out by the Milan Polytechnic with the Holy Family Foundation and IRCCS E. Medea – La Nostra Famiglia Association, thanks to the contribution of TIM Foundation. This is the largest experiment conducted so far on the subject, at an international level.
The findings were presented on World Autism Awareness Day on April 2: i 27 adolescents with Dsa involved, medium/high and high functioning, reported a clear improvement in autonomy measured on the basis of questionnaires. The technological innovation, in this case, has been tested in the field in two scenarios such as subway and railway stations.
The starting point
Virtual Reality allows the person to practice using public transport, immersing himself, through a wearable viewer, in a digital environment that simulates spaces and activities typical of the use of the train and subway. Augmented reality applications they support users while using public transport in the real world, generating visual information on tablets or smartphones which appear as superimposed on the vision of the surrounding environment and help people understand how to move and what to do.
Both types of applications integrate a Conversational Agent , a chatbot renamed Pan, which acts as a virtual companion and talks proactively with the user to guide him both during the simulation of the use of public transport and during the experience in the real world. 5A applications were co-designed by a multidisciplinary team made up of engineers and interaction designers from the Milan Polytechnic, and autism specialists from the two clinical partners – the Holy Family Foundation and IRCCS E. Medea – La Nostra Famiglia Association. At the centers of these partners, rigorous experimentation was carried out which involved 27 adolescents with Dsa, their caregivers and 8 therapists, to empirically evaluate the usability and effectiveness of 5A tools.
Path
The union between the various players in the project born well before 5A. The project takes place in the context of a tender from the TIM Foundation but in our laboratory we had already explored this path through master’s and doctoral degree theses and various research – explains the Professor Franca Garzottoprofessor of Information Processing Systems and scientific director of 5A — in fact we knew the two partners, the Holy Family Foundation and the La Nostra Famiglia Association, because in the past we had worked with them in other contexts.
5A a project truly born together with these two therapeutic partners, and one of the basic ideas of this project born from them: addressing the issue of cognitive rigidity through technology, mitigating the generalization difficulties of people with autism, and helping subjects develop conceptualization skills. Since the theme of the call was the development of autonomy for autism, we thought of theautonomy as the ability to be independent in the fundamental tasks of daily lifeand we have concentrated our effort on mobility in city spaces: the subway, the train, the supermarket, the museum, the library, the hospital and so on, he continues.
Previous searches
In previous searches
virtual reality had already been used to support the training of people with cognitive impairments but the underlying problem is that the person in the real world does not find exactly the scene with which he had experience in the virtual world, and therefore becomes disoriented precisely because he is unable to generalize what he has learned through technology, underlines Professor Garzotto. How to try to overcome this limit? We thought that after the experience in virtual reality, in which the person experiences a simulation of some tasks, when she is in the real world she needs to something that connects it to the previous virtual experience to be stimulated to apply the skills acquired and understand how
to do it replies the expert of the Polytechnic.
Here comes the idea of coordinating augmented reality with virtual reality through the transmedia elements: these elements are common to virtual reality and augmented reality, where the latter is used to help the subject when he is in the real world: here the boy can see, superimposed on the vision of the surrounding world, some visual elements of the scenario simulated in virtual reality through his smartphoneand above all he finds his “virtual companion”, Pan, a nice conversational agent with a funny shape who does not give trivial instructions but interactive, speaks, and helps to remember and generalize. This is what the therapists have clearly suggested to usand we have tried to make it as best as possible, while recognizing the limitation of not being able to map exactly all the situations of the real world in virtual reality.
Choice of boys
In Italy, according to data from the Ministry of Health (2021) it is estimated that 1 in 77 children (ages 7-9) have autism spectrum disorder. In the person with autism spectrum disorder, the different way of managing social communication and the need to adhere to stable routines can have a significant impact on the ability to carry out daily life activities independently and adapt to different environmental conditions, they explain Maria Luisa LorussoHead of the Neuropsychology Unit of Neurodevelopmental Disorders of the IRCCS Medea – La Nostra Famigliare e Monica Contidirector of Innovative Services for Autism of the Holy Family Foundation.
The main rehabilitation methodologies used to date consist of behavioral training aimed at consolidating and encouraging functional behaviours. It is an approach that is certainly effective in producing changes in an adaptive sense, even if experimentation in different contexts is sometimes complicated, they add.
Field training and testing
To achieve the objectives, the children who participated in the experimentation have first followed a training with virtual reality viewers in the centers through applications created by the Polytechnic on the basis of the indications of the therapists. The sessions served to analyze the reactions of the students and identify any critical points. The young people were then taken to the field: the section of the M1 subway in Milan (between Bisceglie and Primaticcio) and some stations of the Northern Railways. The boys were challenged with a series of tasks. As for the subway: find the entrance; to buy a ticket; use a turnstile; find the correct direction; reach the platform; exit the train at the right station; exit the station. As for the train instead: find the station entrance; buy a train ticket: find the train on the timetable board; validate the ticket; reach the platform; exit the train at the right station; exit the station.
All tasks that are generally easily tackled for the “neurotypical” person but which, in the case of people with Dsa, can become insurmountable obstacles. The train timetable, for example, is read from bottom to top and not vice versa. So even in the app design phase, we tried to recreate the virtual scenarios as closely as possible with respect to the real ones. But if the reality that autistic people then face turns out to be different for them perhaps due to a small detail, their “cognitive rigidity” it blocks them, says Professor Garzotto.
The results
The experimentation had encouraging results. All 5A applications showed good usability, both for users and for therapists In both scenarios (train-meter) both the autonomy perceived by users and that measured by caregivers improves. Users have perceived 5A apps as very useful. Caregivers believe that 5A applications have a therapeutic potential; perceived 5A apps as useful in both everyday and therapeutic contexts and would (mostly) like to reuse the apps in the future.
For the children involved, participation had positive effects on their self-esteem and personal motivation because they felt chosen to take part in a project whose development and success depended on their contribution. Caregivers have also enthusiastically joined, moved by the desire to expand the opportunities for their children to have experiences that are a source of well-being and personal satisfaction. This project is in line with the commitment of the Foundation which has always experimented with innovative paths aimed at offering autistic people the best opportunities for growth and autonomy, observes Monica Conti.
Future scenarios
What will become of the 5A project now? At the moment these are actually apps designed, and not born, with therapeutic purposes, but we believe they could be used as an adjunct during more classic therapeutic interventions for autism and more generally for neurodevelopmental disorders. It would be nice to turn it into an app that everyone can then use, says Professor Garzotto. This technically possible: the technological skeleton that we have created very modular. It exists due to a problem of human and economic resources related to the need to increase the number of scenarios and keep applications alive, and we need to find a way to finance the extension and technical maintenance of apps in the medium to long term, he concludes.
March 31, 2023 (change March 31, 2023 | 15:21)
© Time.News