Dhe relationship between pedestrians and drivers is a special one. Collisions between these two species are to be avoided at all costs, not only for the obvious reasons, but also because of the special protection for vulnerable road users. The German judiciary tends to at least partially blame the driver because of the operational danger of his vehicle even if the pedestrian ran straight into the car.
Inattentiveness or distraction of one of the parties involved is not always the cause of the accident. Frequently, there is simply a lack of communication between the car and the pedestrian, a problem that is likely to get worse with the growing use of autonomous cars. In addition to the armada of stereo cameras, radar, lidar and ultrasonic sensors on and in the vehicle, improved early detection of possible misbehavior on the part of pedestrians could be helpful to prevent accidents. This is what researchers at the Fraunhofer Institute for Optronics, System Technologies and Image Exploitation (IOSB) are working on, who want to teach self-driving cars social skills, so to speak. Special artificial intelligence (AI) systems should anticipate the behavior of pedestrians in order to enable interaction with autonomous cars.
“Driving a car is much more than accelerating, braking and steering. Communication with other road users plays a decisive role,” says Manuel Martin, graduate computer scientist at the IOSB. “The autonomous car must be able to recognize early on whether a pedestrian at the side of the road is going to enter the lane or not. What the person at the wheel usually perceives intuitively and without thinking, for example based on the pedestrian’s location, line of sight or gestures, must first be taught to the self-driving vehicle so that it can act independently.” For this purpose, the behavior of the pedestrian must be precisely analyzed and be interpreted.
Although methods of artificial intelligence have the potential to analyze video images, they must first be fed with large amounts of training data so that they can draw the right conclusions. This is currently being worked on at the IOSB as part of the research project “Intelligent Human-Technology Communication in Mixed Traffic”. The institute recently presented a prototype that can assess whether a pedestrian will cross the street, analyze their gestures and thus create the basis for interaction.
The system consists of a stereo camera that can “see” three-dimensionally and thus detect the exact position of the passer-by, as well as an algorithm that detects the position of the limbs and draws appropriate conclusions from this. “Now it’s a matter of continuing to train artificial intelligence,” says Martin. “The system as a whole must be refined in such a way that it can recognize the intention of the pedestrian as accurately as possible in all conceivable situations.”