Human drivers are usually trusted to make better decisions in road traffic than software. The consortium of the project "Product security for cross domain reliable dependable automated systems" (SECREDAS) has set itself the goal of strengthening trust in such networked automated systems. This is not only about physical security, but also about trust in data privacy. 69 partners from 16 European countries are participating in the project, including the Fraunhofer Institute for Experimental Software Engineering IESE.
In autonomously driving cars, neural networks play an increasingly important role in control and situation recognition. The difficulty here is that the way in which these neural networks make their decisions cannot always and never be fully understood. The researchers are therefore developing a safety supervisor that monitors the decisions of the neural network live, so that, if necessary, regulatory intervention can be carried out on the basis of these assessments. This supervisor is based on algorithms that make use of classical approaches. Using these, the researchers do not record the overall situation like the neural networks, but critical key points. In this context, the researchers' first concern in the project is to identify suitable metrics; the introduction of suitable countermeasures to control the risk will then be the subject of further work.
The researchers explain how this is done exactly using the example of a real intersection situation: The neural network is designed to capture the overall situation: Which right-of-way rules apply, is the traffic light red or green, are pedestrians within the danger zone, do other cars cross the planned future roadway? Instead, the Safety Supervisor's algorithms rely on specific metrics. These would be, for example, the "General-time-to-collision (GTTC)", i.e. the time to a collision taking into account the expected trajectory, or the "Worst Case Impact Speed" metric for assessing the severity of damage based on the expected collision speed. If the car is heading towards another road user who should have escaped the neural network, the algorithms of the safety supervisor recognize that the distance is shrinking to a dangerous degree. They can take command and brake the car if the autonomous control fails. In a simulation, the researchers evaluated the suitability of these metrics for various dangerous situations. The result is positive for the researchers: "The approach of checking the neural networks at any time and live using classical approaches, together with dynamic risk management, can significantly increase safety," says Mohammed Naveed Akram, scientist at Fraunhofer IESE.