Highly Automated Driving Demands Sensor Fusion
In current implementations, there are solutions, in which a particular type of sensor provides data for a specific function. More complex functions such as traffic jam assistance systems, however, require shared use of multiple sensors of different types and the associated links. The link between sensors and functions is becoming multidimensional. Given the increasing number of sensors and the ever more complex functions for a highly automated driving mode, one needs a centralised environment model as the basis for different ADAS functions. This model is based on loads of sensor data and their assessment builds the basis for actions that are real-time interventions in the road traffic and thus require hardware and software implementations with a high level of performance, operational safety and low power consumption.
In the process of creating a centralised environment model different types of sensors are used, which provide redundant and supplementary information thereby reducing the dependence upon individual sensors, for example, cameras that cannot achieve the specified detection rate or report objects where none is there, so called “false-positive detection”. Supplementary sensors, in this case, could be Radar or Lidar.
To merge these sensor data into a unified view, one requires massive computing power. Sensors like cameras or Radar can transfer data at a bandwidth from 10 Mb/s to 40 Mb/s, which quickly adds up to 500 Mb/s of net data in a comprehensive sensor system. This data must be fed through the vehicle network and collected by an ECU in real time. For this, the ECU must have access to a Gigabit Ethernet including the corresponding switches with automotive reliability and quality level.
For a correct representation of the surroundings, the ECU must not only receive the data in real time but also process it. Since the sensors may not detect all objects at the same time, the data must be chronologically synchronised with one another; at the same time, since each sensor is uniquely arranged regarding its location and how it detects objects, the data must also be spatially synchronized with each other. On top of that, the ECU must assess the incoming data for plausibility, link it together and classify the detected objects regarding their relevance to the vehicle or to future actions like steering or braking. High performant architectures like System on Chips (SoCs) along with a host MCU are required to achieve all this.
The SoC delivers the computing power while the host CPU controls the actuators. Besides the massive memory bandwidth using DDR3 or LP-DDR4 interfaces, the SoC must also be able to provide the high-speed link to sensors and other computing units via Gigabit Ethernet and PCI Express. The processing and assessment of data requires multiple 1000 DMIPS up to several 10000 DMIPS computing power. This necessitates sufficient dimensioning of the on-chip structures such as the multi-layered buses and cache memories that are connected to MPUs as well as to hardware accelerators. The third generation Renesas devices, like the recently introduced R-Car H3 family, are designed to handle such tasks. The DDR interface can provide a significant bandwidth up to a peak value of 51.2 GB/s with a high average throughput. For data processing, Renesas has implemented the latest ARM-v8A 64-bit Cortex-A57/Cortex-A53 CPU cores with a computing power up to 40,000 DMIPS. Besides standard MPUs, special hardware accelerators have been used to provide additional performance at low power consumption. This is realised through the implementation of GP-GPUs and special vision processors. Thanks to the parallel instruction processing, the vision processors enable a significant speed improvement in handling large data arrays; this is because depending upon the type of algorithm to be used, these processors offer SIMD or MIMD execution modes.
Also, the sensor fusion ECUs operation is safety relevant and must conform to the accepted safety standards. The automotive industry is now widely using the ISO26262 safety standard to achieve the safety goals of safety-critical functions. The architecture underneath, at the vehicle level right up to the internal ECU details, must undergo a safety analysis. It includes the inspection of the development process and the technical architecture of all layers to reduce the risk of systematic or random hardware faults. The result allows classification in safety levels from QM (Quality Managed, without taking any safety measures into considerations) up to the highest automotive safety level ASIL-D (Automotive Safety Integrity Level D). The ASIL levels describe the probability of failure occurrence over a predetermined time: at system level, 100FIT is required for ASIL-B and 10FIT for ASIL-D. ASIL support in MCUs such as the RH850-P1H Series, which supports ASIL-D systems, is becoming standard; however the implementation of safety requirements in SoC is new and has only become necessary with the growing demands for the highly automated cars. Due to the complexity of SoC architectures, the implementation of the functional safety at minimum additional system costs requires detailed know-how and experience about the integration of the underlying safety mechanisms. Thus, for example, the implementation of the larger Cortex-A CPUs including their cache memories as lock-step dual-core, a common solution in designing safety MCU for ASIL-D applications with smaller CPUs, is economically not feasible as SoC. This also applies to the complex circuits like GP-GPUs, which cannot be implemented redundantly. Thus, as the key safety aspect of the automatic driving mode, a detailed analysis of the safety requirements and the target application case must be taken into consideration. This is the only way to assess the relevance of the safety aspects and the cost efficiency of a large-scale production. Another example is the implementation of memories. If one uses the planar technologies of different geometric sizes for that, the FIT level increases with every step towards smaller geometries. In SoCs with multiple megabytes of integrated SRAM, the underlying FIT level before taking the safety mechanisms into account is naturally high. To achieve the target criteria for ASIL-B a judicious selection of error correction measures like ECC and physical measures such as the use of finFET transistors is mandatory.
Another requirement is that all relevant, randomly occurring hardware failures must be detected during the driving cycle, and the system must introduce measures to ensure that these failures do not impair the driving safety. This may lead to a reduction in the range of available functions, but it is important that the life-threatening situations are avoided. For this reason, depending upon the functions affected, provisions for a certain level of functionality in case of failure of a highly automated driving mode must be made, so that even in the event of failure the vehicle remains in a safe traffic condition. The most obvious measure is to provide a fully redundant ECU and the usage of a second computing path when the main path is in safety mode. Due to duplicate implementation such a system is costly and besides the larger space requirement, it would also lead to higher power consumption. Since the control ECUs often have limited cooling possibilities, the available power budget is in general below 20 Watts. This includes a SoC that provides the high computing power and functional safety, an extremely fast network connection with communication switches, large memory, a high-safety MCU and corresponding power conversion stages. Consequently, all components of this ECU must be highly energy efficient. SoCs in particular, must utilise the latest process technologies and the power management techniques available such as the power insulation (to reduce the leakage current of unused circuit blocks), clock gating and dynamic clock frequency adjustments in order to keep the power consumption of the SoC below ten Watts. Dedicated hardware accelerators developed for use in the automotive industry contribute towards this.
At the same time, the combinations of different sensors and functions pose new challenges for the implementation of the software. This leads to new requirements for semiconductor vendors and their partners and calls for novel approaches towards the realisation of highly automated driving mode.
The AUTOSAR framework that is used by MCU software in most of the cases is not well suited for the dynamic memory allocation and multithreading capabilities of modern, sophisticated CPUs. For this reason, the use of complex real-time operating systems, their interaction with existing AUTOSAR mechanisms and the use of the time-slot method and controlled sharing of resources is regarded as one innovative solution. Such architectures allow the ECU to receive and implement updates without affecting the underlying system. Further, this architecture also permits that software from different vendors can coexist side by side. Such mechanisms facilitate the planning not only during development and integration phase, where development is done independently followed by integration but also for deterministic execution during operation. This is mandatory because hardly any single vendor commands the necessary expertise for all services that are combined with each other in a single ECU. Thus different software packages may share the operating resources of available CPUs, hardware accelerators and communication networks – without mutually blocking each other or occupying the shared resources. Such behaviour is one of the most important requirements for sensor fusion ECUs. To accomplish this, one must use special tools already during the implementation phase. Renesas is working together with partners in the R-Car Consortium who command these mechanisms and facilitate a predictable and reliable integration of software modules by defining clear limits for each software module.
Sensor fusion is a complex and literally a crucial component of the highly automated driving mode. It requires a bundling of the latest technologies, highly efficient SoCs and MCUs, the highest functional safety standards coupled with the expertise of experienced integrators and leading partners in the software and middleware fields. Put together, it enables a user-friendly and safe implementation and at the end, gives the driver the option to choose from: Self-driving, assisted by multiple safety mechanisms, or let oneself be driven in an automatic guided driving mode.
If you enjoyed this article, you will like the following ones: don't miss them by subscribing to :
