• This email address is being protected from spambots. You need JavaScript enabled to view it.

Autonomous Driving / ADAS

The behaviour prediction is fundamental in the autonomous vehicle function. This is where the vehicle predicts the future states of nearby objects based on both current as well as past observations.
To do so typically various deep-learning and mathematical approaches are being used.
Common for all is the need to handle vast amount of data in real time.
AI + Signal Processing Necessary For L4 Autonomy

AI + Signal Processing Necessary For L4 Autonomy

The autonomous vehicle must be able to work under all types of road, weather and light conditions. This includes nighttime, low light conditions, fog, rain, snow, black ice etc.

To be able to properly plan the right course of action the vehicle must perceive its location and its environment. The environmental sensing is derived from inputs from its many sensors - cameras, radar, lidar, ultrasonic, GNSS etc. The data from the sensors is then either pre-processed locally and then passed on to the tracking system or sent directly as raw data to the tracking system. The received data will be combined (fused) so that the tracked object(s) can be handled in a rational way by the next stage of the system.

Typically a DOGMa scenario will be used for this - a very good example is a Particle Filter - but there may also be AI implementations involved.

Key is that VSORA's architecture is algorithm and host processor agnostic, thus making algorithm implementions easy, especially since all programming is done in high-level language (Matlab-like/Tensorflow-like/C++) language or by using graphs.

The Power Gap

There is an element that studiously is being avoided when discussing full autonomy (L4/L5) for vehicles. It is the effective compute power required when implementing the enormously complex algorithms that are required to execute in real time. The realistic timing budget can be measured in a few tens of milliseconds.

Listening to the discussions and looking at available media one is easily led to believe that the L4 autonomy ("Eyes off") the market wants is here, and that it is not being deployed for other reasons than availability, be that political, legal, financial etc.

The reality is that what is currently being offered is limited to L2 ("Feet off") and the beginnings of L3 ("Hands off").

There is a simple reason for this - compute power! The compute power required to handle the data from the 40+ sensors one will find in an autonomous vehicle in real time is exponentially higher than what is currently being offered as state-of-the-art solutions.

There is a clear gap between what the market wants and what is being offered on the market - the Power Gap.

The Power Gap


L4/L5 Autonomy Cannot Be Achieved Unless AI Is Combined With Signal Processing

Functional Safety

Functional safety (ISO 26262) is a vital element in an autonomous vehicle implementation. Level 4 and above assumes that the vehicle will be able to behave as a human driver would under the same circumstances. Implementing solutions will require redundancies to exist and that the appropriate ASIL level is met (ASIL-D or ASIL-B typically).

These are decisions ultimately made by the vehicle manufacture in cooperation with its suppliers. In the background one may consider what could happen in case of an accident - in a level 2 vehicle for example the human driver will be responsible for the way the vehicle is being controlled, and that will be considered in the event of an accident. This scenario is different in a level 4 vehicle, since by default the vehicle is responsible. So who will be held accountable in case of an accident?

The best way to minimize this dilemma is to ensure that the vehicle control is designed in a way to ensure the highest possible safety.

Sensor Processing

The sensor data will require processing and this inevitably will involve elements of signal processing. Take a typical example of what is considered an AI centric application and use case: KITTI PointPillars (see image below). This is a #D object detection algorithm that is often used to analyze the AI performance of a solution.

Once one start breaking down the elements included in the algorithm one will quickly find that the backbone is truly AI centric, but the rest of the algorithm involves the use of both AI and signal processing.

Another example would be the use of Particle Filters in a DOGMa scenario. This relies entirely on signal processing. The result however may well be run through a pure AI algorithm.


Source: PointPillars: Fast Encoders for Object Detection from Point Clouds;https://arxiv.org/pdf/1812.05784.pdf

Algorithm Diversity

Since the algorithms being used in its ultimate form will be responsible for human lives it is essential that some sort of redundancy is being built into the system. The technical redundancies are being handled by the ISO 26262 part of the implementation, but what about the algorithms being used?

Running the same algorithm on multiple processing units and then comparing the results would be one way of doing this. Typical setups would either be a lockstep scenario or a voting system.

This, however, will not be able to handle inherent algorithm errors. As the algorithms are becoming ever more complex, the ability to test all possible permutations in all cases reduces. Like all software, the more widely it is being used the more likely it is that a never before seen error pops up. How can you handle that in a vehicle going full speed on the road - especially since the lockstep setup actually does not detect it as an error?

The only realistic way to do that would be to use algorithm diversity. An example of this could be a system using a predominantly AI centric analysis that uses a predominantly signal processing system as a comparison. Two good comparisons would be to look at the output of a PointPillars setup and a Particle Filter setup. Ideally the output, if run in parallel using the same data, will generate the same result.

VSORA Compute Companion IP Core

AD1028 Block Diagram

There are 3 main components to the AD1028:

  • An advanced signal processing unit (DSP)
  • An AI unit consisting of a series of identical cores
  • A tightly coupled memory (TCM).

Advanced Signal Processing Unit

The signal processing part of the AD1028 contains 1024 ALUs. This is a normal signal processing unit, with the main difference that the number of available ALUs are significantly higher than what can be found in any other DSPs of today. The other main difference is that the DSP can access all of the available TCM.

AI Unit

The AI unit consists of 16 cores. Each core is built up of 16k MACs, so the AD1028 provides 256k MACs to the system. The AI unit can access all of the TCM as well, and thus the data can quickly and efficiently be shared between DSP and AI unit.


The size of the TCM is is typically around 30MB. For an IP solution the optimal size can be derived using the development tools, but even if the size of the TCM should be too small for optimal performance the unit will still work, if only with a slower throughput.

AD1028 Overview

AD1028 Chip

Companion Solution Use

Development Tools