Can you introduce yourself in a few words?
My name is Alex GRANDJEAN and I have worked at AVSimulation for 6 years, as product owner and project manager. I am an expert in scenarisation, traffic simulation and HPC. I am involved for more than 4 years in SVA and 3SA collaborative projects in partnership with IRT SystemX to define new methodologies based on simulation to demonstrate that autonomous vehicles are safe.
Can you give us a quick definition of massive simulation?
First, you need to understand why massive simulations are mandatory for autonomous vehicle and ADAS development.
The development of ADAS must prove the safety of these systems by simulating them over millions of kilometers, in all possible cases (through changes of weather, roads, markings, vehicles, etc.)
The only way to do this is to use simulation, and optimize it by running millions of simulations in parallel, on dedicated infrastructure. This allows you to stop relying on a single workstation or driving simulator. Of course, you need to adapt your simulation infrastructure to massive simulation.
What are the different steps in massive simulation?
Everything starts with the ODD (Operational Design Domain) which will define all the situations that the autonomous function will have to manage. This will allow to select from the simulation catalog, which scenes and scenario must be integrated in the test plan: highway/country roads, situations with car/pedestrian/trucks, etc. The ODD, along with safety goals, will also help to identify the ranges of parameter that will be selected. For example, if we only focus on an autonomous shuttle evolving on restricted road, the range of speeds can be limited under 50 km/h.
Then, there is a number of steps in achieving a run of massive simulation:
First, you need to prepare your data. For example, the vehicle you want to use; which ADAS you want to test on your vehicle; which scenario and virtual environment will be used for the test. For the scenario we have developed templates scenario for each family which contains many parameters that can be modified at each run: speed of the ego vehicle or the participant, weather conditions, curvature of the road, dangerousness of the maneuvers, etc.
Secondly, you will create your test plan by selecting the proper scenario and for each parameters the ranges to cover. As the pure combination of all this parameters often lead to combinatorics explosion, a statistical method to explore this wide multi dimension space is often needed. This will help to reduce it and generate millions or more test cases, by creating variability in the parameters (for example weather, other vehicles’ speed or characteristics, road markings, etc.) of the use case.
Then, if you want to simulate your millions of scenarios in a reasonable amount of time, massive parallelization of the computation on local HPC or cloud infrastructure is mandatory.
Last but not least, you will analyze the results of your test plan to see if your ADAS worked well in all situations. This analysis will help you understand which situations or combination of parameters your ADAS can’t handle.
If not all the tests are passed, the system must be improved and the previous steps must be repeated until the analysis shows that everything is successful.
What are the analysis options that follow theavailable after massive simulation tests are run?
As I mentioned earlier, you need different degrees of analysis when a simulation run. First, you want to quickly know whether your system worked well or failed on all the scenarios you simulated. This allows you to take a closer look at those specific cases because you need to understand why the system failed. For example, it may be that the input parameters were entered in the wrong spot and the system cannot work properly. It is very important to analyze all the parameters and outputs of the system in more detail to understand which function has of the system has failed.
You should also check the coverage of your test and check whether you really have explored in detail all possible parameters of your definition space. With this, you can answer the question whether you have done enough testing on your system or not.
Can you tell us what tools are needed to use it and what they are used for?
For each step of the massive simulation presented earlier, there are two dedicated software packages in the AVSimulation products.
To prepare your models (vehicle, sensor, Roads) create the scenario and interface the AD/ADAS system, SCANeR studio is the perfect tool. It will help you setup everything and verify that everything works fine on a few scenarios.
Then, to generate your test plan, with variability generation based on constraints, you need SCANeR explore. It will allow to define the best strategy to explore the wide space of possible combination with advanced statistical methods.
To run the actual simulation in a dedicated cloud infrastructure -like Azure, AWS, Alibaba-, l use SCANeR compute and all the apply that come with it.
Finally, to analyze the simulation data, you can also use the results provided by the API of SCANeR compute and SCANeR studio to use different technologies.
Soon, the new massive simulation platform that we are developing, will execute all these tasks from a single web interface.
What is the API and what is it used for in massive simulation?
API stands for Application Programming Interface. It is usually used to connect different tools. For example, a lot of SCANeR product users with a toolchain of different tools for performing massive simulation have their own tools in their toolchain for specific steps. They need to connect their tools to others and perform a full run of massive simulation. AVSimulation product provides API for connecting to SCANeR. API also allows to customize or extend the capacities of our tools: for example you can define your own statistical methods in SCANeR explore or connect your own computation platform.
We are agnostic with regards to the cloud platform, we already successfully deployed massive simulation on Microsoft Azure, AWS, Alibaba.
Is there a limit to the number of simultaneous tests?
Not really! The only limit is the infrastructure you can build and the budget you have to run all the simulations, you can run as many simulations as you want on a cloud platform.
The important point is to understand what you are simulating and testing, to avoid useless computations.
Can you give us an example of a recent project where massive simulation was used?
A good example of a client using massive simulation is Renault. SCANeR -studio, explore and compute-, the software of AVSimulation is the backbone of massive simulation for ADAS validation platform, computing continuously millions of kilometers to validate theirs systems.
This project shows that massive simulation greatly reduces the number of physical tests, reduces time to market, tests costs and the number of physical vehicle prototypes.
A word of conclusion?
Massive simulation is a revolution in the process of vehicle validation and development for ADAS and autonomous vehicle development. It really opens up a new way of using simulation for developing vehicles. Thanks to the cloud we can very rapidly allocate thousands of computation nodes that will allow to run millions of tests in minutes.
We are starting massive simulation for ADAS validation but we can imagine many other application for such tools and platform like dataset generation for machine learning for example. There is still a lot of things to imagine and build with this new approach to simulation, and we don’t run out of ideas.