How to Construct a Simulation Scene for Testing Autonomous Driving System in a Virtual Environment?

Developing an autonomous driving system is one thing, and testing its efficacy is another thing. It is highly possible that researchers may have made un-realistic or ideal assumptions while designing an autonomous driving system. Such designed systems may look very promising on the paper. But since ground realities could be a bit harsh, it is possible that desired results could not be achieved. But the question arises that how can such developed systems be checked and verified? The classic approach says to run a trial test. However, given the implications associated with the autonomous driving systems, trial tests in real-time could prove to be fatal. If algorithms are not efficient and reality-based, what if it hit a pedestrian? What if it violates the traffic signals and could get in a road accident? There is a lot of risks associated with such trial tests.

Simulation is one fine way to not only avoid such risks but also to test the developed system in a much alike real-time environment. While work on the development of autonomous driving systems is being done on a vast scale, a lot of researchers are working parallel to construct a desirable simulation scene for testing autonomous driving systems. The team of researchers at MIT is successful in creating such a simulation scene. They have developed a photorealistic simulation engine with various steering possibilities. Critics are regarding their simulation scene to be quite helpful in letting self-driving cars to go through numerous worse case scenarios before finally rolling down on the real-world roads.

Construction of a Simulation Scene

To make sure that autonomous vehicles perform close to reality in the virtually simulated environment, the construction of the simulation scene can be divided into three main algorithms or layers. The first algorithm defines the different road surfaces so that cars can be tested on different roads. The second algorithm lays down the network of roads such as intersections, a single road, highways, motorways, and different obstacles. The third algorithm undertakes dynamic behaviours to test how this car and other users interact and how well does designed system responds to these events. To have a deep and proper understanding of the virtually simulated environment, a thorough discussion is required. Following literature provides a brief introduction to these algorithms:

Defining Road Surfaces

To train autonomous vehicles about the different road surfaces, it is very important to define road surfaces. The algorithm takes into account friction and elevation related parameters of the road. MATLAB can prove to be quite helpful in designing such algorithms. Researchers can use the Curved Regular Grid technique to record and map different road surfaces. With this, a curved grid can be superimposed on the target road surface and desired values can be recorded. These records can then be fed to the system to evaluate the performance of the autonomous driving system. The recorded data points can then be extracted both in ASCII and binary format. While ASCII file format can be used for human interpretation, a binary file can be fed to the system, thereby training and improving the autonomous driving system.

Defining Road Networks

While road surface definition takes into account single roads, a network of roads algorithms provides a combination of several roads, resulting in a more proactive approach. In addition to the road surface algorithm, the road network algorithm takes several other parameters to test the autonomous driving system’s performance. It includes features like interconnected lanes and streets, along with the different types of intersections. It also defines whether the subjected road surface routes through a highway or a town. To further improve the algorithm, traffic signals and road signs are also connected along with their interdependencies. Moreover, certain other possible obstacles such as intra-road green belts, trees, or any obstacles on the street are also accounted in algorithm development. The inclusion of such a wide array of road network parameters ensures that an autonomous driving system is tested in close to the real-time situation while being in a virtually simulated environment.

Defining Dynamic Behaviours

While the definition of road networks and surface is the static component of simulation scene creation, the definition of dynamic behaviour is important for ensuring autonomy in real time. The ideal situation will be to define every situation or event that may occur while autonomous vehicles are on roads. The more proactive approach will be to define the behaviours of different drivers. There can be different scenarios, such as it is possible that while one driver may continue to commute steadily, another driver continues to accelerate until and unless both of them come closer and then suddenly overtake manoeuvring occurs. It is one such intense example of how complex dynamic behaviour definition can be. Likewise, there are plenty of other driver behaviours that, if defined well, can not only test the performance of autonomous driving systems, but it can also recommend more behavioural improvements in the algorithm. Nevertheless, such definitions may seem quite easy to define, but the reality is different. The foremost reason is the presence of hundreds of thousands of road users on the road, and so does their respective behaviours.

Assembling the Algorithms

After being done with the creation of different simulation scenes, integrating it with the virtually simulated environment is very essential. Once everything has been integrated, data from different sensors will be brought to the application so as to gauge the performance of the autonomous driving system. Researchers and manufacturing teams can evaluate the performance of their designs and algorithms by strictly monitoring the testing of the autonomous driving system under simulated environments. With the observations, decisions about further improvement or soft launch can be taken more proactively and systematically.


There is hardly any argument against the creation of simulation scenarios for gauging the performance of autonomous driving systems. But having said that, researchers should remember that simulation scenes would remain effective only if different simulation scenarios are designed with close proximity to reality. As observed from the above discussion, the definition of dynamic behaviours is quite a difficult task. Therefore, there is a need to stimulate greater research and observational skills so that subsets of dynamic behaviours can be noted, and eventually, get embedded in the algorithmic form. According to the reports of Rand corporation, it takes about 11 billion miles for an autonomous vehicle to reach the level of human driving. Since simulation provides similar environment, completing those 11 billion miles in a simulated environment won’t just make autonomous vehicles flawless but shall also let automobile manufacturers identify and fix the problems prior to mass production.



Leave A Comment