We humans rely mostly on our eyes while driving – and have good reason to do so, since the human eye is a highly evolved visual instrument. Its impressive specs include a field of view of roughly 180 degrees, a resolution of about 576 megapixels, a palette of 10 million colors and the ability to process up to 1000 frames per second. But there are limits to our vision. For one thing, we have to shut our eyes every four to six seconds – and a lot can happen in the blink of an eye, especially when we're behind the wheel.
Combination of Radar, 2D Camera and 3D Flash Lidar
Machines never have to blink, and they can still see in darkness, rain, snow or even fog. In the modern automated driving systems being tested by Continental, three types of sensors are at work: radar, a 2D camera and 3D Flash Lidar. Unlike the often bulky and fragile moving lidar systems that are typically mounted on top of the car, these sensors are combined in state-of-the-art units that are robust and compact. Moreover, the technology is integrated within the vehicle body at strategic points, providing a constant and safe flow of hyper-accurate visual information.
Each sensor complements the others with extra abilities and features. Thanks to its natural electromagnetic wavelength, radar can see far ahead even in the densest of fog. 3D Flash Lidar works with a laser pulse that is picked up by its own smart pixels, which in turn deliver a detailed image with precise object contours of the road and the vehicles by picking up surface reflectivity, which radar cannot. But even if it provides a high-definition 3D picture, lidar does not process colors, and this is where the 2D digital camera comes in.
Mastering Daily Routines and Difficult Situations
Imagine driving through thick fog in an automated vehicle. Despite poor visibility the radar tells the system about the various objects in the road, while 3D Flash Lidar verifies and refines this data by providing optical resolution that is precise down to a few centimeters. This data is enhanced by color information from the 2D camera, which lets the system know whether the lane markings are white or yellow, whether the next traffic lights are green, yellow or red, or whether the car in front is signaling a turn. Current systems are able to check pretty much everything up to 250 meters ahead.
Another example is a metal chain blocking a parking space. Again, the reflectivity of this thin object is very low, but thanks to the combination of radar, 2D and 3D imaging, the system can register the object and its proximity to the car in real-time.
Most importantly, the fusion of these three technologies allows the system to sense moving objects such as pedestrians or animals, even under difficult conditions. For instance, while the 2D camera might be defeated by bright sunlight, radar and lidar are not affected by light whatsoever.
Tough conditions like these are why Continental, when testing and developing integrated systems for automated driving, focuses on the challenges of unexpected and unusual traffic situations. By using highly advanced radar, 2D camera and 3D Flash Lidar technology, Continental demonstrates the safe and perfect cooperation between all components of the system. It’s something we can rely on at all times – like the human eye, only better.