Map data as co-pilot: Extended scope for autonomous driving
January 03, 2017
In this decade, highly automated driving is the goal of every car manufacturer. New players have entered the market while traditional car manufacturer...
In this decade, highly automated driving is the goal of every car manufacturer. New players have entered the market while traditional car manufacturers gradually improve driving assistance functions until they finally become completely independent from the driver. The newcomers have less experience in actual car manufacturing, but great know-how in machine learning and interaction with the cloud.
The commonality is the need for sensor data to replace human senses. An autonomous car requires information about its environment to act properly. Each type of sensor has different strengths and weaknesses. Cameras give the best overview, but the images must be processed in real-time to provide a proper environment model. Radar can detect objects very well and deliver a proper environment model, but is easily confused by metal objects like drain covers. Lidar, a laser-based system, seems to be a good compromise, but it is very expensive. The answer is to combine multiple sensors to get the most likely picture.
These sensors gain range with every new hardware generation, but they still have physical limitations. They cannot see much further than 200 meters, cannot look around corners, or see through hills or other big obstacles. A car relying only on sensor data will therefore drive like a visitor from another city, as if it has no knowledge about the turnoff lanes, crossings, or curves. The car will drive by its sensors and will probably perform maneuvers very late when it finally recognizes the right path to take.
Additional map data is the only way to avoid this. Such data can be thought of as being like an assistant rally driver. It improves driving by enabling smooth transitions between different conditions. For example, predictive active cruise control would sharply brake the car when the sensors spot a speed limit, but with map information, the speed can be reduced in ahead of time. Special conditions like sharp curves, dangerous inclines, or other obstacles are also known in advance. A detailed and up-to-date map is vital to provide this information. Such a map contains two kinds of information: Static and dynamic.
- Dynamic content changes all the time. Such content includes data about accidents, hazard warnings, traffic jams, or weather conditions. It needs to be reported, processed, and distributed immediately. Otherwise, it would be simply outdated and lead to incorrect action. This information correlates to the traffic news on the radio.
- Static content does not change that often. Examples are slopes, curvature, lane markings, or signposts. This data can be collected gradually, and the processing and distribution may take days or even weeks. This correlates with the information on paper maps.
A feedback loop must be established to provide such a map. The vehicle reports potential map information, which is collected and added to a map, and then distributed back to the vehicle. The feedback loop must be as fast as possible for dynamic content if it is to be useful. In contrast, static content is still valuable after long time, so it could be buffered in the vehicle when an upload connection is unavailable.
The feedback loop is fed continuously with the objects or situations recognized by all participants in both cases. Timestamp and position data is added to the sent messages, but some kind of authentication is needed to avoid misuse of the upload portal. This authentication allows a vehicle to connect to cloud services and prevents uploading of malicious messages by unknown senders. It usually allows the receiver of the message to determine the sender, and make decisions about whether to keep the information about the sender permanently to allow for further evaluation or discard it during the processing chain to ensure privacy.
The uploaded information must also be validated, with every record checked for accuracy and invalid records removed immediately. For example: if a GPS signal is not available, the location data for latitude 0 and longitude 0 in the Atlantic Ocean might be sent. This error is detected easily when it is received. Besides the position data, timestamps should also be treated with care, since the local time of a vehicle may not necessarily be correct. This might be due to wrong time settings or a low quality clock in older vehicles, and newer vehicles might be set to the wrong time zone during travel. Since the dynamic content must be sent immediately, the arrival time of the message on the server can be used. This can be trickier for buffered static messages when the sent time stamp cannot be trusted and the message might have been kept in the buffer for an undefined time. Static content, fortunately, is not as time critical as dynamic content, so the precision of the time stamp is not as relevant.
The information received from all vehicles is collected in a large database, and dynamic data should be distributed as quickly as possible. It makes perfect sense, however, to collect additional information about the same item. For example, the position of spilled oil spotted by a vehicle is always inaccurate. This is because the sensors and algorithms in a car are optimized for correct recognition rather than for accurate location. Recognition of whether or not it is oil may be wrong in itself. This may be due to a broken sensor, bad weather conditions, or because the algorithm was stimulated to recognize something completely different than an oil puddle. The only solution is to aggregate multiple data points about the same item. It is only a mathematical challenge to determine how many cars need to report the same attribute “slippery road” at roughly the same place to determine the correct location and be able to trust the information.
Vanishing information is another problem affecting reliability. An example of this is a construction site along the road. It appears someday, lasts for maybe 2 weeks, and then disappears again. Such a construction site will probably affect even static information like lane markings, lane geometry, speed limits, and so on. The updated values from the cars that recognize the construction site are counted as outliers, or are unable to influence the statistical mean value of the past few years. As a result, no map update or an insufficient one will be created.
The age of the information must be considered in order to address this problem. More recent information must be rated with a higher probability than older information, even if it originates from a less reliable source. A similar problem applies when a sign is removed. The sign will simply not be spotted any more, and no car will send information about it. Therefore, a check must be performed to see if there is an item in the database for which no further acknowledgements arrive. Finally, these items must be removed from the database.
The collected and aggregated information leads to a much more detailed map. This map has more comprehensive and more accurate geometry information such as lanes, exact curvature, or gradients, and more details along the road like zebra crossings, signposts, or traffic lights. These maps are too large to be downloaded every day, even though mobile networks are getting faster with every generation and the costs for mobile data are shrinking constantly. The updates must therefore be sent out in small portions that only contain new data. Version management has to take into consideration all the information already present in the vehicle and avoid sending this information repeatedly. Nevertheless, a general update will still contain irrelevant information because a car in one city will not need to know about a traffic jam in another city. The update thus boils down to a very specific map update package for each vehicle at its given position and expected route. The update package will only contain the latest information about the current track and the side roads that can be taken next.
Minimalistic update packages have the advantage that they can be installed as well as downloaded faster. Since the vehicle may receive update packages at any time, it must be able to handle them even while driving. Either the map database must be stopped, updated, and restarted, or a live update must be performed. If the database is stopped, the vehicle loses the map as an additional source of information and reverts to purely sensor-based driving. After the restart, all the relevant data must be read from the database again and synchronized with the other sensor data. A live update guarantees continuous usage of the map, but brings with it various database synchronization problems, for example the update algorithm may change information in the database while the autonomous driving algorithm is trying to read this data. Although this may work sometimes, there is always a possibility that both algorithms will try to access the same data at the same time. Broken data sets are transmitted when this happens. Just imagine driving along a curve while the curvature is being updated. A typical database approach would be to lock the affected data and thus prevent a read operation while data is updated. But this can lead to a deadlock when each algorithm waits for the other to unlock a needed record. Eventually there has to be a compromise between the database downtime and the availability of the most recent data.
The feedback loop is closed when the most recent data becomes available in the vehicle. The car can make use of all the data collected either from its previous drive or from other vehicles. This is actually the crucial point in the whole feedback loop. It can only work properly if it has enough participants. Static map data is still collected and updated by special cars. The only purpose of these cars is to collect data. They therefore drive accordingly (i.e. slowly, carefully, and over the same road several times). These cars are equipped with highly accurate sensors and transmit very precise information. While this works perfectly to create a basic road map for navigation, it is simply not enough for a map that is to serve as a basis for autonomous driving. Such a map must outline the slightly shifted lanes around a construction site as well as indicating the changed speed limit. A special car cannot collect data often enough to keep the map of a whole country up to date given that a construction site may only last less than a month. This means that every car and every driver must participate to maintain a highly accurate map in the same way that Wikipedia users do to keep the world’s largest encyclopedia updated.
Elektrobit Automotive
LinkedIn: www.linkedin.com/company/elektrobit-eb-automotive
Facebook: www.facebook.com/EBAutomotiveSoftware
