DOI: 10.1145/3241979

Metamorphic testing can test untestable software, detecting fatal errors in autonomous vehicles’ onboard computer systems.

BY ZHI QUAN ZHOU AND LIQUN SUN

ON MARCH 18, 2018, Elaine Herzberg became the first pedestrian in the world to be killed by an autonomous vehicle after being hit by a self-driving Uber SUV in Tempe, AZ, at about 10 p.m. Video released by the local police department showed the self-driving Volvo XC90 did not appear to see Herzberg, as it did not slow down

or alter course, even though she was visible in front of the vehicle prior to impact. Subsequently, automotive engineering experts raised questions about Uber’s LiDAR technology. 12 LiDAR, or “light detection and ranging,” uses pulsed laser light to enable a self-driving car to see its surroundings hundreds of feet away.

Velodyne, the supplier of the Uber vehicle’s LiDAR technology, said, “Our LiDAR is capable of clearly imaging Elaine and her bicycle in this situation. However, our LiDAR does not make the decision to put on the brakes or get out of her way” ... “We know absolutely nothing about the engineering of their [Uber’s] part ... It is a proprietary secret,

and all of our customers keep this part to themselves” 15 ... and “Our LiDAR can see perfectly well in the dark, as well as

Metamorphic
Testing
of Driverless
Cars
key insights

˽ Many software systems (such as AI systems and those that control self-driving vehicles) are difficult to test using conventional approaches and are known as “untestable software.”

˽ Metamorphic testing can test untestable software in a very cost-effective way, using a perspective not previously used by conventional approaches.

˽ We detected fatal software faults in the LiDAR obstacle-perception module of self-driving cars and reported the alarming results eight days before Uber’s deadly crash in Tempe, AZ, in March 2018.