Self-Driving Uber Death: Role Of Video And AI

JH
John Honovich
Mar 22, 2018
IPVM

The self-driving Uber death has caused significant debate about the future and safety of self-driving cars. The car's video is a key factor in it.

Here are the video recordings of the road and driver just prior to the crash:

And here is a snapshot from the car camera a second or two before the crash:

The scene looks dark but a key issue is how dark it actually was for human vision. The issue is that car cams generally are significantly worse in low light than the human eye (e.g., Testing CarCam). Many are commenting on this using the darkness of the video as reference but how dark it really was for the operator is a key issue. Also, the operator appears to be looking down.

The final issue is the car's self-driving system most certainly does not depend on visible light sensors so why did it miss this person?

JH
John Honovich
Mar 22, 2018
IPVM

Btw, it gets worse, from the WSJ:

The Uber operator’s criminal background may also draw scrutiny. On Monday, Tempe Sgt. Ronald Elcock identified Uber’s operator as Rafael Vasquez. It isn’t clear how long Mr. Vasquez has been at Uber, and he couldn’t be reached Wednesday for comment.

He was convicted and given a five-year sentence in Maricopa County for attempted armed robbery in 2000, according to Arizona Department of Corrections records, a sentence served concurrently with a one-year sentence for a 1999 false statement conviction.

He also has traffic violations, pleading guilty in 1998 to driving with a suspended, revoked or canceled license in Tucson Municipal Court, and in 2012 was cited for failing to produce proof of insurance and driving without a current registration.

U
Undisclosed #1
Mar 22, 2018

I haven't read into the details of this incident yet, but in general I'm surprised the car sensors did not pick up the pedestrian/cyclist. Don't they typically have lidar and other sensors working together just for this purpose? The cameras are mostly there for human use, so I don't imagine the car makes any navigational decisions based on that poorly lit video.

I'll have to read up on the car used here because my gut tells me the technology is not up to the safety level of Google's cars or even Tesla's.

While the operator's criminal background has little to do with this incident, how Uber hired someone with that background for this job is beyond me.

(1)
(1)
U
Undisclosed #3
Mar 22, 2018

Don't they typically have lidar

Yes, but lidar still depends on reflectivity of objects, and some clothing has relatively poor reflectivity. Beyond that, you also have the "AI" systems that convert the point cloud data from lidar into "objects", which can be challenging in cases where you have poor reflectivity, or even a dirty lidar lens.

I don't know what was involved in this case, but lidar can have limitations, though I would have expected that at such close range, the lidar would have at least known there was some kind of solid object there. From the exterior video it looks like she may have come from the side of the vehicle in a potential lidar blind-spot, depending on how the sensors were arranged on the vehicle.

(2)
Avatar
Brian Rhodes
Mar 22, 2018
IPVMU Certified

The machine failed to be safe for the first and most obvious rule of self-driving cars:  Do not run over people.

It is a bad accident, and even if/when Uber tweaks the safeties and sensors, there's going to be people who distrust self-driving cars as a general sentiment.

(2)
U
Undisclosed #1
Mar 22, 2018

Agreed. With how few self driving cars are on the road, any death is going to be justifiably scrutinized. Hopefully it will be shown over time that self driving cars will have fewer accidents and deaths compared to human operated vehicles.

Personally I like the idea of all self-driving vehicle engineers working together on a singular standard/AI in order for all vehicles to have a similar and ideally higher level of safety. That way as advancement is made, all manufacturers would be able to update their cars simultaneously. I don't like the idea that Google or Tesla might have the best system and Uber could be allowed to run less safe software/hardware in their cars.

(1)
U
Undisclosed #2
Mar 22, 2018
IPVMU Certified

...for the first and most obvious rule of self-driving cars: Do not run over people.

But is that really the first rule?  What about the safety of the driver/passengers?

Hypothetically, should a self-driving car, with family in tow, attempt to swerve at high speed to avoid a jaybiker?

(5)
(1)
Avatar
Brian Rhodes
Mar 22, 2018
IPVMU Certified

Does a jaywalker deserve to die or get maimed because they violated a misdemeanor?

I'm not saying any death is acceptable, but given the occupants are in the car willingly (and have options to taxi or other less risky transport) and bystanders have no authority over the car being where it is, it seems preventing injury to bystanders is primarily important.

UM
Undisclosed Manufacturer #4
Mar 22, 2018

Not sure I 100% agree with your point here Brian [if I'm understanding it correctly]. Ultimately the Jaywalker was at fault by way of the law. So if there was to be a relative assessment of blame between an auto-swerve [dangerous to passengers and other vehicles] vs a less risky simple best braking effort, it seems to me the former would be preferable to the latter. 

This doesn't speak to the tech and it needing to be more mature and capable before mass adoption, but it was still a person in the street out of the middle of nowhere on a dark night. I expect the tragedy would occur more frequently with human drivers compared to hybrid man/vehicle or fully autonomous.

 

(1)
U
Undisclosed #2
Mar 22, 2018
IPVMU Certified

The final issue is the car's self-driving system most certainly does not depend on visible light sensors...

From: What A Self-Driving Car Sees - Tesla Video Analytics

(1)
JH
John Honovich
Mar 25, 2018
IPVM

The LIDAR supplier for the Uber vehicle is putting the blame on Uber:

 Certainly, our Lidar is capable of clearly imaging Elaine and her bicycle in this situation. However, our Lidar doesn’t make the decision to put on the brakes or get out of her way.

In addition to Lidar, autonomous systems typically have several sensors, including camera and radar to make decisions. We don’t know what sensors were on the Uber car that evening, if they were working, or how they were being used.

 

(2)
U
Undisclosed #2
Mar 25, 2018
IPVMU Certified

The LIDAR supplier for the Uber vehicle is putting the blame on Uber:

Waymo CEO piles on as well:

“I can say with some confidence that in situations like that one with pedestrians — in this case a pedestrian with a bicycle — we have a lot of confidence that our technology would be robust and would be able to handle situations like that,” Krafcik said Saturday during a panel at the National Automobile Dealers Association convention in Las Vegas.

U
Undisclosed #2
May 29, 2018
IPVMU Certified

Update: Camera analytic gets the blame

According to the NTSB report, the Uber vehicle struggled to identify Elaine Herzberg as she wheeled her bicycle across a four-lane road. Although it was dark, the car’s radar and LIDAR detected her six seconds before the crash. But the perception system got confused: it classified her as an unknown object, then as a vehicle and finally as a bicycle, whose path it could not predict.

Just 1.3 seconds before impact, the self-driving system realised that emergency braking was needed. But the car’s built-in emergency braking system had been disabled, to prevent conflict with the self-driving system; instead a human safety operator in the vehicle is expected to brake when needed. But the safety operator, who had been looking down at the self-driving system’s display screen, failed to brake in time...

JH
John Honovich
May 29, 2018
IPVM

it classified her as an unknown object, then as a vehicle and finally as a bicycle, whose path it could not predict.

A person walking a bicycle is likely an edge case that the designers of the system did not anticipate. That's not to excuse them but to point out the complexities of ensuring a system that works under diverse real-world conditions.

New discussion

Ask questions and get answers to your physical security questions from IPVM team members and fellow subscribers.

Newest discussions