WSJ on Tesla Autopilot (probably behind paywall)

[Jonathan S.]

Apologies in advance for the potential tease, since this is probably behind a paywall for anyone who isn't a WSJ subscriber.
(Will probably be posted to Jalopnik soon in summary form, since that's a frequent feature there.)

But the WSJ obtained hacked footage of Tesla Autopilot crashes, many of them fatal:
https://www.wsj.com/video/series/te...as-crash/68D26569-0251-4637-A035-A5131D8883B8

This isn't merely the video that an owner can retrieve, but also the overlaid Autopilot assessment decisionmaking. (Looks like a scifi movie!)

Another video shows how the footage was obtained:
https://www.wsj.com/video/series/te...lot-data/4B08ADEA-F38E-4EBF-91FD-485CE13079A8
In brief, a Tesla salvager extracts the motherboard, then ships it to a hacker (identity not revealed by the WSJ!), who in turn has to physically disassemble certain parts of the motherboard to obtain the heavily encoded data.

More qualitatively, the most interesting part (at least to me) is that the exclusively camera-based system relies on what Autopilot has been trained to identify.
So if, for example, a Lidar-based system detects an overturned semi blocking a highway lane, the system has no idea what it is, but knows it's a fixed object, so better apply the brakes, etc.
By contrast, Autopilot just continues on until the last possible second.
(Which is too late for the deceased driver, although fortunately the four guys standing in front of the semi see the Tesla -- the video is quite dramatic -- with just enough time to run for their lives, apparently successfully.)
(A more innocuous example is rain-sensing windshield wipers. Everyone else just uses a water sensor that costs something like a buck per car. Tesla instead tries to train its camera to detect rain.)

Sponsored

 

[eisenb11]

As someone who had a Tesla, (Model 3 Performance prior to my Taycan) - Tesla's rain sensing was absolutely terrible. Wish they used the $1 rain sensor that everyone else uses.

 

[daveo4EV]

ultimately Musk is "all in" on what I consider to be a logical but ultimately flawed perspective by hard cord AI crowd - the hard core AI crowd's goal is simple:

"if humans can do it, so can AI."

and humans drive via mostly vision only - so following their logic you should be able to achieve the same results with only cameras/vision - I believe this approach to be flawed - in that we do not have to limit ourselves to "human" capabilities, but I know for a fact from former Tesla employees (including one or two former co-workers I've lunched with in leadership and reporting directly to Musk) Musk is "deeply 'all in'" on the perspective of "if Humans can do it, so can AI" - so that leads to the current implementation - unfortunately to train these AI's you need massive amounts of training data - and Musk is using the fleet of Tesla's as a data source for the training - including the low percentage high impact training scenarios that are the rare event you want to train the AI to avoid, but to train it to avoid it, it first has to happen, and then the data harvested and labeled as "bad" - never mind the consequences of what actually happened in that scenario.

the moral, technical, legal, and business practice ethics of this approach are highly debatable - with passionate, smart, well intentioned, and articulate advocates on both ends of the spectrum pushing their agendas - it is unquestionably one of the great policy debates of at least the next 1 orf 2 decades (or more) and we are breaking new ground on a weekly basis with this grand experiment. The proper balance of allowing progress but not pushing too hard and too fast, and avoiding the stifling of promising new approaches is perhaps the most difficult private and public policy decisions for the next 2 or more generations…

ultimately I believe the hard core AI approach/perspective will fail in the market place and techniques and testing need to be conducted safely in private in more controlled circumstances rather than on unwitting customers as part of public transporation infrastructure. It's a serious problem in my opinion that it's being allowed to proceed on public highways in a consumer product with 99.9% of the participants having no clue what they are doing and how it could impact them. Probably time to call a time out on this approach to developing AI.

I used to admire Musk (2010-2013'ish) and believe he was chaotic good, but my perspective and he has changed - I do believe now he is trending towards malicious evil and Tesla should have it's wings clipped with regards to allowing this "pure AI experiment" to continue - Tesla is a great EV force in the industry - it's probably past time for Musk to be removed from leadership so the company can thrive without some of Musk's bias's…

it's also probably time for both government and legal action to thwart the continued undisclosed experiment that is Telsa's FSD and cause some change to happen via external pressure and intervention.

my $0.02

 

[Vercingetorix]

The simple fact is Mercedes is the first in the USA to have level 3 autonomous driving for sale. I trust Mercedes, and have zero faith in Tesla.
https://fortune.com/2024/04/18/merc...-cars-california-nevada-level-3-drive-pilot/#

 

[eisenb11]

The problem is that Tesla's approach to Autopilot (and FSD) is inherently flawed. Vision-only is a total loser of an approach. The counter-argument is that if humans can drive with eyes-only, so can autonomous driving, but that's no good, IMHO, because human aren't really the greatest drivers due to limitations of our eyes. I don't see a level 3 system working well enough with visual-only for the level of trust for me to bet my life on it. You really do need a sensor fusion approach to this problem - a combination of EO, IR, LIDAR, radar, whatever.

Training against models is just no good in the edge cases and it's the edge cases that kill us. Likea while back, I'm going down the highway at 80 mph and I had to dodge a desk that was literally just sitting in the middle of the highway. This is a case where the visual system is not going to recognize the object - at least with radar/LIDAR the car will know "solid object - don't hit it"...

 

[Vercingetorix]

Tesla in Seattle-area crash that killed motorcyclist was using self-driving system, authorities say
https://apnews.com/article/tesla-fu...clist-killed-d3393396521c373fe5df5a44d2d9637f

 

[anonymouse]

I'm going down the highway at 80 mph and I had to dodge a desk that was literally just sitting in the middle of the highway. This is a case where the visual system is not going to recognize the object - at least with radar/LIDAR the car will know "solid object - don't hit it"...

Sorry, but that is just FUD. Tesla Autopilot would correctly recognise an obstacle, and apply corrective action. Furthermore, its corrective action may be smarter than a human — it will know what is in adjacent lanes and decide whether to swerve or screech to a halt for example. It will apply hazard lights too (as would the Taycan if the driver braked hard).

While we are here let’s look at whether Porsche’s active cruise control is better:

 

[Jonathan S.]

^ Then how come Autopilot didn’t detect an overturned semi blocking off more than an entire highway lane? Or a stopped police car? Etc. The WSJ example has many terrifying examples.

(As for whether Porsche ACC is better, that passage is just CYA. Lidar would have easily recognized all the obstructions in the WSJ video. And Porsche ADAS would not have allowed extended hands-free driving.)

 

[eisenb11]

Sorry, but that is just FUD. Tesla Autopilot would correctly recognise an obstacle, and apply corrective action. Furthermore, its corrective action may be smarter than a human — it will know what is in adjacent lanes and decide whether to swerve or screech to a halt for example. It will apply hazard lights too (as would the Taycan if the driver braked hard).

While we are here let’s look at whether Porsche’s active cruise control is better:

Sorry, but this is completely untrue and has been demonstrated time and time again with Tesla’s slamming into things. They’ve slammed into trucks, ambulances, police cars on the side of the road. Mine used to slow down for no reason all the time due to traffic on the sides that was behaving normally. Never once has it done anything smart to avoid any kind of debris in the road in front of me resulting in me having to manually react. My Tesla has *never* performed any kind of emergency evasive maneuver for me other than when it had the phantom braking problem And would slam the brakes at highways speeds for absolutely no reason (fortunately they fixed that after a few months So it became phantom slowing down which is better than phantom braking).

ALK has its own issues too (it’s very primitive, IMHO), but at least it uses radar and is able to better detect the distance from objects directly in front of me. The difference, though, is that Porsche advertises the system as merely a driver assist to reduce fatigue… Tesla makes much loftier claims, so it’s a matter of expectations.

Curious, do you actually have experience with ALK and Autopilot (or FSD)?

 

[anonymouse]

Curious, do you actually have experience with ALK and Autopilot (or FSD)?

Yes. 70,000 miles of road trips in UK, France, Germany, Spain, Norway, Denmark, Australia, US etc. in Model S and Model 3. And although there was an occasional unexpected braking, in 80% of cases I could see why it was doing so and quickly corrected with a tap on the accelerator. Furthermore I believe it may have saved my life at least once due to reacting faster than I did to a situation.

No FSD experience of course, as that is US-only.

In the Taycan I have also had some unexpected braking. A lot fewer, of course, because it is a dumber system that is only looking at the road ahead (vs Tesla will hazard a guess that a pedestrian heading towards the road might step off the kerb).

 

[eisenb11]

Yes. 70,000 miles of road trips in UK, France, Germany, Spain, Norway, Denmark, etc. in Model S and Model 3. And although there was an occasional unexpected braking, in 80% of cases I could see why it was doing so and quickly corrected with a tap on the accelerator. Furthermore I believe it may have saved my life at least once due to reacting faster than I did to a situation.

No FSD experience of course, as that is US-only.

In the Taycan I have also had some unexpected braking. A lot fewer, of course, because it is a dumber system that is only looking at the road ahead (vs Tesla will hazard a guess that a pedestrian heading towards the road might step off the kerb).

Yeah, I also found that an accelerator tap usually got my Tesla to stop misbehaving. Don’t get me wrong, though, Autopilot blows away ALK for most things - ALK stinks at lane identification And is *very* primitive… like on occasions where cars are merging into my lane, for example. Autopilot will slow down when the merge starts while ALK (technically ALK+ACC) doesn’t do it until they merge much further in (Like 3/4 of the way).

My favorite example of using Autopilot was when it started raining crazy hard suddenLy. I couldn’t see a thing. I turned on Autopilot and it was able to successfully drive the car for those few moments.

While it may seem like a double standard, for me it’s what comes down to expectations. Porsche basically says their system sucks and it’s meant to merely reduce driver fatigue. I agree with this assessment - it does suck, so I merely use it as a driving aid to relax a little. Tesla is much more aggressive in its Autopilot claims and way more aggressive in its FSD claims.

Despite this… I still miss having Autopilot, though… lol

 

[anonymouse]

Autopilot blows away ALK for most things

My highlights (Extended Autopilot in non-US version, vs Innodrive + ALK on Taycan) would be:

Tesla AP:
- reacts to a lot more potential signs of danger; and consequently gets more wrong ("phantom braking") ... but could also save a life
- has generally excellent knowledge about context, so can act accordingly, eg auto lane change only when safe, cancel indicators correctly after you have finished a manoeuvre (vs Taycan cancels based on steering wheel turn and knows nothing about why you are indicating)
- with EAP, knows where I am going and will pick the right lane almost every time: big help when in a new country approaching a complex intersection
- is continually improving (vs Taycan which is essentially frozen when your model was released)
- has a bit more courage than Taycan when attempting a sharp bend at speed (although in a Porsche who would want to waste a good bend by being on ALK?)

Taycan ALK/Innodrive:
- allows me to interact even when it is on ALK, eg nudging it away from a potentially sleepy truck driver in the adjacent lane. Tesla requires me to cancel and re-enable autopilot to do this
- Innodrive will correctly reduce speed before reaching a limit change sign (Tesla incorrectly starts braking AT the sign which could result in a traffic offence).
- Innodrive seems to get speed limits right a slightly higher proportion of the time than Tesla's system (but still can't be relied on).

Sponsored