Not enough cameras for FSD - a physic-driven simulation

[firsttruck]

Today I drove a Nissan Versa sedan medium compact car.
Some people drive sitting close to the steering wheel with arms bent but I drive sitting as far from the steering wheel as I can while still being able to control the wheel & pedals.

I did measurements

1. With my head resting on the headrest, near B-pillar and my head turned to the left (I also measured the horizontal distance from my eyes to top of steering wheel which was 24" (not angled distance, I measured only horizontal).

2. From my chin to top of the steering wheel is 24" (not angled distance, I measured only horizontal).

3. Without moving the seat position, I leaned forward and then turned my head to the left.

Result: The eye position difference was 24" (B-pillar to my eyes. Not angled distance, I measured only horizontal).
I could have got more if the A-pillar had not sloped so much toward the driver.
Jeep Wrangler type vehicles have near vertical A-pillar.

Unless there is physics I missed, in many blocked view situations, I could in fact creep 2 feet less into cross traffic lane than the current Tesla FSD that uses B-pillar cameras for direct left/right side views.

In these blocked view scenarios, Tesla FSD could beat the best human driver by an order of magnitude just by the Tesla FSD having a side camera at each vehicle corner because the creep amount total would drop from 2-4 feet (includes hood/frunk length) to a couple of inches.

There is a 2005 Nissan SUV that I don't drive often because it is a gas hog.
Today I had to transport some large boxes so after I was done I took some measurements.

1. With my head resting on the headrest, near B-pillar and my head turned to the left (I also measured the horizontal distance from my eyes to top of steering wheel which was 27" (not angled distance, I measured only horizontal).

2. From my chin to top of the steering wheel is 27" (not angled distance, I measured only horizontal).

3. Without moving the seat position, I leaned forward and then turned my head to the left.

Result: The eye position difference from B-pillar to leaning forward head with head turned left was 27" (B-pillar to my eyes. Not angled distance, I measured only horizontal).

If I move the seat forward so I drive sitting closer to the steering wheel with my arms bent (like how many people drive).
The difference between where a B-pillar camera would be vs what a human driver leaning forward is 32". Reason I can get up to 32" in the SUV vs only 24" in he Nissan compact sedan is the SUV has a larger cabin, longer doors (longer distance between pillars), more headroom / higher roof ( so I can position my head past & over steering wheel) and the A-pillar is more vertical. When leaning all the way forward and looking left my head is actual past the top of the steering wheel.
The hood of the SUV is longer so sometimes I need all those extra inches of peekability to reduce how much the front of the SUV intrudes into cross traffic travel lane.

Sponsored

 

[Crissa]

I have never seen someone put their head atop the steering wheel to see around these corners.

Ever.

If through traffic can't deal with two feet of bumper sticking into the lane, that traffic was going too fast for the terrain.

-Crissa

(This is also why in the mountains we often do a thing where we reverse the right of way: by stopping traffic back from an intersection to let vehicles enter and exit the road.)

 

[JBee]

First up. Well done on the video and animations! What program did you use for the animations if I may ask?

I think the problem is real, and in fact a lot more common than one would think because of trucks/large vehicles pulling up in the lane beside the M3 and blocking the camera view. I always stay further back from the intersection when driving our 16t truck or even the van, so that cars next to me can see the intersection better and pull out into traffic whilst I can't. In the cab over truck/van I can just look over the other smaller vehicles anyway.

The best way is to obviously have more cameras, or better placement of them and/or better field of view. In this scenario I think a bumper mounted 180-220 degree FOV camera (or split in two) should suffice to do both directions and add some confidence to the FSD that it is actually seeing traffic coming from the side. The easiest place to integrate these would probably be in the front lights.

On the subject of intersection behavior and camera coverage, I was thinking that someday if there's enough cars with FSD on a intersection it would actually be worth sharing position and driving data over wireless between them at intersections to keep everyone safe. So basically be able to see with other FSD cars cameras as well. This would also work well for vehicles slipstreaming on highways to increase range.

 

[firsttruck]

I have never seen someone put their head atop the steering wheel to see around these corners.

Ever.

If through traffic can't deal with two feet of bumper sticking into the lane, that traffic was going too fast for the terrain.

Have you lived in Boston or New York City?

Before you could get to 2ft you might lose the front of your car.

And get a traffic ticket.

These types of accidents are part of the millions that occur every year.

We want FSD to be orders of magnitude safer than human drivers.
FSD having better vision than human is part of how it is achieved.

 

[Crissa]

Have you lived in Boston or New York City?

Did you just do the stupid 'but you don't live in new york city' argument? That doesn't change the basic safety issue.

Before you could get to 2ft you might lose the front of your car.

If there is no visible traffic and someone hits you for creeping forward - it's not going to be your fault.

Sorry. That's what a dash cam is useful for. Something all Teslas have built in.

-Crissa

 

[firsttruck]

Did you just do the stupid 'but you don't live in new york city' argument?

I have lived in both places in the past but have not lived or driven in either for over 20 years.

Driving in Caiformia, Oregon, Arizona, Texas is different and yes I have driven in all those states and many others (almost all U.S. states except Alaska).

There are reasons Waymo spent most of thier time testing in Phoenix ,AZ area and the reason WAS NOT because it was tougher test for Waymo FSD.

Even Elon Musk admitted that Tesla FSD AI was overly trained to California and California driving styles.

One example is Tesla FSD very frequently at stop signs does a rolling stop instead of coming to a complete stop.
Tesla FSD does not do that because of the sign itself. The FSD does that because it learned from behavior of California drivers.
I have gotten traffic tickets for rolling stops in other states and I am sure I am not the only one.

That doesn't change the basic safety issue.

Of course the behavior of other drivers makes a difference and the percentage of rude or aggressive drivers is not spread evenly.

I am pretty sure on a per capita fleet basis, There is more vandalism & coaling coal on Teslas in Dallas/Houston, TX than there is in San Francisco, CA or Portland, OR (I have lived in all those places including those Texas cities).

 

[firsttruck]

First up. Well done on the video and animations! What program did you use for the animations if I may ask?

Thanking the wrong person, I did not make the video.

I think Youtuber "KookyBone" made it.
https://www.youtube.com/channel/UCPm-Eop5ooVrRLxsMc21qZA

....
The best way is to obviously have more cameras, or better placement of them and/or better field of view. In this scenario I think a bumper mounted 180-220 degree FOV camera (or split in two) should suffice to do both directions and add some confidence to the FSD that it is actually seeing traffic coming from the side. The easiest place to integrate these would probably be in the front lights.

I do not think a single camera 180-220 degree FOV camera that is forward facing is enough.

Remember the camera needs to see left or right objects good enough to identify what the objects are, AND accurately the object's DISTANCE and speed.
I am concerned that the extreme edges 180-220 degree FOV might have lose of clarity & accuracy for those three things (identification, distance, speed).
Cross traffic could easily be doing 40mph to 60mph. At those speeds it does not leave a lot of time.
Even if the cross traffic legal speed is 20mph, if I am in a robotaxi I want it to detect that someone in cross traffic might actual be driving at 60mph and smash front of robotaxi or directly T-bone the passenger compartment.

 

[firsttruck]

Example of FSD Beta driver leaning all the way forward with head at steering wheel in-order to see high-speed traffic approaching from the left.
Driver can see better than the car's B-pillar camera.

Testing problematic unprotected left with FSD Beta 10.5
Comparing a problematic unprotected and limited visibility left from FSD Beta 10.4 to FSD Beta 10.5.
Nov 23, 2021
Barefoot Tesla



------------------


Difficult unprotected left testing with Tesla FSD beta 10.4
I found this more difficult left turn onto a 45 mph road.
Nov 18, 2021
Josh Perry

 

[Crissa]

That's an example of failure of road design more than bad autonomous sensors, tho.

if you have to lean forward over the steering wheel, your other views are occluded while you're doing that and the safety devices don't work as well in that posture.

-Crissa

 

[Ogre]

This just in: Intersections which are dangerous to humans are likely still dangerous to autonomous cars.

 

[Dids]

I Have a question. Does anyone believe that Tesla who has been working on FSD for years with Elon Musk who is an innovator hasn't thought of optimal sensor placement? That Tesla FSD team would use less sensors if more sensors helped them achieve success?
The problem probably isn't sensors or getting data. It is interpreting and acting on that data. Adding more data doesn't help.
Also FSD and human sensors are not the same. FSD uses optical flow to determine speed and direction of movement. Humans use parallax. FSD pays attention to all 90° of b pillar vision. Humans pay attention to ~2° or the area covered by a thumb held at arms length. Meanwhile FSD is also using forward view 120° and paying attention to it.

There will be situations where no amount of sensors or their placement would allow safe movement that is why they put traffic lights and make it a protected crossing. Humans are bad at estimating danger so often we have to rely on oncoming driver to react, we rely on oncoming to not be speeding etc. We say I can see 25ft is clear speed limit is 25 and oncoming driver can slow, so we risk and go. Accident happens when we have to turn head, refocus attention forward, judge speed forward, ignore bird that flew into vision, go forward while maintaining timing in our head for the amount of time it took to choose to go etc etc.
FSD also relies on other drivers reactions. That is what the aggression setting is. None aggression means don't rely on other drivers at all. Aggressive means rely on them all the time and this guys youtube imagines that with better sensors the problem could be solved?

 

[Ogre]

The problems the simulation shows are problems for humans too. That is why there are laws against building that close to property lines on properties that border streets. The Fences and pillars which are pictured in this simulation are all illegal structures. If you’ve ever tried to get a permit for fences they make this stuff pretty clear.

Tesla’s vehicles are designed to interact with a world designed around humans.

 

[firsttruck]

Another example of human driver leaning head forward to the steering wheel to see past obstruction.
Tesla creeps car too far out into road way.

FSDBeta v10.5 Left Facing Unprotected Left Turns
Nov 24, 2021
Chuck Cook

 

[Ogre]

Another example of human driver leaning head forward to the steering wheel to see past obstruction.
Tesla creeps car too far out into road way.

The guy deliberately and repeatedly approaches really bad left turn situations.

As a driver, that sort of turn is something I avoid. Is this the route the car chose, or is it a route the driver forced on the car by engaging FSD in a place where it was the only way it could go?

Clearly FSD should be better on these turns, but equally clearly, there are some serious navigational shortcomings here.

 

[firsttruck]

The guy deliberately and repeatedly approaches really bad left turn situations.

Yes, Chuck is going there deliberately. How else is there going to be real world test of these types of scenarios. Chuck is the best most conscientious FSD Beta tester I have seen.

Chuck is doing Tesla FSD Beta testing.

Testing what human drivers can safely do VS FSD Beta.

This is a real street that human drivers encounter.
The city road dept has not added additional control measures so they might assume most human drivers can see good enough by leaning forward.
It also depends on the size and design of vehicle. Smaller compact cars poke out a lesser distance. In smaller car, the driver might not need to lean their head to steering wheel to see past obstruction.

There are millions of intersections with limited left views like this in U.S..
I have driven in almost every state in U.S. and found these situations more in east coast cities & suburbs than in western states or rural areas.

As a driver, that sort of turn is something I avoid. Is this the route the car chose, or is it a route the driver forced on the car by engaging FSD in a place where it was the only way it could go?

As a defensive driver I too avoid these situations WHEN I can. If I know ahead of time.
But you don't always know ahead of time. And you can't always backup.

Sponsored