FSD Using Cameras in Rain??

[damnitjim]

So I’ve been wondering. If Tesla only uses cameras for FSD and not LiDAR, how will it work/react in the rain? My 2021 Audi Q5 rear camera looks like this in the rain:

 

[Crissa]

Tesla's camera is tucked away to avoid fouling. Mine on my Mazda is right above the license plate (tho I need to put it on the fin) and it doesn't ever get that bad x-x

-Crissa

 

[damnitjim]

Tesla's camera is tucked away to avoid fouling. Mine on my Mazda is right above the license plate (tho I need to put it on the fin) and it doesn't ever get that bad x-x

-Crissa

That's good. I have never driven a Tesla in the rain, only a Model X once for a short time on a sunny day. But water does distort images. I'm just curious how the tech tackles that variable.

 

[Crissa]

That's good. I have never driven a Tesla in the rain, only a Model X once for a short time on a sunny day. But water does distort images. I'm just curious how the tech tackles that variable.

It's a good question! And I really wish we would get that question at AI day. Unskewing images is a massive undertaking.

-Crissa

 

[CyberGus]

 

[Mike the cyberguy]

This is what I found online, it's some sort of new patent. I'm sure they can incorporate with their fantastic cameras to make their FSD 100%

 

[fritter63]

And let's talk about that California coastal fog......

 

[Rootinboy]

So I’ve been wondering. If Tesla only uses cameras for FSD and not LiDAR, how will it work/react in the rain? My 2021 Audi Q5 rear camera looks like this in the rain:

I drove a model y in the rain using fsd. No different than a clear sunny day...

 

[Bill906]

So I’ve been wondering. If Tesla only uses cameras for FSD and not LiDAR, how will it work/react in the rain? My 2021 Audi Q5 rear camera looks like this in the rain:

FSD will drive in the rain using cameras just like humans with eyes drive in the rain. The main forward looking cameras are behind the windshield and the wipers clear the rain off the windshield.

 

[slomobile]

FSD will drive in the rain using cameras just like humans with eyes drive in the rain.

I've had to pull over and seek shelter under an overpass in severe rain with hail due to lack of visibility. Same in snowstorm whiteout conditions. It makes you a sitting duck target for anyone 'brave enough' (dumb enough) to keep doing 70mph unable to see their hood ornament.

I would pay extra for a radar sensor that keeps me moving slowly in those conditions and avoids hitting or being hit by big things. Autonomously moving over to avoid being hit from behind could avoid some of the most costly massive pileups. Insurance companies ought to recognize that. Perhaps Tesla insurance will look into that when Tesla density results in 2 or more covered vehicles per pileup.

If you are in manual drive mode and your Tesla suddenly changes lanes a split second before a semi barrels through where you just were, would that be acceptable if the cost were a rare unexpected lane change for no apparent reason?

Its an interesting problem because the operational design domain exceeds human capabilities. It requires the vehicle to actively override human inputs and substitute sensors and AI judgement over clear human commands in the moment.
It is not unprecedented. Antilock brakes do the same thing. Stabbing the brake pedal to the floor is a clear human intent, and it produces a sickening feeling as the pedal shudders and groans while you roll slowly into an active icy intersection. It is the feeling of your car betraying you. And it is considered completely ok.

Any light camera can be defeated by a kid with a spray can regardless of how many layers of glass it hides behind. It would be nice to have at least one defeat resistant sensor even if it's vote is ignored most of the time for being generally less reliable.

 

[Bill906]

I've had to pull over and seek shelter under an overpass in severe rain with hail due to lack of visibility. Same in snowstorm whiteout conditions. It makes you a sitting duck target for anyone 'brave enough' (dumb enough) to keep doing 70mph unable to see their hood ornament.

I would pay extra for a radar sensor that keeps me moving slowly in those conditions and avoids hitting or being hit by big things.

Any light camera can be defeated by a kid with a spray can regardless of how many layers of glass it hides behind. It would be nice to have at least one defeat resistant sensor even if it's vote is ignored most of the time for being generally less reliable.

Agreed. That all makes sense in theory.

Question. Camera doesn't see anything in front of you. Radar sees a large object. Do you trust the camera and drive through it, or trust the radar and hit the brakes?

 

[slomobile]

Question. Camera doesn't see anything in front of you. Radar sees a larger object. Do you trust the camera and drive through it, or trust the radar and hit the brakes?

Take conservative action to narrowly avoid the worst case scenario. If you've done everything below and still cant decide, apply the brakes.

You don't trust any sensor fully, they are given variable weight based on trends. If a large portion of camera image has been white or black over the last second, minute, hour then add more weight to radar. If camera images are resulting in high rates of object classification, reduce weight to radar. It is a busy well lit environment. On the highway where car thought an overturned truck was a billboard, classification rate would have been low, and image white, so radar weight would have been high, possibly braking to avoid the incorrectly classified object.

You gather more opinions to influence the decision by alerting driver, check for brake pedal trends, look for corresponding spikes in magnetometer, check if all cameras agree, check for patterns of false radar alerts across the network in this location, or at this time of day, or at this temperature, and weigh them all.

Yes, this has caused undesirable behavior. Yes it is more computationally costly. It is an acceptable cost to me. A cost I chose to bear when I enabled radar. I can opt out of the cost by disabling radar.
I've heard the argument that they require independent models to develop. Hogwash. Just develop for radar sensors allowing 0% weight given to disabled or non existent radar.

 

[Crissa]

That's why they eliminated radar.

Because it couldn't tell the difference between a sign and a truck across the road.

So no, you don't want radar to have you creep along. You're still a sitting duck.

-Crissa

 

[cyberboi]

Take conservative action to narrowly avoid the worst case scenario. If you've done everything below and still cant decide, apply the brakes.

You don't trust any sensor fully, they are given variable weight based on trends. If a large portion of camera image has been white or black over the last second, minute, hour then add more weight to radar. If camera images are resulting in high rates of object classification, reduce weight to radar. It is a busy well lit environment. On the highway where car thought an overturned truck was a billboard, classification rate would have been low, and image white, so radar weight would have been high, possibly braking to avoid the incorrectly classified object.

You gather more opinions to influence the decision by alerting driver, check for brake pedal trends, look for corresponding spikes in magnetometer, check if all cameras agree, check for patterns of false radar alerts across the network in this location, or at this time of day, or at this temperature, and weigh them all.

Yes, this has caused undesirable behavior. Yes it is more computationally costly. It is an acceptable cost to me. A cost I chose to bear when I enabled radar. I can opt out of the cost by disabling radar.
I've heard the argument that they require independent models to develop. Hogwash. Just develop for radar sensors allowing 0% weight given to disabled or non existent radar.

I agree. I think the main reason Tesla doesn’t want to include radar is cost.

 

[slomobile]

That's why they eliminated radar.

Because it couldn't tell the difference between a sign and a truck across the road.

So no, you don't want radar to have you creep along. You're still a sitting duck.

-Crissa

In sudden zero visibility it doesn't need to tell the difference. It just needs to go 15mph to reduce closing speed from the idiot behind me and maintain steering ability, stay between the centerline and shoulder best it can according to GPS, and dont hit anything big. Sign or truck, I don't want to hit either of them and I don't care if my car knows what the blip was.