Explain HW3 v HW4 Implications for CT?

[cvalue13]

I just typed a bunch of stuff that says, the high def stuff gets uploaded and the selective downsampled get used realtime.

This doesn’t make sense to me

the compute is onboard at the sensor, this is where it is easiest, best, and most cost effective to use the high def stuff

I would more soon underatand they are BOTH using the high def on board AND incurring the expense etc of uploading the high def to DOJO/etc, than instead they’re doing what would appear to be the opposite of efficiency (low def at edge, high def at cloud)

I hope it's not in the blindspot.

which means f the thousands?

we’re not in the autonomous driving space, but instead the related edge computing infrastructure running various algos at sensor (with a training backend at cloud)

that said, I’m an exec that runs the thing, not an engineer

Sponsored

 

[Deleted member 17810]

This doesn’t make sense to me

the compute is onboard at the sensor, this is where it is easiest, best, and most cost effective to use the high def stuff

I would more soon underatand they are BOTH using the high def on board AND incurring the expense etc of uploading the high def to DOJO/etc, than instead they’re doing what would appear to be the opposite of efficiency (low def at edge, high def at cloud)

You can do both.

The local computing need as much info as possible. The ML @dojo needs higher quality that can be processed later (wife) stand still.

As far as I know tesla isn't doing image sensor processing. The compute is after that, like all smartphones, webcams, etc.

which means f the thousands?

we’re not in the autonomous driving space, but instead the related edge computing infrastructure running various algos at sensor (with a training backend at cloud)

that said, I’m an exec that runs the thing, not an engineer

that was a joke about the FSD not being able to process stuff it cant see.

As an exec you should be able to see that

 

[Baldey]

It's in the weeds but this is post processing from the sensors:



*etiquette = conforming to the audience


It's also etiquette* to explain acronyms before delving in.

Sure it is post processing, but it is done by ML. Anything you do to raw sensor data means post processing.. That's implied. What we are discussing is if there's any traditional processing done before processing by the ML models.

Again, i think we are on the same page, you just misunderstand what post-proccessing means. It doesn't always have to imply the traditional demosaicing, white balance, color correction, gama correction, sharpening and noise reduction, down sampling, cropping, compressing... it can be done directly from raw to AI like tesla does, by starting with feature extraction. Removing those traditional steps requires more bandwidth, but cuts down on latency quite a bit.. A worthy trade-of, when driving.

I believe you misunderstand the definition of etiquette as well: you are not conforming to your audience. Defining acronyms is only etiquette when speaking to a broad audience. You are on the CT[CyberTruck] Owners forum. Defining FSD and ML here is being pedantic at best, and condescending at worst. I won't insult your intelligence or ability to google by defining those.

 

[Deleted member 17810]

Sure it is post processing, but it is done by ML. Anything you do to raw sensor data means post processing.. That's implied. What we are discussing is if there's any traditional processing done before processing by the ML models.

Again, i think we are on the same page, you just misunderstand what post-proccessing means. It doesn't always have to imply the traditional demosaicing, white balance, color correction, gama correction, sharpening and noise reduction, down sampling, cropping, compressing... it can be done directly from raw to AI like tesla does, by starting with feature extraction. Removing those traditional steps requires more bandwidth, but cuts down on latency quite a bit.. A worthy trade-of, when driving.

I believe you misunderstand the definition of etiquette as well: you are not conforming to your audience. Defining acronyms is only etiquette when speaking to a broad audience. You are on the CT[CyberTruck] Owners forum. Defining FSD and ML here is being pedantic at best, and condescending at worst. I won't insult your intelligence or ability to google by defining those.

I'm not misunderstanding, that stuff gets done by the ISP on the image sensor before being delivered to the GPU.

As far as I know tesla is not doing raw processing from sensor data before it gets to images and videos.

There are two levels, again per their patent, they take image/video data run it through a ML to crop/down scale sections.

If you want to be technical, be technical,.

I know etiquette, I was being passive aggressive about acronyms. Ask CG

 

[Baldey]

I'm not misunderstanding, that stuff gets done by the ISP on the image sensor before being delivered to the GPU.

As far as I know tesla is not doing raw processing from sensor data before it gets to images and videos.

There are two levels, again per their patent, they take image/video data run it through a ML to crop/down scale sections.

If you want to be technical, be technical,.

I know etiquette, I was being passive aggressive about acronyms. Ask CG

So you meant you were using bad etiquette, i see lol.

What i am saying is they disable the ISP for better data AND latency. They combine all the cameras into one vector space, from raw data bypassing the ISP. Lets be technical. Show me this patent. But even if you are right we agree:

-You are saying they use ML to crop the images.
-i am saying they use ML to combine the images into a vector space.

Either way we are both saying they use ML to post-proccess. The only thing we differ on is wether or not tesla begins using ML to process raw data, or data that's already been processed by the ISP.

Disabling the ISP saves latency. I think i read somewhere that the typical ISP delay for android devices is anywhere from 250-400ms. That would literally cause a wreck when driving @ 85 miles an hour. 85mph = 36 meters per second. In 400ms, you would be off by 12 meters. You would literally crash at every turn.

 

[Deleted member 17810]

k

https://www.teslarati.com/tesla-patent-autopilot-enhance-object-identification/

You said, and I quote:

Tesla does not use any kind of image post-processing, its sensor data directly into the ML model. So when the sensor changes, it is an even bigger difference for the machine learning models to deal with.

That all happens before going into the FSD stack for ML

 

[Baldey]

k

https://www.teslarati.com/tesla-patent-autopilot-enhance-object-identification/

You said, and I quote:



That all happens before going into the FSD stack for ML

that patent proves my point. They start with raw data, and crop that. the ISP is disabled, according to this patent.

Also, why are you quoting my original post? Did you read the thread? I already elaborated on this:

Sorry, i'm not sure if i understand your comment either There must be some confusion.. I did say "any kind of post-proccessing" , so i think the confusion may have been my bad. What i meant is that they disable any post processing done by the manufacturer, and do their own processing of the RAW CMOS sensor data.
...
Here is a good explanation on what happens to the light hitting a digital sensor, in 10 steps before the jpeg. I am not sure which step the FSD network is trained on, but i am pretty sure it is 4, or maybe even 3.
https://photo.stackexchange.com/questions/1455/what-is-raw-technically

 

[Deleted member 17810]

ok

 

[Baldey]

ok

god, i love sarcasm. BTW you spelled Kahpernicus wrong, unless its the kerbal variety..
/sarcasm

 

[cvalue13]

You can do both.

you can do both

it just sounded like you were saying they were not only not doing both, but doing the inverse of normal (eg low res at edge, high res at remote)

that said, I would be surprised to learn they’re doing both - it would be bananas if Tesla was moving full sensor data stream to remote

 

[Deleted member 17810]

you can do both

it just sounded like you were saying they were not only not doing both, but doing the inverse of normal (eg low res at edge, high res at remote)

that said, I would be surprised to learn they’re doing both - it would be bananas if Tesla was moving full sensor data stream to remote

I wasn't saying that.

But, in the 40 minutes it took to talk to baldey, the computing power of the next dojo probably can.

e: post to not talk

I'm sorry opt

 

[PilotPete]

Sure! What could go wrong?

Yeah, well... In addition to being stupid, it only hit a Cirrus Jet. Really, $3M is cheap in the airplane game. Especially for a jet.

Sponsored