Store and download onboard cameras video?

[TayTaySD]

It would seem an easy feature to add something that is constantly storing the video from all the cameras
and either manually or automatic sensing using the accelerator video get stored, say 30 seconds before
impact, 30 seconds after. Of course they should allow the user to install a USB stick, easier than storing
onboard then uploading it from the car...Sneakernet

Yes I know devices like this are in the aftermarket, seems like a simple software solution in this car already
having all these cameras. Does a Tesla do this?

Sponsored

 

[whitex]

The short answer is no, it it not easy to add, and yes Tesla does it, but only on their newer cars (March 2018 or later, some earlier cars with hardware upgrades but not full functionality) and even there for best performance you need a MY22 Tesla with the latest 3rd generation infotainment hardware.

The longer answer is that it takes more than "just software" to add a "dashcam" functionality. The devil is always in the details. History of this feature with Teslas is a good illustration why it is no easy. The list of issues includes, but is not limited to:

1. Camera format - not all cameras support compression. Recording uncompressed videos would easily exceed the capacity of USB drives (in terms of amount of data per second to be stored, including how much storage is available). Compression is very intense on CPU's, therefore is usually done using dedicated accelerators, which are hardware, not software.
2. Camera purpose - cameras are often configured differently for automated driving purposes than for daschcam recording. For example, say the vision system to detect pedestrians prefers black and white videos with less frames per second, while the dashcam would prefer color and 30fps. This issue also showed up in Tesla, when some people updated to latest AutoPilot computer and latest infotainment, they realized their dashcam was in black and white (or green/white).
3. Interconnects - not all cameras are connected to the same place. There may not be sufficiently capable interconnects to get the data from the cameras to a USB slot. This is why some Teslas only store some camera views, because they have older hardware and there is no way to pass the video to the infotainment which controls the USB (the cameras are connected to the autopilot computer rather than the infotainment). In a Taycan a bunch of cameras are connected to zFAS module which is not the PCM, and there probably isn't even enough bandwitdh between the two to pass all video footage real time to PCM.
4. Processing resources - even with hardware offloaded compression, it still takes a lot of processing power and memory to process the videos and store them. This is why older generation Teslas have issue with recording gaps, or infotainment become unusably slow when recording, etc.
5. Quality of user provided storage, not all USB drives are created equal, and most of them are not automotive quality. Allowing users to plug in their own USB stick is just inviting a while lot of customer complaints. Tesla learned that, which is why they started shipping Tesla USB sticks and stopped offering support for anyone with issues using non-Tesla USB drives.
6. Non-interference with safety critical functions. Overloading the same hardware which takes care of safety critical features like lane keeping, with non-safety critical feature like dashcam is hard and takes a lot of work to design, implement, verify, and safety certify.

There are more issues to be considered, but the above one give you an idea that it's not "just software". The feature needs to be planned out and appropriate hardware designed and put in the car, often changing the design of other features for which the cameras were intended in the first place.

So many things seem easy, but as always, the devil is the details. To be honest, a dedicated dashcam is actually cheaper and easier to add to a car than reusing existing cameras. For example, the Mercedes EQS just has a dashcam option which adds a dedicated mirror mounted dashcam camera. IIRC you can view it on the main screen, but the camera works independently using the infotainment only as a user interface display.

All that said, different governments, including US NHTSA, are looking to mandate a "black box" functionality to all cars, which will require telematics data (speed, location, controls like steering wheel or pedals, etc) as well as maybe some video to be stored somewhere in the car in case of an accident, though privacy laws may complicate things there too, we'll see.

 

[TayTaySD]

The short answer is no, it it not easy to add, and yes Tesla does it, but only on their newer cars (March 2018 or later, some earlier cars with hardware upgrades but not full functionality) and even there for best performance you need a MY22 Tesla with the latest 3rd generation infotainment hardware.

The longer answer is that it takes more than "just software" to add a "dashcam" functionality. The devil is always in the details. History of this feature with Teslas is a good illustration why it is no easy. The list of issues includes, but is not limited to:

1. Camera format - not all cameras support compression. Recording uncompressed videos would easily exceed the capacity of USB drives (in terms of amount of data per second to be stored, including how much storage is available). Compression is very intense on CPU's, therefore is usually done using dedicated accelerators, which are hardware, not software.
2. Camera purpose - cameras are often configured differently for automated driving purposes than for daschcam recording. For example, say the vision system to detect pedestrians prefers black and white videos with less frames per second, while the dashcam would prefer color and 30fps. This issue also showed up in Tesla, when some people updated to latest AutoPilot computer and latest infotainment, they realized their dashcam was in black and white (or green/white).
3. Interconnects - not all cameras are connected to the same place. There may not be sufficiently capable interconnects to get the data from the cameras to a USB slot. This is why some Teslas only store some camera views, because they have older hardware and there is no way to pass the video to the infotainment which controls the USB (the cameras are connected to the autopilot computer rather than the infotainment). In a Taycan a bunch of cameras are connected to zFAS module which is not the PCM, and there probably isn't even enough bandwitdh between the two to pass all video footage real time to PCM.
4. Processing resources - even with hardware offloaded compression, it still takes a lot of processing power and memory to process the videos and store them. This is why older generation Teslas have issue with recording gaps, or infotainment become unusably slow when recording, etc.
5. Quality of user provided storage, not all USB drives are created equal, and most of them are not automotive quality. Allowing users to plug in their own USB stick is just inviting a while lot of customer complaints. Tesla learned that, which is why they started shipping Tesla USB sticks and stopped offering support for anyone with issues using non-Tesla USB drives.
6. Non-interference with safety critical functions. Overloading the same hardware which takes care of safety critical features like lane keeping, with non-safety critical feature like dashcam is hard and takes a lot of work to design, implement, verify, and safety certify.

There are more issues to be considered, but the above one give you an idea that it's not "just software". The feature needs to be planned out and appropriate hardware designed and put in the car, often changing the design of other features for which the cameras were intended in the first place.

So many things seem easy, but as always, the devil is the details. To be honest, a dedicated dashcam is actually cheaper and easier to add to a car than reusing existing cameras. For example, the Mercedes EQS just has a dashcam option which adds a dedicated mirror mounted dashcam camera. IIRC you can view it on the main screen, but the camera works independently using the infotainment only as a user interface display.

All that said, different governments, including US NHTSA, are looking to mandate a "black box" functionality to all cars, which will require telematics data (speed, location, controls like steering wheel or pedals, etc) as well as maybe some video to be stored somewhere in the car in case of an accident, though privacy laws may complicate things there too, we'll see.

I agree with most of your points, as always the system design needs to anticipate features
like this for it to have any chance of being implemented. I have "surround view" which is
a $1200 feature. The cameras it has are all color, I use the forward mounted camera to
park my car in the garage every night. CAN bus architecture is such that safety items (braking, steering etc) have priority. I suspect the non safety cameras on the Taycan are on their own data bus anyway. I would be shocked if current cameras in the car don't do built in digitization and compression, HW that does that is decades old. Billions of phones have cameras that do that. I guarantee they don't pipe analog video around the car.

My point is in a performance oriented driving computer like the Taycan is, and most cars are these days, "dash cam" functionality should be designed in for a few extra $ BOM. Corvette has had built in performance data recorder functionality for several years. I hope Porsche gets on the leading edge in more features than it history of leading edge gas engines and suspensions.
We consumers should demand this. The Taycan does have an Event Data Recorder (EDR) system already (as you mention). Integrating that into a customer facing tool should be a reasonably small project. BTW I have an app for my iPhone that will do all of these features just using the phone and even map it around a racetrack (Harry's LapTimer).

 

[Genau]

I'm skeptical that any automaker would design dashcam functionality that could compete with dedicated hardware designed by a company dedicated to this purpose. Take a look at the engineering progress report on this Kickstarter campaign for the Dride dashcam:

​

 

[whitex]

The cameras it has are all color, I use the forward mounted camera to
park my car in the garage every night.

Yes, they are capable of color capture, but I suspect when using surround view to park, the camera settings may be changed from normal driving. For example, maybe while driving, the driving assistance vision system works better if you change the settings to high contrast black and white. I don't know that for a fact in a Taycan but maybe. Can you drive on the highway and see the video from all cameras, if so, do they look ok?

CAN bus architecture is such that safety items (braking, steering etc) have priority. I suspect the non safety cameras on the Taycan are on their own data bus anyway. I would be shocked if current cameras in the car don't do built in digitization and compression, HW that does that is decades old. Billions of phones have cameras that do that. I guarantee they don't pipe analog video around the car.

I never said they pipe analog video around. It's all digital, using probably GMSL or some other digital link technology. CAN bus is rather slow and unsuitable for any quality video. I was talking about compression, which yes, many cameras can do, but when using it for vision systems the requirements can be different than when recording it. Video compression is not lossless, so details important to vision AI can be lost, plus compressing the video in the camera and then having to decompress it in the vision system takes time, translating into higher delay/latency, which is not what you want in a real-time safety system - for example you don't want your car to detect a cross-traffic collision half a second after the car crashed.

I was also talking about the system architecture how different cameras are connected to different ECUs/modules in the car and how you'd have to somehow send all that data to whatever ECU is writing it to USB or SD/MMC storage. For example, in a Taycan, some cameras are connected to ZFAS driver assistance ECU, which reportedly has a MobileEye and a Tegra SoC chip in it. So, rather than just be processed by MobileEye and discarded, you'd have to pipe all the video content to the Tegra chip, and I there may not even be a link capable of that much bandwidth between them, and then from the Tegra if would have to be passed via FlexRay A bus to the gateway which would have to relay it to the PCM via Ethernet for storage in PCM I presume. FlexRay A bus has a max speed of 10Mbps and it's used for other communications, including real-time critical modules like RAS, so you cannot monopolize it for video.

The above are just my speculations based on what I know so far about the Taycan architecture, but I'm putting it here to illustrate my point that there are a lot of design challenges. Remember that Porsche is not a high volume car too, so the design and hardware cost would have be amortized over much lower number of cars. The hardware would have to be included with all cars, and option to enable might cost a lot, causing even less people to buy it, which means the price has to go up even more, and how many people would pay say $4,800 for a dashcam function? Software is expensive to design and maintain. The unfortunate truth is that the more exclusive the car, the less software features it is going to have. Check out Rimac Nevera - more exclusive than a Taycan, will beat a Taycan in any race btw, but the software will never match the main stream cars. The hope for Porsche is that they can draw on the VW software features, VW sells way more cars and therefore has much, much bigger software budget than Porsche.

For dashcams, it is much easier to do what MB EQS did, design an independent, click-in, add-on system (which IIRC they sell for $240) which integrates nicely with the car for mounting, and uses the infotainment display for UI. I suspect that will happen eventually with Porsche - they will pre-wire harnesses to front and rear dashcam camera, then sell it as an add on which will be its own product. This will be priced affordably because the dashcam will not have to have safety certified software, because it will not run on any safety critical ECUs or communicate over any safety-critical interconnects.

 

[whitex]

I'm skeptical that any automaker would design dashcam functionality that could compete with dedicated hardware designed by a company dedicated to this purpose. Take a look at the engineering progress report on this Kickstarter campaign for the Dride dashcam:

​

Auto manufacturers could design their own, they even have more expertise in designing electronics for the car than a new startup. I would personally think hardware designed by traditional manufacturers like VW or Toyota would be more robust than hardware from a startup that has never shipped one yet. What the startups have is richer features, which partially stems from their ability to take more risks with the design (both hardware and software) - worst case the startup folds its tents, not something a large manufacturer is going to do over a dashcam product.

Personally, I hope that the auto manufacturers will start providing "dashcam pre-wiring" option, a way for you to order a car with sensors (either separate, dedicated cameras for dashcams but integrated with the car's design, or tapping existing camera feeds if possible but with isolation in a way such that any recorder could not interfere with the camera operation even if it wanted to). Then you could just buy and plug in your favorite recorder from your favorite company. Kind of like Apple Car Play or Android Auto Model, if the phones were actually recording the footage. Heck, if they were dedicated sensors, they could standardize the sensors themselves so that you could upgrade in the future to better sensors but still running over the same wiring, say 1000BaseT1 Ethernet.

 

[Ambroos]

The zFAS takes in all the cameras and has a few ISPs to process the images. The zFAS is what does all the surround / 3D view stitching. That is then displayed on the PCM via a simple LVDS connection. It's more like the PCM acts as a monitor for the zFAS and puts it's own graphics on top of it.

The PCM has no access to actual cameras in the background, and there's no high bandwidth connection between the PCM and zFAS besides the LVDS.

Not possible with the current architecture is the result.

 

[TayTaySD]

I'm skeptical that any automaker would design dashcam functionality that could compete with dedicated hardware designed by a company dedicated to this purpose. Take a look at the engineering progress report on this Kickstarter campaign for the Dride dashcam:

​

I do agree that Porsche should just integrate someone else's camera system.
Porsche really doesn't manufacture much and especially on the Taycan.
I don't really know what the % they mfg, they don't build the motors, batteries and I suspect even all the software is outsourced currently, so my wild guess would be
they manufacture %20 (by cost). So yes, they should work
with a 3rd party and integrate their system, or make it plug and play. Personally I
don't want to record information from inside the cabin, only outside (and telematics of course). Bosch, Bose, Mobileye, Apple etc

 

[Ambroos]

Expanding on my last comment, the best public information on how things work can be pieced together from these two Audi documents:

MIB3: https://static.nhtsa.gov/odi/tsbs/2021/MC-10189329-0001.pdf
Latest generation zFAS: https://static.nhtsa.gov/odi/tsbs/2018/MC-10144686-9999.pdf

In short, all surround view cameras and the front driver assistance camera (R242-R246) all connect to the zFAS C (J1121) via LVDS. The zFAS does processing and stitching, and sends it over LVDS to the MIB3 system (PCM / J794).

The zFAS does have an ethernet output connection to the infotainment that could theoretically be used for this, but I can't find any references about this on the MIB3 side (in publicly available documents). Maybe that connection is not used in reality.

See two attached images for zFAS C and MIB3 diagrams. MIB3 LVDS-in from J928 is what's used for J1121 input in the most recent cars.

This similar structure (sending video over LVDS to a component which overlays it's own graphics) is used for the "virtual cockpit" map.

 

[whitex]

Expanding on my last comment, the best public information on how things work can be pieced together from these two Audi documents:

MIB3: https://static.nhtsa.gov/odi/tsbs/2021/MC-10189329-0001.pdf
Latest generation zFAS: https://static.nhtsa.gov/odi/tsbs/2018/MC-10144686-9999.pdf

In short, all surround view cameras and the front driver assistance camera (R242-R246) all connect to the zFAS C (J1121) via LVDS. The zFAS does processing and stitching, and sends it over LVDS to the MIB3 system (PCM / J794).

The zFAS does have an ethernet output connection to the infotainment that could theoretically be used for this, but I can't find any references about this on the MIB3 side (in publicly available documents). Maybe that connection is not used in reality.

See two attached images for zFAS C and MIB3 diagrams. MIB3 LVDS-in from J928 is what's used for J1121 input in the most recent cars.

This similar structure (sending video over LVDS to a component which overlays it's own graphics) is used for the "virtual cockpit" map.

There is an Ethernet (presumably 1000BaseT1) coming out of the MIB3 on the Taycan (there is an Ethernet connector labeled in the Audi MIB3 diagrams you posted too). It goes to the gateway. Interestingly, I have not seen any mention of an Ethernet on the Taycan's version of ZFAS, only the FlexRay and CAN. I am learning mostly from the internet, for now. As a side note. Taycan's version of MIB3 appears to have an additional connector (see red arrow below), so there are some differences between Audi and Porsche:

 

[Ambroos]

Interesting!

Maybe for the night vision camera?

The only ethernet I was aware of is from the e-call / connectivity module to MIB. MIB2 had it's own data connection, but MIB3 uses the single data connection from the connectivity module. I'm surprised there's another from the MIB3 to the gateway. Maybe for that planned-but-never-happening OTA update support after all?

 

[whitex]

Interesting!

Maybe for the night vision camera?

The only ethernet I was aware of is from the e-call / connectivity module to MIB. MIB2 had it's own data connection, but MIB3 uses the single data connection from the connectivity module. I'm surprised there's another from the MIB3 to the gateway. Maybe for that planned-but-never-happening OTA update support after all?

From my research, there is an "ICAN" labeled CAN bus going between the MIB3 and the OTA module and the Gateway. Then OTA and MIB3 each have an Automotive Ethernet connecting to the gateway. Both Ethernets are supposed to be Gigabit capacity. One thing I have not yet figured out, is OTA part of the conBox High module, gateway module, or a physically separate module. I've seen diagrams showing OTA in conBox, in gateway, and as a separate block, so unsure at this time. Apparently OTA has Ethernet to the gateway, ConnectCAN and ICAN connections. Do you know whether it's a physically separate ECU or just a functional block inside the Gateway or conBox?

 

[Ambroos]

From my research, there is an "ICAN" labeled CAN bus going between the MIB3 and the OTA module and the Gateway. Then OTA and MIB3 each have an Automotive Ethernet connecting to the gateway. Both Ethernets are supposed to be Gigabit capacity. One thing I have not yet figured out, is OTA part of the conBox High module, gateway module, or a physically separate module. I've seen diagrams showing OTA in conBox, in gateway, and as a separate block, so unsure at this time. Apparently OTA has Ethernet to the gateway, ConnectCAN and ICAN connections. Do you know whether it's a physically separate ECU or just a functional block inside the Gateway or conBox?

I know nothing beyond what can be deduced from those Audi PDFs and some messing with MQB CAN unfortunately.

From one TSB I figured out the part number is 9J1907018AD, so I think it's separate? I wonder if it's just a "PIWIS in a box", that can apply prepackaged services. I'd imagine the MIB downloads them and sends them to the OTA module. Slightly weird setup but could make sense if the MIB itself isn't "hardened" enough for a purpose like this.

Most likely it's a Harman OTA system which seems to do exactly that. It also looks like VW group hasn't successfully made proper use of it yet either, since they keep sending people to dealers for all updates.

 

[whitex]

I know nothing beyond what can be deduced from those Audi PDFs and some messing with MQB CAN unfortunately.

From one TSB I figured out the part number is 9J1907018AD, so I think it's separate? I wonder if it's just a "PIWIS in a box", that can apply prepackaged services. I'd imagine the MIB downloads them and sends them to the OTA module. Slightly weird setup but could make sense if the MIB itself isn't "hardened" enough for a purpose like this.

Most likely it's a Harman OTA system which seems to do exactly that. It also looks like VW group hasn't successfully made proper use of it yet either, since they keep sending people to dealers for all updates.

Thanks. You are right. That part number does appear to be a separate OTA module for a Taycan. Below are pictures of what it looks like (from ebay). I probably should do some digging through some of the TSB's, they seem to be a good source of information.

 

[whitex]

Maybe for the night vision camera?

My educated guess is that it's for the infotainment display screen in the dash (possibly both passenger and driver display via some interconnection between them). My guess is based on the fact that the lower display (where the charging information shows) has 2 matching black and white connectors (and there are no other connectors on it other than the ones below, so cannot connect main screen to the lower screen).

Sponsored