"Why America's EV chargers keep breaking" - Article

[daveo4EV]

The fanboy back in full swing!



I rather watch this, then reading your fanboy nonsense!

[delete]

Sponsored

 

[daveo4EV]

The fanboy back in full swing!



I rather watch this, then reading your fanboy nonsense!

[delete]

 

[whitex]

relax the session start criteria when the internet is down and accrue the identified sessions vs. the unidentified sessions - but always let the charge session work for the perception and actual fact of network reliably - I can also put limits in place to mitigate how long I let this go on before the site goes off line due to lack of command/control from the mother ship and accrued costs - and then monitor costs associated with "outages"…and if "outage based" session costs and getting out of hand - work to manage that and mitigate it - but honestly I bet the data shows this is not the biggest problem they have or will have - and lots of other things don't work when the internet is "down" - but i can keep the charge session working no problem - you can fire me when those costs become our biggest problem - I doubt that's the reason however that will come up

I can manage the hell out this problem and makes sure it's not the significant bit in the cost of the network - smart business don't inconvenience their customers with their infrastructure problems - they manage those problems and mitigate their impact, and then fix them when it becomes a big enough problem that it matters.

They could mitigate by giving free energy out (no way to authenticate a CCS car car connected through a dumb adapter) like EA does. The problems is the incomplete outages, something engineers often don't test. E.g. We can reach a server but cannot get a proper response, or authenticate the server (expired cert for example), or it cannot authenticate you (its clock may be set wrong), or always get an access denied response, etc. Such partial failures are often never coded for, and sometimes even cause systems to crash. For example I have an IOT device which crashed every few minutes whenever my home internet was down, but that was puzzling since I could unplug it, or firewall it off the internet no problem, it only crashed when my main ISP went down. Long story short, took me some time to figure out it will work fine with or without internet, but it will crash if it can reach a DNS server but that DNS server cannot resolve some public domains (I have a local DNS server on my home network) - some engineer assumed that if you can reach DNS that means you have internet access and can resolve google.com. If you can't find google, the world must have ended so might as well not bother coding for "unable to resolve google.com" return code, so this return code results in a total crash of the system.

This is why I personally advocate for designs in which the cloud is not required for normal operation, only for additional, non-critical features, e.g. OTA updates, ok if offline for a few hours a day, or even reporting the number of available stalls (nice to have, but most people would prefer not knowing how many stalls but still be able to charge, vs. no-availability information means all stalls are free because no charging can take place). Not only that, the connected part of the device should be completely separate, so that it can crash (or get hacked, it is on the internet after all) without affecting the core functionality. All that said, the current generation of engineers love to design cloud-reliant features by splicing together code examples and importing bloated libraries, because it helps them get there faster, but when the cloud infrastructure malfunctions, all functionality is lost, or worse - the system crashes.

 

[snstevens]

The fanboy back in full swing!

I rather watch this, then reading your fanboy nonsense!

@feye - You're not making a contribution to the discussion with this sort of post.

You've had some good things to add to discussions in the past so I expect more of you...

 

[daveo4EV]

it's my "sense" that in North America the "problems" with CCS are in 3 basic categories:

1. CCS1 plug design lacks ergonomics and is fragile and unwieldy - and when it breaks (which it does often) the entire station is down until the connector/plug can be replaced/repaired - a costly "fault" in both time and direct costs - stations can be routinely offline for weeks/months waiting for cable repair/replacement - NACS greatly improves on the ergonomics and robustness vs CCS1 and I believe this is the _MAJOR_ factor driving Ford/GM after living with CCS1 up until now.
2. Charging session "activation" - EA/EVGo/Charge point and others seem to have complex and fragile charge session activation procedures. These are operational problems and have nothing to do witih CCS1 physical design _OR_ the CCS protocol for charging a vehicle - they have designed/implemented Point of Sale systems that simply are infuriating. Switching to NACS will have no affect on these issues and operational problems with non-Tesla Charging vendors will remain even if we could wave a magicwand and make NACS happen overnight for all existing and future vehicle's and chargers - this is not a short coming of CCS or NACS - it's purely operational "front end" issues with how the charging vendors are managing their charging infrastructures - European CCS based charging vendors seem to be demonstrating much greater compentence in this space. This problem has nothing to do with CCS as a charging protocol
3. Reliabilty of of the stations to remain operational for period of weeks/months with out requiring maintanance. Frankly none of the existing CCS charging stations installed by any of the vendors seem to be demonstrating any robustness in terms of remaining operational - this problem is not helped by item #1 - but even if the CCS1 conenctor and cord is perfect often times stations are offline for days/weeks/months waiting repair for various reasons (including wonky point of sale components that are part of item #2). Charging vendors in North America seem unable to be able to effectively maintain functional charging stations, and even when they are known to be defective, seem to lack motivation and/or ability to address repairs in a timely fashion. Again this does not seem to be a major problem in other markets.

CCS as a charging protocol has nothing to do with items: 1, 2, or 3…

• NACS should improve #1
• Tesla has demonstrated competence for item #2 - other's not so much in North America - the reasons are probably multiple and complex and institutional in nature.
• Tesla has demonstrated competence for item #3 - other's not so much in North America - the reasons are probably multiple and complex and institutional in nature.

effectively tesla has added CCS charging support to their entire north american charging network (as they have in europe) - and today are the defacto "best" and most reliable CCS charging network in North America - they have more stalls, more sites, and more charging than anyone else, and a relability and availability record that is world class and hard to imagine better. NACS has also proven itself to "not be a problem" in terms of impacting charging station availability - CCS1's physical is PROVEN to be root cause in a significant number of reasons for why chargers can be made inoperable and require costly/time-consuming repairs - even worse it can be broken frequently and repeatably with no correlation to "I just fixed that" - there is zero potential longevity with regards to this part (I speculate that network operators have a "budget" for each station's CCS1 cable/connector - 6 repairs a "quarter" - so if it breaks a 7th time we'll wait until next fiscal quarter to fix it - because the lost revenue is cheaper than the repair replacement…it's this sort of twisted but fiscally responsible logic that can lead to the sorts of network operational problems we are seeing in North America) - we know it's broken - but it's cheaper for me to leave it broken than it is to fix it - and if I fix it will just break again - causing a never ending open ended expense cycle. OMG I think this is legit - we can all see it right?

Ford/GM are moving to NACS as a physical standard, but I expect they will remain CCS based vehicle's and would be deeply suprised if they ever adopt/implment the native Tesla supercharger protocols - they don't need to if Tesla is providing native CCS sessions at their supercharger sites. Which they are - because my unmodified 2020 Taycan Turbo charges just fine at the Supercharger in Scott's Valley, CA via the passive MagicDock adapter - this is also how Tesla operates their network in Europe - native Supercharger charging sessions for Tesla vehicle's via CCS2 connector/port/cable - and native CCS charging sessions for non-Tesla's via the CCS2 connector/port/cable…

CCS1 as a physical design standard needs to go away - it has issues and they are affecting the robustness and usability of the entire North American EV charging infrastructure - it's a failed design at this point in time - no question and no one can defend it or it's record in terms of usability/ergonomics or reliability in terms of being resilient for it's use case. NACS wins hands down on those metrics when compared to CCS1.

but CCS as a charging protocol is here to stay and will never go away - nor should it. It's not the problem.

After NACS "happens" what will remain is the stark reality that EA, Chargepoint and EVGo (and others) so far have zero demonstrated competence at keeping their networks operational - and once your remove the "CCS1" induced failures from the equation you're left with operational problems with the organization and their business practices that have nothing to do with CCS, CCS1, NACS, or anything to do with actually charging the vehicle. 90% of the time when I have a problem with EA it's either because - all the stations are broken/offline, or I can't get the session to actually start because EA can't seem to debug their process for starting a charging session (there are 10 ways to start an EA session none of which actually work most of the time).

Tesla never had problem #1, and they have demonstrated competence on items #2 and #3 vs. the other guys.

 

[tigerbalm]

I hadn't appreciated that CCS1 is a worse design and contributes to the unreliability of North American charging stations. I mostly thought the main difference between it and CCS2 was just the "AC" part of the socket had a different layout.

Having never used (or seen) a CCS1 station, I hadn't even noticed the "button/latch" on the plug before.

Are the stations that Tesla have converted to CCS via the MagicDock solution using a CCS1 like latch design too?

Is the thinking that if the latch is on an adaptor that a user owns/brings with them – they'll look after it and it will be more reliable than hanging off the end of a public charging cable subject to abuse?

 

[daveo4EV]

I hadn't appreciated that CCS1 is a worse design and contributes to the unreliability of North American charging stations. I mostly thought the main difference between it and CCS2 was just the "AC" part of the socket had a different layout.

Having never used (or seen) a CCS1 station, I hadn't even noticed the "button/latch" on the plug before.

Are the stations that Tesla have converted to CCS via the MagicDock solution are using a CCS1 like latch design too?

Is the thinking that if the latch is on an adaptor that a user owns/brings with them – they'll look after it and it will be more reliable than hanging off the end of a public charging cable subject to abuse?

CCS1 MagicDock has the "same" problem has CCS1 "cables" - let's ignore MagicDock for the time being and focus on a Ford Mach-E CCS1 based vehicle with the "flawed" CCS1 "adatper" for Supercharging…

if/when the CCS1 adapter "breaks" it only affects that sole Mach-E user - but leaves the NCAS station "operational" for every other user…since the NCAS plug/connector/cable remain functional

the problem with the CCS1 latch is it's attached to the station's cable - if the CCS1 latch is "broken" that entire station is OFFLINE until the cable/connector is repaired/replaced - no other CCS vehicle (including NACS based vehicles such as a Tesla Model 3) can use that station/connector/cord until the connector is repaired.

NACS and CCS2 both have the "latching" mechanism inside the vehicle's - this has two major reliablity advantages (even if it's subject to breakage)

1. if it breaks it only affects that vehicle and that owner - it doesn't render the station inoperable for everyone or require dispatching personal to the site for repairs
2. that latch only "cycles" when that car charges - which is twice a day at most??? twice a week normally - vs. the same latch on a CCS1 cable which cycle's 15-30 times a day 7 days a week 52 weeks a year. Simple MTBF stats demonstrate that latch will in it's life time cause problems due to it's high cycle utilization.
   1. add breakage due to damage vs. MTBF failures on the cycle count and these things can fail at any moment for any number of uncontrolled reasons
   2. the failure mode here is the stations is offline until the latch is repaired -normally by replacing the entire cable…
   3. failures of the latch can occur for any reason at any time with no correlation to longevity - fix the latch on monday with a new cable, customer drops the CCS1 connector on tuesday - and back offline until repair - less than 24 hours of uptime after repair.

Add into the equation the ergonomics of the CCS1 cable (it's big, heavy, and hard to manage) and it gets "dropped" a lot in "normal" used - the drop alone may cause the latch to become broken…

even if the adapter is flawed and subject to "breakage" it's usage profile is so different that it doesn't matter - my personal CCS1 adatper being broken does not render stall #12 of 16 stalls in Scott's Valley, CA inoperable until it can be fixed (days/weeks sometimes months)

your 2023 CCS2 Taycan's internal CCS2 latch being broken and therfore unable to start a CCS2 session does not take the entire CCS2 charging stall offline - it means you can't charge your Taycan, but other CCS2 vehicle's are unaffected.

the fact that CCS2 and NACS have no moving parts, denser designs, and easier ergonomics makes them vastly more reliable _BY DESIGN_ - it seem amazing, but that simple design "oversite" with the CCS1 plugdesign _IS_ one of the major reasons the entire North American CCS charging network is more fragile than either the Supercharger network (in the same region and operational conditions and more charging sessions/cycles) vs. Europe which if a station is offline it is far less likely because of a problem with the connector/cord attached to the station.

I'm 100% sure Ford/GM have the hard data on this - and I'm betting they came to realize the CCS1 design makes it such that stations can NEVER be reliable…the design is fatally flawed, fixing requires "change" - so any change in this space is equally hard - moving to CCS1.1 is just as expensive, time consuming and disruptive as CCS2 or NACS…

CCS1 has a fatal design flaw - it has moving parts, that if broken, render the connector and cord inoperable and required a station repair visit - normally to replace the entire station cable - which is an expensive water cooled item itself and time consuming replacement task…

add in the bad ergonomics take make CCS1 hard to insert and prone to being dropped and you have what we have today. Two major auto makers vested in the standard, part of the SAE organization that designed it, vested interest in NO CHANGE in this space, have called it quits and is doing a wholesale replacement from their main competitor's part's bin…it's mind blowing when you realize how bad it must really be for them to make this change.

it's just sooo bad once you think about it…

NACS and CCS2 does not have this issue.

now focus on "magicDock" supercharges - same problem they can break - but the MagicDock is "not" attached to the NCAS cable - it's modular - so while it can still break in the same way - it will not affect native NACS vehicle's since when they grab the cable the NACS connect detaches from the "broken" MagicDock CCS connector and remains operations - repair while still requiring a "visit" is cheap/fast/easy - detach the broken CCS adapter from the magic-dock holster - replace it with a new operational unit - insert back into the holster - repair complete - no station down time during the "repair" - and NACS based vehicle's are unaffected while the MagicDock adapter is 'broken' …still costly to send someone onsite to replace the broken CCS adapters - but trivial to repair once they are onsite…this may be one reason why Elon has been "resistant" to adding CCS1 cables to Supercharger's - cable replacement is expensive/time/consuming and renders the station "offline" until the cable can be repaired…

if you avoid the MagicDock approach (which I believe is correct and Elon wants) and every CCS1 vehicle is BYOA (bring your own adapter) - any given adatper being broken had no impact on the charging sites/stall relability - only affects the owner of that vehicle for that one charging session - and repairing/replacing the adapter does not require a charging site visit - no impact to me as the operator of the charging network.

CCS1 is flawed - it's just so bad.

 

[or1]

Seen this?

 

[daveo4EV]

OMG - I think I just figured something out - seriously

CCS1 is so bad and so fragile - it makes perfect sense for charging operators to leave their stations broken - because the potential cost of constantlly "fixing" broken CCS1 cables/connector is more expensive than the lost revenue - there is ZERO correlation between "fixing" a broken CCS1 cable and any expectation of longevity (revenue restoration) - you can fix the cable on monday and it can be broken again on Tuesday when a customer drops it again…you can't do this effectively and expect to make any money - it's better to leave it broken than fix it…because it will just break again - so you "budget" a fixed number of replacement cables to control costs - and once you "hit" that budget - well the potential lost revenue is less costly than the repair…

simple math to calculate max revenue for a given stall - 24 hours day @ biling rate per-kwh = max revenue - how many cables does it take to wipe out max revenue - math to do this is trival...and damming probably

CCS1's design incentivizes leaving the station broken…vs. a policy of repair when broken for uptime.

WOW - OMG - WOW - I can totally see it.

why fix it, if it can just break again - we'll visit each station once every 4 months and repair any broken cables, but leave them broken until each "scheduled" visit - because the lost revenue (and saving on electricity not consumed) is a better cost control metric than unbounded repair cycles of repair on demand, because there is no control mechanism for demand -you can not make it more robust and lower repair demand - it's an unmanged expense - business don't handle unmanaged expenses - they manage them - and the ONLY way to manage CCS1 repair expenses is to limit them - so you leave the station offline rather than repair it - because it makes "business sense" - to do otherwise is irresponsible because it could be an unbounded expense.

that boy's and girls has blown my mind for tonight!!

it's cheaper to leave the charging station offline than it is to keep it operational if your major reason for being offline is broken connector/cable - I bet Ford/GM came to understand that.

 

[daveo4EV]

Is the thinking that if the latch is on an adaptor that a user owns/brings with them – they'll look after it and it will be more reliable than hanging off the end of a public charging cable subject to abuse?

short answer - your CCS1 adapter being broken has no impact on my 16 stall charging site's uptime…yeah it sucks to be you if you drop the adatper and the latch breaks, but my charging site and all it's stalls remain 100% functional for everyone else. And it's your expense (not mine) to fix/replace replace the adapter…

if reliability is what you're going for it makes perfect sense.

 

[or1]

OMG - I think I just figured something out - seriously

CCS1 is so bad and so fragile - it makes perfect sense for charging operators to leave their stations broken - because the potential cost of constantlly "fixing" broken CCS1 cables/connector is more expensive than the lost revenue - there is ZERO correlation between "fixing" a broken CCS1 cable and any expectation of longevity - you can fix it on monday and it can be broken again on Tuesday when some one drops it again…you can't do this effectively and expect to make any money - it's better to leave it broken than fix it…because it will just break again - so you "budget" a fixed number of replacement cables to control costs - and once you "hit" that budget - well the potential lost revenue is less costly than the repair…

CCS1's design incentivizes leaving the station broken…vs. a policy of repair when broken for uptime.

WOW - OMG - WOW - I can totally see it.

why fix it, if it can just break again - we'll visit each station once every 4 months and repair any broken cables, but leave them broken until each "scheduled" visit - because the lost revenue (and saving on electricity not consumed) is a better cost control metric than unbounded repair cycles of repair on demand, because there is no control mechanism for demand -you can not make it more robust and lower repair demand - it's an unmanged expense - business don't handle unmanaged expenses - they manage them - and the ONLY way to manage CCS1 repair expenses is to limit them - so you leave the station offline rather than repair it - because it makes "business sense" - to do otherwise is irresponsible because it could be an unbounded expense.

that boy's and girls has blown my mind for tonight!!

it's cheaper to leave the charging station offline than it is to keep it operational if your major reason for being offline is broken connector/cable - I bet Ford/GM came to understand that.

I think you are on to it there. The operators are not in essence incompetent, they are managing their business the "best" they can.

 

[daveo4EV]

I think you are on to it there. The operators are not in essence incompetent, they are managing their business the "best" they can.

CCS1's design flaws mean you have uncontrolled repair costs if you optimize for uptime - you can not be a CCS1 charging operator and have controlled maintenance costs and excellent uptime - by design!

CCS1's failure rate is essentially uncontrolled (random) - and busy stations will be broken more often = more stalls = more potential cable repairs - so we install fewer stalls - and fix the broken CCS1 cables on a fixed schedule.

and potential revenue from each station is "fixed" - hours in the day some one could be plugged in…

OMG a 5th grader with a head for business would realize you don't fix the stations when they are broken - because each station only has fixed revenue potential - there is no incentive to keep the stations "running" if one uncontrolled expensive and labor intensive element can constantly cause them to fail - yeah I'll lose revenue all quarter long rather than spend money to fix that damm cable again this quarter.

CCS1 has no predictable MTBF - and it's one of the most expensive and labor intensive elements to repair/replace - and it subject to abuse…

so you "fix" your repair costs for each stall - and eat "lost" revenue and hope the random cables that continue to not be broken make enough money to make this whole thing work - and the fact that people can't charge their EV's - well they should've bought a gas car or a Tesla if they wanted it to work.

OMG what a sh*t show.

 

[or1]

CCS1's design flaws mean you have uncontrolled repair costs if you optimize for uptime - you can not be a CCS1 charging operator and have controlled maintenance costs and excellent uptime - by design!

CCS1's failure rate is essentially uncontrolled (random) - and busy stations will be broken more often = more stalls = more potential cable repairs - so we install fewer stalls - and fix the broken CCS1 cables on a fixed schedule.

and potential revenue from each station is "fixed" - hours in the day some one could be plugged in…

OMG a 5th grader with a head for business would realize you don't fix the stations when they are broken - because each station only has fixed revenue potential - there is no incentive to keep the stations "running" if one uncontrolled expensive and labor intensive element can constantly cause them to fail - yeah I'll lose revenue all quarter long rather than spend money to fix that damm cable again this quarter.

OMG what a sh*t show.

Put another way: average uptime after a repair is so short that it's not possible to get enough revenue to cover the repair. If that is the case, a worse design than CCS1 can hardly be imagined.

Edit: CCS1 with the latch as part of the charging station cable. A latch on an adapter owned by the car owner is still bad design, but the uptime and repair cost consequences are very different and it may be an acceptable system.

 

[daveo4EV]

Put another way: average uptime after a repair is so short random and frequent that it's not possible to get ever realize enough revenue to cover the repair forseeable operational maintainence. If that is the case, a worse design than CCS1 can hardly be imagined.

agree 100% - minor tweak for emphasis - but yeah - CCS1 means you can NOT have an effective EV charging infrastructure with high availability.

a standards change is required - the euro CCS2 plug would've been another choice. It didn't have to be NACS - but it had to be something.

 

[daveo4EV]

Sponsored