barnard's Toyota MR2 - now Mr240

[barney]

I hope you dont mind, coulomb and terry, but Im not gonna do THAT to any of MY batteries... well, not yet. All I plan is re-seating B44 and B45 (and possibly 22, the other end), using CABAc's approved grey caulking compound.

I do indeed have a big red switch, which should disconnect +ve (if weber is right) and the logging cables are at a loose end, until I attach one logger to them, so they are 'free'. In theory this leaves only the -ve cable, which snakes out of the Upper battery box and, inside orange flexible stuff, onto the Controller box. It would be very convenient if this were the only cable, and I could leave it, and indeed it does seem to me that is the case. See pic of the red switch and orange flexible conduit.


In tomorrow's light I unbolt (carefully, protecting all possible metal surfaces) then scrape with wire brush all connections and finally coat them with Cabac's grey conducting goop and re-attach. No torsion wrench so it will be a guess, I'll try and do them just like they were at undo time.

Because B45 is the end of the string, it has 3 items not 2... the main battery terminator as well as BMU as well as braided link to B44. In fact this may be part of the reason Im working on it. For the same reason, should I do B22 next door? B22 is its at the other end of the same string... And B44 next door was also problematic in the recharging log, so it will get a scratch too. Then, the point of all the work, I can log another re-charge, and post the results. Whoo hoo!     Barney

[T1 Terry]

Any chance of doin a charging log before the work and another after the work? Just to verify the work actually achieved the result required and that it hadn't already self rectified.

T1 Terry

[barney]

Hi T1

I did think about that, but since the only change over the last 6 mo in battery performance has been to get slowly worse, Im hoping the charge log of earlier in the week will suffice. Also, I know it won't have much effect but with the battery pack sitting at 40% not 95% I thought I'd feel happier working on it!

Wish me luck... Barney

[weber]

Here there are effectively two "pouches" in parallel. It would be conceivable for the connection to one of them to be insulated by some oxide layer or similar that was somehow voltage sensitive.

I see no pouches in that image (I note your scare-quotes, Coulomb). Yes, Barney's Sky Energy cells look like that inside too. If you take them out of the case, you can peel them apart from the bottom like the pages of a book (or perhaps a calendar, given the orientation). The pages go copper, plastic, aluminium, plastic, copper, plastic, aluminium, plastic ... A repeating sequence of 4.

But I agree it is possible, that one half could have a high-R connection and the other not, or that the connection resistance could come and go. But not very likely.

[weber]

For the same reason, should I do B22 next door? B22 is its at the other end of the same string... And B44 next door was also problematic in the recharging log, so it will get a scratch too. Then, the point of all the work, I can log another re-charge, and post the results. Whoo hoo!     Barney

If connection resistance proves to be even a part of the problem with B45 and its disadvantaged mates, then you should redo the lot. But there's no need to do anything with B22 for this test.

I still suspect that part of the problem is that the charger's voltage limit is set about 1 volt too low.

[coulomb]

I hope you dont mind, coulomb and terry, but Im not gonna do THAT to any of MY batteries... well, not yet.

Well, that's a Hi-Power cobalt cell, it would be good to see inside a CALB or Sky Energy cell...

Because B45 is the end of the string, it has 3 items not 2... the main battery terminator as well as BMU as well as braided link to B44.

In a single series string of cells as you have, surely every cell terminal has either a link to its neighbour, or a cable off to another box or the controller etc. So B45 is no different in this respect.

The thermal properties of a cable are different to that of a link, perhaps especially a braided link, but I would only expect to see that difference over many charge and drive cycles.

[barney]

I have cleaned, sanded, polished and gooed each and every part on B44 and B45 that was an electrical carrier of current. Photos up next, just gotta take pics of the charging process now going on in the garage. By looking at photos of the terminals, you might get an idea of whether they were the problem?

Barney
[edit: typo >the]

[barney]

Battery pack still charging, so here are the items before and after a scrub and polish with caulking compound. It smells like whale oil (not the charging, the goo), didn't go well with the toasted sandwich I can tell you...

I hope these are small enough to fit side by side, but still big enough to be clear. First the surgeon's table, and typical screws etc as they were removed. My first impression is that they are not too bad?


Next the various posts and connectors...


The takeoff from B45 to the big red switch has a copper spacer so that its BMU has equal height connections at both posts:


The BMUs are also in the mix; they were cleaned with RP7 spray lubricant, polished clean with a cloth and then smeared with white goo, as was the copper spacer....


And of course it all fell back into place without a hitch; the pic is before tightening everything up, but I may not have used enough goo. Fixable, but hope its not a problem for tonight's charge...


I guess I should build another timeline, so comparison is easier with the previous charge. Should be ready tomorrow night, easlier if I can.

Cheers, Barney

[edit - more typos...]

[barney]

In a single series string of cells as you have, surely every cell terminal has either a link to its neighbour, or a cable off to another box or the controller etc. So B45 is no different in this respect.

You are right, of course. The takeoff to the controller is physically different and that is what I was trying to express; you point out it is not electrically so. However, taking the BMU off B45 this afternoon I realised there is a further difference, namely that there are TWO wires running back to the ribbon cable which ends with the Logging connectors. It is noticeable in the final assembled pic of my latest post, uploaded c.11.15 this evening (presumably back one post).

The last pic shows B45 (top) and B44 (bottom), and B44's is like every other BMU, namely a single wire running back to the ribbon. B45's BMU has two wires, one either side of the sealed section in the BMU's centre, both running back to the ribbon and the Logging connectors.

I hope this isn't significant for the purposes of this recharging analysis?

Cheers, Barney

[barney]

Big discovery!

I mused a day or two ago, that B45's bypass red LED seemed not to come on during the previous charge, since when I took a photo of the batteries with red leds in bypass, B45 was not in the list. I was wrong. Tonight it did. But in sad circumstances...

B45's red led did indeed turn on, but much earlier than any other cell; and since the charger was still pumping out 19.4A, it wasn't able to protect the cell at that point. Nevertheless, the bypass light was the first in the whole upper pack. As soon as the V dropped to 4A or less, bypass WOULD have worked.... but at that point the V of cell B45 seems to drop away quickly and as a result, bypass wasn't needed, in fact the cell jumped from being the highest voltage to the lowest in a just tens of seconds.

It occurs to me that this behaviour (since it is AFTER the cleaning of the terminals) might not augur well for the prospects of my batteries being in solid condition?

Time to get the data logger uploaded, charging has now finished and there is a spreadsheet to build.

Barney

[Tritium_James]

Barney, I think I can see a problem - not sure if it's 'the' problem though...

You appear to have the cable lug terminal *above* the BMS. This means your high currents are flowing through the BMS circuit board and the bolt, rather than directly from the battery terminal to the lug. The way you have it is definitely going to be higher resistance than a direct connection.

[barney]

Aha, great observation, Tritium_James. Out of interest I'll have to check to see if that was how it was assembled BEFORE the cleaning. Aaah, before:

and after:

I'll get out there now and return the arrangement to the 'proper' method - thank you for your eagle eyes!

Barney

[barney]

So is it drivable like this or is it super serious and should be changed immediatelyimmediately?

[Johny]

If B45 really is as bad as it looks in the logs then you could be very close to blowing up the B45 BMS module when that cell gets discharged. B45 looks pretty sick. You are quite sure that the bolts are not bottoming out. I.e. You can't rotate the cables under the bolts or anything?

[weber]

Barney, I think I can see a problem - not sure if it's 'the' problem though...

You appear to have the cable lug terminal *above* the BMS. This means your high currents are flowing through the BMS circuit board and the bolt, rather than directly from the battery terminal to the lug. The way you have it is definitely going to be higher resistance than a direct connection.

Hi James,

I saw that, but said nothing, not having seen the "before" photo. I just shook my head as usual. One reason I said nothing is because the "after" configuration uses the stacking order that is recommended in the EV-Power BMS installation manual. In fact it is a slight improvement in regard to resistance, compared to the recommended configuration which is this:



See page 7 of the manual here:
http://www.ev-power.com.au/IMG/pdf/cell ... ion_SE.pdf

To be fair, the resistivity of Stainless Steel is only about 40 times that of copper.

The "before" configuration, or the one that you and I would recommend, is explicitly shown as "incorrect" because "Termination in this way can result in cell module internal electrical damage when the bolt is removed."

Also, I didn't think the configuration was the problem, because there are several other cells behaving like B45, which don't have that terminal configuration.

I was shocked to learn that those expensive braided links have slots in their end-pads, which reduce their contact area by about half. Why would they need slots when the whole point of the braid is that they are flexible?

[weber]

If B45 really is as bad as it looks in the logs then you could be very close to blowing up the B45 BMS module when that cell gets discharged

B45 "only" went down to 1.6x volts in the logged driving test, it didn't look in danger of reversing. And we can see that the data logger sees exactly what the BMS sees in the case of cell B45. So as long as barney doesn't keep driving with the BMS alarm going I figure it should be safe. But Barney, I agree you shouldn't be driving with this pack except for the minimal amount required to actually determine what is wrong.

I'm hanging out for those graphs of the post-cleanup charge.

[woody]

I guess the EV-power recommendation is to protect the BMS against "disconnect destruction":

This is a non-obvious voltage which appears when you disconnect your battery while it is connected to a load or charger.

I think you mention the Weber-Colomb BMS is protected against this.
For the rest of us, we need to isolate the battery to avoid this.

It's good practice to have the battery automatically isolated unless you are charging or driving.

[T1 Terry]

Big discovery!

I mused a day or two ago, that B45's bypass red LED seemed not to come on during the previous charge, since when I took a photo of the batteries with red leds in bypass, B45 was not in the list. I was wrong. Tonight it did. But in sad circumstances...

B45's red led did indeed turn on, but much earlier than any other cell; and since the charger was still pumping out 19.4A, it wasn't able to protect the cell at that point. Nevertheless, the bypass light was the first in the whole upper pack. As soon as the V dropped to 4A or less, bypass WOULD have worked.... but at that point the V of cell B45 seems to drop away quickly and as a result, bypass wasn't needed, in fact the cell jumped from being the highest voltage to the lowest in a just tens of seconds.

It occurs to me that this behaviour (since it is AFTER the cleaning of the terminals) might not augur well for the prospects of my batteries being in solid condition?

Time to get the data logger uploaded, charging has now finished and there is a spreadsheet to build.

Barney

I'm keen to see the data log and a reference to where the BMS red led came on. Was it over voltage or the BMS simply coming on too early resulting in part of the charge that cell would have received being wasted resulting in that cell remaining under charged.

T1 Terry

[weber]

I'll get out there now and return the arrangement to the 'proper' method - thank you for your eagle eyes [Tritium_James]!

This is probably too late, and you probably did the right thing anyway, but don't forget to turn off the big red switch and unplug the ribbon cables, same as last time.

Assuming the answer to Johny's excellent question about bolts bottoming out is "No", it's looking like it wasn't high resistance joints but rather the high resistance is inside B45. If that's the case, then we want to know whether it is permanent or temporary, and we want to know what caused it in the first place.

As I've said a number of times, I suspect that the charger's voltage limit is set too low and this has led to those cells that happened to have a higher self-discharge, or that happened to have a lower state of charge when first installed, never getting fully charged, and so falling further and further behind the others.

To determine whether B45's problem is permanent or not, we need to get it fully charged. i.e. we need to hold it at 3.65 V until the current falls to not much more than the 600 mA its BMU is bypassing.

T1 Terry has made a number of good suggestions about low-cost single-cell chargers of various kinds. That's one way.

The other way is to make the existing charger work as it is supposed to, and do a proper balancing charge on the whole pack, making all the BMU's bypass lights come on at the same time, and stay on until the current falls to not much more than the 600 mA the BMUs can bypass.

This has the advantage that it will help all the underprivileged cells at once, and may eliminate the original cause of the problem. It has the disadvantage that it requires you to adjust your charger, which I believe requires getting access to a specific trimpot inside it and it has to be done "live" because you can't get any output to measure unless there is a battery connected to it.

Good article here:
http://www.endless-sphere.com/forums/vi ... =14&t=6196

But probably better not to do this yourself, if you can help it. Is the company who supplied the charger local? Can you get them to do the adjustment?

[coulomb]

... but I may not have used enough goo.

It looks to me like you have used plenty of goo. Remember, this isn't conductive; its job is to fill the air gaps and exclude oxygen from the clean metal surfaces so they don't oxidise. You use it sparingly, and the act of tightening up the bolt should cause all excess goo to squeeze out of the way. None of it should end up between the cell terminal and the lug or link, except where there are imperfections that would not have made contact anyway.

By the way, did the bolts seem reasonably tight? They shouldn't have been gorilla tight; that would risk damaging the internal thread of the cells. But they should have been noticeably tight; it should have been impossible to undo by hand with no tool, for example. Certainly, you should not be able to move the lugs by hand.

[coulomb]

B45's BMU has two wires, one either side of the sealed section in the BMU's centre, both running back to the ribbon and the Logging connectors.

I hope this isn't significant for the purposes of this recharging analysis?

This is unavoidable. Monitoring 8 cells requires at least nine wires; it's a fencepost and rail thing (there are always one more fenceposts than rails). Computer scientists refer to "fencepost errors" (also known as "off by one errors").

A better way to do it is to use 16 wires, so the measurements are independent of the resistance of the links. But that is not so practical; you need almost twice the number of wires and connector pins, and the differential inputs are more expensive.

With the nine wires, it means that one of the eight cells is measured without interference from the resistance of the links. This happens to be B45, the very cell we are most interested, so that's a happy situation. Of course, there could still be weird effects from resistance of the bolts, especially if the BMU is underneath. But with the charging current limited to less than 20 amps, and newly cleaned contacts, this should hopefully have no measurable effect on the results.

[barney]

Thanks for all the explanations, queries and observations - useful. Now to re-create a timeline, which is unavoidably long (well, there are other ways to do it but this is very easy, by comparison and it matches the previous one; and of course suggestions for better methods are welcome)











*Correction*: At lines 962 & 1470 I identified early-riser cells (reached bypass/led went red) before charger Amps dropped enough to be useful). One was incorrectly referred to as B26, it is B25. The three 'ER' cells in the upper pack are B25, B36, B45.

Note on the column titles, colouring etc.
----------------------------------
The 3 cols on the left are time measured used by different machines; the CellLog shows Time elapsed top right of the screen regularly, hence the Elapsed time in hrs format; and Actual comes from the camera's clock, so I can place the images in the right row.

The colour gradation from red to yellow is determined by the value in the cell. Lowest scores are yellow, anything below 3270 or 3.27V; higher scores are red, anything above 3690 or 3.69V. Screen grab of the formula might explain better.


The column Apparent V is simply calculated from the 26-odd V of the 8 cells being measured, as tho they were the whole pack. Its not super accurate but it shows the trend, as does the coloured blue bar in the same cell, and hopefully even better.

I have tried to make the junction between screen grabs more obvious, and have dropped rows whenever there was nothing that wasn't predictable, going on (eg a steady increase in voltage with no change in current).

As usual the full spreadsheet is available as a zip file from the ZEV240 site (455MB, xlsx format only atm) , but there isn't much extra in it except some formulae and the conditional formatting.

The point of it all seems to be that there isnt much difference that a bear like me can see between the two charging runs. But I did notice that B26 was rating at nearly 4V after the charge. And presumably there are a rougly equal number of dicky batteries under the car as well in the lower pack. Assuming some of these are indeed dicky!

I note weber's (I think) suggestion that I dont drive the car atm. That is the bluntest bit of advice Ive received so far. This is looking a tiny bit serious....

Barney

[edit: Correct weirdly-scaled image in timeline;
pointed out the early-rising cells included B25, not B26]

[weber]

Good work Barney. Now we just need the graph. But I couldn't wait and plotted one myself. I see it is almost identical to last time. So it sure looks like B45, and presumably its 9 mates really have gone high internal resistance.

I'm thinking they may have been overheating under heavy load for some time.

[T1 Terry]

Do you know what range the BMS units turn on and off their red light and start/stop their equalize phase? At row 1419 where it's marked the charger shut down was B45's red light still on? By 1313 the red light should have been off.
the cell that was still high quite possibly isn't discharging via the BMS link so progressively it's become over charged (over full)The high voltage from this cell and the others that are going high results in the other cells in the pack not reaching fully charged together,(average voltage charging draw back) if the BMS links aren't stopping their discharging all at the same voltage more cells will possibly end up being damaged as these cells start their load duty with a lower state of charge.

T1 Terry

[barney]

Hi Ti terry. I guess this is the challenge of individual monitoring vs global (one wire) monitoring, when decisions are made about total voltage even tho it may be one or two or more are actually to high, starving the lower ones yet to get full. This is why Jack Rickards compared a battery pack with a wolf pack, saying the weaker ones are devoured by the stronger ones (if that is how wolves work).

But to answer your question, I think you are referring to the individual BMUs. I do not know their exact figures. But I can confirm that by row 1419 B45's light was already off. It was off by the time pic on row 1295 was taken, and probably went off by row 967 (colour change shows a fall below ~3.6V) and was certainly off by row 1295 according to the photo. Whether it was off at 1270 is moot, as this was the changeover time for B44 and B45. At this point they are BOTH rated OVER (see the 'O' above columns 7 and 8?) , according to the CellLog. This may have been about the time B45's red led went off, and B44's red led came on. I wasn't watching, unfortunately!

Hth, Barney