Odds and Ends: Finding the “root cause;” The cliche about bad pennies

Finding the “Root Cause:” The world waits for the Boeing 787 to return to service, following a series of proposed “fixes” designed by Boeing in conjunction with its relevant suppliers and with help from Ford, General Motors and other experts versed in lithium ion batteries.

The National Transportation Safety Board, investigation the Japan Air Lines battery fire in Boston, and the Japanese investigators trying to figure out the battery melt-down of the ANA battery in Japan, have yet to identify the root cause of the issues.

This disturbs many, who question the wisdom of prospectively returning the 787s to service without know the root cause. We confess we’re not too happy about this, either…the idea of an in-flight fire simply scares the bejesus out of us. (So does Boeing’s insistence no “fire” occurred, just two-inch flames in the JAL case and none at all with ANA. This simply is an eye-roller.)

But Sunday we were watching a program on the Smithsonian channel called Air Disaster. This program examines air accidents and near-accidents and this particular episode was called Turning Point, about Northwest Airlines flight 85 in 2002.

The flight was two hours west of Anchorage on its way to Tokyo (six hours away) when there was a rudder hard-over. Through superb airmanship, the flight returned to Anchorage and a one-shot emergency landing. The flight landed safely.

Investigators determined the Power Control Module (PCM) end cap blew out. The connecting rod went beyond the end of the PCM end cap and jammed the rudder. There was no apparent reason why the end cap blew out.

The long and the short of it: the NTSB could not find the root cause of what happened. But “stops” were added to the PCM to prevent the connecting rod from extending beyond the end cap location should another end cap blow out. Four years later, one did and the Air France 747-400F landed safely. Only then was it discovered that a design defect caused the failure.

This brings us to the 787. Many ask how the Federal Aviation Administration can clear the 787 to return to service without finding the root cause first. Boeing’s redesign has several elements to it, but to us the key one is the element intended to deny oxygen to the battery and thus snuff any fire before it can get started. Basic science tells us if there is no O2, there is no fire.

We recognize that many will say Boeing, its suppliers and the FAA should have designed this system in the first place. But as we have written on more than one occasion, aviation is replete with instances where testing was thought to be adequate only for later service to demonstrate through incident or worse that a flaw worked through the system. We’re just glad this flaw was discovered before any lives were lost or even any serious injuries occurred.

As for the prospect the FAA will allow the 787 to return to service but with ETOPS restrictions: we don’t see why restrictions should be imposed. When Reuters asked for our opinion about the prospect the FAA might restrict the 787 to only over-land flights, we said this would be very damaging and this is true. Since then there has been some speculation the ETOPS would be reduced from 180 minutes to 120 minutes. While this would be inconvenient, costly to airlines and still hurt the 787 business case, this is far less damaging than prospectively restricting the airplane to over-land flights.

Boeing said that it expects no restrictions (Mike Sinnett, 787 engineer, at the Tokyo press conference). Our view is rather pragmatic (or fatalistic, depending on your point of view). Given the information from Airbus in a 2012 fire-and-smoke study (totally unrelated to anything involving batteries or the 787) that a fire can go out of control in eight minutes and you need to land within 15, it doesn’t really matter whether ETOPS in 60, 90, 120, 180 or 330 minutes. If there is an airborne fire, chances are you’re cooked no matter what the ETOPS. (It also might be problematic for land within 15 minutes from a cruising altitude of 41,000 feet to an airport that could accommodate the 787 in any event.)

Note: we caution readers planning to comment on the above to watch yourselves. We’re clamping down on spurious and ill-considered tirades.

Bad Pennies: You know what they say about bad pennies always coming back. This couldn’t be more true with Scot Spencer, the convicted felon who keeps turning up in commercial aviation circles. We knew this guy when he and others purchased Braniff Inc (the second one) from the Hyatt family, ran this into the ground and bankrupted it, then started a third Braniff. Spencer and one of his co-investors went to jail for bankruptcy fraud.

Reputations of several respected airline officials Spencer and his co-investors hired to run Braniff Inc were damaged by their association with Spencer. We then wrote for trade magazine Airfinance Journal and revealed a scheme called upstreaming from a series of aircraft leases whereby Spencer and his co-investors bumped the lease rates they paid to higher rates subleased to Braniff, adding tens of thousands of dollars per month to Boeing 737s leased to the carrier through a separate company owned by Spencer and his co-investors. Once this was revealed, a $100m financing was withdrawn prior to closing. Braniff Inc ceased operations a few months later.

The Department of Transportation banned Spencer from future airline involvement, so he went to San Bernardino (CA) and in a move that still baffles us, persuaded elected officials there to give him millions of dollars in contracts to develop the former Norton AFB into a commercial airport. The project was silly to begin with–the Ontario Airport is just down the road–but even knowing Spencer was a convicted felon didn’t dissuade these stupid officials from giving Spencer contracts.

This story, complete with photo of Spencer in custody and hiding his handcuffs, has links to several other stories.

This old document has some of the sordid history of Spencer’s involvement with Braniff.

Here is a court record of Spencer’s bankruptcy fraud.

52 Comments on “Odds and Ends: Finding the “root cause;” The cliche about bad pennies

  1. I’m hazy on the chemistry, but isn’t Lithium Cobalt Oxide self oxidizing? In other words, it doesn’t need an external source of oxygen to burn?

    • From Wikipedia: LiCoO2 batteries are susceptible to thermal runaway in cases of abuse such as high temperature operation (>130ºC) or overcharging. At elevated temperatures, LiCoO2 decomposition generates oxygen, which then reacts exothermically with the organic materials in the cell. This may pose a safety concern due to the exceptional speed and magnitude of the highly-exothermic reaction, which can induce thermal runaway in adjacent cells or ignite nearby combustable materials

      • Interesting article on battery safety here, referenced by the Wikipedia entry above: http://www.nrel.gov/docs/fy13osti/54404.pdf

        Some things I have learnt from this article:
        – Once thermal runaway starts, there’s nothing you can do to stop it. Your main safety objective must be to protect one cell from affecting the next.
        – Pass / Fail tests aren’t sufficient to determine the safety of a battery. Good design with sophisticated protections are what you need.
        – There is no test against short circuits, but a short circuit is sufficient to kick off a fire.
        – Batteries are somewhat fragile things they need to be treated properly.

      • GREAT point-. Now lets look back at the Boston NO fire jet flames observed by the firefighters.

        What the majority of the public and probably pundits seem to miss is as follows

        Going back many decades- and still in use today are oxy-acetlyene torches. When used to ‘ cut’ steel, or to tap steel furnaces, what happens is that the flame from acetleyne and oxygen ism used to heat the steel ‘ red” hot. Once that steel is red hot, shut off or reduce the acetleyne and use simply oxygen to continue the cutting. When one presses the ‘ lever” on a oxy-acetelye torch, it simply allows a much greater supply of oxygen to continue the process. that is why one does not need to heat the whole piece of ” steel” to melting point before cutting it with an oxygen lance or torch.

        Now think of the multiple ‘ spark holes’ noted by the Japenese agency in their battery issue.

        Bottom line – IMO- Mike S is partially right- partially wrong- but mostly spin re introducing oxygen and saying it does not propogate. IF the oxygen is released as in a pinhole or burst disc- it is in a near jet form and COULD ACT ALMOST LIKE AN OXYGEN LANCE/ CUTTING TORCH !!!!

  2. ..”Boeing’s redesign has several elements to it, but to us the key one is the element intended to deny oxygen to the battery and thus snuff any fire before it can get started. Basic science tells us if there is no O2, there is no fire…

    True as far as it goes- but I seem to recall that the chemical reactions involved with Li-ion are such that they produce their own Oxygen- thus the NO fire jet NOflame observed in boston- and why halon doesn’t really work except momentairily.

    And one still wonders why no heatsink on then boom box like on the eagle-picher cessna battery still undergoing tests and certification.

    • Assume you have enough energy at hand to heat the battery mass to 600..800°C.
      The bomb calorimeter box more than tripples the mass that will be heated.
      thus I would be surprised the box after reaching equilibrium will reach more than 200..250°C on the outside surfaces ( stainless may pose a bit of a problem as its heat conductivity is low
      and hot spots coud develope in places with a heat source in direct contact.).

  3. ” Basic science tells us if there is no O2, there is no fire.”

    This is the problem of many commentators they stick to basic science (when they know & understand it).
    Li-ion batteries have this particularity that the oxidiser is part of the electrolyte: there is no need of an external oxidiser source (like oxygen) to maintain the fire and there is largely enough electric energy stored to initiate the fire that will run away (hence the thermal runaway).

    Anyway, the Boeing box is not a closed system (and must not unless you want an explosion) hence it would not prevent oxygen from coming inside the battery.

  4. I believe this Seattle Times blurb from Saturday is important:

    “Boeing postponed a planned test flight of the 787 Saturday that was intended to test newly engineered power panels. The reason is not known at this time….”

    http://blogs.seattletimes.com/today/2013/03/boeing-postpones-787-flight-intended-to-test-power-panels/
    also:
    http://seattletimes.com/html/businesstechnology/2020671693_boeingtestflightxml.html

    The article notes these are the panels involved in the United and Qatari in flight incidents in December 2012. We know they included the “P100” – the same panel that melted on that test flight in November 2010, nearly crashing the plane in Laredo.

    In 2010 Boeing said that fire was caused by “foreign object debris” in the panel (though no FOD was recovered), and a software patch would reduce the vulnerability.

    Now, however, they’re apparently finally replacing the design. This panel sits adjacent the APU battery in the aft EE bay. Two different components close together that have both caught fire.

    The interesting thing to know would be when and why did Boeing decide to change the panels. Had they been planning the upgrade all along? Did they decide to do this since it’s a quick fix they can do while replacing the batteries?

    Or, is the FAA finally pushing back on Boeing’s assurances that this is all just a battery problem and there is no need for a comprehensive electrical system re-design? I think that is the most likely case.

    What else are they redesigning? Two weeks ago there needed to be just one FAA cert flight to demonstrate the battery. Will this require more? Is there now a moving target of what needs to be “fixed” in the electrical system?

  5. Don, the boom-box itself is a heat sink, but any excess battery heat enclosed within the inner battery-box that is nested inside the boom-box has a somewhat long way to travel before it gets to the outer surface of the boom-box. When it gets there eventually, it can be dissipated into cool cabin air surrounding the boom box, perhaps except for its bottom-side.

    If necessary, a thermo-electric cooling plate attached to the top of the boom box could help to cool the battery box inside somewhat, assuming that the air trapped in the tiny space between the two boxes is free to circulate around a bit between them. Maybe the testing will reveal how hot the batteries are getting if the temperature of each individual battery is being recorded..

    Even a complete future thermal runaway can be contained by the boom-box itself. However, he integrity of the vent tube and its secure attachment to the fuselage would seem to be the weak links in this possible runaway-problem-containment-concept.

    • http://www.ainonline.com/aviation-news/ainalerts/2013-02-05/eaglepicher-li-ion-aircraft-battery-nears-certification

      The cessna style battery is much smaller than the BA battery- yet it has a major heat sink on top

      SS is sort of a poor heat conductor- and if the batteries are surrounded by insulation as in electrical tape plus ceramic dividers, etc- its the battery that will get hot first, etc

      and i’m trying to find a video wherein a model airplane lipo batter was place in a steel ammo box, a 1 1/2 to 2 inch dia tube- a few feet long attached to one side- and then overcharged

      what happened was the lid blew off the box, and very little went out the tube

    • Hmmm- about cert flights and battery

      from the everett herald today april one

      http://www.heraldnet.com/article/20130401/BLOG01/130409990/1010/BIZ01#Dreamliner-flight-Monday-not-part-of-battery-certification-testing

      Dreamliner flight Monday not part of battery certification testing
      Boeing says 787 test flight will take in coming days

      A 787 has taken off from Paine Field on the way to Moses Lake, but Boeing said the flight is not part of the company’s effort to certify the Dreamliner’s redesigned battery.

      “The battery certification demonstration flight will take place in the coming days,” Marc Birtel, a Boeing spokesman, wrote in an emailed statement Monday.

      Birtel didn’t say why the 787 was being flown on two-hour flight south down the coast and east over to Moses Lake. Information on the 787’s flight can be viewed on FlightAware.

      …. Boeing already conducted one flight with the redesigned battery on March 25 and has done some ground testing with a 787 at Boeing Field.

      Maybe tests are not going as planned ?

      or about as accurate as rollout and first flight dates announced for 787 . . .

      • at 12 54 pdst

        http://flightaware.com/live/flight/BOE272

        Snohomish County (KPAE – info)
        Everett, WA

        Grant Co Intl (KMWH – info)
        Moses Lake, WA
        11:19AM PDT

        01:23PM PDT
        Scheduled: 11:00AM PDT

        Scheduled: 12:54PM PDT

        En Route / On Time (229 sm down; 245 sm to go)
        Aircraft Boeing 787-8 (twin-jet) (H/B788/Q – photos)
        Speed 241 kts (planned: 500 kts) (graph)
        Altitude descending 5,100 feet (planned: 35,000 feet) (graph)
        Distance Direct: 146 sm Planned: 952 sm
        Route SEA J523 TOU HQM AST ONP RBG EUG IMB PDT PELLY (Decode)

  6. The 787 is a young aircraft and I cannot imagine it will be flying with Lithium-Cobalt batteries for the next thirty years. I find Boeing’s solution acceptable for the short term, but I cannot possibly see it as a permanent fix.

    The only viable alternative is to change the chemistry of the battery. The way the 787 electrical system is designed may require Lithium-ion, I don’t know. But if that’s the case I would suggest they use a more stable form of Li-ion, like Lithium-Phosphate. But in my opinion the best long-term solution would be to revert to Nickel-Cadmium.

    Leeham:

    – This disturbs many, who question the wisdom of prospectively returning the 787s to service without knowing the root cause. We confess we’re not too happy about this, either…

    Frankly, I am disappointed that the root cause has not been identified yet, but I am not overly concerned by it. I believe Boeing has taken appropriate measures to mitigate the risks, enough so to justify a quick return to service. We have to be safe, but also have to be practical. Boeing did a fantastic job to extricate itself from a desperate situation in order to return the Dreamliner to service after an interminable period where it sees its corporate image tarnished more and more every day.

    But even if I understand that I am still not comfortable with the idea of having the Dreamliner fly around the world with Lithium-Cobalt batteries, even if they have been locked inside a safe. It stresses me just to think about it. I hear it tick like a time bomb. Boeing should start working on an alternate solution that would be more acceptable. Something that would skip the problem of containment.

    The analogy with the Northwest incident is interesting but only partially applies to this case. What Boeing did at the time was the most logical thing to do. Prevent the hard-over from reoccurring until we understand what happened with the cap. On the surface this is exactly what Boeing is doing today: prevent a battery event from harming the aircraft and its occupants until we understand what’s going on with the batteries. And modify the batteries in hope that this would mitigate the risks further.

    This reminds me of what de Havilland did after the Comet was initially grounded. They had no idea what had happened but they reviewed the whole design and made over 50 modifications to the aircraft, hoping it would get rid of the problem. It didn’t work. But Convair had a similar approach with the Atlas rocket and it apparently solved the problem even if they never found the root cause of the explosions. In the case of the Dreamliner we all know what the “root cause” is, it’s the chemical Boeing selected: Lithium-Cobalt.

    My reasoning is that if Boeing gets rid of Lithium-Cobalt it will eliminate any problem with the battery itself. But that does not mean the root cause would then be eliminated, because it could very well lie outside the battery. So getting rid of Lithium-Cobalt would only be a first step towards identifying and understanding the problem. You work by elimination until you get to the root cause.

    To get back to the Northwest incident, to prevent another hard-over from happening was the main goal. Preventing the connecting rod from jamming the rudder was the solution. In other words to CONTAIN the rod. But for the 787 the aim is not to contain a fire but to PREVENT one from happening in the first place. That’s where the comparison stops.

    I will not discuss the flawed reasoning about the oxygen generation because it has already been addressed by other posters.

    Leeham:

    – As for the prospect the FAA will allow the 787 to return to service but with ETOPS restrictions: we don’t see why restrictions should be imposed.

    As I have mentioned in a previous post I have a different view on that. I will not dwell on this, but briefly there are two reasons for imposing restrictions:

    1- It would allow a progressive return to service. For example the first six months could see the 787 flying on ETOP 120. And then fly the next year or so on ETOP 180.

    2- It would force Boeing to redesign the battery system using a more benign chemistry.

    http://leehamnews.wordpress.com/2013/03/28/odds-and-ends-787-service-return-lionairs-a320-order-race-to-paris-777x-v-a350/#comment-36286

  7. http://www.nfpa.org/assets/files/pdf/research/rflithiumionbatterieshazard.pdf

    I found this report, prepared for the Fire Protection Research Foundation, very informative. It corrected me of the notion that Li-ion electrolyte decomposition produces O2. What is produced is mostly CO2, with CO and C2H4 being the second most common constituent. Very little, if any, O2 is produced. Of course, C2H4 is a flammable hydrocarbon, but without O2, it cannot burn.

    This report also had some interesting things to say about cell chemistry with regard to safety. Numerous studies have been done to measure the thermal stability of various cathode materials with various electrolyte materials at full charge. The studies are then often presented as a way to rate the relative safety of various cathode materials. This report criticizes that methodology because it ignores the reactivity of the anode, cell construction details that affect the likelihood of developing an internal short, and the probability of manufacturing defects to cause internal shorting.

    While lithium cobalt dioxide cathodes will begin to react exothermically with electrolyte at lower temperatures than other cathode materials, this does not mean they are less safe. The electrolyte will begin to exothermically react with the anode (typically carbon in powder form combined with a binder) at a much lower temperature than for the cathode. So, it is the anode material, which is very similar across all Li-ion types, that kicks off the often talked about thermal runaway decomposition reaction.

    I’m not saying that the cathode material does not matter, it does. Lithium cobalt dioxide cathodes will get the thermal decomposition reaction going faster than other cathode materials, but it is a popular misconception to think that other cathode material are somehow inherently safer.

    • from that same report . . .When a cell vents, the released gases will mix with the surrounding atmosphere, and depending upon a number of factors including fuel concentration, oxygen concentration, and temperature, the resulting mixture may or may not be flammable. The flammability limits of a gaseous fuel/air mixture are the prime measures for ascertaining whether that mixture is flammable. Fuel/air mixtures have two flammability limits: a lower flammability limit (LFL) or lean limit, below which the concentration of fuel is too low to allow flame propagation, and an upper flammability limit (UFL) or rich limit, where the concentration of fuel is too high for the available oxygen to support flame propagation. If the fuel concentration in a particular gas mixture is between the LFL and UFL, that mixture is ignitable. If a competent ignition source is present, a flame can propagate through the mixture.

      +++

      Now keep in mind that the vent diaphram probalby has a burst pressure above about 10 psi to prevent breaking at 35,000 to 40,000 ft altitude

      and there is some oxygen at altitude – – so flamability limits apply

      Bottom line – supposedly the BARF team has taken all the above into consideration-

      Including an organ pipe effect of a burst diaphram- tube at various airspeeds ?

      so mush so that it will be 1000 percent safe ?

      • 1000% safe. I just love political appointees and their penchant for hyperbole, especially ones that don’t have a technical background.

        Agreed about the flammability limits. I would think the vent gasses should be considerably cooler at the vent opening before they enter the air-stream, and since there should be no competent ignition source at the vent opening, there should be no flame.

        With regard to the organ pipe effect, I know this is not the first time Boeing has had to design a vent to the aircraft exterior. I would hope they could get this one right. We shall see.

        • Mike Bohnet :
          1000% safe. I just love political appointees and their penchant for hyperbole, especially ones that don’t have a technical background.
          Agreed about the flammability limits. I would think the vent gasses should be considerably cooler at the vent opening before they enter the air-stream, and since there should be no competent ignition source at the vent opening, there should be no flame.
          With regard to the organ pipe effect, I know this is not the first time Boeing has had to design a vent to the aircraft exterior. I would hope they could get this one right. We shall see.

          re the vent opening and flame – I was thinking more of the battery vent diaphram bursting into a sea level pressure sealed battery box and then burning until the internal SS box pressure gets to above 10 psi differential. Although insulated – consider that more than one cell will most likely be warm, an enclosed box of SS with poor heat transfer- battery cell burst impinging on a ceramic “plate” which heats up, etc. possibly hot enough to react with inner aluminum box and then to ??? Would the aluminum then react with gases , etc ?

          More and more to me at least- the SS boom box looks more like a bomb calorimeter- and may not stop the domino effect from one cell, but simply slow it down?

          And where does the excess moisture drain to inside the SS box…?

      • Burst plate release pressure for the Yuasa Types were it is published ( space qualified “single bale” cells ) is ~660 psi.

        • Uwe :
          Burst plate release pressure for the Yuasa Types were it is published ( space qualified “single bale” cells ) is ~660 psi.

          could that be a misprint or conversion error ?

          consider that over 40 to one volume increase to 15 psi not counting temperature effects

          so such a burst would fill the aluminum box and the ss boom box ( depending on free volume ) to at least 20 to 30 psi ( very rough guess ) with hot reactive gas/chemical composition

          and absent a boom box burst diaphram break, increase by a like amount the pressure to burst another cell diaphram.

          Sure makes one wonder ??

      • The nickel burst disk on the GS Yuasa LVP65 cell opens at about 600kPa (87psi), not 660psi.

        • Good Morning Mike,
          thanks for the info. ( where did you find it ?)
          Early in january I had found a document on Yuasa Space Qualified cells “LSE190” ( about 1/3 LVP65, similar stainless welded lid casing. Burst plate spec was 3..3.5MPa ~350..500psi.
          ( a bit sheepishly I have to confess that I can’t remember how I got to 660psi )

      • I remember getting stuck on the 1kA max current draw ( slight dissonance with the Yuasa datasheet, dissipates 2.5kW best case in the battery) but then the burst pressure given didn’t “overwrite” my minds previous assumption.

    • To some extent we are struggling with terms. We might refer to the events in Japan and Boston as fires following a thermal runaway, but Boeing rejects those terms. My question is, does the box design reduce the possibility of those events being initiated again? Leave aside other improvements to the battery that might reduce the chances of thermal runaway and leave aside the argument that better containment obviously benefits the overall safety of the aircraft. Does the box design of itself reduce the possibility of those events being initiated again?

      It seems to me that Boeing claims that starving the battery of oxygen will prevent fires as they define it. But they don’t define what happened as a fire anyway.

      • These are the very types of studies that are criticized in the report I cited above. These diagrams are applicable to bulk heating induced cell failures which, of course, can happen, but are not the most common cause of thermal runaway. The most common cause is manufacturing defects leading to internal shorting, leading to localized heating, leading to electrolyte/anode reaction that generates more localized heat, leading to electrolyte/cathode reaction and even more heat.

        Another aspect that is completely overlooked is the fact that one manufacturer’s lithium cobalt dioxide battery is not the same as another’s. Each manufacturer has their own “secret sauce” in all three components, anode, cathode, and electrolyte. For example, the particle sizes and shapes can be varied over a wide range for both the anode and cathode, and there are binders present in both as well. These factors can greatly affect the reactivity.

        Again, I’m not saying that chemistry does not matter for safety, I’m merely pointing out that in the view of the Fire Protection Research Foundation, cell chemistry is not the characteristic that matters most. They have observed severe thermal runaway events in all types of Li-ion battery chemistries.

      • Well, the “heat to destruction” tests show the energy potential of a thermal runaway for various typical chemistries. Obviously variations will behave slightly different. But I would expect the results over variations to “group”.
        In one of the NTSB docs is an indepth discussion on fault emulation problems and solutions.
        Beyond chemistry the form factor of the cell will further influence behaviour and susceptibility to problems.
        forex the prismatic cells will get significant palpation from pressure changes ( diaphragm type sides ) while cylindrical
        arrangements will see significantly less volume change.

      • You could be right that chemistry is a huge safety issue. I’m only basing what I’m saying on a quick several week long self-taught education in lithium-ion battery chemistry that I have been undertaking. So it is honestly worth asking the question: “What do I know?”

        I can say, however, based an many years of experience, that particle sizes and binder composition can have huge effects on the reactivity and the shock and thermal sensitivity of a given energetic substance.

        Cheers!

  8. Your sequence of words “Spencer in custody and hiding his handcuffs, has links …” is surely intended as a pun, rather than journalistic oversight? Go on, tell me it’s your April Fool’s day prank on the world.

    Regards from early fall down here,

    Gerard

    ________________________________

  9. Don Shuper :
    …possibly hot enough to react with inner aluminum box and then to ??? Would the aluminum then react with gases , etc ?

    Since the decomposition products are mostly CO2 and CO along with fuels such as H2, CH4, C2H4, and C2H6, I do not think reaction with the aluminum battery pack enclosure is a concern at all. Even if O2 were present in sea level quantities, with a solid Al slab the heat from the reaction zone is too easily transferred away to maintain the proper melt layer for a self-sustaining reaction. The melt layer is needed in order to induce cracking the outer oxide layer. It is these cracks that allow the O2 access to fresh Al.

    Aluminum powders are a whole different animal, however, and can be quite reactive with O2 and even H2O. The melt layer is much easier to maintain because the high surface area to volume ratio.

    • great answer Mike .. but the question of the Boston NO fire hot jet described in the NTSB report still remains. That ‘jet” was due to a ‘hole” melted in the aluminum box surroounding the battery cells.

      Given that the ‘ final fix” Boom box still uses the aluminum box- but the cells are supposedly wrapped with ‘ insulating’ tape and possibly ceramic ‘ plates” on the large side of the cells,- it would seem unlikely that a cell burst diaphram would allow hot gases ( NO flame of course ) would impinge on the aluminum box. So perhaps THAT issue goes away.

      But the overall heat problem- from charge- discharge cycles seems to have gotten worse>

      Granted the 787 is designed for long thin routes with many hours between takeoff and landing, giving adequate time for cooldown between uses at takeoff and after landing ( towing, etc)- I would hope that any reasonable test regime would include a significant series of start apu- start engines- takeoff- fly for say one hour- or two hours- land- tow- brake refuel – reload 1/2 to 1 hour – start apu start engines etc.

      No matter how one does it – charge and discharge cycles produce heat in the cells- the current scheme seems to make what little heat transfer arrangements were in place in initial design- use much worse.

      Again- with all the ex-sperts ( used to be drips over 50 feet from desk ) involved, and the phony NON system tests using resistors instead of APU and possible feedback as APU starts and comes up to speed, etc – I still wonder

      Guess we all will have to wait and see what Next move is- already delays in the so called cert flight do NOT give me ( or others? ) a warm (pun? ) feeling.

      • Don,
        After refreshing my memory by reviewing the NTSB Interim Factual Report, the JAL mechanic reported seeing two distinct 3″ long flames emanating from the two connectors on the front of the battery pack case. The hole in the aluminum pack case corresponded to a hole in one of the cell cases that was consistent with arc damage.

        Obviously, the flames occurred early in the incident because none of the ARFF personnel ever reported seeing flames. They did report first seeing a softball sized heat signature on their portable thermal imaging camera. After a shot of Halotron, the ARFF personnel reported that the heat signature was reduced. Granted, the white smoke severely limited visibility, but it is fairly certain that no flames existed after the battery pack was removed from the aircraft.

        Question is, when exactly did the hole happen on the right side of the battery pack case. I would guess that it happened early, but that is just my guess.

        Cheers.

  10. Don Shuper :
    More and more to me at least- the SS boom box looks more like a bomb calorimeter- and may not stop the domino effect from one cell, but simply slow it down?

    I think you are right. The containment box cannot stop a thermal runaway in one cell from propagating to other cells in the battery pack.

    In my opinion, relying only on the containment of a battery thermal runaway failure for safety is not sufficient to meet all of the special conditions. In order to certify this new battery system, Boeing is going to have to show that the likelihood of any one cell within a battery pack suffering excessive pressure and temperature due to any failure event, is less than once in 10 million flight hours. Putting the battery in a thick containment box does not change this, although I have to say it makes me feel more comfortable.

    The way I see it, the purpose of the containment box is threefold:

    1) It starves the battery of oxygen, thus preventing fire within the box, even in the event of a thermal runaway failure. The chemical reactions are limited to the ones between the electrolyte and anode, and between the electrolyte and cathode. These decomposition reactions occur at much lower temperatures than combustion with oxygen, aka fire.

    This distinction is important because Boeing has to show that the likelihood of battery fire is less than once in 1 billion flight hours, 100 times more stringent than for gas or smoke generation.

    2) In the event of any cell thermal decomposition failure, the box and corresponding vent will prevent gasses, smoke, or liquid electrolyte from touching the aircraft interior. The box also provides thermal isolation between the failing cell/cells and the aircraft interior.

    3) The box controls the battery pack environment, mainly pressure and humidity. Boeing stated the Tokyo press conference that moisture was considered one of the possible causes of external shorting that can lead to cell decomposition failure. Thus, properly controlling the humidity can result in reduced failure rates.

    Again, I want to emphasize that if Boeing cannot show that the likelihood of a cell thermal decomposition event is less than once in every 10 million flight hours, it will be back to the drawing board, box or no box. The containment box really just insures that the battery fails safely when it fails.

    • “Again, I want to emphasize that if Boeing cannot show that the likelihood of a cell thermal decomposition event is less than once in every 10 million flight hours, it will be back to the drawing board, box or no box. The containment box really just insures that the battery fails safely when it fails.”

      Nail perfectly hit, thank you!
      I expect the bomb calorimeter box to perform to expectations ( being defined by the FAA special conditions ).
      Still the probability of loosing the battery must be below 1/10e6 hours.

      Will be interesting to see if the ancillary ” just in case” changes ( reducing condensation, narrower limits on over and undercharge, current limits .. ) will work their magic.

      Also if the changes to the swithing panel have an influence on the battery !?

    • “Will be interesting to see if the ancillary ” just in case” changes ( reducing condensation, narrower limits on over and undercharge, current limits .. ) will work their magic.”

      It will indeed be interesting to say the least. If Boeing misses on this, then who knows what will happen!

      I don’t think some of the less obvious changes are ancillary, they are just not emphasized in the press. In my opinion, the number one thing that Boeing is doing to greatly reduce the likelihood of cell decomposition failure is putting tighter controls on the manufacturing process. Apparently, independent auto industry battery experts have claimed that GS Yuasa was not tight enough in their quality standards, and was passing way too many cells that should have been rejected. The number one root cause of thermal runaway failures in consumer Li-ion cells is manufacturing defects. The only caveat to this is that this assumes the battery charging and monitoring circuitry is mature (works properly).

      “Also if the changes to the switching panel have an influence on the battery !?”

      Honestly, this is the aspect that makes me a bit nervous. I know the NTSB cleared the charger and the FDR data does not indicate that the battery was electrically abused. However, I’m not convinced that enough data is actually recorded by the FDR to say definitively that the electrical system functioned correctly with regard to the battery.

      • Ancillary low profile in the way it is presented.

        manufacturing quality: IMHO a reason to use COTS type cells with reasonably longstanding ( and good 😉 pedigree.

        I noticed that the charger now sits in direct vicinity of the battery(box)

        Next noteworthy item imho is that Boeing shows a sudden flurry of activity towards solving this problem cluster ( going beyond the battery ).
        From my noob outside view this looks like a massive stepup in actual interest from the previously leaned back stonewalling tactics: keep it mum, provide enough spares to keep on flying.

      • Yeah, I hear you. Obviously Boeing’s assessment of the technology maturity lead them to believe they could do it. I still think they could make it work, but they better be darn sure they get it right this time.

        It’s hard to know sometimes where and when reliability will intersect with safety. While one can’t overreact every time something fails, one also can’t wait until a full blown crisis occurs before action is taken. I think Boeing was a bit too focused on the rate increase. Doubling the rate in one year was very ambitious. They achieved it, but at what cost?

        I’m not familiar with GS Yuasa, do they not have a strong rep in aerospace? Thales has a pretty strong rep. I’d be curious to know their real opinion on all this.

        • . . . I’m not familiar with GS Yuasa, do they not have a strong rep in aerospace? Thales has a pretty strong rep. I’d be curious to know their real opinion on all this. . ..

          Somewhat ironic

          http://www.nytimes.com/2013/01/26/business/selection-of-the-boeing-787s-battery-maker-raises-questions.html?pagewanted=all&_r=0

          Japan’s Role in Making Batteries for Boeing
          By JAMES B. STEWART
          Published: January 25, 2013 . .

          …. Landing the 787 contract was a huge boost for GS Yuasa and for Japan’s efforts in aviation technology. Though not well known outside of Japan, the company describes itself as Japan’s leading manufacturer of batteries and said it had supplied lithium-ion batteries for over 50 satellites “without anomaly or failure.” Still, this was its first effort in commercial aviation. “I had never heard of Yuasa in the aviation context,” Mr. Aboulafia said. GS Yuasa has since been chosen to supply lithium-ion batteries for the International Space Station. Although Pratt & Whitney nominally chose the company, Boeing is the prime contractor on the space station.

          There were better-known and more experienced suppliers of lithium-ion batteries for aviation uses when GS Yuasa was chosen, including the Saft Group of France, Mr. Aboulafia said. Saft describes itself as “the premier battery supplier to the aviation industry,” and Airbus is using Saft lithium-ion batteries for its new A350 aircraft. ..
          ++++++
          And somewhere else I read that the Li batteries for the space station are man-rated ( which is a much higher standard then airplanes ?? ) and are shipped( flown- launched ) at about 50 percent charge prior to installation which is exterior to space station . .

  11. Don Shuper :
    And where does the excess moisture drain to inside the SS box…?

    I assume it will pool in the bottom of the containment box, below the tray that will hold the battery pack in place.

    I sincerely hope Boeing is at least considering the inclusion of some sort of desiccant pack mounted inside the containment box. This could really help to eliminate any possible humidity related issues.

    • Hmm- IF the box gets hot- then even the moisture trapped in the desiccant turns into ‘ steam” although it should be cooler at the bottom then at the top of the box.

      And then a sudden pressure- depressure at altitude due to an overheated cell(s) and bufrst diaphram would also convert moisture into flash steam on its way out .

      But on ground would be another issue.

      Perhaps someday – Mother B will explain all the issues they considered AND tested with power point presentations vetted by technically qualified people other than the corner office rangers ??

      Nuff for now

  12. Mike Bohnet :<STRONGAgain, I want to emphasize that if Boeing cannot show that the likelihood of a cell thermal decomposition event is less than once in every 10 million flight hours, it will be back to the drawing board, box or no box. The containment box really just insures that the battery fails safely when it fails.

    That seems to be the issue here. Boeing is making large issue of the containment and seems to be addressing the issue of the frequency of these events, which I think is much more important, in a very low key way. I believe they are making it low key because, to put it honestly, I think they are guessing as they do not really know the cause for these events.

    I am also a bit suspicious of this business with the power panels, which Boeing asserts is not related to the battery issue. If it is not related, why are they doing this in the middle of the grounding of their fleet? More importantly, why did they have to flight test this power panel before they performed the “final” certification flight test of their battery containment system?
    I thought their highest priority was getting the grounding lifted. They aren’t going to accomplish that by performing flight tests of the power panels. So why did the power panel testing become such a high priority item?

    Now for the FAA; were they aware of this extra flight test to check out the new power panel? If not, how do they see this issue? I mean, was the FAA informed of these power panel flight tests when Boeing submitted their test plan for certification of the battery? Does this extra flight test not “violate” the grounding of the 787 fleet as it had, by Boeing’s own admission, nothing to do with the battery?

  13. I think Boeing is well aware that even in case they get out of the battery malaise with a black eye, they cannot afford another emergency landing due to an electrical issue. So I guess there is a frantic effort underway to solve all ‘known unknowns’ while the battery issue is providing cover…

  14. Well the question would what battery this flight was using during that flight. I’m sure Boeing isn’t stretching the AD and testing the FAA.

  15. Aero Ninja :
    I am also a bit suspicious of this business with the power panels, which Boeing asserts is not related to the battery issue. If it is not related, why are they doing this in the middle of the grounding of their fleet? More importantly, why did they have to flight test this power panel before they performed the “final” certification flight test of their battery containment system?

    That’s an excellent observation. Indeed there might be a link between those tests on the power panels and the battery. If so it’s all done in secrecy. It would be something neither Boeing nor the FAA would want to make public, for it would put pressure on both organizations.

    Aero Ninja :
    Now for the FAA; were they aware of this extra flight test to check out the new power panel? If not, how do they see this issue? I mean, was the FAA informed of these power panel flight tests when Boeing submitted their test plan for certification of the battery? Does this extra flight test not “violate” the grounding of the 787 fleet as it had, by Boeing’s own admission, nothing to do with the battery?

    When I learned that, the first question I asked myself was how come the FAA authorized this flight which apparently has nothing to do with the battery? My understanding is that each flight must be authorized individually and separately by the FAA until the ban is lifted. No airplane is authorized to do even a single ferry flight to get back to its home base, so why would one be authorized to do a test flight that has nothing to do with the battery fix? It doesn’t jive.

  16. Mike Bohnet :
    This distinction is important because Boeing has to show that the likelihood of battery fire is less than once in 1 billion flight hours, 100 times more stringent than for gas or smoke generation.

    Mike, do you have a reference that says Boeing has to show a likelihood of 1/1billion flight hours? My understanding is that this was not the case the first time around. In other words Boeing did not have to demonstrate 1/billion flight hours when the 787 was initially certified. What would now legally bind them to do so? Or what has changed since the emergency AD was issued? I am just asking, because all this is not clear for me at this stage.

    • Normand,
      I apologize in advance for how long this response is. Please bear with me.

      This issue is a bit confusing to me as well. The NTSB Interim Factual Report section 1.7.3 starting near the bottom of page 31 is what I tried to interpret to arrive at my conclusions.

      http://www.ntsb.gov/investigations/2013/boeing_787/interim_report_B787_3-7-13.pdf

      The section talks about how the functional hazard assessment, performed by Boeing, identified and classified two hazards associated with the 787 lithium-ion battery:
      1) “battery vents smoke/fire”, which was classified as catastrophic
      2) “battery vent and/or smoke (without fire)”, which was classified as hazardous

      Then, based on the functional hazard assessment, Boeing defined three failure requirements for the 787 lithium-ion battery, which can be found in Table 3 on page 32 of the report:
      1) The battery shall have a probability of less than 1 x 10^-7 for gas emission.
      2) The battery shall have a probability of less than 1 x 10^-7 for smoke emission.
      3) The battery shall be designed to prevent spilling a flammable fluid, a hazardous event with occurrence with a probability of less than 1 x 10^-9

      I am assuming that the FAA originally approved these Boeing defined failure requirements. I’m also assuming that reduced probability of requirement 3 is because spilling flammable fluid is considered tantamount to allowing a fire to start outside the battery pack enclosure (blue box that we have all seen)

      I think my assumptions are supported by two footnotes taken from the bottom of page 32:

      46 The harmonized requirements for 14 CFR Part 25.1309 define a catastrophic event as one that normally involves a hull loss with multiple fatalities and is assigned an allowable qualitative probability of being extremely improbable and an average quantitative probability of less than 1 x 10^-9 per flight hour.

      47 The harmonized requirements for 14 CFR Part 25.1309 define a hazardous event as one that normally involves a large reduction in functional capability or safety margins of the airplane with serious or fatal injury to a small number of passengers or cabin crew along with physical distress or excessive workload impairing the ability of the flight crew and is assigned an allowable qualitative probability of being extremely remote and an average quantitative probability of less than 1 x 10^-7 per flight hour.

      Section 1.7.3 of the report also provides some insight into Boeing’s original stance on how to go about addressing the hazards associated with the 787 lithium-ion batteries. The following is a long (I apologize) quote from the report:

      “Boeing’s 787-8 electrical power system safety assessment also included an analysis of lithium-ion battery cell failure modes. This analysis determined that overcharging was the only known failure mode that could result in cell venting with fire. As a result, Boeing established additional design requirements to ensure that the likelihood of occurrence of an overcharge event was extremely improbable.48 Boeing further determined that cell venting without fire could be initiated by several different failure modes, including external overheating, external short circuiting of appropriate impedance, internal short circuiting, recharging a battery that has been overdischarged, high-rate charging, or charging at cold temperatures. To evaluate the effect of cell venting resulting from an internal short circuit, Boeing performed testing that involved puncturing a cell with a nail to induce an internal short circuit. This test resulted in cell venting with smoke but no fire. In addition, to assess the likelihood of occurrence of cell venting, Boeing acquired information from other companies about their experience using similar lithium-ion battery cells. On the basis of this information, Boeing assessed that the likelihood of occurrence of cell venting would be about one in 10 million flight hours.”

      With the benefit of hindsight, I think that Boeing’s safety assessment was based on insufficient testing and was probably too heavily influenced by the results from one particular test, the nail puncture test. This caused them to focus on overcharge protection and perhaps not focus enough on other protection aspects. My guess here is supported by the third footnote on the bottom of page 32:

      48 The risk of fire was addressed through overcharge protections. For example, Boeing required that “the battery monitoring unit when combined with the overall battery protection subsystem shall prevent undetected over-charge (over-voltage) a catastrophic event with a probability of occurrence of less than 1 x 10^-9.”

      Obviously, Boeing overlooked the possibility of the flammable gasses igniting into a torch-like flame when passing through a failed main connector seal where electrical arcing could act as a competent ignition source.

  17. Reuters:

    All Nippon Airways, the biggest customer for Boeing Co’s grounded 787 Dreamliner, will put its pilots through training to resume flights in June.

    The Japanese carrier, known as ANA, is also likely to use the Dreamliner initially for cargo flights once the new battery system is installed, to reassure the public about safety before restarting passenger flights.

    Anticipating regulatory clearance, ANA will put its roughly 200 Dreamliner pilots through flight resumption simulator training so that they will be ready to fly the jets again in June.

    “It’s not that we have decided to resume flights, but rather that we have not decided on cancelling flights,” spokesman Ryosei Nomura told Reuters.

    “More than half of the testing is complete with the remaining ground and flight tests set to occur within the next several days,” Marc Birtel, a spokesman for Boeing, told Reuters by email.

    It is still unclear how long the FAA will take to approve Boeing’s battery fix. After the FAA’s certification, Japan’s Civil Aviation Bureau is likely to certify the fix around the same time.

    A few dozen Boeing engineers are already in Japan so that they can start work on the battery fix as soon as approval is received, the sources said. ANA estimates it may take a month to install the new battery system for its 787 fleet.

    “By making a track record, the company wants to provide a sense of security to passengers. What it is concerned about is whether passengers will fly the Dreamliner like they did before,” the source said.

    “We will probably conduct test flights before carrying passengers onboard. Some, though not all, of the pilots will have to fly in order to keep their pilot’s qualifications,” Nomura said. He declined to comment on details of the possible test flights.

    http://www.reuters.com/article/2013/04/02/us-boeing-dreamliner-ana-idUSBRE93106U20130402

    • What common property joins some /majority/?all? of the suppliers?

      Maybe bringing their own money or a lower offer because they expected
      a leg up in the market from the Dreamliner project.?

  18. Aviation Week:

    Boeing today conducted a flight test on LN86, the aircraft being used for 787 tests, which the company says were unrelated to a fix for the aircraft’s lithium-ion batteries.

    The aircraft, also known by its Boeing production designation as ZA272, originally was expected to make a test flight on March 30, but for unknown reasons this was cancelled.

    The interim testing appears to be focused on verifying the functionality of specific systems that could play a key role in the upcoming battery demonstration flight, as well as other electrical systems unrelated to the battery modification. A revised power panel design was expected to be tested as part of the canceled March 30 flight from Paine Field at Boeing’s Everett, Wash., production facility.

    The power panel has been the source of early in-service issues with the 787.

    Power panel malfunctions also were behind a string of events in December.

    Issues with the power panel where soon overshadowed by the battery failures in January.

    Commenting on the power panel events in January, Boeing 787 VP and Chief Project Engineer Mike Sinnett said, “The event on United Airlines was a surprise to us. We’d only seen one thing that looked like that before, then we saw three more.”

    Other systems understood to be targeted for testing include back-up standby instrument displays and the crew information system.

    http://www.aviationweek.com/Article.aspx?id=/article-xml/awx_04_01_2013_p0-564734.xml#

  19. Mike Bohnet :
    46 The harmonized requirements for 14 CFR Part 25.1309 define a catastrophic event as one that normally involves a hull loss with multiple fatalities and is assigned an allowable qualitative probability of being extremely improbable and an average quantitative probability of less than 1 x 10^-9 per flight hour.

    47 The harmonized requirements for 14 CFR Part 25.1309 define a hazardous event as one that normally involves a large reduction in functional capability or safety margins of the airplane with serious or fatal injury to a small number of passengers or cabin crew along with physical distress or excessive workload impairing the ability of the flight crew and is assigned an allowable qualitative probability of being extremely remote and an average quantitative probability of less than 1 x 10^-7 per flight hour.

    I assume that when the FAA will have certified the battery fix we will find out what standards exactly they have used.

    What has always been clear to me is that the chance of a battery fire must be reduced to 1/1 billion. I would not accept a higher probability, even if the battery is contained in a box.

    Except on a temporary basis. Because with the modifications that Boeing made to the original battery design an event would be less likely to happen the first few months.

    I would have more concerns if the Boeing solution was accepted as a permanent fix with a probability of less than 1/1billion. But I am afraid that is what is going to happen if the battery container is part of the equation. It could be used to lower the standard for a bare battery.

    1- Is the battery container something that would be nice to have in case something extremely improbable (1/1 billion) happen?

    2- Or is it something absolutely essential to protect the aircraft and its occupants in case something extremely remote (1/10,000,000) happen?

    My impression is that the FAA will go for no 2 to get the aircraft back in the air. But the NTSB would probably recommend no 1 as a permanent fix.

    In my book no 2 would be an acceptable compromise with reduced ETOP qualifications. And ETOP 360 should be sought only with no 1.

  20. Just read an interesting comment on PPRune on their 787 Grounding blog.
    A Seattle based contributor states that electromagnetic compatablity and portable computers are a 787issue.
    Don’t recall this coming up before.

Leave a Reply

Your email address will not be published. Required fields are marked *