Bjorn’s Corner: The challenges of airliner development. Part 18. Certification work after program launch.

By Bjorn Fehrm, Henry Tam, and Andrew Telesca.

August 27, 2021, ©. Leeham News: Last week, we started looking at our work during the Detailed Design phase after Product Launch. We outlined the exacting work needed to design all parts of the aircraft and how we must keep everything in sync.

An essential part of keeping everything in sync is the Certification Compliance Planning we do in this phase, Figure 1.

Figure 1. The program plan for our project. Source: Leeham Co.

Certification Compliance Planning

As our engineers are heads down with Detailed Design, our Certification team is working with the regulator to align the design with the certification requirements in what is typically called “Certification Planning.” This work results in an accepted certification plan between our certification office and us.

Depending on the scale of the program, this could be captured in a single document, or for a large airplane like the 787 could require over 100 individual plans to certify the different systems/functions that must integrate into a single plan.

What is in these plans? It’s a mapping of each certificate regulation and how we can have the airplane, its systems, or individual components demonstrate they comply with the regulation.

It can be straightforward or very involved. Do I need to do environmental qualification testing for temperature, vibration, or other conditions to be part of this demonstration? Then it needs to be in the plan.

Do I plan to create flight deck simulations to test pilot responses to utilize their reactions as part of my safety case demonstration of compliance? Then these simulations need to be described in detail in the plan.

The software problem

Am I developing software to monitor or control different systems? Then not only does the software development need to be in the plan, but I need to establish how safety-critical the software functions are (design assurance level).

I need to write an additional set of plans for managing the software development and have the development plan audited by the authority before I even start the functional specification writing, followed by coding.

Complex software generation and integration are one of the most closely regulated and hardest aspects of modern aircraft development — just see the leaked May FAA letter to Boeing on the 777-9’s Type Inspection Authorization (TIA). Insufficient maturity and the difficulty of change control for the 777X’s central software architecture called “Common Core System” (CCS) were cited as the basis to delay the entire program’s certification flight testing.

Preventing this type of delay starts during this planning phase and continues throughout development and testing.

What we need to do to get Certification

When we sum it all together, it’s not uncommon for the project (“the applicant”) to have to generate tens of thousands of “demonstrations of compliance” evidence items that are gathered in thousands of documents to receive a Type Certificate.

These compliance documents map verification data for a component, system, or part of the product to a specific regulation. Our generated compliance evidence items during the project then verify we are compliant.

Even our smaller project under Part 23 can expect to make thousands of these compliance evidence pieces gathered in hundreds of documents, and all of them must be planned out during this phase, and the approach we take in each case agreed with our regulator.

To make our project successful, we will need to focus on three things during this effort:

Thoroughness:

It is essential during this phase that we identify all activities, especially testing activities, we need to do to reach Type Certificate. Late discoveries can come with large program impacts from unbooked test facilities, insufficient purchasing of test articles, or undesigned test equipment.

One of the famous examples of missed testing came from the space side of the industry when NASA failed to conduct sufficient lens alignment testing before the launch of the Hubble telescope, resulting in a $250 million in orbit repair mission.

Efficiency:

Testing is expensive. Finding efficient methods to demonstrate compliance is a big cost lever. Maybe a supplier has previously approved data on their parts we can show is valid and applicable for our product without new testing.

Maybe we can select a less ideal material for our design, but it is eligible for an “equivalency campaign.” In this, we establish the design values (allowed stress levels, margins, fatigue assumptions…) based on the previous acceptance data of this material, saving us nine months of testing and allowing us to have access to accepted design values earlier in the program.

Prudence:

It is also crucial during this phase to consider how we manage cost vs. schedule risk. The more substantiation activities we propose to do, the less risk we miss something. The more aggressive we are with the use of simulation, analysis, etc., instead of testing, the more time and money we can save, but all must be agreed upon by the regulator.

And the more novel our approach to saving time and money with, e.g., simulation instead of actual tests, the longer it can take to get an agreement from the regulator. It means we have a program time-use uncertainty for longer.

It’s critical to program cost control

The certification activities we list can account for as much as one-third of our development costs, so we must find the right balance on all these tasks to get our project to stay on schedule and within our budget.

23 Comments on “Bjorn’s Corner: The challenges of airliner development. Part 18. Certification work after program launch.

  1. You need to keep track of changes and how they effect the certification testing and reports. As changes comes late in the test program it is essential to have time and money spared to redo required tests to reflect the new configuration and maybe one more time as testing might show another round of changes are needed. For software heavy system just look at the MTU FADEC for the TP400 engine and prop that needed to be certified according EASA rules and not just military ones causing delays, similarly Cobham got stuck when their KC-46A boom needed civil FAA certification.

    • Well done series sir….. When Certification testing is planned, advanced aircraft are also at risk of “outrunning the regulators'”. Electric aircraft run into this often as regulations based on recip and turbine aircraft engine design and construction “interfere” with smooth integration of electric motors. This usually occurs when decades old rules must be rewritten as the technology evolves beyond the rule creators vision… This adds risk to the program as your proposed design may be viewed differently by the regulators than how you anticipated…. The 787 had a lot of this going on….

      • @Scott: In the future please add something to your screen name to differentiate from me, so as not to confuse who’s commenting. Thanks very much.

        Hamilton

      • Scott:

        Good points. I think the 787 biggest basis of failure was the battery and the self certified test process (I would go so far as to call it a joke process).

        It seems to me that the Radio Technical Committee (RTC) that was called in to correct the 787 battery debacle would be where electric aircraft should start (though I think the electric fad is going to fade away in a couple years)

        Sans the 787 panel failure, the rest of the more electric system worked, little reporting on any issues.

        The electrical panel failure has me puzzled as fault isolation has been around for along time. A phase to phase short is a known issue, be it wire to wire or a conductive object between them (don’t ask me how I know!)

        That said I have also been involved in failures that no one foresaw and we were the test subjects.

          • IMHO you’ll first see more orthogonal designs.
            Airbus rather clean 2H 2E systems layout.

          • jbeeko:

            I was referring to the battery fueled/power machines aka local air taxis stuff popping up everywhere.

            Fully agreed 787 is the wave of the future though Airbus elected to stay with bleed air on the A350.

            I was surprised Boeing did not have a vastly lighter 787 as the power routing was much easier with electric. Maybe very conservative with a new material.

        • “electrical panel failure”

          rain in the plane, condensation. aggravated by
          the requirement for electronics cooling afai could research.

          What Boeing seems to have managed rather well in the end is an embargo on bad news re the 787.

          • Uwe:

            You have not a clue what you are talking about.

            The incident was well documented.

            The hysteria of any time a 787 had a hiccup is well documented as well, that is the press norm for any aircraft that has an issue. We see the same thing on the MAX that has equipment that is also NG and it makes headlines.

            Boeing in fact did not build that power system. I believer Rockwell Collins did.

            Equally Airbus did not acknowledge the major changes on the A350 after the first 17, but they happened.

          • The A350 was never grounded.
            The B787 has been subjected to two groundings…and it’s currently in a months-long delivery hiatus.

            Apples and oranges (currently lemons).

    • TP400 FADEC software was a BAE Systems effort.
      ( together with the Spanish Hispano Suiza. )
      They completely botched the qualification of the SW toolchain.

      I would not put it beyond BAE to have done a bit of industrial sabotage in that case.

      • But it was under MTU contract and responsability?
        The TP400 engine was a M88 core (initially a F-404 copy) with the regular EU split of modules between partner nations causing a pretty expensive engine. One could think the A400M should move to a MkII or MkIII as the MkI’s are so modified to meet most promised requirements that they can be MkII std. It was designed in the age of the A380/NH-90 design, today they could improve the design with more payload, more advanced props or wait for the RISE engine a more efficient wing and updated systems. The US are normally better in swallowing their pride on projects not meeting initial brochure promises and pour money over new versions that step by step come closer to what the services want.

        • Is this another one of these “others can only copy US” 🙂

          Amusing.

          • Just a examples of original and succesful French aerospace companies are Turbomeca, Safran Nacelle, Ariane Group, Airbus with its helicopters and Dassault. Safran has had its issues after the name change from Snecma when designing new engines by themselves.

          • I was questioning your statements of derivation.

            The M88 is afaics not a copy of the F404.
            Though the target airframe first flew with F404
            as the M88-2 were not ready yet.

            I see posters here often spouting a caricature of reality
            to make it fit their assumption of superiority.

          • @Uwe. I once was quoted a GE engineer saying “they even copied my misstakes”… but I agree it evolved from the initial version with a new fan section and on.

  2. On the subject of new aircraft certification:
    FG: “China Eastern Airlines expects to take delivery of the world’s first Comac C919 narrowbody in the second half of the year, as it removes the Boeing 737 Max from its delivery forecast through 2023.”

    https://www.flightglobal.com/fleets/china-eastern-expects-first-c919-delivered-in-second-half-2021/145266.article

    Here’s an equivalent article, without a paywall:
    https://gettravelfares.com/china-eastern-expects-first-c919-delivered-in-second-half-2021/

    Interesting:
    “In the same fleet forecast, the SkyTeam carrier removed any delivery forecast for the 737 Max, which remains grounded in China following two fatal crashes in 2018 and 2019. ”
    “Compatriot Air China, which also has 737 Max aircraft on order, did not indicate when it expects deliveries to resume, casting uncertainty on the fate of the popular narrowbody.”

  3. And regarding another plane that has yet to be certified, Ryanair brazenly continues to dangle the carrot outside the local B-Mart store:

    “BRUSSELS (Reuters) – Ryanair is “very keen” to make a significant order for Boeing’s 737 MAX 10 aircraft, but there has been no agreement on pricing so far in ongoing talks, Group Chief Executive Michael O’Leary said on Tuesday.

    “We’re certainly very keen to place a MAX 10 order but only when the timing and the pricing is right,” O’Leary told Reuters ahead of a press briefing in Brussels. “Boeing needs an order.”

    https://uk.finance.yahoo.com/news/ryanair-very-keen-boeing-max-075631586.html

    • does this go beyond a paper derate?
      ( i.e. any changes to the airframe? Looks like
      it might lose the center tank (plumbing) just like early A333 )

Leave a Reply

Your email address will not be published.