Bjorn’s Corner: The challenges of airliner development. Part 15. The Certification application.

By Bjorn Fehrm, Henry Tam and Andrew Telesca.

August 6, 2021, ©. Leeham News: Last week, we went through the OTA, Authorization To Offer milestone.

Now we look at another important milestone in the Pre-Launch phase, the design Certification application. It involves some tricky differences in rules and procedures between the primary Certification authorities.

Figure 1. The Program Plan for our Green 19 seater. Source: Leeham Co. Click to see in full scale.

The Certification application

The FAA, Transport Canada, and EASA use different principles on how to arrive at a Type Certificate (TC) for the aircraft design.

The FAA system is a direct oversight system. FAA experts do the design oversight of a Type Certificate application until it reaches a Type Certificate. Alternatively, the applicant may hire FAA designees to do the FAA direct oversight, so-called Designated Engineering Representatives (DER) and Designated Airworthiness Representatives (DAR).

FAA states in 14 CFR Part 21.13: “Any interested person may apply for a Type Certificate.” It will then help the applicant to either hire the competence needed via DERs and DARs or assist in building up an organization that can get the project to a Type Certificate.

It’s a softer start system than the EASA system, which is an indirect system. It asks the project to build the necessary competence for self-certification before the application is approved. Then EASA follows the project with an indirect oversight, doing periodic inspections and sample tests of documents.

To get to where an organization can do its oversight, EASA demands you reach a Design Organisation Authorization (DOA) level before the application is accepted.

It has a comprehensive list of requirements for how to reach a Design Organisation Authorization level. You need to establish:

  • A Certification organization with competent personnel
  • The internal Procedures for this organization
  • A Design Organization Handbook that everyone shall follow
  • A Quality system that can verify that designs, including your suppliers, are in line with EASA requirements

Transport Canada is somewhere in between these two.

TCCA 521.26 says: “An applicant for a type certificate in respect of an aeronautical product shall have, or have access to, the technical capability to conduct the design analyses and tests required to demonstrate the conformity of the aeronautical product with its certification basis.”

If it deems the applicant doesn’t have this competence, it can refuse the application.

The end result is the same

In the end, all three organizations certify to very similar requirements, and the hurdles to get a Type Certificate are the same.

It’s how you get there that differs. The FAA method allows an upstart project to gradually build its competence, whereas EASA requires a minimum level before a Certification application is approved.

For an investor in an aviation upstart like ours, the information we have been granted an FAA application doesn’t have the same value as an EASA approval. It would mean we have an organization that can hold a DOA with all it entails regarding competence, procedures, and documentation.

Processes and Tools

The certification process requires you to issue and keep track of thousands of documents and files. It ranges from CAD/CAM/PDM files, Requirements, Specifications, Reports, Meeting protocols, Communication logs, Certification documents, etc.

All these files must be tracked with versions; which are the valid versions at a particular time, and which are checked out of the system for revision? Where are these in the revision flow, when do they check in again, and will they then become the valid version?

For this, the established OEMs have hundreds of IT applications, often homegrown. There are workflow applications available that support such flows. Some are from established CAD/CAM/PDM vendors, others from document management and workflow companies.

Before we go to the Post-Launch phase, such applications must be up and running, and people must be proficient at using them correctly.

We are soon transitioning from the conceptual design phase where we work with fewer than 100 engineers to the detailed design phase, which explodes our work packages to where we need perhaps a thousand engineers. The amount of documents and files increases accordingly, and we must be able to handle it.

There are large airplane projects that have not managed this process well, with dire consequences. The cable crisis of the Airbus A380 was because different organizations didn’t have the correct files to work from and one of the many reasons the Boeing 787 got into design problems was because the project lost track of where the design was at each point in time.

16 Comments on “Bjorn’s Corner: The challenges of airliner development. Part 15. The Certification application.

  1. Terrific concretization of the phase barriers of entry.

    With respect to management, tracking and merging of digital design iterations, Linus Torvalds developed a system called git which is beyond amazing for this purpose. A no-cost insurance against emotional and economic calamity.

  2. This (certification) seems a daunting organisational task, enough to challenge even the largest, most experienced OEMs as the author suggested. There exists however, an AI protocol that properly trained and configured would render this task trivial. It is called Generative Pre-trained Transformer 3 (GPT-3). It is a transformer-based deep learning neural network software architecture that is capable of processing, mining, organising, connecting, contrasting, understanding and generating answers to questions in human language such that no human can determine if it is a machine providing answers to your inquiry.

    Moreover, such a system could combine all known certification standards and ensure your design is compliant in each from the get-go.

    You may be sceptical in which case let me test your credulity to its limit; it is already possible to say to a trained GPT-3 system in plain English “design a 19 seat turbo-prop aeroplane that complies with all current certification standards and which exceeds the performance of all known designs”. You need not elaborate further but, within seconds you will have an output of technical drawings of such an aircraft which will be fully complaint and which you can build and fly.

    Ask it to “print flying manuals in all languages”. It will do so.

    • Fastship — GPT-3 will, if instructed, provide “technical drawings of such an aircraft which will be fully complaint [sic] and which you can build and fly.”
      Yes, yes, yes – but, if given the Max job, would it have been trained to offer MCAS as part of the ‘solution’ or generated it automatically? Who gets to check all the calcs, and will regulators insist on a human signature on those drawings? Forgive me, but in my case any intelligence actually is artificial.

      • Who got to check all the drawings, calcs and compliance’s for the Max and whose signature signed them off and what has been the consequences for those individuals?

        If the AI in your future car swerves to avoid a child in the road but in doing so hits and kills a pedestrian in a wheel chair by what mechanism did it determine which of them was the better to save? Was it a moral choice? Did it commit murder or manslaughter? Was the writer of the code culpable or society whose collective experience the AI sampled?

        GPT-3 AI can prove the presence of knowledge but not the absence. The imminent GPT-4 will be orders of magnitude more powerful, its sampling greater and will perhaps give the answer to your prompt “I don’t know”. If it does then we truly will be in a new age.

        • Fastship — “If the AI in your future car swerves to avoid a child in the road but in doing so hits and kills a pedestrian in a wheel chair by what mechanism did it determine which of them was the better to save?” To anticipate such hypotheses, it is to be hoped that it will have been programed with the standard principle of “brake, but do not swerve.”

    • This is an absolutely comically ludicrous overstatement of GPT-3’s capabilities. While it is surely an impressive piece of technology, it is miles away from being able to design a toaster, let alone an aircraft.

  3. Bjorn — “There are large airplane projects that have not managed this [certification] process with dire consequences.” You cite, but briefly, A380 and 787; can you put a little more meat on those bones and, perhaps, also cite other examples, please?

  4. Pundit:

    Rather than a reply to your question, I will put in what I read while tracking the 787 (via Av Week, other sources) as I was intensely interested.

    My take was it had dual failure modes, each causing its cascade issues.

    1. A technically new approach (or rarely used) for all CRFP (within the limits of course) as well as spun fuselage. More than challenging enough by itself though amazingly it went well overall.

    2. The other one was the scattered organization of a outsourced contracts and the suppliers doing the engineering.
    It could work, but you would need a centralized control and teams to monitor and manage it. Boeing had none of that in place to start out with.

    A subset was the fastener problems, it was the wild wild west and some suppliers corralled what they needed and others did not. But the contracts were all based on a given supplier providing its part/s and not what the impact on the whole project.
    Eventually Boeing created something like 20 teams that had engineers, electrical, logistics experts to visit suppliers, assess where they were at and work at solutions. They found that they had to have a team as they never knew what aspect or aspects were involved in issues and often needed several disciplines to work a problem.

    The Battery problem was the one that got a good public documentation but there were other areas that did not, but its a good example.

    Thales if I remember right got the battery overall. Why SAFT (French battery mfg that is the gold standard) was not part of the report. My view a couple of paragraphs down

    In turn, Thales put something called Securi Plane (Burgler Alarms for Business Jets!) to build the charging board (they never had a thing to do with batteries and built a division to do the work – they had some odd link back to Thales). They also burned down a building in a mad scientist testing situation.

    The monitor board was build by a separate Japanese company.

    The Batteries were farmed out to Yuasa (probably part of the Japanese work share). While Yuasa does make Li Ion batteries, they did not make that type (deep draw startup).

    When the factory was checked for quality as part of the fix, they found that rather than a near clean room environment a Li Ion battery requires the place was filthy.

    Equally the internal forms were being made by hand via a hammer and a support (zero quality control on what you get out of that process, it works for a black smith and horseshoes but not Li Ion batteries)

    From memory they were passing 80-90% of the batteries as good (other Li Ion mfg get 60%)

    Boeing or one of the contractors determined the way to test a Li Ion battery was to drive a spike into it. Based on NO science, just someone opinion.

    When the RTC got done, they had a full on quality control process in place with multiple tests and scans of the batteries and the more known beef up of the box, outflow, separation and meltdown test to ensure that there was enough separation so a failed cell would not cascade into burning up other cells.

    The other fairly well documented on was the total electrical system failure.

    The belief is that a wrench was left in a panel and went across the mains.

    The protection should have disconnected that panel and left the other elecrial system intact, instead it took down the whole primary electrical system (battery backup is the third I believe, I think only two systems but 3 possible feeds from APU and each engine)

    Upshot was it took the whole electrical system and they landed on RAT power and battery backup which did work.

    A total short should be a standard test, no reports publicly as to what the tests were and why they allowed the short to propagate, just that it did and they then corrected it (or at least that known fault was corrected)

    Compound that by the entirety of the 787 systems, the far more elecric nature of it and the flip is its amazing it worked at all let alone successfully once they got the issues resolved (so a lot went right).

    While there were some technical failures (to be expected) vast majority of it was a huge management failure thinking there was a free lunch in out source and there was not.

    They could not get Chance Vought to correct its fuselage and assembly issue in Charleston and bought them out and that in turn lead to a whole new assembly plant when management decided to de-unionize (ironically it was the Unions in Everett that figured out how to make the assembly work)

    Charleston assembly added billions to the cost and as far as I know all the assembly issues came out of Charleston (Spirit had two shim issues on the nose they build) – how much could they have saved just working with the Unions?

    And now Everett is working to just 767 builds (well that and fix it work on the Charleston 787s which may be a growth industry for Everett)

    • TW said:
      “The other one was the scattered organization of a outsourced contracts and the suppliers doing the engineering.”

      It takes common values, whereas for example Smiths Yakima/Cheltenham were working against Boeing in the 787 program – cutting off their nose to spite their face by delaying cash flow from production deliveries. Very troubled operation.
      (Some divisions of Smiths were far better, variation in a bureaucracy is typical including within Boeing.)
      And some Boeing executives were sneaky.

      And of course it takes sharp people, some on the project in Boeing and some suppliers should have been dumped.

      Beware that organizations change over time, one supplier with a good history was not proficient by the time of the 787 program.

      • Keith — “And some Boeing executives were sneaky…” Surely not?

    • The ex GE top managers at Boeing thought they could copy the GE Risk and Revenue system for the 787. As they were finance people they did not see the difference of experienced and well funded suppliers like MTU or IHI designing and making parts since the J-79 days to a rockie supplier as they looked the same in the spreadsheets. They then too late experienced the difference but it was too late to get in reverse gear and they could see time and money flowing away while trying to get all problems fixed.

    • @Transworld @claes, In my present industry we carry out a FAT “Factory Acceptance Test”. You have to develop a checklist and a series of tests and physically visit that factory and witness the test.

      I’ve found out the hard way that you can not trust people. (Some people you can but you can’t identify them immediately). One must physically visit the factory.

      Eventually you can leave it to the suppliers factory in some cases once they delivered quality product that has proven itself.

      Prior to that there needs to be an overall quality plan and the suppliers factories need to be inspected when its a case of a first purchase of a critical item so that the ability of the supplier can be understood.

      I feel sorry a little for Boeing on the battery issue probably had 1000s of hours of flight test campaign with no battery issues and in this instance in that Boeing, not being battery experts, trusted Thales->SAFT->YUSA but in reality there should have been an assessment on such an item that raised some red flags.

      I also note that when I have something made in Australia and if I’ve made a mistake in part of the design or provided ambiguous information I will receive a phone call high lighting the issue well in advance. If made overseas it hardly ever happens. The distance is too great, cultural issues exist and people can hide.

      I note Thales also made the dodgy AF447 pitot static probes which makes me question them They had one job. How did those things get certified? That’s likely to be a story as infuriating as the MAX MCAS if anyone delves.

      • William:

        A correction, SAFT was no involved, just that they should have been as they are the Gold Standard for Aviation battery systems (not sure where they started, my brother worked with them on NiCads on aircart and I had one SAFT Nicad backup for a fire alarm system that was wonderful – add a bit of water once in a while and never an issue)

        I started out as a labor but got to see construction contractors at their often worst. So early on got lessons in watching anyone closely until you got some sense of what their quality was.

        One contract to haul out overburden got a change to the main contracts that allowed our Foreman to inspect the equipment they would use and reject it if it was sub par (and oh yes it was, what a mess).

        Boeing problem the battery was while they did not know, they self certified it. When all was said and done something called the RTC (Radio Technical Committee) was convened to come up with a Li Ion standard.

        Boeing’s standard (lack) were laughable and RTC were what you would hope for in an Aircraft system. I shudder to think what would have happened if the battery melt downs took place in the air.

        Equally, people keep harping on Boeing and the ADs. I cam across one on an A310 issue with throttles issue that indirectly lead to a crash and loss of life. Airbus has futzed with the issue for years and not fixed it.

        Like the 787 battery, it was a pure roll of the dice and the A310 crash was hidden by a pilot issue (and the PNF probably had a heart attack so the FP had multiple issue)

        No one is immune though Boeing attitude at least until recently was many steps beyond the ebb and flow of safety.

        • Transworld – “I cam[e] across one [AD?] on an A310 issue with throttles issue that indirectly lead to a crash and loss of life. Airbus has futzed with the issue for years and not fixed it.” If this was the Tarom aircraft in 1995, was not the ‘issue’ covered by a DGAC AD?

  5. Thank you for sharing.
    Concerning the Design Organisation Approval in the EASA system, my understanding is that the DOA is not necessary at the time of application. It is necessary at the time of obtaining the TC. The reason is that the work to obtain the TC is a validation of your DOA.

Leave a Reply

Your email address will not be published.