August 27, 2021, ©. Leeham News: Last week, we started looking at our work during the Detailed Design phase after Product Launch. We outlined the exacting work needed to design all parts of the aircraft and how we must keep everything in sync.
An essential part of keeping everything in sync is the Certification Compliance Planning we do in this phase, Figure 1.
As our engineers are heads down with Detailed Design, our Certification team is working with the regulator to align the design with the certification requirements in what is typically called “Certification Planning.” This work results in an accepted certification plan between our certification office and us.
Depending on the scale of the program, this could be captured in a single document, or for a large airplane like the 787 could require over 100 individual plans to certify the different systems/functions that must integrate into a single plan.
What is in these plans? It’s a mapping of each certificate regulation and how we can have the airplane, its systems, or individual components demonstrate they comply with the regulation.
It can be straightforward or very involved. Do I need to do environmental qualification testing for temperature, vibration, or other conditions to be part of this demonstration? Then it needs to be in the plan.
Do I plan to create flight deck simulations to test pilot responses to utilize their reactions as part of my safety case demonstration of compliance? Then these simulations need to be described in detail in the plan.
Am I developing software to monitor or control different systems? Then not only does the software development need to be in the plan, but I need to establish how safety-critical the software functions are (design assurance level).
I need to write an additional set of plans for managing the software development and have the development plan audited by the authority before I even start the functional specification writing, followed by coding.
Complex software generation and integration are one of the most closely regulated and hardest aspects of modern aircraft development — just see the leaked May FAA letter to Boeing on the 777-9’s Type Inspection Authorization (TIA). Insufficient maturity and the difficulty of change control for the 777X’s central software architecture called “Common Core System” (CCS) were cited as the basis to delay the entire program’s certification flight testing.
Preventing this type of delay starts during this planning phase and continues throughout development and testing.
When we sum it all together, it’s not uncommon for the project (“the applicant”) to have to generate tens of thousands of “demonstrations of compliance” evidence items that are gathered in thousands of documents to receive a Type Certificate.
These compliance documents map verification data for a component, system, or part of the product to a specific regulation. Our generated compliance evidence items during the project then verify we are compliant.
Even our smaller project under Part 23 can expect to make thousands of these compliance evidence pieces gathered in hundreds of documents, and all of them must be planned out during this phase, and the approach we take in each case agreed with our regulator.
To make our project successful, we will need to focus on three things during this effort:
It is essential during this phase that we identify all activities, especially testing activities, we need to do to reach Type Certificate. Late discoveries can come with large program impacts from unbooked test facilities, insufficient purchasing of test articles, or undesigned test equipment.
One of the famous examples of missed testing came from the space side of the industry when NASA failed to conduct sufficient lens alignment testing before the launch of the Hubble telescope, resulting in a $250 million in orbit repair mission.
Testing is expensive. Finding efficient methods to demonstrate compliance is a big cost lever. Maybe a supplier has previously approved data on their parts we can show is valid and applicable for our product without new testing.
Maybe we can select a less ideal material for our design, but it is eligible for an “equivalency campaign.” In this, we establish the design values (allowed stress levels, margins, fatigue assumptions…) based on the previous acceptance data of this material, saving us nine months of testing and allowing us to have access to accepted design values earlier in the program.
It is also crucial during this phase to consider how we manage cost vs. schedule risk. The more substantiation activities we propose to do, the less risk we miss something. The more aggressive we are with the use of simulation, analysis, etc., instead of testing, the more time and money we can save, but all must be agreed upon by the regulator.
And the more novel our approach to saving time and money with, e.g., simulation instead of actual tests, the longer it can take to get an agreement from the regulator. It means we have a program time-use uncertainty for longer.
The certification activities we list can account for as much as one-third of our development costs, so we must find the right balance on all these tasks to get our project to stay on schedule and within our budget.