Leeham News and Analysis
There's more to real news than a news release.
Leeham News and Analysis
- Boeing unlikely to meet FAA’s 90-day deadline for new safety program April 18, 2024
- Focus on quality not slowing innovation, says GKN April 18, 2024
- Boeing defends 787, 777 against whistleblower charges April 17, 2024
- Dissecting Boeing CEO’s statement next new airplane will cost $50bn April 15, 2024
- Bjorn’s Corner: New engine development. Part 3. Propulsive efficiency April 12, 2024
Bjorn’s Corner: The challenges of airliner development. Part 23. Certification Flight tests
By Bjorn Fehrm, Henry Tam, and Andrew Telesca.
October 1, 2021, ©. Leeham News: Last week, we described how we produced our Flight Test Articles, FTAs, and how the company flight tests are made.
Now we have finished the Company flight tests and made any modifications required to the FTAs so we can apply for Certification Flight tests with the regulator.
Figure 1. VMU flight test with the Airbus A350. Source: Airbus video.
Certification Flight tests application
When we apply for the start of Certification Flight tests, we typically must have flown through all the test points that we will have to fly with the regulator on board, and passed the regulatory requirements for all test points.
This is an enormous task and we can calculate that we must fly something like five hundred sorties of about one to two hours each to achieve this for an aircraft of our complexity level, certified to Part 23.
Now that we are convinced we can demonstrate compliance to all Part 23 requirements we make the formal application with the regulator. The regulator’s personnel have followed our flight test and know well our status. By first conducting the tests as a company, we have taken out risks of failure during certification testing and we can resolve design issues we discover prior to the “for score” testing.
We have also proven that the envelope is safe and the FTA is mature so that the regulator can comfortably get on board and make their findings of compliance.
It’s this knowledge and our continuous documentation on achieved certification requirements together with the formal application that forms the basis for a clearance to begin formal Certification Flights. When a company proposes progressing to certification flights without sufficient proof of maturity first the regulator will often refuse in order to not waste their resources on testing that may need to be repeated. The FAA did this quite publicly earlier this year when refusing to grant approval to start flight testing — Type Inspection Authorization — on the Boeing 777-9.
Certification Flight testing
In essence, we go through the same flight test program again but now with the knowledge and confidence of what we can achieve for each parameter. All parameters are logged, evaluated, and documented to form the basis for the information base for the Type Certificate.
The tests we do serve to demonstrate that all systems and protections work correctly (including at the corner points of the envelope), that the handling and control qualities of the aircraft are appropriate to allow operation without inordinate piloting skill, and that the product is reliable enough for practical use.
There are some common exceptions to this repetition philosophy. Particularly high risk testing, such as the testing to find the absolute minimum takeoff speed, is often only run once to reduce risk to the flight test pilots and the aircraft ( https://www.youtube.com/watch?v=4FSm_SL2n9o&t=255s ).
Additionally, some testing that is very expensive or difficult to set up may be run only once with the regulator present — this increases failure risk but reduces program cost if successful. One example of this would be Cold Soak testing for large aircraft, as the temperature extremes that are needed can be very difficult to find ( https://www.youtube.com/watch?v=e3sWVRwofOA ).
Another example would be Water Ingestion testing due to the time and expense of building the water trough, as well as the risk of runway excursions ( https://www.youtube.com/watch?v=VhtUqsDojRM&t=290s ).
During such simultaneous testing, company and regulator pilots may alternate runs at the controls in order to both make their assessments.
The difference between own flights and Certification flights
In most cases, the certification flights should be run very similarly to our own. However, they require significantly more formality:
- Test plans must be documented in great detail and approved prior to the flight
- Configuration control and inspection evidence must be presented (to prove the conformity of the FTA)
- Data acquisition and reporting plans must be agreed upon
- For many tests, agreed versions of operating and maintenance manuals must be followed to ensure the tests are representative of service conditions, and
- Failures must be formally dispositioned to the satisfaction of the regulator, often requiring a retest.
These measures ensure the validity of the testing to the final product to be certified, and also serve as records that can be used to support future change approvals and safety investigations. Most of this data must be retained for the life of the type certificate in case there is ever an accident on the aircraft type.
In addition to the control measures, there are also some differences in actual test cases between the programs. In particular, our engineering program will typically involve taking additional measurements to validate our analytical models or gather data for future product development assessment that is not required by the regulator.
Does the FAA/EASA test the Iron Bird/Full flight siumlator with full systems for multiple failures in all different flight profiles with min/max TOW, min max speed and max fwd aft allowed center of gravity. Hence when in the process should they discovered the 737MAX MCAS system logical errors if Boeing did not redo the full Failure mode cause and effect analysis?
IMU There was no MAX changes covering simulator initially.
Later the MAX changes coverage was still incomplete?
..
Iron Bird like teste equipment is associated with an upcoming “new” designs.
Creating an easily introspectable setup much later could well be seen as good design practice.
Don’t they keep the iron bird to speed up testing of equipment enhancements and safely test system errors as they pop up in service. Like the 737 rudder hardover, thrust reverser deployment at climb or different bus bar errors. The full flight simulator mainly simulate the boxes functions thru software. I can think of situations with multiple failures including engine failures and some systems shut downs that you don’t want to flight test in the air but test in the full flight simulator connected to the iron bird.
Suppliers have their own test rigs for their components. No need for Boeing to keep a full iron bird or ITV system active.
claes”
I will leave the direct reply to Bjorn under your question.
There would not have been an Iron Bird on the MAX. That is used for new aircraft.
What I do know is the Simulator is supposed to have 100% Fidelity with the aircraft in the air.
I was the mechanic/engineer responsible for a MD-11 support system (cooling, Cockpit and and facility power)
In the case of MCAS, I have not seen anything that says it was not present and I suspect 100% (?) that it was tested in the air.
The issue of course is, tested with and AOA Vane failure in a Sim.
Obviously an AOA Vane failure was not tested in the Simulator. MCAS did exactly what it was designed to do, the tragedy is how they slathered over the failure aspects and did a (avoidance offensive language) a horrid job of the failure modes and what that would do. Covered up is not too strong a term. Playing Russian Roult3ette with people lives come to mind.
That said, the one part that was missing and have yet to see explanation of, was the Trim Wheel lock up at high loads. Between MCAS and that lockup the pilots on Ethiopian could not get it to work. I have seen Mentour Pilot U Tube on it and its virtually impossible for one person and someone has to fly so they can’t both crank on it (and still would be difficult sans a yo yo which you have to get to yo yo control to do so)_
The Ethiopian Pilots would have had no training in that trim wheel lock up as it was not there even in the NG Simulator.
Having had a few fun moment in the air and open the ground I know what that does to your brain when something changes behavior of what your experience (and or training) said it would do. Brain Freeze.
What most people do not know is a Flight Simulator is not only certified but re-certified on a regular basis (yearly I think but could be wrong)
Our cooling fan had a noise issue on the MD-11 sim. We fixed the issue but by that time they had been hit with a warning and built a sound enclosure to deal with the fan noise that was at issue (FAA inspection heard the non normal sound)
That enclosure then became required equipment or the flight ops for pilots was not legal (hate to think what that would have entailed in failed pilot check outs who were then flying routes).
That became a problem as we had no access to the fan to inspect short of a middle of the night shutdown. Mostly we did emergency repair as the unit failed, we could take the enclosure off but they had to put it back on.
Eventually we got them to re-run the sound test without the enclosure proving we had corrected it.
It was a hard test to run and it was easier for them to put the enclosure back on (it was not built to be taken on and off and it became an issue where it was falling apart)
For something like the NG or MAX where an AOA Vane failure has a major cockpit change (shifting to the other flying position) you would think it would be a scenario item and looks not to have been)
This has been a fascinating series. Thank you.
thanks Björn, again for these clear explenations of difficult processes!