September 15, 2023, ©. Leeham News: Last week, we described the beginning of the Detailed design phase of an airliner development program. We discussed the importance of a good information set from Preliminary design, ideally as a collection of digital models forming a Preliminary level digital twin.
We now discuss how the work is managed in the Detailed design phase and how to speed up the work and make it more efficient.
The traditional management of large aircraft programs has followed a structured and controlled method called the Waterfall method. It means the design engineer gets his job to-do through a hierarchical breakdown process where his manager takes part of the Preliminary design information that touches the designer’s task, clusters it, and complements it for him to have as input for his design job.
The designer makes a 3D design of the part and adds information about material, treatments, tolerances, etc., into the Product Data Management system, where it forms a complete design information for the part.
The production engineer then takes the 3D model with information and morphs the data into a production specification that includes what methods and work steps shall be used to manufacture the part.
The OEM or a contracted machine shop then produces the part. The OEM’s Quality Assurance (QA) department measures the part when exiting the own shop or when it arrives from the external shop. Any QA deviations are recorded and reported to the production engineer and the designer.
Is the part acceptable, it’s built into the assembled portion of the aircraft where it belongs. Any problems or findings at the assembly process are noted and, if grave (fit or access problems, part not optimally designed), reported to the production and design engineers.
The part in its context is then tested for its function in the aircraft by the test department, which writes test reports that go to the designer of the part, among others.
The whole sequential process of breakdown, design, manufacturing adaptation, manufacture, and assembly into a test rig or prototype, where a test report is the final feedback on whether the part is correctly designed, constitutes a very long feedback cycle. It can take a year or more before the designer has full feedback on his work. The after-sales service aspects (can the part be easily removed to give access to maintenance behind the part, etc.) are only received when the maintenance department starts working on prototypes to check if the aircraft is maintainable.
The fundamental problem of the classical method is the long feedback loops, where design changes are made very late based on how it shall function in the aircraft and during maintenance.
The strength of the procedure is a strict step-by-step transfer of responsibilities and information. Project and Configuration management is straightforward.
The above describes hardware development, where the final product is physical. It’s relatively easy to define how it shall look and fit (dimensions, materials, tolerances, mass, surface treatment, etc.).
Functions realized in software are harder to define to 100%. It made the waterfall method less suitable as the end result was often suboptimal (This is not what I expected or need! Yes, but it was what you specified (six months ago)!).
As a reaction, Agile methods appeared for software development. In essence, these break down the software functional specification into small blocks that can be specified, programmed, and tested with the customer within a week or two. Now the feedback time was weeks, not months or years.
The Agline method allows cross-functional teams where the customer of the function (let’s say it’s the cabin team with lead customer representatives as the software is controlling the air conditioning) can be part of the team, as can the maintenance department so it can look at BITE (Built In Test) functionality for the functions from the Get go.
The success of Agile in software development, its increased development speed and customer satisfaction made program management for other disciplines wanting to test it as well. Initially, it was thought, “Finally, we have found the medicine against the sequential and long loops of the waterfall method.”
In practice, the results have been mixed. As we described last week, the projects now span many thousands of people inside the OEM, further thousands from external consultancies supporting the OEM, and even more people in the supply chain working on the project.
Once the work spans several organizations, forming short-loop Aigle teams gets complicated. Expected work results versus money paid is part of contracts with the external organizations. Such contracts have specified deliverables with attached costs.
But Agile is about seamless cooperation in small self-governing cross-functional teams where teamwork rules over organizational belonging and contract terms. Agile teams are thus less defined in their outputs (it’s the point of Agile; it follows the customer’s need), making Project and Configuration management difficult.
The overall result is that Agile is now part of airliner projects but hasn’t replaced the more traditional sequential methods. These are used for overall Project and Configuration management.
As more and more functions in an aircraft are realized in Software, Agile methods are used for a larger and larger part of the aircraft project. It’s also used in other areas where short feedback loops are needed and effective.
And in general, the strict hierarchical “over the wall “ transfer of development process steps has changed. Cross-functional teams are formed much earlier in the development, and feedback loops are as short as possible.
Has Agile made the development of aircraft more efficient? Yes, but not to the extent once thought. As the importance of software increases, the effects of Agile increase.
Hi Bjorn,
Congrats for your great job of providing a clear and understandable view of such a complex activity of devlopping an airliner!
Will you introduce us with APQP approach (which is now being applied and required by most of OEM) in the classical V model developpment life cycle you described ?
You also have the concurrent engineering method where you speed up prototype manufacturing in soft tools/forging hog-outs in parallel with traditional optimized production designed parts to be able to instrument and test early before production parts arrive. That way changes are less costly but you spend more up front.
Claes:
I am totally sourced on concurrent production. Recent examples were the F-35, they had something like a 100 built before they got to testing (short story part) and none of them will be combat capable because of the major structural changes. Only good for practice (not a bad thing as we need stealthy practice but huge cost)
The other one at a almost zero production rate that was still horribly messed up was the Ford Carrier, munitions elevators did not work, they did not have
a straight shot to the upper deck (I know but they managed that) and the catapults and arrestor systems did not work.
Boeing and Airbus early builds are not liked (up to about 20).
Granted there are ways to not have those outcomes but you need a true prototype and test it first.
The F-35 early production before full capability certification was political to have the Ft. Worth production rolling. That is an exception when money is not an issue.
Claes:
Interesting view as I have seen nothing but negatives on that. Do you have an example that worked? In years gone by when I was working I just saw the general trend of programs and did not get into the details of what the issue were.
US has tried it on other programs and it was a failure there as well.
I don’t disagree it may work but the evidence is at least for defense related it does not.
In the case of the Ford, the weapons elevator shafts were not direct up and down and how you manage to build a ship and not notice one of the key advances does not work?
And money is always involved as well as politics, its just a problem that goes back forever.
I think a true prototype program is the best way and then refine that prototype into a pre production, test it and then finalize the design.
Boeing and Airbus do go into pre production by any other name and they at least understand its going to result in sub optimal airframes you sell as a discount (airlines might not take them as they know the issues)
Boeing wants to start low rate initial production on the T-7A and the GAO says they should not do that. In this case there are two prototype flying but the 5 test planes for the USAF are just getting the first one into the air.
The first C-17 was so oddball from the rest they retired it as soon as they had enough production airframes.
“The airplane, designated T-1, first flew on Sept. 15, 1991, six years before McDonnell Douglas merged with Boeing, and was expected to last about five years. On Wednesday, it finally made its last flight, into retirement at the National Museum of the U.S. Air Force, in Dayton, Ohio.”
That was April 2012 when retired,
https://www.seattlepi.com/business/boeing/article/Air-Force-retires-first-Boeing-C-17-Globemaster-3513345.php
Think the GE90 was one of the first concurrent engineering designed engines. In the end sold pretty good.
Duke: Thanks for the update. Its odd to see such a troubled program go onto such great success. Always hate to see the lines closed but that is the nature of defense programs.
Claes:
Definitely not familiar with how GE went about the 90, it seems most engines are somewhat concurrent. Only 8000 testing hours in a year and we see the end results with PIPs needed.
Then they change things for production and as we saw with the shaft coating on the GenX, bad news can happen.
I will agree its possible concurrent can work but it surely needs being managed carefully or like the F-35 or the Ford, get out of hand. I can’t fathom (pun intended) how the Ford coule have offset elevators that were obviously not going to work. The EMALs and arrestor gear should have had land based test versions and only late in the game, seriously stupid.
It is always a pleasure to read Björns articles, for the content but also as an example of clarity. Great work.
Agreed..
Agreed
I’m highly skeptical of Agile management techniques for airplane development. Friends of mine who work in the super-computer field writing software and in chip design state that Agile doesn’t work where creativity and innovation is required. Essentially you can’t force a schedule onto problem solving. Furthermore, engineers quickly learn how to game the system to set easy goals during the Agile sprints (short term schedule milestones). My own experience has been that managers are not incentivized to challenge engineering teams to do better than what is required to meet the schedule. Keeping the stoplight chart green means each cycle’s goals will be easy. So in effect, Agile management techniques could be counter-productive to schedule and quality.
Behavioral economists have conducted detailed studies which demonstrate innovation and problem solving becomes more difficult under stress. So telling teams that they have 3 weeks to solve a difficult problem might not get you the best results.
Agile techniques may best work where the technical problem is modifying something that has already been done before. For example, maybe it is just porting code from one platform onto the next.
It seems gimmicky to me to state that there are transformative benefits to cost and schedule using Agile.
JB:
Oddly, I have had my most brilliant work under severe stress (impending doom and its amazing)
True that is not innovation but working out of a crisis.
I have also come up with some good ideas when my first ones were turned down (usually cost related).
It may be that a certain amount of stress but not overloaded is a good thing. Some people were good to talk to, not because they had good ideas but they got me thinking in different directions.
Part of the issues are that while you can specify a part, tolerances, dimensions and properties (and test those) the longevity of a part is only provable through in service time (note all the engine issues as well as early wear-out all mfgs are seeing)
A couple come to mind.
The more recent one was the shedding of turbine blades on an A380 powered by GP7000 engine. Kudo to Airbus recovering the bits on the icefield (Greenland?) and they found a hydrogen embritlement issue that they did not know existed or occurred.
RR in the 777 that had the fuel heater freeze up they had never seen.
GE with the super cooled droplets on the engine vanes.
And if its an all new structure you don’t have the base knowledge.
I think its a matter of doing the best you can and not using process that has been proven not to work (or modify the process to account for its shortfalls)
To me the amazing thing is they do get it done and machines fly mostly successfully.
Boeing attempted to Concurrent Engineering on the 747-8. It failed because the scope of work kept changing, and with it, the schedule was never firm. Boeings last successful major program that was executed flawlessly, was the implementation of the “Paddle Fitting” revamp of the 767 fuselage splicing process. There was a tightly defined SOW, an on dock date for the first parts to load, and no safety net. It had to work on the day the schedule required. the team brought it home at 74% of budget. It can still be done today, but the project must be very tightly defined and change orders must be kept to an absolute minimum. Doing things for a second time is roughly 4 times as costly as getting it right the first time. Thats the challenge before us today
Scott C:
As I understand it, a lot or all the 747-8 engineering was handed off to the Moscow center and had to be redone.
I saw the same thing on our building controls area (and in depth discussion with oil field items done in India). Not used to US approach and no understanding of what really was presented so it all had to be re-done.
All that stuff is hard enough in person or on site.
Moscow center was destined to fail as documentation made available did not describe the plane as built on the shop floor.
garbage in .. garbage out.
The jingoistic path taken is badmouthing those that are “off continent” and stating your own innocence.
Uwe.
Boeing Moscow did great work and the loss of their capabilities is still felt today as a result of the Ukrainian conflict. Interestingly, Boeing is standing up their Ukrainian Engineering capabilities in Poland. The Polish government (God Bless Em, they are rock stars) revised national labor laws to allow for immediate work permits for “Refugees in Professional Employment” to make it happen.
I think you may misunderstand why the airplanes didn’t match the drawings. There is a gap between what you draw and what actually works from time to time and the aircraft is built on a NCR instead of as planned. The Nonconformance Record instructions allow the airplane to be built until the Corrective action fix is implemented. The corrective action fix is a supplemental engineering release to bring the drawings up to date to include the fix. Boeing Moscow was tasked with the incorporation of all of these changes into the drawings. There were literally thousands of these to be worked and some fixes didn’t work well and were done 2 or 3 times. The correct sequencing of all these ABRs (as built records) was difficult, and as far as I remember they did a good job on a difficult task. Im curios about how jingoism has any place here because. The concurrent engineering failure was strictly a function of Boeing losing the handle on the statement of work and not stopping mission creep dead in its tracks. Boeing Moscow delivered exactly what they were tasked with.
You need both a very skilled chief engineer and an experienced project manager that works well together and both have clear roles. In the old days those roles was merged into “The Project engineer” personally signing all design drawings of the project.