By Scott Hamilton and Laura Mueller
Update, March 9, 2023: Some readers have interpreted this story as reporting that new deliveries directly from Boeing are being delayed. The wording is somewhat ambiguous. To clarify, airplanes purchased by lessors–who have taken delivery from Boeing–are experiencing delays in delivery to their lessees due to the issues with the Boeing software reconfiguration described.
March 6, 2023, © Leeham News and Airfinance Journal: A new issue with a software program is delaying deliveries of some Boeing 737 MAXes by up to a year, joint reporting by Leeham News and Airfinance Journal learned. The Federal Aviation Administration views its use as a safety matter that must be resolved before delivery on aircraft undergoing reconfiguration. It is not a safety issue when aircraft are delivered to the originally intended operator.
The Boeing software, called Option Selection Software (OSS), is used by Boeing to identify software installed on 737 MAXes that must be reconfigured when the airplanes are going from one airline going to another. For example, if a 737 was built for Airline A and instead it will go to Airline B, reconfiguring the cockpit display and related systems may be necessary. We are told that MAXes and 787s are impacted, given their large inventories of airplanes that have been stored long enough that some original customers no longer wanted the aircraft. When sold or reconfigured for a different operator, Boeing uses the OSS to reconfigure the software and identify related parts for any changes.
This issue has not been reported previously.
A retired Boeing employee said that the reason that fully functional OSS software is so critical to Boeing is that the assembly paperwork systems used by Boeing in the factory (Product Data Manager and Catia/Enovia) to define the job sequences lack the ability to indicate the software revision levels loaded into the black boxes. That data is not found on the face of the engineering drawings.
The retired Boeing employee continued that Boeing identifies and installs each black box per their engineering drawings and then has the vendor supply the correct software for each specific. Boeing has separate assembly jobs in the factory that load and record the software program and revision level for each black box installed. This data is supplied by the black box maker to Boeing outside of Boeing’s Drawing Control System.
The ship’s record assembly paperwork that is stamped and inspected showing this data is not electronically searchable for the software info. While the Boeing drawing tree tells you the wire bundles and black boxes installed in each airplane, the drawing does not tell you what software was loaded. This seems to be the crux of the issue, the retired employee said.
Boeing, which did not answer specific questions, instead provided this statement through a spokesman:
“As the commercial market recovers, there have been delays in execution of certain modification work. We continue our relentless effort to deliver on our customer commitments and this focus has helped identify areas that we can improve.
“We’re taking comprehensive action, from an increase in engineering capacity to process improvements that enable on-time delivery. As a result, we are enhancing quality, stability, and predictability throughout our modification business and our customers are starting to see our progress.”
The FAA declined to comment on the specifics, but said, “our interest is what you’d expect it might be for the safety ramifications of any aircraft-related software.”
Additionally, reconfiguration may also be required when going from one regulatory jurisdiction to another—for example, from the Federal Aviation Administration’s authority to the European regulator, EASA. The reconfiguration requirement does not affect the cabin; it’s only for the cockpit and related systems and parts.
Initially, it was thought all 7-Series airplanes were impacted. But only the MAX and 787 appear to be affected.
New production airplanes that are switching operators before delivery appear to be affected. New 737s ordered by the company 777 Partners, which planned to lease them to Canada’s Flair Airlines, instead were sold by Boeing to other lessors before delivery to Partners. The new owners had their own customers in mind. Initially told that the cockpit could be reconfigured within two months, the software issue emerged and blew up this timing, lessors said. Now, reconfiguring the software won’t happen until January 2024 (plus or minus a matter of weeks).
Lessors said that historically if a lessor wanted to repurpose an FAA configuration to EASA metrics, it might be weeks and no more than 2-3 months for manuals and pin selection changes. Today, the lessor is in a queue of perhaps six months to receive attention and then a quote of six months to complete the tasks.
Lessors Altavair, BBAM, and BOC Aviation are affected with MAXes. Lessor AerCap is understood to be affected with Boeing 787s. There may be others.
Boeing has 138 MAXes in storage ordered by and built for China’s airlines and an unknown number for other airlines that ceased operations during the pandemic. LNA/AFJ are told that these airplanes may also be impacted, though Boeing denies it.
Boeing CEO David Calhoun announced in September that Boeing was going to remarket these airplanes, which the Chinese government has so far refused to authorize delivery. But Calhoun reversed course in January, announcing on Jan. 26 that remarketing the aircraft is “paused.” Calhoun cited appearance of a thaw in US-China relations as the reason. (Days later, tensions heightened again when a Chinese spy balloon entered US territory. It was eventually shot down off the coast of South Carolina.)
However, the timing suggests the pause may be related to the software issue. LNA/AFJ are told the issue was discovered in the Autumn 2022, shortly after the initial decision to remarket the Chinese airplanes. Multiple sources say the software issue is why Calhoun reversed course.
“I am not surprised that Boeing denies that the pause in remarketing white tail Max’s is linked to the problems but it definitely is a huge part of it,” said a person familiar with the situation.
What happened to cause this disruption?
As a legacy of MAX and 787 gaffs, the FAA insists upon complete validation that the system standard is flawless. Experience with this process is unproven and has rarely been necessary in the past. The software is supposed to produce data on a number of operating functions and identifies certain systems and parts that need reconfiguring. LNA/AFJ is told that the OSS isn’t necessarily identifying all that needs to be reconfigured.
The software challenge seems to be rooted in the ability to validate that the existing standard is the current standard. LNA/AFJ is told that the FAA put a “pause” on the process (to borrow Calhoun’s word) while Boeing figures out how to validate the process.
“The FAA has lost all confidence in Boeing,” LNA/AFJ are told.
The repurposing effort runs head-first into a cavalcade of demands on engineering resources (e.g., certification completion of MAX 7/10, 777X, engineering replacement on P2F work that had previously been assigned to Ukraine/ Russia, and a plethora of return to service demands on Boeing from the pandemic’s parked planes). The queue of projects is wildly outstripping the availability of engineering resources. And, the pandemic led to an exodus of proven/experienced talent. Now, the replacements may be learning these tasks and assignments for the first time.
The question being posed by the FAA is “if this software is not recording MAX display software configuration software correctly, what else is impacted?” said a person with knowledge of the situation.
The problem applies to 787s as well, but far fewer are involved. However, this is unrelated to the most recent pause in deliveries, which Boeing attributes to paperwork and analysis of the forward pressure bulkhead.
Boeing previously paused deliveries of the 787 when small gaps, the thickness of a piece of paper, were discovered on some newly built aircraft. Boeing built 115 787s during the pause. The FAA put Boeing through a meticulous review process (some complained of FAA nitpicking) before authorizing the fix and allowed Boeing to resume deliveries. It takes five months to rework the built airplanes.
Nearly all customers kept their orders, but there are a few airplanes that are going to a different airline than originally intended. These must go through the cockpit reconfiguration process.
All the customers were given revised delivery schedules after Boeing received authorization to resume deliveries. Now, many customers are being notified that there will be another round of delays of six to as many as 18 months. It’s unclear if this new round of delays is exclusively for the reconfiguration issue. At least one customer points fingers at Rolls-Royce for delays in delivering its Trent 1000 engines, which have been subject to technical issues dating to pre-pandemic times.
Some airlines are complaining that it is taking Boeing up to two years to complete the cabin reconfiguration paperwork, an issue apparently separate from the OSS.
The OSS is also having trouble reconfiguring the Boeing 747-8-based VC-25, used as the US presidential airplane Air Force One. Two whitetail 747-8s were purchased by the government for conversion to Air Force One. These were ordered by the Russian airline TransAero, which was unable to accept delivery. (This long predates the Russian invasion of Ukraine.) The VC-25s, at Boeing’s facility in San Antonio (TX), must be reconfigured from the TransAero system and configured for the military’s specifications. Operators of the OSS have been trying for two years to identify what needs to be changed. Inexperienced personnel due to turnover is identified as the principal reason for this problem.
It took Boeing 20 months to recertify the MAX and nearly as long to receive FAA approval to resume deliveries of the 787. Now, it’s facing another lengthy process. Although the issue was discovered in the Autumn (the exact month is unknown to LNA/AFJ), “Autumn” runs from September 21 to December 21. Boeing hopes to have the Software validation and approval by the end of April. There is no guarantee this goal will be met.
Laura Mueller is content director at Airfinance Journal. Scott Hamilton is editor of Leeham News.