The debate over using DOT Form 41in comparing A320 and 737 costs

Long time readers know that we don’t think much of the veracity of data filed with the US Department of Transportation under Form 41, detailing maintenance and fuel costs. We know from a consultant who helped an airline create its methodology for filing the data that the airline “fudges” the methodology to distort the data for competitive reasons.

Boeing has cited Form 41 data for years in making its comparisons between the 737 and the Airbus A320. A key piece of information Boeing uses is the maintenance cost data, in which Boeing claims the 737 was up to 29% less costly to maintain than the A320. This comparison in particular drives Airbus crazy, and officials say the Form 41 data is “garbage.”

737 v A320 maintenance

Boeing also previously used a study from IATA from 2006-2009, but dropped that in this year’s comparisons at the pre-Paris Air Show briefings.

Boeing also relies on DOT 41 data for fuel comparisons.

737 v A320 fuel

Beverly Wyse, VP and general manager of the 737 NG program, emphasized the reliance on DOT data in the comparisons.

“You don’t just have to believe us,” she said during the briefings. “We know that out there in the market ‘Airbus says,’ ‘Boeing says,’ and it’s hard to find the truth, so wherever we can we’re trying to use independent sources to substantiate the information that we are providing you. This data actually comes from the US department of Transportation Form 41. Depending on the model, and the configuration of the aircraft, the Dash 800 compared to an A320 or the A321 compared to the 900ER, you’ll see between 5% and 8% better fuel efficiency for the 737.”

Wyse credited the 737’s rival with good dispatch reliability, a rare Boeing compliment of the A320. But she claimed the 737 was nonetheless better.

“The A320 and 737 both have incredible reliability. Anything with 99 point something reliability is something to be proud of. The simpler design of the 737 contributes to 20% lower maintenance costs resulting in 67 fewer days out of service,” she says.

Given the reliance on DOT Form 41 data and our open doubts about using this data, we followed up with a series of questions to Boeing. Here is the exchange.

Boeing cites DOT 41 data in claiming the 737NG costs 20%-24% less on maintenance than the A320. First, DOT 41 data is crap. The airlines manipulate the data to mask what their true costs are from competitors. So how does Boeing purport to un-crap it?

DOT F41 data is a good source because it has rigorous reporting processes, contains constant costs in U.S. dollars, and includes reported data by operators who maintain both aircraft fleets, insuring consistent accounting practices.  But it is not the only source of data for our claims.   That goes toward your second question…

Secondly, Bev in one-on-one said, well, Boeing compares maintenance schedules between the 737 and the A320. Which is it? DOT 41 or company manuals?

The Maintenance Schedules or Check Intervals (that is the interval between performing various levels of Checks, typically classifieds as A, C and D-Checks) are obtained from our company manuals and for the A320, from industry publications and conferences. Specific A320 check intervals are periodically announced in Airbus press releases, journals and conference papers. It may be noted that F41 data provides a look at historic data while the Maintenance Planning Document (MPD) provides the check intervals for the future occurrence of check events. The continuous improvements that occur in the Boeing MPD usually result in even lower maintenance costs to airlines than being currently reported in F41 data. Our focus is to enable the airlines to continuously reduce their operating costs including maintenance costs through better product and services.

Third, I know from my own sourcing from within Boeing that to get to 24%, Boeing has compared new 737s to the oldest A320s using the earliest engines. When new-to-new is compared, the delta is 4% in favor of Boeing. Bev responds to this that this is a fleet-wide analysis.

Bev is absolutely correct; we use fleet-wide data in our comparison. There are many differences in the fleets of different airlines, like airplane age and the average stage length (Fhrs/Trip). Both affect maintenance costs. To enable true comparison, maintenance cost is normalized for a common airplane age. Since A320 is usually an older airplane the normalization process actually reduces the A320 costs to bring it to say a level of 737NG airplane age (say 8 years). Without normalization for age, the cost difference would be an additional 4 to 5% to the advantage of 737NG, but we diligently apply our processes while showing the relative cost and remove this fleet age based difference .  Similarly, costs are normalized for stage length and some other factors. All these normalizing processes are approved by the IATA Maintenance Cost Task Force of which both Boeing and Airbus are also members and uniformly used by most of the industry. We compare costs and do modeling in our tools on basis of the normalized costs.

In summary: All maintenance costs are compared on basis of like airplane age. Our multi level data analysis is for “as reported data”, normalized data and data from airlines operating both airplane types. In each case, the airframe costs for the 737NG are lower and on an average 20 percent lower. 

Fourth point: If this is a fleet-wide analysis, how is this possibly fair? The NG didn’t enter service until 10 years after the A320, so maintenance on aging aircraft skews everything.

Maintenance cost data is normalized to the same age level for instances where the airplane ages are different. This is done for each airline and then grouped together. The analysis ensured we were comparing like-aged aircraft not the entire fleet. By that comparison, the 737 has about 20% (+/-) lower airframe maintenance cost than the A320. But consider that there also are a number of other factors …

Five facts that drive lower maintenance cost for the 737:

1. Newer simple value added design

2. Lighter weight, smaller size;

3. Fewer systems

4. Fewer installed components

5. Longer intervals; fewer maintenance visits

Airlines believe and give us credit for our maintenance story. 

If there were truly a 20%-24% maintenance delta, why would anyone buy the A320? It doesn’t all come down to price, you know. The A320ceo evenly splits the market with the 738 but the A321ceo outsells the 739 by a wide margin and the neo widely outsells the MAX.

Maintenance and its cost is only one of the criteria by which airlines judge airplanes, and a relatively small one at that. Airframe Maintenance is usually 4 to 5 percent of cash operating cost. However, maintenance and reliability plays an important role in the operational experience for an airline. Price is an important factor in airline decisions, but you are correct that it doesn’t all come down to that. As you’re aware, for example, there can be political issues, or even funding issues.  As for the division of the single-aisle market, we are very aware of the inroads Airbus has made in its relatively short existence and are very much motivated by that.  Arguably, that’s another reason some airlines opt for one plane or another. So, too, is the ability to deliver an airplane when an airline wants or needs it. The on-time reliability of one model or the other can be a factor (while both companies produce reliable airplanes, Boeing’s 737s have better on-time statistics).  Too, some airlines prefer to stick with airplanes compatible with their installed fleet to reduce maintenance, training and scheduling costs. The choice is very complex.

I note, too, that this year Bev dropped citing the 2006-2009 IATA maintenance data. How come?

IATA data is used extensively within our own internal analysis. However, we acknowledge that there are some issues with IATA data. IATA airlines do not report data consistently over a long period. They may drop out entirely, miss reporting in some years and new airlines may join for a period. The Form 41 (US DoT) data is mandatory to report and usually aligns with the airline’s Annual Reports reported costs. In addition, variation in currency exchange rates becomes come into the picture. Airlines report in their own currency and that is converted to U.S. dollars. Inconsistency of reporting and the additional currency variable frequently clouds the data.

Bev says, though, that what tipped its use in the end in these slides, however, was that the data we had at the time was old.

We asked Airbus for a response to some of these questions. Engaged in the Air Show this week, we have yet to hear back.

29 Comments on “The debate over using DOT Form 41in comparing A320 and 737 costs

  1. How the 737-800 can be twice cheaper to maintain than the 737-700 on a per seat basis when it only has roughly 20% more seats

  2. I does not surprise me that airlines could be “fudging” their methodology for filling out the Form 41. It would surprise me greatly, however, if this “fudging” on average benefited one manufacturer over another.

    • The nonsense data might be random, but Boeing’s use of it isn’t. Boeing (in this case) only highlights data that is nonsensically to their advantage…

      • No, surely not, but claiming a dataset shows one thing objectively, while using the data selectively is not quite square either.

        As the saying goes: “with statistics you can prove anything.”

      • For scientists using data selectively without disclosure or justification is a definite no-no. However, this is’t science here, this is marketing.

        I find it very hard to believe that Airbus is taking some sort of high road here because the data set has potential problems and they don’t want to risk misrepresenting their product. If this Form 41 data supported a lower maintenance cost for the A320, then I’m sure we would be hearing about it from Leahy.

        It is not yet clear to me that the Form 41 data set is inappropriate for relative comparisons between different aircraft types operated by the same airline. That is one of the questions Scott is trying to address by questioning Boeing. The data set is obviously not not appropriate for comparisons across airlines due to the “fudged” methodology.

        The bottom line is that Airbus does not like what the data says about their product, so the are attacking the validity of it. If Boeing truly is only showing the part of the data that supports the 737, then Airbus would show the part that supports the A320.

    • True, marketing is not science… (I have a few select, non-usable in polite conversation words on what i think marketing IS)
      Unfortunately many non industry insiders, or partisan insiders do base much of their discourse on these presentations.
      and I think a little scientific honesty always benefits any discussion.

  3. Most here know my views about Boeing’s claim regarding the B73X versus the A32X.

    For those who don’t:

    The A32X sales speak for themselves.

    • But that may be driven by acquistion cost and not actual operating and performance cost. If one airframer is offering a discount and warranty programs the other is not, the actual cost of operating one might be skewed. Every airline I’ve spoken with has said the performance of both are equal, but the acquisition costs of one impacts the ownership cost more. That might explain why more of a select airframe sell better? Also it might also explain why the resale value of the other company’s product remains higher. And, finally why one frame has a shorter life expectancy than the other. Driven by the writeoff value being less than the other, and then buying a new one versus reselling the older model. If I can get in to a Caddy cheaper than I can get into a Bimmer, which am I going to buy?

    • Low prices my friend. A bird in the hand may be preferred over two in the bush…if you understand my point.

  4. “Relativ fuel burn/per seat” – nice chart but I can’t find a way to get the freight out of the equation or range. By comparing Airlines with quite different networks I can show any fuel burn rate per seat

    • An airline is not trying to share their operating cost with the competition so you’re not going to get an apples to apples comparison. All depends on how I define the assumptions, and I’ll never tell you what drives my assumptions.

  5. Hey Scott- I see why you are so against Boeing and their marketing info. I can also see why you might have a hard time getting the straight skinny from them too. I bet it is easier for you to get info from Airbus because you’re willing to challenge the Boeing approach. it’s good for the, industry, the reader, and the overall market analysis, if it’s done fairly. Fairly may be defined differently on both sides of the pond.

  6. Great post and tough questioning. Let’s see if Airbus reacts. They are not straight on facts, either.

    Just as note for readers not familiar with the issue:
    Currently maintenance is responsible for roughly 25% of the cash operating cost (that is: without aircraft finance). Rest is fuel (~50%), crew and charges. Of the maintenance cost, about half is caused by engine maintenance. So important to compare the CFM operators. The rest is caused by airframe, with component maintenance being one proportion. This is of course a quite large advantage for the B737, as it has indeed a simpler system design (like fewer computers, different MEL*). Another large proportion is caused by heavy checks. So, comparing maintenance is a very tricky business. Using Form41 data skews the facts, but also has some merits as quite some maintenance cost is caused by overhead. So the fleet size and maintenance organisation influences the cost.

    *MMEL: Minimum Equipment List, it says what you need to actually fly. The Airbus FBW requires more components to work.

  7. From the state of the respective order books it seems that Boeing are the only believers in Form 41

  8. Who is Boeing trying to convince by this? Airlines? Hardly, unless they think they will convince all these airlines that they are wrong and actually SERIOUSLY regret ever buying an Airbus. Boeing alludes to ‘other factors’, but if the numbers are this huge, then no airline would ever buy Airbus on a commercial basis. I mean, there are mixed fleet operators or those who otherwise have access to realworld Airbus data, who make further Airbus orders. But then I guess Boeing really believes that any Airbus customer anywhere in the world is really a political lackey to Airbus and any commercial based decision would go in Boeing’s way.

    What is the point of this ridiculous comparison between different fleets of no longer being produced jets? Boeing’s supposed to be in the business of selling new jets, and stats for previous models just don’t matter. If they want to make a comparison of historical trends in maintenance cost changes over the life of the product, OK, and that can be overlayed on the performance of current/new models (based on OEM) although that should be taken with a grain of salt, but that’ s not what they’re doing.

    And what is with the childish references to Airbus not being in existence (in it’s current form, at least) for as long as Boeing. Shouldn’t they be embarrassed to still be using that line? I mean, when Airbus first started or even with A320 that was perhaps relevant, but nobody in the real world cares about that anymore. I mean, it’s normal for every OEM to promote their product, and the type of comparisons we saw from each re: NEO vs MAX is pretty much expected, and legitimate… Airbus certainly can play the selective comparison thing, but they don’t attack the legitimacy of Boeing as a company, and they don’t use just completely ridiculous and POINTLESS figures like this. (it’s not even relevant to selling new planes) I think this issue pretty much makes it clear: Boeing operates with a superiority complex to Airbus.

    Sorry to say Boeing, but the tangentially alluded to factor of production slots and delivery schedules must certainly be playing in Boeing’s favor or Airbus would be selling even more NEOs.

  9. First, By Boeing’s admission airframe maintenance is ~5% of total cost.
    By Boeing’s claim, Airbus aircraft are ~25% more costly in that area.
    That is 1% on total cost. Certainly a small enough delta to be overcome by many other factors

    Then, @ Mr Scot
    Am I correct in assuming the DOT 41 information is only mandatory for US based airlines or US connecting flights. That would naturally skew the data in favor of Boeing which has historically optimized their designs more for that part of the world.

    That would mean all we can say is that the airbus single aisles are ~1% more expensive to operate in the US (if all other cost factors are constant between the a/c – which I doubt)

    • Airbus lost 100 A320NEO commitments from Lufthansa.
      Lufthansa operates A320 and B737. I guess Lufthansa Technik knows very well the true maintenance costs for both aircraft types. Lufthansa ordered 30 CEOs and 70 NEOs.

      • If you firm commitments you loose commitments and gain firm orders.
        Or do I overlook something here?

      • No you don’t.
        I’m still trying to figure out what this table by UBS is worth: http://leehamnews.files.wordpress.com/2013/06/ubs-paris-2013-day-1.jpg
        Airbus got 20 new commitments but “lost” 100 because the commitments were translated to orders. That is for UBS -80 commitments. From that kind of counting nobody can see a difference between canceled commitments or affirmed ones. I had to look at the orders to notice that it was a good “loss”.

        Btw. in 2008 UBS made the biggest single-year loss of any company in Swiss history, CHF20 billion.

    • Agree with your point that Lufthansa and other dual operators will know all of the performance, maintenance and other costs/numbers of both these aircraft types. But I guess it is in their interest to keep such things to themselves.

      If you are getting your lost 100 commitments from the USB chart, then perhaps you should also look at the orders column. USB is keeping track of orders and commitments together. I wonder if that includes MoUs?!

      • In the late nineties I used to meet regularly with Lufthansa Technik to review Boeing support. Nearly every time I would get beat up about the higher maintenance costs of the 737 compared to the A320. What was comical however, in my always friendly discussions with my Airbus peers, they complained that they, too, were always getting beat up about the higher maintenance costs of the A320 compared to the 737. We used to chuckle about it because Lufthansa used the same data set to support the two differing arguments and we understood it was Lufthansa’s way of keeping both of us on our toes.

  10. Just a little more info, from Michael O’Leary of Ryan Air:

    When Conner added that the 737 also is more fuel efficient, O’Leary interjected: “I actually disagree with that.” The 737 and A320 come out about even on overall fuel efficiency, he says, but the 737-800 and 737 Max 8 are more fuel efficient per seat.

  11. “Who is Boeing trying to convince by this? Airlines? ”

    It is truly comical; Boeing using Form 41 data to convince the airlines who know first hand the worthlessness of the Form 41 data because they were the culprits who submitted it.

    The data was fairly useless 20 years ago, before the DOT bowed to the US airlines and redacted all aircraft transaction data, thus making the data COMPLETELY useless.

    Back then we complained to the DOT about the data, and even offered to write a simple program for DOT to identify out-of-bounds input data so they could require correction.
    DOT was supremely uninterested.

    It seems there is a legal requirement for the US airlines to report the data, but no legal requirement at all that the data be complete, correct or factual.

  12. Pingback: Airbus comes up for air, responds to Boeing’s Form 41 reliance | Leeham News and Comment

  13. Interesting commentary , I Appreciate the facts . Does anyone know where my business can access a template 2013 Boeing X36217 example to type on ?

Leave a Reply

Your email address will not be published.