By Scott Hamilton
Special Coverage of the Boeing crisis
Jan. 24, 2024, © Leeham News: What began as a non-fatal accident with an Alaska Airlines 737 MAX 9 on Jan. 5 has blown into a full crisis for Boeing. The company was once considered the gold standard of commercial aviation.
Today, 171 737-9s remain grounded in the US by the Federal Aviation Administration (FAA). There is no end in sight as the FAA and National Transportation Safety Board (NTSB) investigate the accident of Flight 1282 in which a door plug (an inactive emergency exit) blew off the 10 week old Alaska MAX 9 on climb out from Portland (OR).
It is the second time the MAX has been grounded. All MAX 8s and MAX 9s were grounded from March 2019 for 21 months. This grounding only affects the MAX 9.
Evidence points to Boeing quality assurance flaws in final assembly. An anonymous Boeing employee posted on LNA a detailed scenario how Boeing failed its own processes in final assembly of the Alaska plane. (His post follows this article.)
The FAA is booting more inspectors on the ground at the 737 Renton factory. On Jan. 24, Boeing shut down the 737 assembly line for a “safety stand down.” CEOs of Alaska and United airlines, the two US carriers with the 171 MAX 9s on the ground, publicly eviscerated Boeing.
Related Articles
- Errors at Boeing (Seattle Times)
- Unplanned Removals, Installation Process at Boeing (Leeham News)
- Boeing turns to Admiral for the second time (Leeham News)
At the first aviation conference following the Alaska incident, the Aviation Week suppliers event, some speakers called for leadership changes at Boeing.
Culture, leadership failures at Boeing
Richard Aboulafia, a consultant with Aerodynamics Advisory, is a long-time critic of Boeing CEO David Calhoun. He’s called for Calhoun’s ouster—a move other panelists didn’t endorse. Nor did any call for the ouster of Stan Deal, the CEO of Boeing Commercial Airplanes. But several believe that heads will soon roll, including Calhoun’s.
But, said Michael Bruno, an editor of Aviation Week and a panelist, there is no obvious successor to either Calhoun and Deal. Boeing has a surprisingly thin bench to elevate an internal officer.
(The appointment in December of Stephanie Pope from CEO of Boeing Global Services to EVP and COO of The Boeing Co. caused puzzlement. While not mentioned at the AvWeek event, few understood her appointment. Her background is entirely finance and despite a long career at Boeing, she’s never been on the “front line.”)
Boeing’s repeated quality assurance failures at all three of its assembly plants in Renton (737s), Charleston (SC) (787s) and Everett (777s and 767s/KC-46As) has been well reported. Panelists repeated a long-stated criticism: Boeing’s culture shifted from engineering and quality final assembly to shareholder value.
And, some panelists said, there doesn’t seem to be a solution or a willingness to change. Aboulafia and Bruno compared Boeing’s problems to the morass of the US war in Iraq in 2004 when US troops reached not only an impasse but didn’t have equipment needed to prosecute the war.
Robert Gates was brought in then as Defense Secretary and he slashed through bureaucracy, effecting changes that benefited troops.
We have let our customers down
“We have let down our airline customers and are deeply sorry for the significant disruption to them, their employees and their passengers,” said Stan Deal, CEO of Boeing Commercial Airplanes.
This has become a familiar refrain at Boeing that predates Deal’s appointment. Boeing officials have said some variation of this with the first MAX grounding, the suspension of deliveries of the 787 (for 20 months, due to quality issues), delays on the 787 from 2008 and the 747-8 and during various labor strikes.
“I’m more than frustrated and disappointed,” Alaska CEO Ben Minicucci told NBC News two weeks after the accident. “I am angry. This happened to Alaska Airlines. It happened to our guests and happened to our people. And — my demand on Boeing is what are they going to do to improve their quality programs in-house.” Alaska has 65 MAX 9s.
“I’m disappointed that the manufacturing challenges do keep happening at Boeing. This isn’t new,” said Scott Kirby, the CEO of United Airlines. United has more MAX 9s than any other airlines—79.
“I have a lot of confidence in the people at Boeing,” Kirby said. “There are great mechanics, great engineers, [and a] great storied history. But they’ve been having these consistent manufacturing challenges, and they need to take action to get it.” Kirby pointed to an order for 277 MAX 10s that is already five years late. Certification by the FAA isn’t remotely in sight and is likely to be delayed further. He said United is now planning its fleet expansion without the MAX 10, on the assumption deliveries will be further delayed.
“The MAX 9 grounding is probably the straw that broke the camel’s back for us,” Kirby said. “We’re going to at least build a plan that doesn’t have the MAM 10 in it. Now, we’ll hope that Boeing gets it certified at some point, but we’re going to build an alternative plan that just doesn’t have the MAX 10 in it.”
Safety stand down
On Jan. 23, Boeing announced a safety stand down for the Renton factory. This is the full announcement:
First, Boeing’s 737 factory teams will hold a “quality stand down” in Renton, Wash., this Thursday, Jan. 25. During the session, production, delivery and support teams will pause for a day so employees can take part in working sessions focused on quality. This is part of the immediate quality actions recently shared by Boeing Commercial Airplanes CEO Stan Deal.
Here are excerpts from internal communication to all Commercial Airplanes employees:
- The first of the stand downs will be held Thursday for the 737 program. Production, delivery and support efforts will pause for a day so teammates can take part in working sessions focused on quality.
- The sessions allow all teammates who touch the airplane to “pause, evaluate what we’re doing, how we’re doing it and make recommendations for improvement,” said Stan Deal, BCA president and CEO.
- Over the next few weeks, quality stand downs will take place at all commercial airplane factories and our fabrication sites. During the stand downs, teammates will participate in hands-on learning, reflection and collaboration to identify where quality and compliance can be improved and create actionable plans that will be tracked to closure. [End.]
A new, independent safety committee lead by a retired rear admiral has been appointed. This is the second time such a committee has been formed; the first was in September 2019 during the first MAX crisis. Boeing declined to explain what the first panel achieved and why a second panel is necessary.
Whistleblower reveals what happened
A reader of LNA posted a long comment on our story about Unplanned Removals and Reinstallation processes. He’s anonymous. LNA checked his post with our own Boeing contacts, who said the comment accurately reflects Boeing’s process.
The whistleblower’s post is here in its entirety. It has been slightly edited for clarity.
Current Boeing employee here – I will save you waiting two years for the NTSB report to come out and give it to you for free: the reason the door blew off is stated in black and white in Boeings own records. It is also very, very stupid and speaks volumes about the quality culture at certain portions of the business.
A couple of things to cover before we begin:
Q1) Why should we believe you?
A) You shouldn’t, I’m some random throwaway account, do your own due diligence. Others who work at Boeing can verify what I say is true, but all I ask is you consider the following based on its own merits.
Q2) Why are you doing this?
A) Because there are many cultures at Boeing, and while the executive culture may be thoroughly compromised since we were bought by McDonnell Douglas, there are many other people who still push for a quality product with cutting edge design. My hope is that this is the wake up call that finally forces the Board to take decisive action, and remove the executives that are resisting the necessary cultural changes to return to a company that values safety and quality above schedule.
Why did this happen?
With that out of the way…why did the left hand (LH) mid-exit door plug blow off of the 737-9 registered as N704AL?
Simple- as has been covered in a number of articles and videos across aviation channels, there are 4 bolts that prevent the mid-exit door plug from sliding up off of the door stop fittings that take the actual pressurization loads in flight, and these 4 bolts were not installed when Boeing delivered the airplane. Our own records reflect this.
The mid-exit doors on a 737-9 of both the regular and plug variety come from Spirit already installed in what is supposed to be the final configuration and in the Renton factory. There is a job for the doors team to verify this “final” install and rigging meets drawing requirements. In a healthy production system, this would be a “belt and suspenders” sort of check, but the 737 production system is quite far from healthy. It’s a rambling, shambling, disaster waiting to happen.
As a result, this check job that should find minimal defects has in the past 365 calendar days recorded 392 nonconforming findings on 737 mid fuselage door installations (so both actual doors for the high density configs, and plugs like the one that blew out). That is a hideously high and very alarming number.
If our quality system on 737 was healthy, it would have stopped the line and driven the issue back to supplier after the first few instances. Obviously, this did not happen.
The accident aircraft
Now, on the incident aircraft this check job was completed on 31 August 2023, and did turn up discrepancies, but on the RH side door, not the LH that actually failed. I could blame the team for missing certain details, but given the enormous volume of defects they were already finding and fixing, it was inevitable something would slip through. And on the incident aircraft, something did. I know what you are thinking at this point, but grab some popcorn because there is a plot twist coming up.
The next day on 1 September 2023 a different team (remember 737s flow through the factory quite quickly, 24 hours completely changes who is working on the plane) wrote up a finding for damaged and improperly installed rivets on the LH mid-exit door of the incident aircraft.
A brief aside to explain two of the record systems Boeing uses in production. The first is a program called CMES which stands for something boring and unimportant but what is important is that CMES is the sole authoritative repository for airplane build records (except on 787 which uses a different program). If a build record in CMES says something was built, inspected, and stamped in accordance with the drawing, then the airplane damn well better be per drawing.
The second is a program called SAT, which also stands for something boring and unimportant but what is important is that SAT is *not* an authoritative records system. It’s a bulletin board where various things affecting the airplane build get posted about and updated with resolutions. You can think of it sort of like a idiot’s version of Slack or something.
Wise readers will already be shuddering and wondering how many consultants were involved, because, yes SAT is a *management visibility tool*. Like any good management visibility tool, SAT can generate metrics, lots of metrics, and Boeing managers love their metrics. As a result, SAT postings are the primary topic of discussion at most daily status meetings, and the whole system is perceived as being extremely important despite, I reiterate, it holding no actual authority at all.
Fixing the accident aircraft
We now return to our incident aircraft, which was written up for having defective rivets on the LH mid-exit door. Now as is standard practice in Renton (but not to my knowledge in Everett on wide bodies), this write-up happened in two forms, one in CMES, which is the correct venue, and once in SAT to “coordinate the response” but really as a behind-covering measure so the manager of the team that wrote it can show his boss he’s shoved the problem onto someone else.
Because there are so many problems with the Spirit build in the 737, Spirit has teams on site in Renton performing warranty work for all of their shoddy quality, and this SAT promptly gets shunted into their queue as a warranty item.
Lots of bickering ensues in the SAT messages, and it takes a bit for Spirit to get to the work package. Once they have finished, they send it back to a Boeing QA for final acceptance, but then Malicious Stupid Happens! The Boeing QA writes another record in CMES (again, the correct venue) stating (with pictures) that Spirit has not actually reworked the discrepant rivets, they *just painted over the defects*. In Boeing production speak, this is a “process failure”. For an A&P mechanic at an airline, this would be called “federal crime”.
Presented with evidence of their malfeasance, Spirit reopens the package and admits that not only did they not rework the rivets properly, there is a damaged pressure seal they need to replace (who damaged it, and when it was damaged is not clear to me). The big deal with this seal, at least according to frantic SAT postings, is the part is not on hand, and will need to be ordered, which is going to impact schedule, and (reading between the lines here) Management is Not Happy.
Pressure seal issue
However, more critical for purposes of the accident investigation, the pressure seal is unsurprisingly sandwiched between the plug and the fuselage, and you cannot replace it without opening the door plug to gain access.
All of this conversation is documented in increasingly aggressive posts in the SAT, but finally we get to the damning entry which reads something along the lines of “coordinating with the doors team to determine if the door will have to be removed entirely, or just opened. If it is removed then a Removal will have to be written.” Note: a Removal is a type of record in CMES that requires formal sign off from QA that the airplane been restored to drawing requirements.
If you have been paying attention to this situation closely, you may be able to spot the critical error: regardless of whether the door is simply opened or removed entirely, the 4 retaining bolts that keep it from sliding off of the door stops have to be pulled out.
A removal should be written in either case for QA to verify install, but as it turns out, someone (exactly who will be a fun question for investigators) decides that the door only needs to be opened, and no formal Removal is generated in CMES (the reason for which is unclear, and a major process failure).
Therefore, in the official build records of the airplane, a pressure seal that cannot be accessed without opening the door (and thereby removing retaining bolts) is documented as being replaced, but the door is never officially opened and thus no QA inspection is required.
Verification job not opened
This entire sequence is documented in the SAT, and the nonconformance records in CMES address the damaged rivets and pressure seal, but at no point is the verification job reopened, or is any record of removed retention bolts created, despite it this being a physical impossibility.
Finally with Spirit completing their work to Boeing QAs satisfaction, the two rivet-related records in CMES are stamped complete, and the SAT closed on 19 September 2023. No record or comment regarding the retention bolts is made.
I told you it was stupid.
So, where are the bolts? Probably sitting forgotten and unlabeled (because there is no formal record number to label them with) on a work-in-progress bench, unless someone already tossed them in the scrap bin to tidy up.
There’s lots more to be said about the culture that enabled this to happened, but that’s the basic details of what happened, the NTSB report will say it in more elegant terms in a few years.
