Boeing is breaking the rules of crisis management and making what may well prove to be a bad “bet the company” wager.
The gold standard, as most readers know well, is Johnson & Johnson’s response to 1982 tampering with its product, Tylenol. Seven people died in Chicago due to Tylenol containers having been put on the shelves with cyanide pills inside. A few more people died in what were believed to have been copycat incidents.
Tylenol was Johnson & Johnson’s most important product and then the nation’s leading OTC painkiller. Johnson & Johnson immediate issued warnings, telling consumers not to use any Tylenol until the company had ascertained how big the tampering problem was. It recalled all Tylenol nationwide, cooperated with the police, and set up an 800 number for consumers.
Johnson & Johnson re-introduced Tylenol with triple-safety packing that has become an industry norm. Even though its market share plunged during the crisis, it rebounded to an even higher level a year later.
As one account of the Johnson & Johnson crisis response stressed, the company put safety first:
Johnson & Johnson chairman, James Burke, reacted to the negative media coverage by forming a seven-member strategy team. The team’s strategy guidance from Burke was first, “How do we protect the people?” and second “How do we save this product?”
It isn’t hard to infer that this is not how Boeing is setting its priorities. It is important to recognize that the global grounding of the 737 Max is the result of trying to compensate for questionable, profit-driven engineering choices by adding a safety feature (the MCAS software system) and then going cheap on that, in terms of selling planes not kitted out fully and acting as if it was perfectly fine to install software that could take control of the plane and barely tell pilots about it. Two paragraphs more than 700 pages into a manual does not qualify as anything approaching adequate disclosure.
Boeing is taking steps that look designed to appear adequate, when given the damage done to the 737 Max and its brand generally, this isn’t adequate. No one has any reason to give Boeing the benefit of the doubt. The scale of this failure is so large that it’s called the adequacy of FAA certifications into question. Until this fiasco, aviation regulators deferred to the judgment of regulator in the country where the manufacturer was headquartered. But with China embarrassing the FAA by (correctly) being the first to ground the 737 Max, foreign regulators will make their own checks of Boeing’s 737 Max fixes….and that practice may continue with other US-origin planes unless Boeing and the FAA both look to have learned a big lesson. So far, Boeing’s behavior says not.
One does not have to do much in the way of review to see that Boeing has lost the plot.
Boeing compromised on sound engineering with the 737 Max. Recall the origins of the problem: Boeing was at risk of losing big orders to a more fuel-efficient Airbus model. Rather than sacrifice market share, Boeing put more fuel-efficeint, larger engines on the existing 737 frames. The placement of the engine created a new safety risk, that under some circumstances, the plane could “nose up” at such a steep angle as to put it in a stall. The solution was to install software called MCAS which would force the nose down if the “angle of attack” became too acute.
Before getting to today’s updates, experts have deemed the 737 Max design to be unsound. For
The word “kludge” keeps coming up when pilots and engineers discuss Boeing’s 737 Max, from Quartz:
Again and again, in discussions of what has gone wrong with Boeing’s 737 Max planein two deadly crashes within five months, an unusual word keeps coming up: kludge.
Merriam-Webster defines kludge—sometimes spelled kluge—as “a haphazard or makeshift solution to a problem and especially to a computer or programmingproblem.” Oxford defines it as, in computing, “A machine, system, or program that has been badly put together, especially a clumsy but temporarily effective solution to a particular fault or problem.”…
In the case of the 737 Max, it’s the combination of how two separate problems interacted—a plane whose design introduced aerodynamics issues and what now appears to have been a poorly designed anti-stall system—that seems to be drawing many to turn to Granholm’s term. The problems were compoundedin many ways, including by the fact that pilots were not told of or trained for the Maneuvering Characteristics Augmentation System (MCAS) before the Lion Air crash, which killed all 189 on board.
“My concern is that Boeing may have developed the MCAS software as a profit-driven kludge to mitigate the Max 8’s degraded flight characteristics due to the engine relocation required to maintain ground clearance,” commented Philip Wheelock on a New York Times story about the plane’s certification process this week. “Not convinced that software is an acceptable solution for an older design that has been pushed to its inherent aeronautical design limits.”
“Indeed, it seems the 737 MAX was a kludge to an existing design, and that MCAS was a kludge on top of that,” said a commenter on Hackaday.
Lambert found more damning takes, which he featured in Water Cooler yesterday. First from the Seattle Times:
Boeing has long embraced the power of redundancy to protect its jets and their passengers from a range of potential disruptions, from electrical faults to lightning strikes. The company typically uses two or even three separate components as fail-safes for crucial tasks to reduce the possibility of a disastrous failure. So even some of the people who have worked on Boeing’s new 737 MAX airplane were baffled to learn that the company had designed an automated safety system that abandoned the principles of component redundancy, ultimately entrusting the automated decision-making to just one sensor — a type of sensor that was known to fail. Boeing’s rival, Airbus, has typically depended on three such sensors. “A single point of failure is an absolute no-no,” said one former Boeing engineer who worked on the MAX, who requested anonymity to speak frankly about the program in an interview with The Seattle Times. “That is just a huge system engineering oversight. To just have missed it, I can’t imagine how.”
And the second, from software developer Greg Travis…who happens also to be a pilot and aircraft owner:
That no one who wrote the MCAS software for the 737 MAX seems to have even raised the issue of using multiple inputs, including the opposite angle of attack sensor, in the computer’s determination of an impending stall is mind-blowing. As a lifetime member of the software development fraternity, I don’t know what toxic combination of inexperience, hubris, or lack of cultural understanding led to this. But I do know that it’s indicative of a much deeper and much more troubling problem. The people who wrote the code for the original MCAS system were obviously terribly far out of their league and did not know it. How can we possibly think they can implement a software fix, much less give us any comfort whatsoever that the rest of the flight management software, which is ultimately in ultimate control of the aircraft, has any fidelity at all?
And we’re giving short shrift to how Boeing compounded the problem, for instance, by making it an upcharge to have the 737 Max have a light showing that its angle of attack sensors disagreed (the planes did have two, but bizarrely, only one would be giving data to the MCAS system on any day), or hiding the fact that there was a new safety automated safety system in two paragraphs after page 700 in the flight manual. As Wall Street Journal reader Erich Greenbaum said in comments on an older article, How Boeing’s 737 MAX Failed:
No – this isn’t about “planes that fly by themselves.” It’s about an airplane manufacturer that put engines on an airframe they weren’t designed for, having to add a flight control override to guard against said airplane’s new tendency to nose up, and then adding insult to injury by driving that system with a single sensor when two are available. Oh – and charging airlines extra for the privilege of their pilots being told when one of those sensors is providing bad data.
The 737 Max has gotten a bad name…not just for itself but also for the airlines that were big buyers. Southwest had taken the most 737 Max deliveries, and American was second. I happened to be looking at American for flights last night. This is what I got when I went to aa.com:
I came back to the page later to make sure I hadn’t hit the 737 Max message randomly, by loading the page just when that image came up in a cycle….and that doesn’t appear to be the case. I landed on the 737 Max splash a second time.
This result suggests that American has gotten so many customer queries about the 737 Max that it felt it had to make providing information about it a priority. If you click through, the next page explains how all 737 Max planes have been grounded, that American is using other equipment to fly on routes previously scheduled for those planes, but it has still had to cancel 90 flights a day.
Evidence is mounting that the MCAS system was responsible for the Ethopian Air crash in addition to the Lion Air tragedy. From the Wall Street Journal this evening:
Officials investigating the fatal crash of a Boeing Co. BA 0.06% 737 MAX in Ethiopia have reached a preliminary conclusion that a suspect flight-control feature automatically activated before the plane nose-dived into the ground, according to people briefed on the matter, the first findings based on data retrieved from the flight’s black boxes.
The emerging consensus among investigators, one of these people said, was relayed during a high-level briefing at the Federal Aviation Administration on Thursday, and is the strongest indication yet that the same automated system, called MCAS, misfired in both the Ethiopian Airlines flight earlier this month and a Lion Air flight in Indonesia, which crashed less than five months earlier. The two crashes claimed 346 lives.
Boeing is doubling down on its mistakes. The lesson of the Tylenol poisoning is that if a company has a safety problem, even if it isn’t its fault, it needs to do everything it can to rectify the defects and protect customers. If there is any doubt, the company needs to err of the side of safety.
Here, unlike with Johnson & Johnson, the failings that led to 737 Max groundings all originated with Boeing. Yet rather than own the problems and go overboard on fixing them to restore confidence in the plane and in Boeing, Boeing is acting as if all it has to put in place are merely adequate measures.
Reuters, which has a bias towards understatement, has an atypically pointed discussion of Boeing’s refusal to recommend pilot simulator training for the MCAS:
Boeing Co said it will submit by the end of this week a training package that 737 MAX pilots are required to take before a worldwide ban can be lifted, proposing as it did before two deadly crashes that those pilots do not need time on flight simulators to safely operate the aircraft.
In making that assessment, the world’s largest planemaker is doubling down on a strategy it promoted to American Airlines Group Inc and other customers years ago. Boeing told airlines their pilots could switch from the older 737NG to the new MAX without costly flight simulator training and without compromising on safety, three former Boeing employees said.
Specifically, the Wall Street Journal reported that Southwest, which is the biggest buyer of the 737 Max, got Boeing to agree to a financial penalty if the new plane required additional simulator training:
The company had promised Southwest Airlines Co. , the plane’s biggest customer, to keep pilot training to a minimum so the new jet could seamlessly slot into the carrier’s fleet of older 737s, according to regulators and industry officials.
[Former Boeing engineer Mr. [Rick] Ludtke [who worked on 737 MAX cockpit features] recalled midlevel managers telling subordinates that Boeing had committed to pay the airline $1 million per plane if its design ended up requiring pilots to spend additional simulator time. “We had never, ever seen commitments like that before,” he said.
I’ve never flown Southwest and now I will make sure never to use them.
I hope the pilots in our readership speak up, but as a mere mortal, I’ve very uncomfortable with pilots being put in a position of overriding a system in emergency conditions when they haven’t even test driven it. When I learn software, reading a manual is useless save for learning what the program’s capabilities are. In order to be able to use it, I have to spend time with it, hands on. Computer professionals tell me the same thing. It doesn’t seem likely that pilots are all that different.
In other words, Boeing’s refusal to recommend simulator training looks to be influenced by avoiding triggering a $31 million penalty payment to Southwest. This is an insane back-assward sense of priorities. Boeing had over $10 billion in profits in 2018. A $31 million payment isn’t material and would almost certainly be lower after tax.
Boeing does not seem to comprehend that it is gambling with its future. What if international flight regulators use the Max 737 as a bloody flag and refuse to accept FAA certifications of Boeing planes, or US origin equipment generally? Do you think for a nanosecond that the European and Chinese regulators wouldn’t use disregarding the FAA as a way to advance their interests? Europe would clearly give preference to Airbus, and the Chinese could use Boeing to punish the US for going after Huawei.
Boeing’s comeuppance is long overdue. The company’s decision to break its union, outsource, and move to Chicago as a device for shedding seasoned employees was a clear statement of its plan to compromise engineering in the name of profit. Something like the Max 737 train wreck was bound to happen.