Procedures Must Be Followed to Halt Mounting Toll of Maintenance Mistakes

'All is not well,' safety board member John Goglia declares
Airline flying is becoming less safe and maintenance errors are playing an increasing role in the reduced operational level of safety, according to a new study of the role maintenance mistakes play in aircraft accidents and incidents.

The examination marks an unprecedented effort to take a comprehensive look at the impact of maintenance error on operational safety, said professor Gary Eiff, who directed the study at Purdue University's Department of Aviation Technology in Lafayette, Ind.

The findings - that maintenance errors were a contributory factor in far more incidents and accidents than previously thought - come at a time when financially-strapped airlines are contracting out more maintenance to save money, and in a period when federal oversight of repair stations has been found lacking (see ASW, Jan. 21, 2003, Oversight Needs to Keep Pace With Expansion of Contract Maintenance).

Although adherence to approved procedures is one of the most effective defenses against errors, the Purdue study found that in 76 percent of the maintenance-related events, failure to follow established procedures was cited as a contributing factor.

Although inspection is often the last line of defense, the study found that inadequate or missing inspections were a contributing factor in nearly 20 percent of the maintenance-related accidents and incidents.

Organizational factors, such as inadequate procedures, documentation, training and supervision accounted for more than 15 percent of the maintenance-related contributions to accidents and incidents.

The study was undertaken with the encouragement of John Goglia, the resident maintenance expert on the National Transportation Safety Board (NTSB). The NTSB has found maintenance error contributed to a number of recent fatal air crashes, notably the Jan. 21, 2000, fatal crash of an Alaska Airlines [ALK] MD-83, the Feb. 16, 2000, crash of an Emery Worldwide Airlines DC-8 freighter, and the Jan. 8, 2003, crash of an Air Midwest Beech 1900D (see ASW, March 1). Together, these three accidents killed 112 people.

As a consequence of the investigations into these accidents, Goglia pushed for a wider examination of maintenance error. The Purdue study is a result of this concern that recent maintenance-related crashes are symptoms of a more widespread and sinister trend.

"Since coming to the safety board, and seeing the raw data, my gut was telling me the rate of maintenance-related problems was much higher than generally believed," Goglia said. "This study tells us that all is not well."

The Purdue study found that, indeed, there are worrisome trends that ought to prompt a hard look at the level of training and oversight of aircraft maintenance. The safety culture in aviation maintenance also is being scrutinized in a separate study under way at Missouri's St. Louis University (see ASW, April 12). According to a draft copy of the Purdue study obtained by ASW, "Maintenance contributions to aviation accidents and incidents are increasing."

"While the increase is not at an alarming rate, we should take action now to arrest this trend," the report said. One of the more significant revelations in the report is that while incidents are trending toward fewer per year, the accident trend is in the opposite direction. Thus, while the total number of accidents and incidents remains relatively the same, the report notes that "the criticality of [the] outcome of such events is becoming more severe."

The assertion that maintenance errors are on the rise is a contributing factor in the larger conclusion in the Purdue study: "As a whole, the air carrier industry is experiencing a slight trend toward becoming less safe as a mode of transportation."

This finding flies in the face of the fervently held belief that the safety level is improving. For example, at the recent Federal Aviation Administration (FAA) Forecast Conference, Nicholas Sabatini, associate FAA administrator for regulation and certification, asserted that 2003 was the safest year ever (see ASW, March 29). However, Sabatini's assertion was based on fatal accidents, of which there were only two in 2003, while the 50 other non-fatal accidents for 2003 pushed the total number of air carrier accidents last year to the highest ever recorded by the NTSB.

How the system's performance is measured can largely drive perceptions about safety. Eiff and his graduate students, who did the grunt work in the Purdue study, looked at all accidents and all incidents in the NTSB database over a 20-year period from 1982 to 2002. The Purdue study focused on scheduled passenger operations; it excluded cargo aircraft and charter operations.

The purpose of the study was to determine if maintenance played a more prominent role in aviation accidents and incidents than was previously thought.

The Purdue researchers analyzed roughly 1,300 NTSB records of airline accidents and incidents. The study found that maintenance problems were factors in nearly a third (29 percent) of the events. One of the key assumptions in the study was that mechanical failures were maintenance-related. "While the reports do not directly link these mechanical failures to maintenance, it is reasonable to believe that many result from maintenance shortcomings," the report said.

The study found that maintenance was a contributory factor in far more cases than previously thought. For example, a recently published Boeing [BA] safety summary attributes only about 3 percent of crashes to faulty maintenance. The Purdue study suggests that maintenance problems may be some 10 times greater.

A major reason for the difference is that the Boeing data includes only hull losses. Not all accidents are hull losses. In addition, according to Dustin Wilcox, one of the students involved in the Purdue study, "We looked at incidents as well, which might be described as accidents that got lucky."

Goglia believes the findings of the Purdue study are closer to the reality. "The students who did the work did not change the NTSB findings in all those accident and incident reports," Goglia said. "This was a straightforward exercise, extracting what the NTSB said and compiling it. That is significant, and a big reason why we need to pay attention to the results of this data-mining effort." As an example, the data mining found more than 100 cases of nose landing gear failures to extend for landing.

In a telephone interview, Goglia said the study's findings are consistent with other inquiries. For example, an engine manufacturer found that 50 percent of engine-related turnbacks (return to departure airfield) were caused by installation error, Goglia said.

What can be done? "We need to start putting some mitigations in place," Goglia said. He recalled that after a spate of crashes from pilot error, primarily from a failure to follow procedures, aircrews were enjoined to stick to procedures.

Now, for maintenance, Goglia said, "We need to pound into everybody's head the importance of following procedures." Secondly, he said, the procedures themselves need to be evaluated for currency, consistency and ease of understanding. Following the Air Midwest crash, which was attributed to improperly rigged elevator control cables, the relevant section of the Beech 1900D maintenance manual was revised significantly (see ASW, March 1, , What a Difference an Accident Makes). Goglia, e-mail gogliaj@ntsb.gov; Eiff, e-mail geiff@purdue.edu

Growing Concerns
Research at several airline operations suggested that:

Growing numbers of aircraft delays were maintenance-related.
An increasing number of aircraft turn-backs and diversions were due to maintenance.
Several recent accidents and incidents occurred with maintenance as a causal factor.
Source: Purdue Univ.


--------------------------------------------------------------------------------


Space Agency Seeks Safety Culture Change

Survey finds fear of retribution for voicing safety concerns
Many employees at the National Aeronautics and Space Administration (NASA) believe that speaking up about a perceived safety issue could jeopardize their careers, according to a new survey of agency employees.

The survey's findings have direct applicability to the aviation industry, where cost and schedule pressures also play a significant role akin to the pressures of space launches, and where a widespread cultural commitment is vital to the robustness of safety programs. With only a few word changes, the survey could be employed by any airline to evaluate its safety culture. For NASA, the results showed a dismaying gap between declaratory rhetoric from the top and management's credibility as seen from the bottom up. The survey documented many strengths of NASA's culture but, nonetheless, a failure to communicate, and a reluctance of many in the ranks to speak out. Similar instances - both good and bad - abound in the airline industry.

It is ironic that NASA is seeing safety culture issues while at the same time the agency manages - on behalf of the Federal Aviation Administration (FAA) - the aviation safety reporting system (ASRS) for the aviation community. Under this system, employees can voluntarily submit aviation incident reports, safety issues and concerns (see http://asrs.arc.nasa.gov/overview.htm#1).

The NASA survey, which allowed respondents to answer anonymously, was conducted by a contractor, Behavioral Science Technology (BST), of Ojai, Calif., as part of a $10 million multi-year contract. NASA retained BST to take a hard look at the agency's safety culture. This effort is an outgrowth of the loss of Shuttle Columbia in February 2003 and the findings of the Columbia Accident Investigation Board (CAIB). The CAIB uncovered numerous deficiencies in hazard analysis and safety management that parallel similar problems in the aviation industry (see ASW, Sept. 8, 2003, Special Report: Risk Tolerance).

The CAIB in particular criticized the safety culture at NASA, saying the prevailing norm "was a reverse of the usual circumstance - instead of having to prove it was safe to fly, [engineers] were asked to prove that it was unsafe to fly." In effect, the CAIB charged that NASA's safety horticulture grew a tangled, choking bureaucracy that was complicit in Columbia's loss. NASA's disjointed safety "programs" had grown into an inward-looking and cost-conscious ideological facade.

BST conducted the survey electronically via the Internet in mid-February (with protocols to assure anonymity). The firm received some 9,500 responses from nearly half of NASA's workforce (contract personnel supporting NASA were not involved). Given that the survey was open for only seven days, the response suggests a high level of mission commitment as well as concern about the agency's safety culture. BST will use the responses to craft a three-year program redressing NASA's cultural shortcomings. It is the first agency-wide survey of the safety culture in many years.

The results show that while NASA is committed to transforming its safety culture, the road ahead will be difficult, as evidenced by the findings in the BST report:

"Safety is something to which NASA personnel are strongly committed in concept, but NASA has not yet created a culture that is fully supportive of safety. Open communication is not yet part of the norm, and people do not feel fully comfortable raising safety concerns to management."

While the overall response on the management credibility scale was an encouraging 77th percentile, the survey results for one notable subgroup were not nearly so heartening. Responders who declined to identify their organization within NASA tended to rate the agency lower in the 11 major areas the survey explored. The BST report said:

"The largest difference is on the management credibility scale, which is expected among people who are especially concerned to protect their anonymity.

"The responses of this group ... also differ from the overall survey response on many of the NASA-specific questions. These differences are a 'red flag.' Where open communications are recognized as critical to the culture NASA wishes to create, having a group that feels inhibited from speaking up is problematic. This tends to confirm that an important target for the culture change effort will be building leadership behaviors at all levels that encourage and reinforce open communications."

The survey asked employees to respond on a scale of one to five, in which one meant "strongly disagree" and five signified "strongly agree." Responses ranged on average from 3.60 to 4.09. The BST report said, "Given the objective of cultural excellence, it would be desirable to achieve consistent scores above 4.5 on these scales."

Of interest, the survey revealed that the higher the person is in the organization, the greater the perception that organizational support, management credibility and the safety climate are shipshape. For example, the "safety climate" scores tended to correlate with the respondent's rank within the chain of command:


Executive Leader 4.21
Senior Leader 4.16
Manager or Supervisor 3.98
Team/Group Leader 3.89
Individual Contributor 3.89

In addition, according to the BST report, the survey results indicate that employees "see the least consistency between words and actions at levels highest above them."

The report also noted, "There is a clear perception that budget constraints compromise engineering and mission safety." This finding emanated from five specific questions included in the survey. "With the current safety climate, the lack of firm agreement in the responses indicates continued uncertainty about NASA leadership's 'real' intent," the report noted.

The survey results suggest that the NASA activity centers more closely engaged in flight operations have a healthier safety culture than NASA headquarters, which is further removed from flight operations but perhaps more aware of budget and schedule problems. Survey respondents at Kennedy Space Center and Johnson Space Center gave some of the highest marks to the NASA safety culture, while the score for headquarters was markedly lower. Similar results were obtained for the question regarding upward communications about safety, where the ranking for NASA headquarters was among the lowest of the sites surveyed.

NASA Administrator Sean O'Keefe embraced the survey results. In an April 13 telephonic roundtable with NASA employees, O'Keefe said the BST work provides "a very clear indication that ... what we say about the foundation of safety ... and what we do is a good commentary, but we don't actually live it as deeply as we say we do." O'Keefe had previously laid out five core values for safety, and BST has fleshed out his concept with a proposed philosophy of safety to underpin the cultural reform.

A transcript of the roundtable vividly underscores the dry and abstract statistical results of the survey. The effort to reform the culture at NASA, where expressing a minority view is widely regarded as inviting transfer to a career-ending backwater job, faces monumental skepticism among the workforce.

Given the desire to dramatically change the safety culture in a three-year time span, BST has taken the approach, endorsed by O'Keefe, that behavioral changes must start at the top. BST is presently involved in conducting one-on-one coaching sessions with NASA senior leadership, including O'Keefe. Personal action plans will be developed for each official.

Thomas Krause, BST chairman of the board, said the objective of the coaching sessions is to "help align the behavior of the leader with his or her stated objectives."

"They may not be aware of behaviors that help or hinder how they are perceived," Krause said. He added that this coaching is "more of an awareness issue than a motivational question."

"We are just starting the coaching process, and the method is fairly extensive," he told ASW. "For some, it may be as simple as doing a better job communicating what they already believe and value. For others, it may involve a deep examination of beliefs and values in order to better align themselves, and their behaviors, with the objective of the agency."

Indeed, a highly motivated tyrant who brooks no dissent may embody behaviors that are the very antithesis of the more open communications culture NASA leaders say they want. Rather than an inward-looking and cost-conscious ideological facade, the agency's leaders say they want a dissent-seeking and safety-conscious ideological reality. >> Krause, e-mail tom.krause@bstsolutions.com. For the full BST report, see http://www.nasa.gov/pdf/57382main_culture_web.pdf
Cultural Contrasts in the Airline Industry
"The culture [at this airline] is very oppressive. Management is always right, no matter how wrong they are. They believe the way to motivate their mechanics is to threaten layoffs or by farming out repair work." - Aircraft and powerplant mechanic

"Management's lack of respect really stresses me. They will not listen to input from the worker unless it is their idea - or unless it involves an outside study done by an independent agency." - Aircraft mechanic

"I have wonderful managers who truly have an 'open door' policy. If something is bothering me, I know I can talk to them about it. Best of all, I can honestly express doubts and frustrations without being told, 'You just don't have a good attitude.' "

- Training designer

"Immediate supervisor: quiet, firm. Upper managers: untrustworthy liars." - Captain

"We live by our corporate culture: Safety, Caring, Passion, Fun, Integrity. We always look to the values to decide if things are 'right.' " - Captain

Source: Duh! Lessons in Employee Motivation That Every Business Should Learn from the World's Best (and Worst) Airlines, by David Forward, ReachForward Publishing Group, N.J., 2003

'A Broken Safety Culture'

The CAIB report devoted an entire section to the organizational factors contributing to the loss of Shuttle Columbia. Page numbers in parentheses below refer to pages in the CAIB report (extracts):

NASA's initial briefings to the CAIB on its safety programs espoused a risk-averse philosophy that empowered any employee to stop an operation at the mere glimmer of a problem. Unfortunately, NASA's views of its safety culture in those briefings did not reflect reality. (P. 177)
Engineers and managers incorporated worsening anomalies into the engineering experience base, which functioned as an elastic waistband, expanding to hold larger deviations from the original design. Anomalies that did not lead to catastrophic failure were treated as a source of valid engineering data that justified further flights. These anomalies were translated into a safety margin that was extremely influential, allowing engineers and managers to add incrementally to the amount of damage that was acceptable. (P. 196)
Multiple job titles disguised the true extent of safety personnel shortages. The CAIB found cases in which the same person was occupying more than one safety position, which compromised any possibility of safety organization independence because the jobs were established with built-in conflicts of interest. (P. 199)
The organization structure and hierarchy blocked effective communication of technical problems. Signals were overlooked, people were silenced, and useful information and dissenting views on technical issues did not surface at higher levels. (P. 201)
Source: CAIB, see http://boss.streamos.com/download/caib/report/web/full/caib_report_volume1.pdf

'Engage the Collective IQ'

Roundtable discussion April 13 of employee perceptions (extracts)

Cast of Characters:

Sean O'Keefe, NASA Administrator
Jim Weatherby, NASA astronaut
Scott Stricoff, BST
Questioners - various NASA employees

O'Keefe: We need to create a climate ... in which open communications is not only permissible, it is actively encouraged.

Questioner: A lot of stuff that is in this report, I read and I said no kidding. I have gotten five or six responses before I left the house this morning from [people] saying, 'Yeah, this [report] is great, but don't quote me.'

Weatherby: The first thing that I think we should do is to find out why people are reluctant to speak up.

In large part, it is not because they are afraid of being fired. They are afraid of being rendered ineffective and being moved to a different job, which to somebody at NASA is the equivalent of being fired.

One of the things we can do is create an environment where a launch hold is viewed as a successful manifestation of a good safety culture.

You don't have to do what the person is voicing dissent or a minority opinion. But you must engage that person and find out what is going on behind what they are saying. If you do that, you engage the collective IQ of the whole group, not just the leaders or the supervisors. When we engage the collective IQ, you come up with a better decision.

Questioner: How is this different from every darn management fad that the agency has been through in the last 15 years?

Stricoff: If I want individuals to be more open in their communications upward, I can't get that by training them, and I can't get that by telling them or putting posters on the wall. Source: http://www.nasa.gov/pdf/57770main_roundtable.pdf

'Committed to Safety as Parents to Their Children'
Attributes of a culture of safety excellence (extracts):

We proudly accept the challenge of the experimental exploration of space, knowing of its inherent risks while convinced of its over-riding value to our country. We are committed to mission safety as parents are to their children. We accept the responsibility to manage the risks of space flight, such that no accident will ever happen through lack of diligence.

1. Open and clear communication is encouraged and modeled

People at every level of the organization must be committed to the free and unobstructed flow of information up and down within the organization. This means having the courage to question assumptions, and the willingness to ask even the seemingly obvious questions, to listen actively, and be ready to learn. It describes a value ... that is unimpeded by concern about "looking bad."

2. Rigorously informed judgment is the sole basis for decision-making

The only basis for confidence is properly understood data that meet safety and reliability criteria.

3. Personal responsibility is taken by each individual

Each individual is responsible for upholding a safety-supporting culture in what we do and how we do it. It is unacceptable to assume that someone else will handle your issue or questions.

4. Integrated technical and managerial competence is our shared value

Mission success is accomplished by integrating all aspects of program management: safety, engineering, cost and schedule, across functional and organizational lines.

5. Individual accountability is the basis for high reliability

Mission safety results from actions, not just words. Our credibility is built on the consistency between our words and our actions. Source: BST report, Appendix A


--------------------------------------------------------------------------------