2.1The first side: accidents made by the organisation
Barry Turner's Man-Made Disasters (1978) proposed that a disaster has an incubation period. During it, events that are at odds with the accepted beliefs about a hazard happen and are not noticed, or are noticed and misread. They accumulate until a precipitating event. For Turner, disasters are failures of foresight produced by organisations, not acts of fate (Turner 1978). The weight of the account falls on the gap between what is happening and what the organisation believes about it.
Charles Perrow's Normal Accidents (1984) moved the weight from belief to structure. Perrow treated interactive complexity and tight coupling as two independent dimensions of a system; in systems high on both, accidents are "normal" in his sense: to be expected from the structure, not from a single error (Perrow 1984). This article draws one consequence of its own from that account: if accidents in such systems come from the structure, a long quiet in such a system cannot by itself say much about the system's safety.
James Reason's Managing the Risks of Organizational Accidents (1997) added a picture of how the two accounts meet. Latent conditions in an organisation line up with active failures, and defences in depth have holes (Reason 1997). Reason's latent conditions are, on this reading, what Turner's incubation leaves behind.
Diane Vaughan's study of the Challenger launch decision (1996) named the mechanism this article is about. The normalisation of deviance is the process by which an organisation comes to accept as normal a departure it once treated as a warning (Vaughan 1996). This article reads the term as describing a change in what counts as normal rather than a lapse from a standard that stays fixed: the departure is not ignored but reclassified.
Scott Sagan's The Limits of Safety (1993) set normal accidents theory against high reliability theory and tested both on the command and control of United States nuclear weapons, including incidents during the Cuban missile crisis. He found the record closer to normal accidents theory: there were dangerous near-accidents, and organisational learning from them was limited by secrecy and by blame (Sagan 1993). This article reads that finding in its own terms: a near-accident is the kind of event that could break a custom of calm, and the limits on learning that Sagan found, from secrecy and from blame, are limits on how far such an event can do so.
2.2The second side: organisations that hold the line
Todd La Porte and Paula Consolini's paper of 1991 begins from an observation that the first side struggles with. Some organisations, among them air traffic control and aircraft carrier flight operations, run hazardous work with very few failures. Organisation theory as it stood, which expected error and expected organisations to learn by trial and error, struggled to explain them (La Porte and Consolini 1991). The paper is a founding text of what came to be called the high-reliability school.
Karl Weick's Managing the Unexpected (2001) gives the school's account of what such organisations do. They attend to failure; they refuse to simplify; they stay close to operations; they keep the capacity to recover; and they defer to expertise where it is (Weick 2001). This article reads each of the five practices as a way of keeping the organisation from settling into its own picture of itself. Attending to failure means treating small lapses as information. Refusing to simplify means resisting the comfortable summary. Staying close to operations means keeping the people who decide near the people who see.
2.3The two sides set side by side
The disagreement is easier to see laid out. The table is this article's summary of the reading notes, not a claim the authors make in these words.
| Question | Incubation, normal accidents, normalisation | High reliability |
|---|---|---|
| What is stressed? | Structure (Perrow), misread events in an incubation period (Turner), departures accepted as normal (Vaughan), latent conditions (Reason) | Hazardous work run with very few failures (La Porte and Consolini); practices that attend to failure (Weick) |
| What does a long quiet show? | Little; the quiet is part of the incubation | Something, if the organisation was attending to failure during it |
| Can learning break the drift? | Limited by secrecy and blame (Sagan) | Yes, through the practices the school describes (Weick) |
| Cases named in the reading notes | United States nuclear command and control, with near-accidents (Sagan); the Challenger launch decision (Vaughan) | Air traffic control and aircraft carrier flight operations (La Porte and Consolini) |
The table hides an asymmetry in evidence, as this article reads it. The cases named on the first side are ones in which something went wrong or nearly did; those named on the second are organisations working with very few failures. Each side therefore sees most clearly the stretch of time the other cannot see. Turner's incubation is visible only once the precipitating event has occurred. La Porte and Consolini's reliability is visible only so long as it lasts. Neither side has direct evidence about the stretch that matters most for the question here: the period when nothing has yet happened and the people involved must decide what the nothing means.
2.4What both sides assume
Both sides need some view about what a run of uneventful experience licenses. The first side needs it to explain why the run deceives: Turner's misread events and Vaughan's normalised departures are misread because the organisation treats a run of harmless deviations as evidence that the next will be harmless. The second side needs it to explain how reliable organisations avoid being deceived: Weick's attention to failure is a refusal to treat a quiet run as conclusive. Neither side, in the reading notes on which this article relies, says what the run is actually worth as evidence, or why the mind treats it as worth more.
Here a much older argument becomes useful. Hume's Section IV, Part II argues that every inference from experience supposes that the future will resemble the past, and that this supposition cannot itself be proved by experience; Section V answers that custom, not reasoning, carries the mind from past to future (Hume 1748). If that is right, the drift Vaughan describes is not a failure of reasoning, because no reasoning was ever available to stop it. It is the ordinary working of custom on a run of harmless departures. And the practices Weick describes are not better reasoning either. They are institutional arrangements that place in front of the mind something custom does not supply: a fact, present to the senses, that contradicts the run.
Two adjacent literatures bear on the same point from the side of judgment. Baruch Fischhoff showed that people told how events turned out judge that outcome more likely than they would have without the knowledge, and do not notice that the knowledge changed their judgment (Fischhoff 1975). Jonathan Baron and John Hershey showed that people rate the same decision, made on the same information, as better when its outcome was good than when it was bad (Baron and Hershey 1988). Both findings concern the afterwards. The quiet warning concerns something slightly different: a long string of small afterwards, each of which was good, feeding a judgment that has not yet been tested by a bad one. Hume's argument covers both, and the second of the two records read below sits squarely where outcome bias meets custom.
John Maynard Keynes's A Treatise on Probability (1921) offers one more distinction worth keeping in view. For Keynes, probability is a logical relation between evidence and conclusion, not always numerical, and the weight of evidence differs from probability (Keynes 1921). A long quiet may add weight to the belief that the next period will be quiet without making that belief any more probable in the logical sense. This article does not lean on Keynes, but the distinction helps to say what the records ask for when they ask for more data.