Chapter 1. Reversal Conditions in Twelve Authored Training Cases

This chapter answers the paper's question from the records. Its first section says what the records are and how they were read. The second shows that the set writes every reversal as a strict count over a small test and that the rules lean toward keeping restrictive choices, which places the records on the side of precommitment in the debate on binding future decisions, and close to the zero-numerator problem that the literature on small samples describes. The third shows that the same records state where each of their numbers comes from, which gives their thresholds a pedigree in the sense of the uncertainty literature, and qualifies the impersonal authority that numbers otherwise lend a rule.

1.1Data and method

AEGIS publishes its authored cases as data files of one record a line; the eleven published parts held 6,540 records when they were read for this study. The cases studied here are one programme within them, which AEGIS files under the heading of a corrective review of policy and preparedness candidates and describes in each record in the same words:

The selection rule is one sentence: every authored AEGIS case of the programme AEGIS-RVD-20260921. The programme holds twelve cases, and the rule takes all of them, so no case was chosen for what it says. The twelve fall under seven of AEGIS's domain headings, from national security decision-making to procurement and facilities. Its scope statement announces both things this paper asks about, supplemented thresholds and a statement of where each figure comes from, so the paper treats the presence of numbered reversal rules as the programme's design and reads only their shape and wording.

All twelve records are synthetic, and they say so in several places. Each is typed realistic synthetic; none has a source address; each says that it has not been compared with public records and that its fact check verified no real event or authority; and each carries the same statement of its role:

The records describe themselves as corrected from an earlier list of candidate items, which they call the original; AEGIS's source line names the file that held that list, and this study does not reproduce the line because the file name carries the name of a model. No record carries a completed decision. Each holds three choices and an author's judgment mark that recommends one and lists the other two as alternatives, and each ends that mark by saying it is the author's record and not the output of a calculation. Every record waits for input, and every record says its predictive performance was not measured. None draws links to other cases; each says instead why it declares none. The set marks a recommendation; it does not make a decision.

AEGIS wrote the records in Korean. This study put them into English itself, once, sentence by sentence and without additions, and each case's English seed is labelled as the study's translation and names the Korean original by its hash. Every block quote in the paper is that English, word for word.

From each record the study read the parts that answer its question: the condition on which to reconsider (what would reverse the recommendation), the unknown (where the numbers come from and what is unconfirmed), the three choices with their stated costs, the author's judgment mark, the decision owner and the deadline, and what the record asks someone to observe. Every number in the paper was counted by a script from the study's own copies of the records and is listed in its numbers file; nothing was coded by a model. Classifications were made by stated rules on the Korean words. A reversal counts as a single-failure rule when it is triggered by one or more adverse results in the sample, as an item-by-item rule when it reverses the judgment only for the request or requirement that fails, and as a clean-result rule when it requires zero adverse results or success across the whole sample. A recommended choice counts as the more restrictive one when it stops, mandates, switches everything, or runs a backup that the alternatives would do without.

Because a studio's records can repeat a template, the study measured how much of each part is template before leaning on it. The measure is the one the RISCON series uses: break a text into runs of five words, and count a run as frame when it recurs in at least 5% of the records that fill the same part, and in at least three. Measured against every authored AEGIS case, the twelve records' conditions for reconsidering, unknowns, deadlines, questions and observations were 0% frame, and their judgment marks 2%. Because Korean attaches particles to words, runs recur less in Korean and the measure is a floor; the study therefore measured again among the twelve themselves, in Korean and in English. The conditions for reconsidering stayed at 0%, the unknowns at 0% in Korean and 1% in English, and the choices and needed inputs at 0%. Two parts did not. The statements of source role and of programme scope were 100% frame, and the line that joins each unknown to a standard caveat was 39% frame in Korean and 44% in English, because the caveat is the same in every record. The paper uses none of these as a finding and reads the unknown itself rather than the joined line. Finally, the presence of three choices, of a recommendation and of a condition for reconsidering is guaranteed by AEGIS's template across nearly all its authored cases (98% of them open their judgment with the same author's mark), and the paper treats none of these as a discovery.

1.2How the reversal is written, and which way it leans

Every one of the twelve records states, before the decision, the result that would overturn its own recommendation, and every one states it as a count. The condition is not a change of circumstances to watch for; it is a test with a sample, a tolerance and a consequence. The case of the data centre permit shows the form at its plainest. The record recommends approving a foreign-owned data centre near a protected zone on verifiable security conditions, and writes:

Three issues make the sample, one unenforceable condition is enough, and the consequence is named: the council chooses again. The test is run before approval, not after, so the rule decides whether the recommendation survives the deciders' own preparation. The same form recurs across the set. The samples are small and named: three issues, three mock consultations, three mock tests, three transport requirements, three requests, three restart runs, four functions, six cases, twelve cases, twelve requests, twenty systems, twenty test cases. Half the samples are of three items, and none is larger than twenty.

The tolerance is one failure or none, and this is the first thing the set commits its deciders to. The succession case puts it in a phrase whose like recurs, in one form or another, through half the records:

Six of the twelve rules reverse the recommendation on a single adverse result in the sample. Two more reverse it item by item, for the request or requirement that fails, while letting the rest stand; in the case of the rail control outage the restricted-operation judgment is undone only for the transport requirement that lacks safety approval or runs past its time. The remaining four reverse only on a clean result across the whole sample. No rule tolerates two failures, and no rule weighs a share of failures against a share of successes.

Read beside the literature on second-order decisions, this is a deliberate choice of rule over discretion. Sunstein and Ullmann-Margalit list rules and presumptions among the second-order strategies, some of which put high costs before the decision and low costs at the time of deciding. On this paper's reading, the AEGIS records spend their effort in advance: the sample, the count and the consequence are fixed before any trial is run, and nothing is left for the deciders to weigh when the result arrives. Elster's case that people may benefit from being constrained in their options describes, on this paper's reading, what such a rule takes away. A council that has written "even once in 3" can no longer decide, after a near miss, that the miss did not count. The preregistration argument of Nosek and others points the same way from another field: a condition stated before the outcome cannot be fitted to it afterwards.

The four clean-result rules show which way the set leans, and they are the paper's central finding. In each of them, the recommended choice is the more restrictive one, and the record will reverse it only if every item in the sample comes back clean. The lost cryptographic device recommends switching all authentication, and writes:

The counterfeit parts case recommends stopping and quarantining a delivery, and sets the same kind of bar at a larger scale:

The fuel-cell case recommends running auxiliary power and will step back from it only if the fuel cells restart in time on every run:

And the case of prior consent for base use recommends requiring consent case by case, and will reverse the prior-consent method for a scope only if the scope of prior delegation is confirmed in all three mock consultations and only the individual procedure runs past the agreed deadline. In the other eight cases the recommendation is a middle course accepted under conditions, such as conditional approval, restricted operation, receive-only delivery, joint review or an internal substitute, and a single failure, or a single failing item, reverses it. The pattern is complete across the set: a restrictive recommendation stays until a clean result frees it, and a conditional permission falls at the first failure.

Two works bear on this lean, and they pull against it from different sides. Hanley and Lippman-Hand examine inference from a zero numerator, a study in which none of the events of concern occurred, and observe that no events seems to be viewed as very different, in quantity and in kind, from one or more. The clean-result rules turn on exactly that difference: three runs without a late restart, four functions without shared exposure, twenty systems without a defect. On this paper's reading, the records ask a small clean sample to carry the weight of relaxing a protective measure, which is the situation in which, Hanley and Lippman-Hand say, making inferences seems particularly problematic. One record says as much about its own sample:

The records also guard the clean result against being read too generously. Every one of the twelve conditions adds a second sentence, and nine of these second sentences end by saying what will not be done or presumed. The cryptographic case refuses the most tempting reading of a quiet interval:

and the counterfeit case refuses the reading that a working part is a lawful one:

These sentences are, on this paper's reading, the records' own answer to the question Hanley and Lippman-Hand put in their title: if nothing goes wrong, is everything all right? The records answer that the absence of a visible failure is not the clean result their rules require; the clean result is a confirmed one.

Sunstein's argument in Laws of Fear presses, on this paper's reading, from the other side. If risks lie on all sides and precautionary steps create dangers of their own, a rule that makes precaution hard to undo needs to show that it has counted the cost of the precaution. The records do count it, at the level of the choices. Every one of the thirty-six choices carries a clause naming what it gives up or what it accepts, and the restrictive choices are not spared:

Thirty-three of the clauses name a loss and three name an exposure accepted; one record goes further and says, in describing its setting, who would carry an unresolved risk:

So the lean of the reversal rules does not come from ignoring costs. The costs of caution are written beside each choice. What the rules decide, in advance and without weighing the named costs against one another, is where the burden of proof lies when the test comes back: on relaxing a restrictive measure, and on keeping a conditional one. That is a second-order decision in Sunstein and Ullmann-Margalit's sense, and the paper reads it as the set's distinctive position in the debate on precaution: the records accept Sunstein's point that every option has a cost, and then choose a presumption rather than a balance.

Each case gives the decision to a fictional council or committee, eight councils and four committees, and each asks a single fictional officer, never the deciding body, to check and record the results before approval. The succession case names what that officer is to time:

Eleven of the twelve deadlines fall before an approval or signature. On this paper's reading, that is close to what Haasnoot and the other authors of the pathways method call a signpost, monitored to see whether adaptation is needed, with the difference that each AEGIS signpost comes with its trigger written beside it.

1.3Where the numbers come from

A reversal rule that turns on "1 or more of 3" or "0 of 20" takes its authority from its numbers. Porter argues that quantification becomes most important where private negotiation is suspect and trust is short, as a strategy of impersonality. On this paper's reading the AEGIS rules have that character: once the count is written, no member of a council needs to be trusted to read the result. The same records, however, tell the reader exactly how much the numbers themselves deserve to be trusted, and this is the second finding of the paper. Every one of the twelve says, in what it lists as unknown, where its figures come from. Ten name something as the original's, eleven name a figure as added by the author, and three call a figure a design value or setting. The data centre case is typical:

The contact-reporting case separates the two sources within a single figure set:

The second of these sentences reaches past the figures to the threshold itself. The rule that one mandatory collection of a private contact outside the reporting scope, among twelve cases, reverses the reporting category rests, the record says, on an added exercise value, the misclassification threshold. The record writes a strict rule and, in the same breath, tells the reader that its strictness was set for practice and not from evidence.

Five records go a step further and say that something in the original was not adopted. The rail case declines an inherited figure:

The cryptographic case declines an inherited conclusion:

and the internal substitute case declines an inherited certainty:

The counterfeit parts case declines a claim that a particular inspection removes all risk, and the fuel-cell case declines a prescription, saying that the original's prescription of modifying and concealing piping was not adopted; the maintenance-drawing case says that the original's claim of blocking leakage at the source is not guaranteed. Read together, these sentences describe records edited with their own numbers in view.

This is what the NUSAP literature calls a pedigree. Van der Sluijs and others describe joining a quantitative analysis of a parameter with a qualitative review of its strength, so that uncertainties can be ranked by what is known about the limits of the available knowledge. The AEGIS records do not grade their figures on a scale, but each figure in a reversal rule comes with a statement of its origin and standing: inherited, added, declined. On this paper's reading, the records thereby hold apart two things that a numerical rule tends to fuse. The rule binds the decision, as Porter's impersonal number would; the pedigree tells the reader that the binding figure is an exercise value and not a measured tolerance.

How rare this is can be counted. Across all 6,540 authored AEGIS cases, only 58, about 1%, state the source of a number in these terms, and the twelve records of this programme are all among them. Most of the others stand in two groups: the programme AEGIS-RVE-20260921, with 12, and three review sets whose names this study withholds, with 13. Numbers in reversal conditions are not rare in AEGIS: 73% of all its authored conditions for reconsidering name a number. What is rare is a record that names a number and then says where it came from. By contrast, in AEGIS's X100 set, only 2 of 100 conditions for reconsidering name a number at all.

Dewar's account of assumption-based planning gives a further way to read the pedigree statements. It holds that surprises often trace to assumptions an organisation did not anticipate or had forgotten it was making, and it asks planners to bring those assumptions into the open, monitor the vulnerable ones and prepare for their failure. The unknowns of these records do the first of those steps for their own numbers. The medical allocation case marks out what its rule does not decide:

and the civilian backup case refuses to let a single figure stand in for a judgment of capability:

What the records do not do is the later part of Dewar's sequence. They monitor the result of each test, through the officer who records it, but they do not say how the threshold itself would be revised if the exercise values turned out to be wrong. Every record says its predictive performance was not measured. The rule is fixed and its pedigree is declared; the path from a declared exercise value to a tested tolerance is left unwritten.

This answers the second half of the paper's question. The rule commits the decider to the count; the pedigree tells the later reader not to mistake the count for evidence. That is the set's contribution to the debate on the standing of numbers: it shows a way of writing a rule that uses a number's impersonality while openly limiting its authority.