A pile of reports looks like more evidence than any one of them. Whether it is depends on where the reports came from. If three of four reports were written from one ledger, the pile holds less than it seems; if they were written by three people who looked at the same event separately, it may hold more than any one. Before a pile of reports can count in a decision, someone has to say how many things it really holds, and what follows from that number.
This paper asks how one designed set of decision records answers that. AEGIS, the safety and protection studio of the RISCON family, publishes 6,540 authored cases, each a decision record with a question, three choices, an author's marked recommendation and a list of what must be known before deciding. One programme among them, a set of 196 cases, attaches to each case a small calculation that has actually been run, with its input, its output and a check that running it again gives the same result. The programme uses 12 such rules. One of them counts sources: it takes a list of reports, each tagged with the original source it rests on, and returns two numbers, how many reports were compared and how many distinct original sources they rest on. Ten cases of the programme run that rule, and they are the subject of this paper. Their reports are maintenance histories and maintenance performance figures, personnel, welfare and medical returns, press releases, fuel consumption returns, training accident reports, supply statistics, situation reports, public records of past decisions and the documents of an alliance consultation.
They are authored synthetic training cases, not records of real events. AEGIS types each one realistic synthetic and calls each an authored question seed whose judgment mark is the author's record, not a factual basis from public records. It says of the programme that every number in it is a value the author assumed, and each run says that the mapping of reports to their sources is the author's assumed value. The offices and ledgers named are part of the setting. What can be learned from these cases is how a designed record set frames the counting of sources, not how any real office reports.
The research question is this: when an authored decision case counts the reports before it and finds that they rest on fewer original sources, what does it take that count to mean, what does it recommend doing with the reports, and what does it rule out?
The argument in brief
The ten cases agree on the arithmetic and divide on its meaning. In every case the count comes out smaller than the pile: across the ten, 41 reports rest on 27 original sources. In five cases the gap is read as waste. The question asks how far the reporting can be simplified, and the author's reasons speak of clearing out duplicate reports and lightening a burden. In three cases the same gap is read as a doubt about evidence: the question asks whether the reports can be trusted as independent of one another. The remaining two ask which source should be the standard when several feed one figure. What no case does is read the repeated report as a safeguard, a second channel that would carry the information if the first failed. Two recommended options keep a second record as a check, but only as a check on the first, not as a reason to keep the repetition.
The cases also agree on what to recommend and divide, in a patterned way, on what to rule out. Every recommended option sorts the reports by their original sources: it confirms the roots, keeps one strand or one representative document for each, or marks the root on every record. The excluded option is always one of two opposite errors. In three cases it is binding reports from different sources into one, so that which source stands behind a figure can no longer be recovered. In the other seven it is leaving the reports many and unsorted, so that copies of one source pass for several independent voices. The recommended option sits between them, with the original source as its unit.
Finally, the count rests on a judgment the records do not contain. The calculation only counts labels; which reports share a root is written into its input by the author. Nine of the ten cases list, among the things still needed before deciding, something that would settle that judgment: a register of original sources, a document defining them, criteria for judging them, grounds for fixing them. The cases recommend sorting by a sameness that they assume and ask for, but do not hold.
The debate this paper enters
Whether more sources mean more is an old question in two literatures that rarely meet, one about building reliable systems and one about weighing testimony.
The first treats repetition as a resource. Martin Landau, Redundancy, Rationality, and the Problem of Duplication and Overlap (1969), sets redundancy beside the familiar administrative complaint about duplication and overlap; the article opens with a plane landed safely on an impaired rudder, and with the remark that a commercial airliner is a very redundant system, which is why it is reliable and adaptable. In software engineering, A. Avizienis, The N-Version Approach to Fault-Tolerant Software (1985), reviews an approach in which several versions of a program are run so that the design faults of one can be tolerated. The second voice in this literature asks what the repetition is worth. D. E. Eckhardt and L. D. Lee, A Theoretical Basis for the Analysis of Multiversion Software Subject to Coincident Errors (1985), model the propensity of separately written versions to fail together and give a condition under which several versions are better than one. John C. Knight and Nancy G. Leveson, An Experimental Evaluation of the Assumption of Independence in Multiversion Programming (1986), test the assumption that programs developed independently fail independently, and find many more tests than expected in which more than one version failed.
The second literature asks when agreement is evidence. Krishna K. Ladha, The Condorcet Jury Theorem, Free Speech, and Correlated Votes (1992), starts from the theorem that a majority is more likely than any one member to choose the better of two alternatives, notes that it assumes independent votes, and generalises it to correlated ones; the article applies the result to the question of whom a chief executive should choose as advisers, and finds that for small groups the conditions are severe. Alvin I. Goldman, Experts: Which Ones Should You Trust? (2002), considers how a novice can choose between disagreeing experts and holds that going with greater numbers seems promising but depends on problematic assumptions. Luc Bovens and Stephan Hartmann, Bayesian Epistemology (2004), build models of the reliability of information sources and apply them to jury voting, to the variety of evidence and to the value of testimony. Michael Weisberg, Robustness Analysis (2006), defends the search for predictions common to several independent models against the charge that it is a nonempirical form of confirmation.
A third observation explains why piles of reports exist at all. Martha S. Feldman and James G. March, Information in Organizations as Signal and Symbol (1981), report that organisations systematically gather more information than they use yet continue to ask for more, and suggest that the use of information is embedded in social norms that make it highly symbolic.
The positions do not divide into those for and against counting sources. They divide on what a repeated source is. For Landau and Avizienis repetition can buy reliability. In this study's reading, for Eckhardt and Lee, Knight and Leveson and Ladha, what it buys depends on how far the repeated parts fail, err or vote together, and that is a matter of degree to be estimated, not a label to be read off; for Goldman, going with greater numbers rests on problematic assumptions. For Feldman and March, information gathered beyond what is used is not measured by its evidential worth alone, because its use is embedded in norms that make it highly symbolic. A designed set of decision records has to take a position on this, and it does so in what it calls its count, what it recommends and what it rules out.
Contribution
This paper adds to that debate a reading of how a record set, rather than an engineer or an epistemologist, encodes the step from many reports to a judgment about their worth. It makes three claims. First, the ten cases read the gap between reports and sources as waste or as doubt and never, in this study's reading, as the reliability Landau's opening sets out; the record set has absorbed the administrative complaint about duplication and the epistemic worry about dependence, and left out the engineering case for redundancy. Second, the cases guard against error in both directions: they rule out collapsing different sources as firmly as they rule out counting copies as independent, and they make the original source the unit of every recommendation. Third, the independence that the cases count is binary and assigned, where the literatures from Eckhardt and Lee to Ladha treat dependence as a matter of degree; the cases ask for the register or the criteria that would ground their labels, and do not hold them. That gap between a count that can be run and a sameness that has to be judged is the paper's main finding about what a template can and cannot do.