This paper asked what a decision record takes a small executed calculation to have checked, and what it does where the calculation and its own arithmetic part. Read in thirteen authored synthetic cases from AEGIS, the answer is that the calculation is taken to have checked itself. Its replay confirms that the same input gives the same output, its assumptions disclaim the figures as measurements, and its interpretation restates the author's result. Where the run's output and the author's result differ in sign, as they do in seven of the thirteen, nothing in the record notices.
Discussion
The first finding is that one figure carries two readings. The minutes of rework can fall on each case or on the change as a whole, and the sentences that state them admit both. The run adds the rework to every case. Six records read it the same way, some of them in so many words, and their runs report savings that the authors' own interpretations repeat. Seven read it once for the whole, and in each of them the rework figure is larger than the saving on a single case, so that the run reports an increase where the author states a saving. These seven are the only runs of the rule among all 93 in AEGIS whose output is an increase, and all seven stand in the programme AEGIS built around executed calculations.
The second finding is that the record keeps the two results apart. The run's interpretation, which is the run's reading of itself, gives the author's figure in all seven. The replay reports a match. Five recommendations rest on the author's figure by name, and every mark closes by saying that it is the author's record, not the output of a calculation. The increase stands only as a number under a label. This is the gap the first strand of the debate describes. Richard A. De Millo, Richard J. Lipton and Alan J. Perlis, Social Processes and Proofs of Theorems and Programs (1979), hold that the complexity of specifying real programs makes formal verification hard to justify; here the specification that matters is one word of a sentence, and no check built into the record reaches it. Naomi Oreskes, Kristin Shrader-Frechette and Kenneth Belitz, Verification, Validation, and Confirmation of Numerical Models in the Earth Sciences (1994), hold that models can be confirmed only in part and are of value chiefly as heuristics. The programme scope of the nine records in AEGIS-CALC-v1 claims about as much: that the runs can show how the result changes if the assumptions change. In this study's reading, the seven records are an instance where a change of assumption, from once to each, has already been made between the author's sentence and the run's input, and its consequence is in the record unread.
The third finding is that the records place their doubt in the world. They ask for measurements of time and rework, they name rework among what is unknown, and they tie their conditions for reopening to it. In the four cases whose questions name the author's figure, the condition repeats that figure as a threshold in the same ambiguous words. The records' concern is the concern of the third strand: that a forecast built on a plan, in the sense of Daniel Kahneman and Dan Lovallo, Timid Choices and Bold Forecasts: A Cognitive Perspective on Risk Taking (1993), and of Roger Buehler, Dale Griffin and Michael Ross, Exploring the "Planning Fallacy": Why People Underestimate Their Task Completion Times (1994), may not survive contact with the work. The disagreement that is already in the records is of another kind. It is not about how large the rework will turn out to be but about how many times it is counted, which this study reads as a quantity of the kind that Winston Harrington, Richard D. Morgenstern and Peter Nelson, On the Accuracy of Regulatory Cost Estimates (2000), name when they trace the difference between their total-cost and unit-cost results to frequent errors in estimating quantities. Bent Flyvbjerg, Mette Skamris Holm and Søren Buhl, Underestimating Costs in Public Works Projects: Error or Lie? (2002), set error against strategic misrepresentation. The records are authored and synthetic, and this study imputes no motive to their author; it notes only that the question the records leave open, which reading of the figure was meant, is one that a reader can settle by asking, and that the records give no place to ask it.
The debate on reproducibility gives the sharpest frame for this. Roger D. Peng, Reproducible Research in Computational Science (2011), offers reproducibility as a minimum where replication is not possible. Thomas Herndon, Michael Ash and Robert Pollin, Does High Public Debt Consistently Stifle Economic Growth? A Critique of Reinhart and Rogoff (2013), report a replication by other hands that found coding errors, excluded data and inappropriate weighting in two published studies; this study reads that as an instance where running the original computation again would have given the same figures. Raymond R. Panko, "What We Know About Spreadsheet Errors" (1998), puts testing after development above assumption sections. The thirteen records meet the minimum, carry the assumption sections, and replay themselves; the reading of the figure is left for a reader outside the record. This paper is one such reading.
Contributions
The paper contributes three things. To the debate on verification and validation, it adds a small, fully visible case in which a calculation is verified in the narrow sense, run again with the same result, while its data are read by the calculation and by the record's author in two ways that the record never compares. To the debate on reproducibility, it shows a replay built into a decision record that confirms a result no sentence of the record repeats, and so marks the distance between a computation that repeats and a reasoning that consults it. To the debate on forecasts of time and cost, it shows a record set that places its doubt about a forecast in the size of a quantity to be measured later, while the doubt already present in its own figures concerns how many times that quantity is counted.
Limits
The cases are thirteen authored synthetic records from one studio, chosen by one rule; nothing here is a finding about real offices or real savings, and nothing here says which reading of the rework figure is right or what any office should do. The rule depends on the word for rework appearing in the situation or question; it happens to keep every run of the rule whose output is an increase, but a different rule could find a different balance of readings. The split between the two readings is computed from the figures each record states; a record that stated no such figure would fall outside it, though none of the thirteen does. The records are Korean, and the English quoted is this study's translation; the ambiguity of the rework figure is present in the Korean and has been kept, not added, in the English, but a reader of the Korean may weigh the sentences differently. The works are read through their published abstracts only, and every reading of them beyond those notes is this study's own.
Further research
Three questions follow. The first is whether the other 80 runs of the same rule, whose records do not speak of rework, read their figures as these thirteen do, and whether the other rules of the calculation programme carry figures that admit two readings. The second is whether records of real decisions that keep both an author's arithmetic and an executed calculation set the two side by side, and where; this study could not ask that of synthetic records. The third is what a decision record would need in order to hold the meaning of a figure as it holds the figure: a place to say whether a quantity falls once or on each case, and a check that compares the run's result with the result the record states. Those are questions for the design of records, and this paper asks them without answering them.
The contribution, stated again, is a reading of where one designed record set puts its check: on the repetition of its arithmetic, which it verifies with care, and not on the meaning of its numbers, which it leaves to a reader.