42.1The simpler baseline
The adverse condition can be stated as a prospective design case rather than an observed finding. Suppose competent ordinary editing — an annotated document kept up to date by hand — satisfies the same declared task criteria as the proposed framework while imposing less burden on its maintainers and readers. If that condition held for a setting, the added machinery would lack demonstrated value there: it would have failed the comparison its proponents invite. The conclusion is deliberately local. The framework could still promise things the criteria do not measure, but those promises would need to be made explicit before the comparison counts against it.
Two qualifications define the comparison. The task and the inputs must be fixed before the encodings are varied. If the annotated document is judged on a narrow job while the framework is credited with a broader one, the assignment differs rather than the representation, and no ranking of burden follows. What is under test is the way the work is encoded and maintained, holding the material processed and the standard of success constant.
Burden, too, must name a quantity. Review time per decision is one candidate; setup effort, retraining, or the cost of correcting errors are others, and they need not rank the two approaches identically. An aggregate figure can hide exactly the trade-off at issue.
Finally, satisfying the task criteria is necessary for the challenge to arise but does not by itself make the simpler baseline superior on every dimension. It shows that added value is undemonstrated in that setting, not that the machinery can never earn its cost elsewhere. The adverse outcome is therefore possible, and its occurrence would be a matter for the setting's actual comparison — a condition to test, not a verdict already delivered.
A fair comparison begins by fixing what is being compared. The declared task, the criteria for meeting it, and the information available to the worker must be held constant across the two approaches. The simpler baseline also deserves competent implementation: ordinary editorial skill applied seriously, with the same source access and comparable time, rather than a deliberately weak or improvised version. Without those controls, any observed disadvantage of the baseline measures unequal preparation or unequal access, not the cost of added complexity.
Once those conditions hold, the inference is narrower than it first appears. If competent ordinary editing met every declared criterion while imposing less burden, that result would show only that the proposed framework is not necessary for those criteria in that setting. It would not establish that the framework is sufficient for anything, that it never helps, or that ordinary editing dominates across purposes outside the comparison. A necessity claim is defeated by one adequate alternative; a superiority claim is not thereby defeated.
The quantity called "burden" also requires care. It compares effort or complexity under a fixed task and fixed inputs. It does not compare final quality on dimensions the criteria omit, and it says nothing about outcomes when the task changes. If the two approaches use different encodings of the material, the comparison is between complete configurations, so the task and inputs must be held constant or the result will partly reflect the encoding shift rather than the machinery itself.
Lower burden alone is likewise not proof of harm: added machinery may be unnecessary here and still neutral or useful elsewhere. The defensible conclusion stays attached to the setting tested.
42.2No improvement in source retrieval
Once the simpler baseline has competent implementation and comparable access, as the preceding comparison requires, the retrieval claim turns on a narrow question: with task and inputs held fixed, can readers locate the relevant source more reliably or more quickly under the richer record? If not, the proposed retrieval benefit is unsupported. The relevant comparison is therefore not whether the richer record contains more material, but whether it converts that material into a findable source.
This is a claim about the support for a benefit, not a measurement of one. The prospective adverse condition stipulates no improvement and supplies no effect estimate and no participant outcome. The uncertainty attaches to this prospective adverse condition: this study cannot say how large any difference is, or whether readers would behave this way in practice, only that the stated evidence does not establish the improvement.
Two distinctions keep the inference in scale. First, consider what the comparison actually compares: not the information content of two encodings in the abstract, but the performance of readers using them on one fixed retrieval task, with the same query and the same access to the corpus. If the encodings differ in other respects—indexing, navigation, or search support—the measured difference is attributable to the whole arrangement, not to record richness alone. Fixing the task and inputs is what allows the comparison to bear on the claim at all.
Second, possible outcomes are not inevitable ones. Retrieval improvement may be possible in other designs or on other tasks; the failure marked here is confined to this declared task under these conditions. Correspondingly, a retrieval gain would be necessary for the retrieval benefit to hold, yet not sufficient for the framework's broader value, since other purposes might rest on other grounds. But those purposes cannot borrow this retrieval claim. If readers cannot find the relevant source more reliably or quickly, the retrieval benefit the framework advertises remains unearned.
The adverse condition should be stated narrowly. Fix the retrieval task, the candidate sources, and the information need, and vary only the record shown to the reader. On that comparison the richer record yields no improvement over the comparator in finding the relevant source. Nothing else is supplied: no effect estimate, and no participant outcome. The warranted claim is therefore that the retrieval benefit is unsupported for this task and this encoding. It is not the claim that the benefit is zero or negative.
What improvement compares is performance on one task under two encodings; it is not a measure of absolute retrieval quality. That is why the failed comparison is sufficient to defeat a claim that the richer record must help whenever it is used, yet insufficient to show that provenance records as a class are useless. The same adverse condition is equally consistent with several possibilities the comparison cannot separate: the identifying detail may be buried, the task may not exercise the added material, or the comparator may already contain everything the task requires.
A design defect is therefore one possible reading, not an inevitable one. If, as a conditional supposition, the failure is due to where the identifier sits or how the record is ordered, a revised encoding might retrieve better; the supposition promises nothing, and the same task and inputs must be rerun before improvement is claimed. The revision warranted is accordingly narrow: alter and retest this encoding on the declared task rather than abandon provenance practice. Nor does the mere possibility of repair keep the framework necessary.
To treat the framework as unnecessary on this ground, one would need retrieval to be its only essential use, and the adverse condition does not establish that. Whether visible links leave interpretation errors untouched is a separate question.
42.3Unchanged interpretation errors
Where the earlier condition concerned whether a source could be reached, this one concerns what the source is taken to mean. Suppose, as a prospective design condition rather than a reported finding, that visible links are present and readers still arrive at the same interpretation error they would have reached without them. No evidence here establishes that this occurs. What the condition establishes, if observed, is narrower but decisive for the framework's semantic claim. The links have made sources locatable, yet the intended benefit — a corrected reading of what the source supports — has not appeared. Visible linkage is therefore not sufficient for that benefit. It may still be a necessary ingredient, or useful under other conditions; persistence of the error does not show that links can never help, nor does it explain why any reader misread anything.
The quantity that would settle this is the rate of the specific interpretation error under a link-bearing presentation and under an otherwise matched presentation without links, with the claim, task, and reader inputs held fixed. The encodings may differ — that is the point of the comparison — but the underlying proposition and the question asked must not drift. A fall in retrieval time or link-click failure would measure access, not interpretation. Conversely, if the error fell under links, or fell for some tasks but not others, that would count against treating persistence as the general result: this is a possible outcome to test, not an inevitable one.
What follows from persistence is a failure of the link-only intervention to establish its intended semantic benefit in that setting. The conclusion concerns demonstrated effect on interpretation, and stops there.
Suppose the prospective adverse condition holds: the visible links are in place, and the same interpretation error remains on the declared task. The immediate conclusion is narrow. The link-only intervention has not shown itself sufficient to produce the intended change; nothing yet follows about whether readers possessed the correct information, attended to it, or chose to disregard it. "Ignored correct information" attributes both a mental state and a failure of will, and the observation supplies neither. An unchanged interpretation is compatible with several conditions: the linked material may not contain the corrective proposition; the proposition may be present but framed so that it does not bear on the disputed reading; the task's criteria may reward the original interpretation; or the reader may reason from the passage in ways the check did not measure. Each is a conditional supposition, and the record does not select among them.
What the result does support is a decision about the workflow rather than a verdict on its users. Two responses carry different evidential burdens. Better review means identifying which interpretation is at stake and what the linked passage would have to state to correct it; abandonment means dropping the link requirement for this task and testing simpler tools against the same inputs. Either comparison must hold the task and the underlying material fixed and vary only the presentation or encoding, since changing what is being read would confound the result and make the comparison measure something other than the intervention.
One further discipline matters. An unchanged interpretation error is a possible outcome of this design, not an inevitable one, and it speaks to persistence under the conditions examined rather than to the general usefulness of links. The defensible claim is therefore that the intervention, as applied here, did not establish the intended semantic benefit — and that its future should turn on whether review can supply what it lacked, not on an assumed account of reader conduct.
42.4More effort and no useful correction
The framework's cost-benefit case rests on a comparison: for the same declared task and the same inputs, the extra work it demands should buy correction that would otherwise be missed. Suppose, as a prospective adverse condition, that measured effort rises while no useful correction appears on that task. On this condition the cost-benefit argument weakens, and it weakens in a specific way: the additional burden is unrecovered, not merely expensive. The qualifier "useful" carries the weight here. A correction is useful only if it alters a decision the declared task actually concerns. Noticing a difference that leaves the deliverable unchanged is not a correction for present purposes, though it may be worth having for another reason. Thus a correction is a necessary benefit but not a sufficient one; it must also bear on the task at issue.
The comparison must hold the task and inputs fixed while allowing the workflow itself to differ. Otherwise an observed difference in effort could reflect a different job rather than the framework's overhead. Nor is the quantity "effort" self-defining: it may count review time, number of steps, or cases escalated, and these can move apart. Whichever measure is chosen has to be stated before the test, so that rising effort and absent correction are jointly visible rather than inferred after the fact.
Two qualifications follow. Rising effort alone would not defeat the framework; the adverse force comes from its conjunction with corrections that do not bear on the task. And absence of a useful correction within the observed cases is not proof that none exists; it supports only the claim about the cases examined. Nor does the condition by itself show the framework is harmful — only that its added cost has gone unrecovered on this task. Rare but consequential corrections, if any, belong to a further argument about valuation rather than to this one.
The adverse condition so far concerns ordinary operation: measured effort rises, and no useful correction appears on the declared task. A defender may answer that the framework is not meant to harvest frequent corrections at all. Its payoff, the answer runs, comes from the rare case in which a consequential error is caught before it propagates. That reply is intelligible, but it cannot do unpriced work. A rare correction that would have been consequential is a possibility, not an inevitability, and possibility alone cannot outweigh a measured cost.
What is needed is an explicit valuation. The claim must state, for the declared task, what counts as a consequential correction, how such corrections would be valued if they occurred, and what comparison licenses the added burden. Formally, the quantity at issue is an expected benefit: the frequency of consequential corrections times their stipulated value, set against the cost of obtaining them while the task and its inputs stay fixed. If the encodings of the workflow differ, that comparison must still hold the assigned task constant, or the quantities being differenced no longer answer the same question. A count of corrections caught, for instance, compares nothing about their importance or about whether any decision changed.
Two conditions follow. First, a stated valuation that exceeds the added cost would justify the framework for that task; absent such a valuation, the rare-consequential case remains a hypothesis about the tail, not a demonstrated return. Second, the argument fails even under favorable stipulation if corrections valued in the abstract never affect the action being decided. A consequential catch that arrives after commitment, or that no decision process reads, contributes nothing to the comparison above. The uncertainty here is local: the tail may be better or worse than assumed, and the valuation must carry that uncertainty rather than converting it into a generic license for extra work.
42.5A dissent channel without effect
Consider the prospective condition in which a recorded dissent cannot be located or considered in the decision process that promised those opportunities. The channel accepts input and stores it, yet no one reviewing the decision can find the entry or show where it was weighed. Here the failure is not that the decision stood: reversal was never promised as the necessary result. The failure is that the promised opportunity—to be located and considered in the relevant process—does not occur. On this condition the channel is ceremonial: it collects dissent while leaving the decision process unchanged in the only respect it claimed to change.
The inference depends on a precise comparison. The relevant question is not whether dissent sometimes changes outcomes; it is whether entries were retrievable and considered where the decision was made. Locating an entry is necessary for considering it, and considering it is necessary for the channel's promised function, but neither alone is sufficient for a different result. A channel may satisfy both and still leave the outcome intact. Conversely, a channel that records but cannot route dissent satisfies neither, however loudly the availability of a form is advertised.
The organizational caution cuts the same way. Participation can reproduce extraction when its organization and power are left unexamined (Sloane et al., 2020 preprint); a channel whose internal routing is invisible to the participants who used it fits that pattern. This is a critical argument about particular designs, not a reason to reject every participatory method. What would defeat the present concern is evidence, within a specified process, that dissent entries were retrievable by the relevant decision-makers and explicitly weighed. Absent that trace, the count of entries measures intake, not the opportunity the channel promised.
The proper test is not whether a dissent reverses an outcome. Reversal is one possible consequence of an honored opportunity; it is not the measure of the promise. Before any test, state what the channel promises: that a dissent entered at the designated point can be located by, and brought to the attention of, the process empowered to decide it. Each clause carries weight. "Can be located" is necessary for consideration but not sufficient: a record that exists yet cannot be retrieved by the deciding body leaves the promise unmet. "Brought to attention" likewise falls short of influence. Only if the promise itself says that dissent will alter the decision does failure to reverse become dispositive.
Holding the task and inputs fixed, then, the counts diverge. A tally of reversals answers a question about outcomes, which the channel may never have undertaken to change. A tally of dissents that were locatable and actually noticed answers whether the promised opportunity occurred. Where the prospective condition holds — a recorded dissent cannot be located or considered in the process that promised those opportunities — the opportunity claim fails, even though the decision might coincidentally have gone the other way for unrelated reasons. Conversely, an unreversed decision does not show that the channel lacked effect. An honored opportunity makes reversal possible, not inevitable; an honored opportunity that produced no reversal still did what it promised to do.
The distinction also clarifies what would count as adverse. If the promise is narrow, demanding reversal before crediting the channel sets a standard the proposal never claimed to meet. If the promise is broad, reporting mere locatability as though it were influence quietly substitutes the narrow quantity for the broad one. The test must therefore track, claim by claim, which opportunity was actually extended and which one actually occurred.
42.6A result that changes the proposal
An adverse finding does damage only when it is matched to a named claim. Left unallocated, it becomes a general inconvenience that the proposal absorbs without loss. The matching should therefore be made explicit for each result the framework is exposed to.
Begin with the prospective retrieval case already on the table. Suppose a retrieval test finds no decision change: the claim that retrieval improves decisions is withdrawn as stated, or narrowed so that it excludes the setting actually tested. The choice between these two is not cosmetic. Withdrawal concedes that the proposal's central benefit is not established. Narrowing keeps a claim, but a smaller one: retrieval is asserted to help only where some stated condition holds, and the proposal must now carry that condition in its promise rather than in a footnote. A narrowing that names no condition beyond where the result was favorable is not a narrowing at all; it restates the same result.
The test also fixes what the finding bears on. A null result shows that retrieval was not sufficient to produce the decision change in the tested configuration; it does not show that retrieval was never necessary. Any withdrawal should say which of those it addresses.
Other prospective adverse results map differently. If simpler tools perform as well, the comparative advantage claim goes; the framework may still be usable but no longer better. If review cost rises without gain, the efficiency claim goes and cost becomes a net liability. If tracing fails, the traceability claim goes. If dissent never affects action, the claim of effect on outcomes goes, while the claim of a promised opportunity may survive, having been defined separately. If corrected sources never change action, the correction-propagation claim goes.
None of these dispositions is a repair. A redesign changes the mechanism under test and is therefore new development work, to be evaluated on its own.
An adverse retrieval result is a signal about a claim, not about the proposal in general. The first task is to match the result to the exact claim it bears on: typically, that adding a retrieval step improves a named outcome relative to a stated baseline. If the result shows no improvement, that universal claim is withdrawn; it is not weakened into "improves sometimes" unless independent evidence supports the qualification. Showing that failure is possible refutes an unrestricted claim, but it does not establish that failure is inevitable in every setting.
Narrowing is available only when the failure is tied to a stated condition—say, a particular kind of input or a particular baseline—so that the revised claim specifies that condition and excludes the failing case. The condition must come from the design or from evidence, not from a search through the result for anything that separates successes from failures. Where the task or the inputs are not held fixed, the comparison is confounded and supports less than either withdrawal or narrowing; and where the encodings differ in more than the component under test, the result speaks to the whole arrangement rather than to that component's contribution alone.
A redesign responds to failure by changing the proposal—new retrieval policy, new decision rule, new integration point. That is development work. Its virtue is that it produces a fresh claim with fresh predictions. Its danger is rhetorical: presenting the redesigned version as though the original had been confirmed by the very result that defeated it. Relabelling cannot do this. When the success criterion is altered after the failure—say, a different outcome is named, or the threshold moves—what follows is a new test on new evidence. The old result neither defeats nor confirms the new criterion; it is silent about it. Until that test exists, the honest status is that the proposal's benefit remains unestablished under its revised terms.
42.7When to abandon the design
The competing proposal is the simpler competent annotated document. The integrated design should yield to it under a specific joint condition: once the task and inputs are held fixed and only the encodings differ, the simpler document delivers equivalent results while the integrated version imposes additional review cost and offers no distinct task need that its extra components alone serve. Each half matters. Equivalent results by themselves do not compel abandonment if a component addresses a requirement the simpler form cannot meet at all; conversely, a distinctive purpose does not rescue a component whose outputs no reviewer uses or checks.
The comparison must also be read correctly. "Equivalent results" names a comparison between two encodings of the same material, not a claim that either encoding is adequate in absolute terms, and "additional review cost" names a comparison of effort on matched material, not a general measure of difficulty. Neither figure, taken alone, licenses the abandonment decision. The decision is sufficient only when the quality comparison and the cost comparison point the same way and the distinct-purpose test returns nothing.
Equally, abandonment is possible without being inevitable. A finding that one added layer duplicates another's work is a reason to prune that layer, not yet a reason to discard the integration; only the accumulated failure of all distinct purposes crosses the line. As a conditional supposition: if the integrated document proved harder to trace through while producing no change in the decisions it informs, that would remove the main claimed benefit of connecting the layers, since tracing is the mechanism by which a correction is supposed to travel from source to action. Even then, the supposition would condemn the tracing apparatus first, and the remaining structure only if no separate task need supports it. No such comparison has yet been run, so the criterion states what would justify abandonment rather than reporting that it has been justified.
Abandoning the integrated design need not be an all-or-nothing verdict. The proposal bundles several claims: that the components belong together, that their combination yields something worth its added complexity, and that the distinctions drawn earlier are conceptually sound. The last claim is independent of the first two. A distinction may hold as an analytical matter while the architecture assembled from it turns out to be unnecessary. Retiring the architecture therefore leaves the distinctions standing unless they are separately shown to fail.
The retention test is reduced effort for the same need. A component stays only if it lowers the cost of meeting a task need that the simpler competent annotated document already meets, or if it meets a need that document does not meet at all. Serving a new need alone is not sufficient for retention without weighing the added costs and harms. The comparison must fix the task and the inputs and vary only the encodings; otherwise an apparent advantage may be an artifact of asking the two designs different questions. Whether any component passes this test here is not yet known, since the comparison has no outcome.
Suppose, as a conditional supposition, that one component reduces effort for a need the simpler document already meets. Retention of that component is then justified while the rest of the integration claim is abandoned; partial survival is the expected shape of a negative verdict. What does not follow is that the whole framework collapses conceptually. Loss of the integration claim licenses dropping the combination, not retroactively erasing distinctions that still do work in describing and comparing documents. Architecture failure does not by itself refute a distinction, but conceptual validity is not thereby proven.
One qualification limits this reprieve. A distinction that survives only as vocabulary, and never alters how a document is produced, reviewed or acted on, may support no component at all. Conceptual survival is weaker than design survival. The two can come apart: descriptive distinctions may remain useful while the integrated apparatus is retired piece by piece.