MERIDIANDECISION INTELLIGENCEOwner deskOwnerPublic screen
MERIDIAN / EXPLORE

From a question,
to the next decision.

View

Skin

Reading language
The screen, the questions and the judgments read in English where an English reading has been made. The Korean originals are unchanged, and evidence stays in the language it was written in.

What a Summary Cannot Decide: Question Compression, Pluralism, and Accountable AI · Part I · Questions before summaries

Chapter 6. Evidence and the research contract

By RISCON Labs AI. This conceptual study uses constructed examples and bounded readings of published work. Its proposed evaluations were not conducted. Model-assisted editing and translation do not constitute independent peer review. · First edition · October 2026

AI-generated editorial illustration: A woven cloth is folded into a small frame while some threads remain outside.
In this part
  1. 6.1 Kinds of claim and their warrants
  2. 6.2 A bounded literature map
  3. 6.3 Synthetic examples and their role
  4. 6.4 What a counterexample can show
  5. 6.5 Limits of source and quote verification
  6. 6.6 Negative findings and alternative explanations
  7. 6.7 A testable contribution statement

6.1Kinds of claim and their warrants

The manuscript separates its sentences according to the warrant that would support them. An attribution is warranted by accurate reporting: the manuscript can establish that an author asserts P without thereby establishing P. The attributed argument may itself supply reasons or evidence. Accurately reporting it does not automatically establish the strength of those reasons or the truth of its conclusion. A deduction is warranted by stated premises together with a valid inferential step. Its conclusion follows from those premises; whether the premises are well supported is a separate question, and a valid form does not rescue an unsupported starting point.

An invented construction is warranted by the stipulated setting in which it is defined. A synthetic counterexample can show that a proposed principle permits a specified case, or that two desiderata can pull apart under declared conditions. It says nothing about how often such cases arise, or whether any real protocol would produce them. A proposed observation is warranted, once carried out, by a record of something actually investigated. This manuscript's new reader study and protocol remain proposals: neither has been carried out here. Findings from earlier publications retain their separate status as bounded reports of prior work.

These are roles a sentence can play rather than exclusive bins for propositions. A single passage may attribute a construction to an author, reproduce a deduction using it, and point toward an observation that would bear on the result. What must remain clear is which warrant is invoked at each step. The example in the preceding chapter illustrates the discipline required: its binary flags and weights were declared choices, so the resulting ranking is interpretable but not a measurement of real losses. The same care applies here. Claims remain tied to the support actually offered—source interpretation, formal entailment, stipulated construction, or planned investigation—rather than borrowing authority from one role while resting in another.

Once a sentence has been assigned a role, the further question is how that role travels when a construction acquires additional support. The two-question collision in this manuscript is invented: it is a deliberately arranged interaction of questions, not a report of questions that were posed to anyone. Suppose later a reader study were conducted using such questions. Its records could support claims about those participants under those conditions, and that support may coincide with a proposition the manuscript had already asserted. The coincidence does not convert the original questions into a historical report. Their origin remains synthetic, because what is claimed about them here is that the construction is coherent and does particular argumentative work, not that it occurred.

This is why scope must be tracked at the level of the claim rather than the identity of its words. The same proposition can have more than one source of support: a synthetic construction can show that a possibility is available, while a later observation can show that some readers responded in a specified way. Neither source absorbs the other. Evidence licenses claims about the conditions that were actually examined; it does not license relabelling an invented event as an observed one.

A further limit follows. A future study could directly document responses under its administered conditions. Extending its findings beyond those conditions would require an explicit sampling or transport argument; the construction's generality would not supply that argument. Conversely, the invented construction continues to bear whatever argumentative weight its design supports, without thereby predicting how later readers will behave. Origin, then, is not a stain to be erased by accumulating support but a fact about which warrant each assertion actually has.

6.2A bounded literature map

The map organizes the selected readings by the question each one helps examine, rather than by discipline, chronology, or the prominence of their authors. A reading that bears on more than one question may appear in both places, but each entry records the particular passage consulted and the particular question it was consulted for. This keeps the role of a source legible without pretending that one passage settles several matters at once. It also preserves the origin of a construction: where a construction is developed here and a later reading happens to support a related claim, the map records the construction's origin and the reading's supporting role as separate entries, so convergence does not silently become attribution.

That organization depends on a distinction that is easy to lose. Discovering a source and reading its relevant passages are different acts. The working map therefore carries fields for source identity and version, the question the source informs, the passage actually read, a bounded claim the passage supports, and an unresolved limit on the reading. A record obtained only in abstract remains an abstract-only record; it can register discovery and can mark a question as awaiting closer treatment, but it cannot warrant unread methodological details or a claim that the whole work has been verified. A central argument explicitly stated in the abstract may still be reported within that scope. A source located but not yet read to the relevant passages likewise enters as discovery alone.

What this map can establish is correspondingly modest. It can show how selected readings bear on the questions this manuscript actually asks, and it can identify where a claim rests on a bounded interpretation rather than on the manuscript's synthetic constructions. It cannot establish what the literature at large says. It is a bounded working map, not an exhaustive review, a census of scholarship, or evidence that unlisted works are irrelevant, and no search counts are supplied to suggest otherwise. What a reproducible search record would require before any broader coverage claim is a separate matter, taken up next.

Organizing readings by the questions they help examine does not yet license a claim about the literature as a whole. That larger claim requires a different artifact: a reproducible search record, assembled in future work rather than reported here. Such a record would identify the research question that motivated retrieval, the collections searched, the query text used in each, and the dates on which the searches were run. It would state the eligibility rules, the procedure for handling duplicates, and the reason recorded for each exclusion. Where rules changed during the process, the earlier and later versions would both be retained, as would retrieval outcomes. A successfully executed query returning no items would be distinguished from an unreachable collection or an incomplete export; absence of matches is not itself a technical retrieval failure.

Two fields should remain separate in that record: selection and reading depth. A source may pass eligibility and still be consulted only for one passage relevant to a narrow question; another may be read across the whole work. Collapsing these into a single "included" marker would inflate apparent coverage, because a count of selected items is not a count of passages examined.

The inferential payoff is modest but real. A record of this kind makes the search inspectable: a reader can see which questions prompted retrieval, what was searched, and why items were kept out, and can therefore calibrate coverage claims to the material actually retrieved and read. It also exposes where a claim of broader coverage would outrun the evidence—for example, when the eligible set was assembled under rules that later changed, or when a collection could not be searched at all.

The limit is equally important. Recording a query improves inspectability; it cannot guarantee identical future retrieval or establish complete coverage of a literature. What such a record supports is a bounded claim—this search, these collections, these dates, these rules—not a census.

6.3Synthetic examples and their role

Compression can be tested here without waiting on a corpus, because the relevant collision is finite and can be enumerated. Two questions are supplied. The first asks, "Should the room close at 18:00 or 20:00?" and offers the alternatives 18:00 and 20:00. The second asks, "Should the room close at 18:00 or 22:00?" and offers 18:00 and 22:00. There are thus 2 distinct inputs and 2 distinct exact alternative pairs. The pairs share one member, but the first contains 20:00 where the second contains 22:00, so they are not the same pair.

Compress each question to the same sentence: "Which closing time should the room use?" The result is 1 distinct compressed sentence for 2 distinct inputs. The broad closing-time topic survives intact; the compressed sentence still asks which closing time the room should use. What does not survive is the attachment between a question and the pair of alternatives it offered.

The inferential step is short. Take a single-valued decoder whose only input is that shared compressed sentence. Because the sentence is compatible with both {18:00, 20:00} and {18:00, 22:00}, the decoder cannot return both distinct pairs correctly: no single output can answer two different pairings at once. This is a deduction about what the stipulated interface carries, and it is a deduction from the example as constructed rather than a general result about every possible decoder.

The relevant limit is built into that interface. The decoder receives the compressed sentence alone, without an input identifier and without the original question; the conclusion follows under that condition, and changing that condition would change the information at issue. The construction is finite and invented, and its two input counts and one compressed-sentence count are fixed by the example exactly as given.

The construction that has been set out has a narrow evidential role. It is a finite invented example, and what it delivers is a conditional impossibility: when distinct questions and their exact alternatives compress to one sentence, a single-valued decoder given only that sentence cannot return both pairs. The deduction concerns information availability alone. It supplies no prevalence estimate. The example shows that such a case exists; it does not show how often it occurs among genuine questions, how much of any corpus it touches, or whether it is a marginal curiosity or a routine hazard. Incidence would require enumeration or survey, and neither has been performed here.

Nor does the construction speak to readers. It does not establish that a person notices the ambiguity, treats the compressed sentence as misleading, or experiences the missing alternatives as consequential. No participant study has been run and no human performance was measured. A study in which readers encounter the two questions alongside the compressed sentence would be a proposal, with its outcome left open. Even a task supplying both inputs would leave unresolved how attention is distributed, what is inferred from the sentence alone, and whether alternatives are recovered without prompting.

A third limit concerns representations that also preserve original access. If a representation provides the original questions, or identifiers together with an accessible mapping to their pairs, exact recovery becomes possible in principle. Distinct labels alone do not tell a decoder which alternatives each label denotes. That alteration still does not establish how the representation performs when people use it. Access to an original changes the information available but does not guarantee that a person notices or reads it; whether either occurs, and what follows, remains empirical. Claims about such a representation must therefore stay conditional on its content and its use, rather than being read off the collision.

The example's application is accordingly limited: it establishes possibility and conditional impossibility, while corpus incidence, reader response, and the performance of a richer representation remain undetermined.

6.4What a counterexample can show

The collision supports one conclusion, and its force is exactly that of a counterexample to a universal claim. Suppose it is claimed that whenever a compression preserves the broad topic of a question, it also preserves the exact named alternatives presented with that question. The two checked inputs bear directly on this claim. Q1 asks whether the room should close at 18:00 or 20:00; Q2 asks whether it should close at 18:00 or 22:00. Each input therefore supplies its own pair of named alternatives. Yet both compress to the identical sentence, Which closing time should the room use? The broad closing-time topic survives in each case, while the surviving sentence does not say whether the associated pair is 18:00/20:00 or 18:00/22:00.

The inferential step is modest but decisive within its stated domain. The universal claim asserts that topic preservation is sufficient for preserving the exact named alternatives. Here the antecedent holds—two distinct questions compress to one sentence that retains their broad topic—and the consequent fails, because one compressed sentence is compatible with two distinct alternative pairs. A single in-domain instance of that shape is enough to reject the universal. The example is invented and finite; it carries no claim about how often such collisions arise in a corpus, about how readers handle them, or about the performance of a representation that also supplies original access or additional identifiers. Information of that kind would change what the task has to work with.

What remains established is the narrower negative: broad topic preservation by itself cannot guarantee recovery of exact named alternatives. That result does not make topic preservation worthless for other purposes; it blocks only the sufficiency claim. The productive question is therefore not whether some representation preserves everything about a question, but whether it preserves enough for a specified task.

Rejecting the universal claim still leaves a smaller positive result. In the checked construction, both inputs ask about the room's closing time, and each compresses to the same sentence, "Which closing time should the room use?" So the compression retains the broad closing-time topic in these two stipulated cases. A reader can identify that stated topic without recovering the two original pairs. This is a positive property of the two constructed outputs, rather than a demonstrated classification accuracy over unfamiliar questions.

The positive claim needs its own domain boundary. A collection containing only closing-time questions would not test whether a classifier can distinguish that topic from lighting, staffing or access-card questions. A wider classification claim would require wider material and a specified rule for assigning topics. The present construction instead fixes a limited comparison: its wording retains the closing-time subject while leaving an exact-alternative task unresolved. Changing that task does not repair or invalidate the earlier deduction; it changes which retained information counts as adequate.

The contrast matters for scope. It would be wrong to conclude from the construction that compression destroys everything, since the broad topic survives here. It would be equally wrong to conclude that the construction somehow demonstrates reliable recovery of named alternatives; it shows the opposite for the stipulated case. Nor does the construction establish that every summary preserves the topic, or assign any prevalence to such collisions. It is a finite invented example, not an experiment with readers, and it measured no human performance.

Applied to this manuscript, the counterexample licenses a narrow inference: some compressed question can support topic-level classification while failing exact-alternative reconstruction. The manuscript cannot claim, without unperformed work, how often that gap arises in practice or how people cope with it when it does.

6.5Limits of source and quote verification

Verification is often treated as one act, but three questions have to be kept apart. The first concerns identity: which work is meant? Sufficient bibliographic information, including a version where needed, can distinguish the intended item from similarly named works or later revisions. A title or author list alone may leave that identity ambiguous. Resolving it still does not establish what the relevant passage says.

The second question concerns location. Once the work is identified, a particular passage can be inspected, and the result is knowledge of what text is present and where. A remembered phrase, a search snippet or a secondary description does not yet supply that. Yet even a carefully located passage answers only what the inspected text says; it does not decide how the manuscript may use it.

The third question is a comparison between a manuscript sentence and the passage beside it. Here subjects, conditions and strength of assertion must match. A source statement about one population does not license a sentence about people generally; a qualified or conditional formulation does not license an unconditional one. Each dimension can hold or fail independently, which is why agreement in wording is not yet agreement in claim.

A bounded example shows the separation of dimensions without settling any of them. Gao et al. (2023) (Gao et al., 2023) describe ALCE as evaluating fluency, correctness and citation quality as separate dimensions. This is offered only as a brief example of multidimensional evaluation: it is not evidence that a bibliographic match certifies semantic support. This use is confined to the checked abstract's stated dimensions.

The taxonomy itself belongs to this manuscript; ALCE supplies neither the distinction nor its certification. Its relevance here is that identifying a work, locating its words and testing a sentence against them are distinct procedures, each succeeding or failing on its own terms. A failed comparison does not make the cited source wrong; it shows only that the manuscript sentence is not yet licensed by the text inspected. Their separate records make the remaining check identifiable.

A quotation can match perfectly and still fail to support the sentence that follows it. Identifying the work and locating the relevant words settle only which text is in play. The further question is whether that text, with its own qualifications and restrictions, licenses the scope claimed for it here.

Consider an invented methodological note stating that the procedure was checked only for task T. If a manuscript sentence states that the procedure was checked only for task T and cites the note, wording and scope agree. If the sentence instead states that the procedure was checked for every task, attaching the note verbatim does nothing to repair the claim. The string matches; the application does not. The note confines the check to task T, while the sentence extends it to every task, and faithful transcription cannot close that distance.

The defect is therefore an unsupported change of scope, not a mismatch of characters. Exact quotation can establish which words were used and can preserve a qualification such as "only." It cannot enlarge the range of cases to which the source's claim applies. Whenever a manuscript sentence broadens a condition, drops a restriction, or moves from one task to all tasks, wording verification may report success while support for the manuscript's own sentence remains untested.

The note is invented rather than evidence about a real publication. Its role is to make an unsupported extension inspectable without alleging that a cited author made it. A review can therefore record string fidelity and evidential support separately: success on the first check leaves the second question open.

6.6Negative findings and alternative explanations

Because no participants have yet completed the task, the outline can still specify what a future comparison would have to resolve. The task is concrete: for each supplied source question, the participant must recover the exact alternatives stated in it. An item counts as correct only when the stated pair is recovered; omitting one alternative or adding another is an error. That rule is observable at the item level and does not depend on a judgment about how much time was saved.

The comparison itself must then answer several questions. First, do the compared conditions use the same source materials and the same task, so that differences cannot be attributed to easier questions or a different demand? Second, how do time and correctness move together, or apart? They must be recorded separately: a faster response that produces a missing or added alternative is not equivalent to a slower one that reproduces the pair. Third, what role do prior access to the original and prior familiarity with the task play, and are these balanced or measured rather than left as untracked background? Fourth, are the errors the same kind across conditions, or does one condition trade fewer omissions for more additions?

These questions would distinguish possible outcomes. A comparison might find no demonstrated advantage, a gain in time alongside unchanged or worse correctness, or a difference associated with access or familiarity. Whether a tradeoff counts against the prediction depends on the declared criteria. Identifying a causal explanation would require a design that separates the relevant influences, not merely a list of measured background variables.

The relevant limit is that this remains an outline, not a completed or fully preregistered protocol. Group-level acceptance thresholds and sampling have yet to be specified before data collection. Until they are, the item-level rule fixes what counts as an error, but not what result across items would justify accepting or rejecting a prediction.

The relevant distinction is evidential, not rhetorical. Since no human study has been run and no protocol deployed, nothing here has produced an outcome that the proposed representation failed to predict. A study that was never run leaves the comparison's questions unanswered; it does not answer them in the negative. Consequently the manuscript cannot report a null effect, an adverse result, or a failed replication, because each of those would require measurements that do not exist in this record.

That distinction governs how a later negative finding would have to be read. A measured result would need its conditions and uncertainty reported before it could count against the representation at all. Even then, an absence of detected difference would not by itself establish equivalence between the representations under comparison: it might reflect a difference too small for the design to resolve. Establishing equivalence would require a separate argument about the precision the comparison could sustain, together with the item-level error rule that counted a discrepancy as an error in the first place.

Alternative explanations are likewise scoped by how the disappointing outcome arose. If the measurement procedure broke down, or the comparison changed conditions it was meant to hold fixed, attribution would be uncertain. That failure would neither establish the representation's defect nor automatically exonerate it. If a well-executed comparison failed to demonstrate the proposed advantage, that advantage would remain unsupported under the tested conditions. Preferring the simpler alternative would additionally require the declared adequacy and burden criteria; an inconclusive difference alone would not prove its superiority. Keeping these cases apart prevents the flexibility of "alternative explanation" from absorbing every unfavorable result. For now the distinction is a constraint on interpretation: what is established is the set of inspectable constructions and the questions the comparison must answer, while its outcome remains open.

6.7A testable contribution statement

The contribution can therefore be stated without borrowing strength from evidence that does not yet exist. This manuscript supplies conceptual distinctions, explicitly synthetic constructions that expose scope errors, bounded interpretations of primary sources, and a proposed empirical comparison. None of these elements requires pretending that the comparison has already yielded a result.

The distinctions do their work by making different claims separable: attributing a claim, deriving it, constructing an example and proposing an observation invoke different warrants. A passage can perform more than one of these roles, but a proposed observation is not already a finding. The constructions then make one narrower contribution inspectable. Some constructions expose an unsupported inference; others provide a positive baseline or a comparison. Their stated premises let a reader check the particular conclusion and identify where a revision is needed. That is a conceptual result at its actual evidential level, not a surrogate for participant data.

The proposed study does different work. It addresses practical usefulness by comparing how specified tools perform under specified tasks. The outcome remains open. If participants perform poorly, that would challenge the usefulness of the particular design tested; it would not by itself refute the scope arguments, nor would good performance validate general claims beyond the tested conditions.

This split matters, as 06.6.2 set out: not running a study and running one that fails to support a prediction are different states, and here only the first obtains, so the empirical work remains proposed. The specification gaps identified in 06.6 also remain open before data collection.

What remains testable is not a vague promise of future validation but a set of bounded claims with identified revisability conditions. The constructions can fail under scrutiny of their premises; the design can fail under task performance; the literature interpretation can be wrong at its stated scope. Those are distinct vulnerabilities, and keeping them distinct is what makes the contribution checkable now.

The proposed arrangement has a serious competitor, and it is not a rival theory. Careful ordinary editing, backed by accurate source pointers, may perform the same task with less effort. That possibility deserves more than a gesture. If a reader can inspect the relevant sources, keep distinctions straight, and avoid errors with nothing more than a well-edited summary, the mere availability of additional machinery does not establish an incremental benefit. Its case depends on what it adds under the same task and resource criteria.

A future comparison can provide evidence for that choice, although imprecise results may leave it unresolved. Both the proposed arrangement and an ordinary edited summary should receive adequate access to the same sources, and both should face the same declared criteria. What matters is task-relevant error on the one side and total preparation, reading, and maintenance effort on the other. These are the terms under which the machinery can lose. If it produces no relevant gain while adding burden, the right response is to simplify it or abandon it for that application, rather than to defend it on grounds the comparison was never designed to test.

The resulting statement is conditional, not a finding. No comparison has been run here, so it announces neither a measured result nor a general refutation of conceptual distinctions. Its scope is practical and local: it governs what to do when added structure buys nothing that ordinary editing cannot supply under the declared criteria. Distinctions may still clarify assumptions or expose hidden choices even where they fail an efficiency test.

The formal models in Part II can establish consequences of stated assumptions and make those consequences inspectable. They cannot supply observations of participant behavior or deployed effectiveness. Their evidential role is therefore useful but different from that of the proposed empirical comparison.

RISCON Labsrisconlabs.com