19.1A persuasive sentence without support
Consider a supplied record containing one sentence stating that digitization of the parish register began in March. A hypothetical rewritten summary could combine the supported March start with an added claim that the building dates from 1912. Nothing about 1912 appears in the record. The added date is a detail a reader might find useful, it fits the subject, and it arrives in the same smooth cadence as the supported clause. That is exactly why it is dangerous: the sentence offers one voice, one rhythm, and one surface of confidence, while the record licenses only half of its content.
The inferential step is simple. Once an unsupported claim is welded to a supported one by ordinary style, the reader receives no structural cue that their evidential positions differ. The fluent presentation supplies no explicit mark separating their evidential status; whether a reader overlooks the difference has not been tested here. This is why faithfulness must be assessed separately from how well a summary reads. Maynez et al. (Maynez et al., 2020) evaluate hallucinated content in summaries and treat source faithfulness as distinct from ordinary generation quality measures. A polished summary can therefore score well as prose while importing content that its source never supplied.
The relevant limit is that this argument does not require summaries to repeat their sources literally. Useful abstraction compresses, selects, and generalizes; forcing verbatim reproduction would trade one defect for another. What the sentence above needs is not a more wooden style but a markable gap: one can point to the 1912 clause and say that the record does not supply it. That identifies a concrete point for correction. An explicit disclosure could make the distinction visible, although its effect on readability remains to be assessed.
These are authored contrasts rather than outputs from a named system, and no failure rate for current models is implied here. The date in the example is stipulated for the purpose of the demonstration; whether it is independently true is a further question that the record-based analysis does not settle.
Fluency is not the only thing at stake: an addition can be true and still fail as a report of the record it accompanies. Consider a digitization summary that states the building was constructed in 1912, although the source it summarizes gives no construction date. The date is stipulated to be independently true. That makes the sentence useful and harmless as history, but it does not make the sentence supported by the record. Its defect is one of scope: it speaks as though the record covered a fact on which the record is silent.
The practical move is therefore to distinguish two questions that tend to collapse into one. Was the claim true? Here yes, by stipulation. Did the record support it? Here no. Inspection of the supplied record can answer the second question; it cannot establish the first, and absence of a date in the record is not evidence that the building had none.
This matters for application because support is a relation between a report and a specific record, not a property a sentence carries with it. If the summary instead attributes the date directly to the record—stating, for example, that the building was constructed in 1912 according to the record—it adds a further error, misattributing the claim—but the fix is not forced literal repetition. The summary may keep the addition if its status is disclosed: for example, by marking it as external to the record. Disclosing provenance preserves useful abstraction while keeping the boundary visible.
One limit deserves stating. The demonstration stipulates the date's truth rather than deriving it from any source or observation; if the date were false, the same summary would carry two distinct defects—false content and unsupported reporting. The point is not that true additions should be deleted, but that truth alone cannot convert them into faithful reports of their records.
19.2Factual truth versus source faithfulness
Summarization systems can introduce content that their sources do not support, and Maynez et al. evaluate such hallucinated material while treating source faithfulness as distinct from general generation quality (Maynez et al., 2020). That study is historical in scope, and its findings are used here only to motivate a conceptual distinction, not to characterize any present-day system. The distinction itself is best made concrete, because the two failure modes it separates are easily confused: a summary can be right about the world yet wrong about its document, and it can be right about its document yet report something false.
Consider a hypothetical summary task. A supplied document describes a municipal archive's recent digitization program and says nothing about the building's construction date. A summary states that the archive, completed in 1912, has digitized its holdings. Stipulate that an independent record confirms the date. The added fact is then true, but that stipulation tells a reader nothing about how the summarization system might have obtained or guessed the date. It is nonetheless unfaithful as a summary, because the reader who takes the sentence as a report on the document will attribute to that document a fact it does not contain. World truth and source support answer different questions, and a correct guess about the world does not retroactively supply the missing warrant.
The converse case separates the same axes from another direction. Suppose a fictional document asserts that a river froze solid, while the example separately stipulates reliable observation that it remained liquid. A summary writes that the report claims the river froze solid. The summary faithfully represents the source's assertion, including its status as an assertion; yet the underlying proposition is false of the world. Faithfulness here consists in accurate reporting of what the source says, not in confirming that what the source says is correct.
Three notions should therefore be kept apart. Source faithfulness asks whether summary content is licensed by the document; world factuality asks whether the content holds outside it; endorsement asks whether the summary commits to the content as its own claim. Clear attribution can report an assertion without endorsing it; the surrounding prose must maintain that distinction. The embedded claim’s factual status remains a separate matter, while an unsupported true statement remains a faithfulness failure despite its accuracy.
Source support tells a reader what a document licenses; it does not by itself settle whether a reader should adopt the licensed content as a factual conclusion. Take the river case. A source states that the river froze, and the statement is false. A summary can faithfully write that the source reports the freezing without committing to the event itself. Attribution preserves the distinction between reporting and endorsing. But once the summary drops the attribution and asserts that the river froze, the summary has crossed from source reporting into factual commitment. The source no longer protects that commitment, because the source is wrong. What is needed at that point is checking outside the document.
The digitization case shows the converse. A summary supplies a construction date of 1912, and the date is stipulated to be true, yet it appears nowhere in the source. The sentence is factually correct and still a source-faithfulness failure, since the source does not license it. The date can enter the summary's own factual account only on independent evidence. Its truth does not retroactively make the summary a faithful rendering of the document.
The application rule follows. A source-supported assertion may require outside checking before it is adopted as the summary's factual conclusion, and an assertion supplied from outside the source requires outside evidence before it enters the summary as source content. These are two different failure routes to the same requirement: once the summary's voice is attached to a claim, documentary license is no longer the relevant test.
The limit matters. Factual checking concerns the claim's truth, not its wording. A verified point may still be compressed, paraphrased or abstracted; it need not be repeated in the source's language. What verification disallows is silent substitution of the summary's endorsement for an attributed report, or of outside knowledge for source content. The invented sentences here are defect demonstrations, not measurements of any model's behavior.
19.3Attribution and altered subjects
The contrast between truth and source faithfulness leaves a further question open. A summary may carry a predicate over from its source while quietly changing who is presented as producing it. Take the invented case: a single named contributor requests later hours. A rewritten summary reports that the group requested later hours. The predicate survives almost word for word, yet the sentence now attributes the request to a collective rather than the named person. That attribution need not mean every member agreed, but it still requires a basis absent here.
Notice what has happened in the rewrite. The requested scheduling change stays the same, but changing the subject changes factual content: who is said to have made the request. The collective formulation may sound natural while removing the named contributor from view. The sentence remains coherent and easily reads as a faithful condensation. It is also, on the stipulated record, unsupported: no group endorsement is supplied.
This is not an objection to compression as such. Summaries must abstract, and a source recording an adopted group request could license that collective subject. Several individual requests might justify reporting several requesters without establishing a group mandate. The relevant limit is that those materials are absent here; even if some other member happened to want later hours, that supposition would not by itself make the group the requester. A conditional fact about unrecorded agreement cannot widen the attribution of the sentence under review.
The demonstration is accordingly narrow and deliberate. These are authored contrasts rather than samples of any named model's behavior, so the point is the inspectability of the defect, not its measured frequency. Its defect is visible from the contrast alone, without supposing how the summary was produced. What is established is that a retained predicate leaves the enlarged subject unlicensed. What that enlargement costs the summary is the next matter.
Altering the subject enlarges the field of speakers behind a claim, and the untouched predicate does nothing to restrain that enlargement. In the invented case, one named contributor requests later hours; the rewritten summary says the group requested later hours. The predicate, requesting later hours, is carried over accurately. What changes is who stands behind the request. Since no group endorsement is supplied, the rewritten sentence attributes to the group an act performed by one participant. This is a defect demonstration, an authored contrast rather than a measured output from a named model, and its point is diagnostic: the action is described correctly, so surface agreement with the source makes the sentence look safe.
The inferential step is to hold the subject and the predicate to separate checks. The predicate answers what was said; the attribution answers who made the claim. A summary can be right about the action and wrong about its speaker, and re-inspecting the action will never expose the second error. Here the predicate has been retained unchanged precisely because that retention creates the illusion of fidelity. The scope has drifted from an individual request to collective commitment, and no fact in the invented source supports the wider subject.
The application, then, is to return to who actually made the claim: a named contributor requested later hours, or more simply, a request was made. That repair does not require literal repetition of the source sentence. Compression may recast phrasing, drop detail, or group related facts, provided it does not widen the set of speakers or add endorsements that no source establishes. The relevant limit is narrow and strict: when collective support is absent, predicate-level fidelity cannot license a group subject. Once the subject is restored to the person who spoke, the summary can still be brief without claiming authority it never had.
19.4Qualification and modality
Take an invented source sentence: a measure may help. A rewritten summary says the measure will help. Nothing else needs to change for the summary to assert more than the source did. "May help" leaves the outcome open; "will help" closes it. The subject, the measure and the predicate remain intact, yet the modal force has moved from possibility to prediction.
The step is easy to miss because the strengthened version reads better. A sentence that promises an effect is tidier than one that declines to promise it, and that tidiness supplies no evidence that the qualification was redundant. In the stipulated notice case, repairs may finish Friday with inspection pending; a candidate summary says they will finish Friday. The deadline survives, but the pending inspection — the condition that made the notice hesitate — has no counterpart. What remains is not a paraphrase of the notice's claim but a stronger claim built from most of its words.
This does not mean the stronger sentence is necessarily false. Repairs might finish Friday, and the measure might help. If independent evidence settled either question, a writer could responsibly assert the stronger form. But that evidence would have to come from outside the source, and the summary as written supplies none. The defect is therefore one of source fidelity rather than factual error: the text presents the source as more definite than it was, while every individual fact it mentions may still be accurate.
The distinction between source fidelity and factual error is the practical limit on aggressive checking here. Demanding literal repetition would destroy useful abstraction, so the test is not whether the summary uses the source's words but whether it preserves the source's asserted strength. Because these contrasts are authored demonstrations, they show how a modal change alters a claim; they do not measure how often any system makes it. The remaining question is whether a shorter modal form can carry the same uncertainty.
The comparison just made turns on one contrast, but the notice contains two separable pieces of hedging: the modal verb "may," which withholds assurance about Friday, and the qualification "inspection pending," which supplies the reason the timing remains unsettled. A summary may legitimately compress these into "repairs may finish Friday." That is shorter, it drops the condition, and yet it preserves the strength of the claim: Friday is offered as a possibility, not a forecast. The reader loses the ground for the uncertainty, but not the uncertainty itself.
Compression becomes a defect when it removes the modal rather than the condition. A candidate that reports "repairs will finish Friday" has upgraded possibility to expectation; one that reports "repairs finish Friday" asserts a settled outcome. In either case the surviving qualification, if any, cannot do the work the verb has already undone. Appending "inspection pending" to a categorical verb leaves the dominant assertion untouched, because modal strength is carried by the predicate, not by a trailing clause. Deleting "may" is therefore not a stylistic tightening; it changes what is claimed.
The limit is that omission of a qualification is not automatically a change in asserted strength. Where the task requires only the scheduling possibility, "repairs may finish Friday" may be exactly the relation the reader needs, and insisting on the full notice would be literal repetition rather than fidelity. The qualification matters when the summary's purpose extends further—say, to conveying readiness or permitting planning—because then the missing condition misleads even though the modal verb is intact. These are authored contrasts, not samples of measured model behavior, and they diagnose the scope of a compression decision rather than its frequency.
19.5Useful abstraction
Abstraction is doing its job when it preserves the relation a task turns on while dropping wording that carries no part of that relation. Take an invented source sentence: the digitization review began in March and paused after two trays. A rewritten summary — the digitization review was interrupted after two trays — omits the month. If the task is to report the interruption and its sequence, nothing needed has been lost; demanding the month back would mistake sentence-literalness for fidelity.
That does not make omission costless. The test is whether the omitted material is load-bearing. In the contrast case already in view, a digitization summary adds a 1912 construction date that is independently stipulated true but absent from its source. The addition is not rescued by its truth: the summary now attributes to the source something the source did not say. Omitting an incidental detail and inserting an unmarked detail are not mirror defects. The first can leave the task-relevant relation intact; the second changes the record of where a claim came from. These are authored contrasts, chosen to make the distinction inspectable rather than samples of any model's behavior.
The same line runs through scope and strength. If one named contributor requests later hours, a summary saying that the group requested later hours has generalized past the support supplied, even though a vaguer report of the request would have been legitimate abstraction. And a shorter modal expression can retain uncertainty — repairs may finish Friday with inspection pending — whereas stripping the modal asserts more than the notice did. So the boundary of useful abstraction is not brevity but preservation of the relation the task needs, including who is claimed to have said what and how firmly. Checking that respects this boundary does not force literal repetition; it distinguishes load-bearing content from wording that merely accompanies it.
Abstraction that preserves the relation a task needs will rarely reproduce its source wording. Consider the notice stating that repairs may finish Friday with inspection still pending. A summary saying that Friday completion depends on an inspection that has not yet occurred reorganizes the sentence considerably, but it carries the same conditional relation. A verifier demanding literal identity would reject it, and in doing so would reward copying over understanding. Support should therefore be assessed against the claim the summary actually makes, not against surface resemblance to the source.
That framing also separates two questions that literal matching tends to blur. Suppose the source says the river froze and the statement is false. A summary reporting that the report says the river froze is supported by the source and faithful to it, because the claim actually made is about what the report says. The summary neither endorses the freeze nor adds a new factual assertion. Conversely, the 1912 construction date may be independently true, yet a digitization summary that supplies it is unsupported by that source: the date is absent from the claim set the source offers. Truth and support come apart here, and only the latter is at issue in checking a summary against its sources.
The same claim-relative test catches the defects that abstraction might otherwise conceal. Where a single named contributor requests later hours, saying that the group requested later hours asserts an endorsement nobody supplied; the paraphrase's fluency does not repair the scope error. Where repairs may finish Friday, "they will finish Friday" converts a conditional into a commitment. Neither error is word choice alone.
The relevant limit is this: judging support by the claim made widens what counts as a legitimate paraphrase, but it does not license strengthened modality, enlarged attribution, or added premises. These are authored contrasts for inspecting defects, not measurements of any model's behavior.
19.6A synthetic correction
Support is judged against the claim actually made, not against literal overlap with the source. Take the invented notice, which states that repairs may finish Friday provided the inspection is satisfied. A synthetic draft says, "Repairs will finish Friday." The sentence is smooth, and Friday may indeed turn out to be the finish date; that is beside the point. What the draft asserts is unconditional completion, while the source offers only a conditional possibility. The confidence has been added rather than carried forward.
Two changes need to be separated. The draft drops the inspection condition and strengthens a possibility into a prediction. Even a satisfactory inspection would not, by itself, make completion certain; the notice still says only that repairs may finish. Repair does not require literal repetition of the notice. A verifier allowed to work on the claim actually made would not demand that the summary repeat the proviso verbatim; it would check both the condition and the modal strength of the claim. A verifier combining consistency prediction with supporting or conflicting spans could mark the disputed passage and point to the conditional span as the relevant boundary (Kryscinski et al., 2020). Span extraction and classification remain fallible, so the apparatus locates the disagreement rather than certifying a verdict.
A narrower hypothetical correction would report that the notice allows for completion on Friday only if the inspection is satisfactory. This gives up the confident timetable but reports what the source licenses. The gain is inspectable in two places: the summary now asserts less, and the reader can see which part of the notice supports it. Nothing in this correction predicts whether the repairs actually finish Friday. The point is narrower and more useful. A summary that preserves a source's conditionals has not failed to abstract; it has abstracted to the level the evidence permits, and the residual gap between possibility and prediction is now visible instead of hidden inside a well-formed sentence.
Once the digitization defect is on the table—the 1912 construction date is stipulated true, yet the source does not supply it—the question is what may count as a repair. Not a plausible empirical fact offered to fill the gap, and not a second unverified detail offered as corroboration. Either device repeats the original move at a larger scale: it converts an unsourced addition into something that reads as an attested record, and the summary becomes more convincing precisely where its basis has not improved. The same logic governs the other authored contrasts here, including the river, hours, and repair notices; they are stipulated demonstrations of defective sentences, not samples of any system's output.
Two repairs remain legitimate, and both preserve abstraction. The sentence can retreat to what the source gives, dropping the date and reporting only the scope of the surviving materials; or it can keep the date explicitly as supplied from outside the source, so its provenance is stated rather than implied. If a construction date is independently established, it may enter the summary as a labeled addition, never smuggled in as source content. Neither repair demands literal repetition of the source's wording. Aggressive checking would destroy useful abstraction only if it made paraphrase itself suspect; the force of the check is narrower. It asks whether each widening of scope or strengthening of modality is licensed.
The river case shows why truth is not the test the summary must pass. Because the source falsely says the river froze, a summary that reports the claim as a claim is faithful to the record without endorsing the freeze. In the hours case, one named contributor requested later hours, so 'the group requested later hours' adds endorsement nobody supplied. In the notice, repairs may finish Friday with inspection pending; 'will finish Friday' deletes the condition.
One qualification constrains all of this: a summary need not reproduce every available detail. Omission becomes a defect when a dropped premise is what an inference depends on, not when the omitted material is merely present in the source.
19.7From defect detection to revision
Detection alone leaves the review incomplete. A useful review records three things: the defective move, the source sentence that licenses or fails to license it, and the revision that removes the defect. Without the middle term the revision is guesswork; without the third the defect is merely catalogued.
Consider the digitization summary that adds a construction date of 1912. The date is independently stipulated to be true, but the source sentence does not contain it. The defect is therefore one of source fidelity rather than factual accuracy, and the revision is correspondingly narrow: the summary must stop presenting the date as something its source provides. A conditional alternative exists if a separate source were permitted to supply the date, but this contrast does not license inventing one. The river case runs the other way. The source falsely claims that the river froze; a summary may report that claim as a claim without endorsing it. The defect appears only where assertion-by-a-source is converted into established fact, and the revision restores the attribution.
The same three-part record sharpens the remaining contrasts. When one named contributor requests later hours, a summary saying that the group requested later hours enlarges the scope of attribution beyond anything supplied; the revision names the individual and adds no group endorsement. When the notice says repairs may finish Friday with inspection pending, the candidate's statement that they will finish Friday strips away both modality and condition; the revision restores the possibility and the outstanding inspection.
Linking defect to source to revision does not require forcing the summary into literal repetition of its source. Abstraction can remain, provided that omission is not converted into implied support and that a qualified claim is not flattened into a certain one. What the link secures is a targeted repair with a stated reason; it does not by itself certify the revised summary as a whole.
Linking a defect to its source and to the revision needed is necessary, but it is not the end of the review. The revised summary must then be inspected for two further things: whether the correction actually removes the defect, and whether anything necessary has disappeared along with it. A passing classifier score settles neither question.
Take the digitization summary that supplies a 1912 construction date absent from its source. Suppose a revision deletes the date and, in the same place, also drops the independently supplied account of when digitization began. A detector trained to flag unsupported construction dates may award the revision a clean score, because the flagged date is gone. Yet the omission is real and consequential. The score records the fate of the detected defect; it says nothing about content the detector was never asked to track. Adequacy therefore requires a separate omission check, directed at the claims that were present before revision and should still be present afterward.
The same limit applies to corrections that alter attribution or modality. Where one named contributor requests later hours and the summary attributes the request to the group, a revision that simply deletes the sentence is not thereby adequate: it removes the false collective endorsement but also loses the individual request. Better is a revision that restores the narrower claim. Likewise, where the notice says repairs may finish Friday with inspection pending, a revision that recasts the prediction as a certainty corrects nothing; it merely exchanges one modal error for another. These are authored contrasts for illustrating the inference, not outputs measured from any model.
The qualification is narrow but important. A classifier score is evidence about the particular relation it measures—unsupported construction dates, perhaps, or dropped clauses under a specified rule. It cannot certify that the correction is substantively adequate or that omission has been ruled out. Those must be checked explicitly, and checked against the source rather than against the score.