22.1The comparison task
A paired comparison of two summaries of the same material does not ask a single question. It can ask which version is easier to read, which is more accurate, or which is more agreeable, and these are distinct tasks. Readability concerns the effort of following the text; accuracy concerns its fidelity to the source; agreeableness concerns its fit with the reader's views or tastes. A person may find a concise summary easy to use because it saves time while preferring a longer version because it preserves a condition. Those are stipulated interests, not collected ratings, and neither preference settles the other question.
The proposed evaluation therefore has to match what the comparison is meant to isolate. If readability is under test, the two versions should have matched source access and matched factual content. Suppose instead that one version also changes a disputed conclusion: a reader's preference then responds partly to that difference, and the answer cannot isolate readability. Likewise, an agreeable summary may be the one that softens a conclusion the reader dislikes, while the accurate summary is the one that retains the source's qualification. Asking "which is better?" without specifying the criterion gives no way to tell which standard the answer reflects.
The same distinction bears on whose preference counts. A personal reading aid can answer its user's declared purpose, and that purpose is enough to define the task for that user. A committee brief has a separately assigned remit and affected parties; the preferences of its immediate audience do not automatically supply that authority. So the comparison task must name both the criterion and the constituency before the resulting ranking means what this study takes it to mean.
Even once the comparison task has been named, a single overall preference answer leaves the reason for the choice underdetermined. A reader asked which of two summaries is better may be responding to length, tone, organization, factual coverage, or some balance among them, and the preference record alone does not tell a reader which. Attributing a choice to readability therefore requires additional evidence collected alongside the ranking: either a question that asks specifically about readability, or a paired design in which source access and factual content are held matched across the two versions. In the latter case the inference to readability is supported only under those matching conditions. If one version also changes a disputed conclusion, preference cannot isolate readability, because accuracy and agreement now travel with the choice.
The stipulated interests show how easily reasons diverge. A fictional reader may prefer the concise summary because it saves time, while a contributor prefers the longer one because it retains a condition that the short version drops. These are declared interests, not collected ratings, so no ranking result is available here; but the example marks the interpretive gap. Were such readers actually to rank the versions, a preference for the concise text would register a preference for speed, not an endorsement of what the omission costs, while a preference for the longer text would register concern about the condition, not a verdict on readability.
This matters for scope. A personal reading aid answers its user's declared purpose: if that user wants speed and accepts the trade-off, the preference is responsive to the task set. A committee brief has a separately assigned remit and affected parties, so its quality cannot be settled by whatever its immediate readers happen to prefer. Neither context automatically supplies the other's authority. Preference learning remains useful here as evidence of how specified readers choose under specified instructions; it does not by itself establish factuality, agreement on contested claims, or permission to authorize representation.
22.2Who provides the preference
A single overall preference answer leaves the reason for the choice underdetermined. The next question is equally basic: whose preference is being recorded? Contributors, intended readers, hired raters and decision owners each stand in a different relation to the material, and a preference study must name the group from which its judgments come before it can say what those judgments are evidence of.
Consider the stipulated case. A fictional reader prefers a concise summary because it is faster to use, while a contributor prefers a longer version because it retains a condition. These are stated interests in the example, not collected ratings; turning them into data would require an actual study with named participants and stated questions. Even so, the example shows why provenance matters. The two judgments answer different questions: one concerns the experience of reading, the other concerns what representation of the material is acceptable to the person represented. Averaging them into one score would hide the disagreement rather than resolve it.
Hired raters occupy yet another position. They can apply a task consistently, but their answers speak only to whatever the task instructions asked them to judge, and the population from which they were recruited still limits generalisation. Decision owners bring a remit: a personal reading aid answers its user's declared purpose, whereas a committee brief has separately assigned responsibilities and affected parties. Neither context automatically supplies the other's authority.
Accordingly, a proposed paired evaluation should keep source access and factual content matched when comparing readability. If one version also changes a disputed conclusion, the resulting preference cannot isolate readability at all, whatever group supplies it. Where judgments from several roles are collected, they should be reported separately, with the question each group was asked attached to its answers. If a study instead elicits one pooled preference, any claim about a particular constituency must be treated as conditional on the composition of that pool. Naming the source of a judgment establishes its provenance; how far the judgment binds anyone remains a further question.
Even when the reader is identified, the scope of the answer must be specified. In the proposed paired comparison, the reader sees two summaries of the same material, with source access and factual content held constant, and answers which is easier to follow. That judgment bears directly on readability. The stipulated facts make the boundary visible: a reader may prefer the concise version because it is faster, while a contributor prefers the longer version because it retains a condition. These are interests of the two parties, not collected ratings, and there is no requirement that they agree.
The inferential step is narrow but consequential. A reader's preference for the concise summary is evidence about ease of processing under the stipulated question. It is not consent to the representation of the contributor's condition, because the reader was not asked to give, and did not hold, authority to consent on the contributor's behalf. Preference answers the question posed to the participant; it does not extend beyond the participant's standing. If one version also changes a disputed conclusion, then the preference cannot isolate readability, since the reader is responding to a difference in content as well as presentation.
This scope matters in application. A personal reading aid answers its user's declared purpose, and that purpose may legitimately be speed. A committee brief has a separately assigned remit and affected parties, and the reader's ease does not by itself settle what may be represented to or about them. Neither context automatically supplies the other's authority. So the useful judgment is one whose question and judge are both specified: readability from a reader who can compare matched versions. Whether preference-trained methods then produce summaries worth adopting is the separate, positive question to which the paired study is directed.
22.3The positive evidence for improvement
The literature supplies a positive result worth preserving. Stiennon et al. train a summarization policy from a model of human comparisons, offering evidence that useful compression can improve (Stiennon et al., 2020). To clarify the kind of claim at issue, consider a separate hypothetical comparison in which declared readers rate paired summaries of the same material under a fixed preference question.
The intended inference is specific. A well-designed comparison could show that readers prefer the trained policy's summaries under the declared task. Holding source material fixed helps compare candidates, but it does not alone establish a reliable training effect: sampling uncertainty, the comparator and the evaluation procedure still matter. The measured outcome would still be preference under that procedure. The preference question and the participating readers are therefore part of the result, not incidental to it. Nothing in that inference shows that the summaries are more accurate, or that the readers speak for anyone else; the success would concern the declared preference evaluation.
The boundary follows from the ingredients. The preference task, readership and dataset define the directly evaluated setting. Extending the result requires a separate argument or evidence. A preference judgment is not by itself a factual accuracy claim, and it is not consent on behalf of someone represented in the material. A reader may prefer a concise summary because it is faster to read, while a contributor may want the longer version because it retains a condition; those are stipulated interests, not collected ratings. The positive literature claim and the proposed comparison thus have distinct roles: one motivates taking preference improvement seriously, while the other identifies what a new evaluation would need to establish.
The bounded result is that one summary was preferred under the stipulated questions and participating readers. That result belongs to the comparison actually made: paired summaries of the same material, assessed as the study specified. It does not follow that the preferred version improves every aspect an editor must protect. The additional test needed is not a reason to discount the finding; it is a way to identify what the finding covers.
Readability is the clearest example. A fair test would give both versions access to the same source and match their factual content. Under those conditions, a difference in preference can reasonably bear on readability. If one version also changes a disputed conclusion, however, preference cannot isolate readability: a reader may like the version precisely because of its conclusion. An additional evaluation should therefore hold source access and factual content fixed while varying the feature under examination. Only then can the reported preference be assigned to that feature rather than to a change in what the summary says.
This qualification matters when the result is extended to representation. Preference may show that a summary is easier to use, but it does not by itself establish that the summary is accurate, that it reflects genuine agreement, or that the contributor consents to being represented. Consider the stipulated interests already on the table: a fictional reader may prefer a concise summary for speed, while a contributor prefers a longer version because it retains a condition. Neither preference settles the other's concern, because each answers a different declared purpose.
Application accordingly depends on the task. A personal reading aid answers its user's declared purpose, and preference can reasonably optimize for it. A committee brief has a separately assigned remit and affected parties; it does not inherit that authority from the personal case. The improvement is real within its measured comparison, and the limit is real at the boundary of what that comparison tests.
22.4Disagreement among raters
Improvement on a reported comparison does not by itself settle whether a summary preserves what a particular reading task requires. The next test is to keep the raters' disagreements rather than average them away, and to ask which of those differences bear on the declared purpose.
Consider the stipulated case: a fictional reader prefers a concise summary because it is quick to use, while a contributor prefers a longer version because it retains a condition. These are declared interests, not collected ratings, but they show why a single preference score can conceal a question worth preserving. Speed and a retained condition are not the same criterion. If the evaluation asks only which summary is preferred overall, the resulting number mixes two tasks.
So disagreement matters only relative to a specified reading task. When the task is fast personal use, the concise preference is directly relevant; when the task requires noticing conditions on a claim, the contributor's preference identifies material the concise version omits. Neither preference displaces the other, and neither establishes that either summary is better outside its remit.
A proposed paired evaluation makes this sharper. Keep source access and factual content matched when comparing readability. If one version also changes a disputed conclusion, preference cannot isolate readability, because raters are then responding to content as well as presentation. Matching readability, conversely, certifies neither factuality nor authority to represent.
Authority is likewise scoped. A personal reading aid answers its user's declared purpose; a committee brief has a separately assigned remit and affected parties. Neither context automatically supplies the other's authority. Disagreement can therefore locate where declared purposes diverge, which is a finding about the task at hand. It does not, by itself, say which purpose governs a representation that someone must stand behind.
Retaining raters' variation answers the question only when the disagreement survives a check of the task itself. In the stipulated comparison, a fictional reader prefers a concise summary for speed while a contributor prefers a longer version because it preserves an explicit condition. Those preferences are real as stipulated interests, but they may reflect different instructions rather than a substantive clash about the material. A prompt that rewards concision and one that demands retention of every condition invite different answers from the same pair.
The proposed paired evaluation makes the distinction testable. Source access and factual content are kept matched across the two summaries so that preference can track readability. If one version also alters a disputed conclusion, the responses no longer isolate readability; they blend style, interpretation, and the underlying dispute. Agreement scores then describe a mixture, and preserving the mixture as "variation" postpones a question that can be settled by consulting the material.
Some disagreement, in other words, comes from ambiguous instructions or plain mistakes. If the source states a condition that one summary drops, adjudicating the omission is not an imposition on raters' freedom. The task was to summarize the same material; a demonstrable source error in the output is a defect, not an alternative reading preference to be averaged into the result. Where a source check establishes the error, the system should correct it and say so.
The limit is equally clear. Rater disagreement alone is not evidence that one side erred. Variance should remain visible when readers are pursuing genuinely different declared purposes, and a personal reading aid may legitimately serve its user's own purpose. But that authority is local: a committee brief has a separate remit and affected parties. Neither setting supplies the other's warrant, and neither permits an unexamined factual error to hide inside preserved disagreement.
22.5Preference for congenial conclusions
Rater variation can persist after ambiguous instructions are repaired, so the next question is what a preference signal actually measures. Take a stipulated paired evaluation over one piece of source material, which asserts that a measure is justified only when the affected group has been consulted. Summary A reads that the measure is justified. Summary B reads that the measure is justified only after the affected group has been consulted. Both are drawn from the same material, and both are quick to read. But A is smoother: it has one clause, a firm cadence, and a conclusion that a reader may already find congenial.
The gap is not merely stylistic. B preserves an explicit condition; A drops it and thereby strengthens the claim from conditional to categorical. If A is preferred, that preference may register ease of reading, agreement with the unhedged conclusion, or both at once. The result would then not isolate readability, because the smoother version is also the one whose conclusion becomes more congenial through losing the source's own qualification. If trained on such pairs, a system would be rewarded for treating that confluence as quality.
This stipulated pair shows only how such a preference could arise. No ratings were collected, and the reader's taste for concision and the contributor's insistence on retaining the condition are asserted interests, not observed choices; nothing here licenses a claim about anyone's motives. The qualification that matters is authority. A personal reading aid may take its user's declared purpose as its remit, while a committee brief answers to a separately assigned remit and to affected parties, and neither context borrows the other's warrant. Preference learning can therefore teach an editor to sound fluent and agreeable; it cannot by itself establish that the dropped condition was dispensable, or that the editor was entitled to state the conclusion without it.
Suppose the smoother version also states the congenial conclusion. A reader's preference for it is compatible with two mechanisms: liking the conclusion itself, or liking the ease with which it is read. As paired, fluency and conclusion travel together, so the observation establishes only that raters favored that version. It is not evidence that they chose on ideological grounds. The confound is structural, and no preference question posed after the fact can separate its ingredients.
The comparison should therefore cross the factors rather than vary one while conceding the other. Keep source access and factual content matched, retain the explicit condition in every version, and hold fluency as nearly constant as the writing allows. Then contrast a fluent and a less fluent version stating the same conclusion with versions of matched fluency that state different conclusions, including one the reader is stipulated to dislike. If preference tracks fluency across conclusions, readability is doing the work; if it tracks the conclusion once fluency is held roughly constant, a congeniality effect is indicated.
The limit is that this distinguishes mechanisms only among the stipulated readers answering the stipulated questions. It does not show that raters generally reason ideologically. Nor does it decide what a preference licenses. A fictional reader may prefer a concise summary for speed while a contributor prefers a longer version because it retains a condition; reporting both faithfully still leaves open whether the output should answer one user's declared purpose or serve a committee brief with a separately assigned remit and affected parties. That question concerns authority to authorize representation, which a preference comparison does not settle.
22.6Personal and public use cases
When the goal is to serve one reader, preference learning has a defensible and limited role. A personal assistant can adapt a summary to that reader’s stated interests: a reader who wants speed may prefer the concise version, while a contributor may prefer a longer one because it preserves a condition. These are stipulated interests, not collected ratings, and the example therefore shows what the method could be asked to do, not what readers generally choose.
That adaptation is legitimate as a response to a declared purpose. It remains important to say which faculty is being tuned. Matching source access and factual content between paired summaries allows a comparison of readability; if one version also changes a disputed conclusion, preference for it cannot isolate readability, because agreement may have done the work. Readability is therefore a fit between expression and the reader’s purpose, not a proxy for factual correctness or shared approval.
The same care applies to representation. A personal assistant is authorized by its user to organize material around that user’s interests. It is not thereby entitled to decide that those interests ought to govern anyone else’s account, nor to certify the resulting summary as factual. The relevant limit is simple: personal usefulness answers the question posed by the user’s declared purpose. It supplies no general selection rule.
This leaves a distinction to carry forward. A committee brief has a separately assigned remit and affected parties; its selection of material must answer to that remit rather than borrow the personal assistant’s authority. Conversely, the existence of a public need does not remove the legitimacy of a limited personal tool. The method earns its place when the purpose is stated and the requested adjustment is kept distinct from the claims it cannot settle.
Given that a personal aid may adapt a summary to one reader's stated interests, the question is how far the same standard of selection travels. A fictional reader can prefer a concise summary for speed while a contributor prefers a longer version because it retains a condition. These are stipulated interests, not collected ratings, but they show that the same material can support two defensible selections without either preference being mistaken. For a single-reader assistant, the user's declared purpose is enough to settle what counts as useful, provided the assistant does not quietly add purposes the reader never assigned.
A shared public brief cannot borrow that authority. Its selection procedure needs an explicit account of whose interests shaped it: which readers it was meant to serve, which contributors it was meant to represent, and who bears the cost when a condition is trimmed away. A committee brief has a separately assigned remit and affected parties that a personal reading aid does not, while the personal aid answers only its user's stated purpose. Neither context supplies the other's mandate, so personal usefulness is not a universal selection rule. The fact that one reader benefits from concision does not establish that concision should govern a document written for several constituencies.
This matters for preference-based training in a specific way. In a proposed paired evaluation of readability, source access and factual content should be held matched across the two summaries. If one version also changes a disputed conclusion, then higher preference cannot isolate readability: the reader may be agreeing with the conclusion rather than preferring its expression. The limit is sharp here. Even a well-matched preference comparison tells a reader which version readers favor under the stated question; it does not decide whose interests a public selection ought to serve, nor convert a user's declared purpose into a remit for others. That decision has to come from the mandate the brief announces.
22.7Learning under a declared mandate
Preference learning answers a question that must be declared before the ratings are collected or used for training. The declaration should identify whose preferences count, what material they compared, and which task the comparison serves. Without that account, a preference score mixes several judgments: ease of reading, agreement with an interpretation, accuracy, and willingness to have one’s interests represented.
The distinction appears in the stipulated comparison of two summaries. A fictional reader may prefer a concise version for speed, while a contributor prefers a longer one because it retains a condition. These are declared interests, not collected observations. If the proposed evaluation matches source access and factual content, the resulting choice can bear on readability. If one version also changes a disputed conclusion, however, preference cannot isolate readability: agreement or disagreement with the conclusion enters the same response.
This limit matters even when the preferences are genuinely useful. For a personal reading aid, the user’s declared purpose can legitimately govern what counts as a better summary. That narrow authorization does not transfer automatically to a committee brief, which has a separately assigned remit and affected parties beyond the person providing ratings. Nor does a majority preference settle whether a statement is factual. Preferences may reveal what readers find usable or acceptable under a stated task, but factuality still depends on the matched material and evidence available to them.
The mandate therefore precedes the learning method. It fixes the question that the paired summaries and preference labels are meant to answer. A useful model can then be trained toward readability or personal usefulness without being credited with authority it was never given. What remains uncertain is whether the mandate itself was properly assigned and whether the people affected by the resulting representation had standing in that assignment.
The ratings answer the question they were collected to answer: which summary the participating readers preferred when shown the stipulated pairs. A change of mandate does not alter those ratings or the model trained on them, but it changes what they can justify. Under a personal reading aid's declared purpose, a preference for the concise summary can support the claim that this reader, seeking speed, favored that version. The same evidence does not support a claim about which version should be issued under a committee brief's separately assigned remit, where contributors and other affected parties have interests of their own. Neither context automatically supplies the other's authority.
The stipulated interests make the point concrete. A fictional reader may prefer concision for speed while a fictional contributor prefers a longer version because it retains a condition. These are stipulated interests, not collected ratings, so they cannot be counted as votes for either summary. They show instead why the mandate determines whose purposes the preference question is meant to serve. Ratings gathered from one audience answer the question asked of that audience; changing the audience or the purpose requires asking again.
A proposed paired evaluation can isolate one narrow claim. If source access and factual content are kept matched across the summaries, a preference difference bears on readability rather than on the substance of what the material says. The limit matters: if one version also changes a disputed conclusion, preference cannot isolate readability, because the reader may be choosing between conclusions rather than between presentations. The mandate therefore does not make preference evidence stronger or weaker in itself. It fixes the scope of the license the evidence provides, while the unchanged ratings continue to record only preference under their original question.