14.1Four different denominators
Before any compression of the consultation record, four quantities must be kept apart. A document count asks how many texts have entered the corpus. A contributor count asks how many people produced them. An eligible-population count asks how many people could have been invited or could have responded. An affected-population count asks who bears the consequences of the eventual service design, including people outside the intended service population.
The fictional invitation illustrates the separation without resolving it. Eight respondents submitted material: five request later hours and three do not. Five and three sum to the positions taken in those messages, and eight sums to the identifiable contributors, provided no one submitted more than one message. None of these totals supplies the denominator for eligibility. The number invited is not given, and no account of nonresponse is available. It would therefore be wrong to report five-eighths of service users, or even five-eighths of a relevant population of respondents; the defensible arithmetic is five of eight respondents, and only if the corpus records exactly one message each.
Eligibility and affect come apart as well. Suppose, conditionally, that some people served by the program were never in the invitation frame, while others who do not use the program are affected by its hours. Then the eligible population is a matter of who could have been reached under the recruitment design, whereas the affected population is a matter of the decision's consequences. Neither can be recovered by adding documents.
This separation of the four denominators has a limit rather than demanding impossible certainty. The consultation may still be useful for surfacing possible problems, and identifying contributors and their messages is genuine evidence. What it cannot do is convert unobserved people into estimates merely by naming the intended population. Recruitment evidence missing at the outset does not reappear inside the corpus.
Even complete coverage of the collected documents would not establish coverage of the people who might use the service or of the concerns they might raise. Semantic exhaustiveness is a property of the corpus: every collected message has been read, classified, and retained. The people and their concerns lie outside that boundary unless they are separately established.
Take the fictional invitation. Eight respondents answer an unknown population of service users; five request later hours and three do not. The five-to-three split describes the respondents and nothing further. The number invited was not given, and the nonresponse mechanism is unknown, so no denominator for comparison exists. Nor does the range of topics among the eight settle the question: diversity of subjects among submitted messages is not evidence that the contributors cover the intended population or its concerns.
Three failures of coverage must also be kept apart. A missing participant never submitted anything; an unelicited concern was never asked for or prompted; an omitted collected concern exists in the corpus but drops out in summary or processing. Only the third is discoverable by closer semantic inspection of the documents. The first requires evidence about recruitment and nonresponse; the second requires evidence about what the consultation asked and how concerns were invited. Compression cannot supply either by reading the corpus more thoroughly.
The stated purpose matters here. If the aim is to gather possible problems, the eight responses can be useful as a source of questions to examine, and later hours can be one such question. That usefulness does not make them a population estimate, and no estimate should be attached to the five-to-three split. The relevant limit is therefore purpose-relative: a voluntary consultation may legitimately supply candidate problems while leaving unanswered whose problems are still missing.
14.2Voluntary contribution
Voluntary contribution permits useful questions to enter the record without establishing that those who contribute are a representative sample. In the fictional consultation, eight respondents send messages: five request later hours and three do not. That description is complete only for the respondents themselves. The number invited is not given, and the nonresponse mechanism is unknown, so the split among the eight cannot be converted into a share of the intended service population. Nor does the variety of topics among the eight demonstrate representativeness; diversity of content and representativeness of people are different properties.
What voluntary contribution does establish is a narrower and still useful thing. A collected message raises an entrance concern absent from a different summary. As a record of possible problems, that message matters because the corpus contains the concern, not because some known proportion of users holds it. A stated purpose of gathering possible problems fits this evidential role: the consultation asks what might go wrong, and the answers supply candidates for examination.
Three apparent gaps now separate accordingly. A missing participant is a recruitment gap and requires evidence about who did not contribute. An unelicited concern is an elicitation gap and requires evidence that no available prompt drew it out. An omitted collected concern is a record gap and requires comparison with what was actually collected. None of these can be repaired by enlarging or reorganizing the semantic coverage of the corpus, because the first concerns people, the second concerns the questions asked, and only the third concerns the treatment of received material.
The relevant limit is therefore not that a voluntary project must represent everyone in order to count. It is that its output supports possible-problem claims, not population estimates. The impossibility of proving representation from the eight respondents does not cancel their contribution; it fixes the scope of what the contribution can show.
A voluntary archive may therefore be useful as a record of what its actual contributors raised. In the fictional consultation, the eight respondents—five requesting later hours and three declining that change—establish only that these questions appeared among those eight. The stated purpose of gathering possible problems is compatible with treating their messages as a source of candidate concerns. It does not turn the five-to-three split into an estimate of the service-user population, and diversity of topics among the messages cannot make the contributors representative.
The archive’s defensible claim is narrower but real: it can say that these contributors raised these concerns, and it can serve them by preserving those concerns and keeping them visible in later deliberation. This scope works when the archive is used to identify problems encountered by the people who supplied the questions. The missing population estimate is then a limit on extrapolation, not a defect in their contributions. The same record supports a different application only under stronger conditions. If it is used to rank the needs of all service users, for example, the five-to-three division carries no such authority.
Limiting the claim also leaves open the chapter’s question: whose questions are missing? In this fictional case, neither the number invited nor the nonresponse mechanism is given, so the gap cannot yet be classified. A person absent from the invitation process is missing at recruitment; a concern that no participant was prompted to raise is unelicited; a concern contained in a collected message but left out of a summary is omitted from the record. These failures call for different evidence. Even conditionally, if every invited user received the same prompt and every submitted message were retained, recruitment absence and elicitation gaps would remain unresolved.
The archive thus merits its modest role without an impossible mandate. Any stronger use requires specifying who was invited, who responded, and how possible concerns were elicited.
14.3Sampling and recruitment conditions
A voluntary archive may serve its contributors by retaining and organizing their material. Having collected their statements does not, by itself, authorize the archive to speak for them or for anyone who stayed away. To move from that modest service claim to a generalization about a wider consultation, recruitment and selection become the conditions to specify first.
Consider the stipulated invitation. Eight people respond out of an unknown population of service users; five ask for later hours and three do not. The figures directly describe those eight respondents; a population inference would require additional assumptions and evidence. Because the number invited is not given, no response rate can be computed. Because the nonresponse mechanism is unknown, the record does not establish what nonrespondents would have requested. The three respondents who did not ask for later hours are present in the data; their positions must not be confused with the unknown positions of absent users. The variety of topics raised does not supply representativeness either: a diverse set of concerns is still a set of concerns from this self-selected group.
This is why four things must be held apart: the text corpus that was collected, the identifiable contributors who produced it, the intended service population the project hopes to serve, and the nonparticipants who are absent from the record. Recruitment describes how the invitation reached people; selection describes which of them ended up contributing. Both need to be investigated as empirical questions — who was approached, through what channel, with what coverage of the intended population, and under what self-selection or nonresponse — before any population estimate is offered. Semantic coverage of the corpus cannot repair a recruitment gap, however rich the messages are.
A documented study shows why such specification matters without supplying it here. In Fishkin, Siu, Diamond and Bradburn's account (Fishkin, Siu, Diamond and Bradburn, 2021), changes after a national deliberative event were reported alongside a pre/post control comparison. Note the limit: the abstract-level record available here does not document recruitment or assignment in detail, and it does not transfer to this consultation.
The qualification that survives is modest but sufficient. A consultation can be useful for surfacing possible problems — as a stated purpose permits — without estimating what a population wants. Its value does not require an impossible mandate; it requires that its claim stay inside the recruitment it actually achieved.
Four things must stay apart in this record. The corpus is the collected messages; the contributors are the eight respondents who produced them; the intended population is the wider service-user group named by the project's purpose; and the nonparticipants are everyone in that group who contributed nothing. The description supplies no count of people invited and no account of how respondents came to participate, so the boundary between contributors and nonparticipants cannot be drawn numerically here.
What the numbers license is correspondingly narrow. Five of the eight respondents request later hours and three do not. This is a property of the collected messages, not a prevalence estimate: without a denominator for those invited or eligible, and without any description of who declined or was never reached, the split cannot be projected onto service users at large. Nor can effect sizes, response rates or recruitment parameters be reconstructed from an abstract-level description of a deliberative exercise. Filling those gaps by inference would manufacture precision the record does not contain.
A voluntary consultation can still be useful, and nothing here requires it to represent everyone. A stated purpose of gathering possible problems makes the eight messages evidence of concerns that were actually raised; five requests for later hours is a concrete signal worth weighing when hours are set. That value does not rest on representativeness. But the diversity of topics among eight respondents is not a sampling argument, and the same split cannot tell a service manager how many users want later hours.
The appropriate use is therefore local and conditional: let the record open design questions and register what respondents asked for, and hold population claims back until the invitation process and its selection conditions are documented. The qualification that matters is simple. The claim that eight respondents said something and the claim that service users want something are different claims, and only the first is supported by the material before a reader.
14.4Nonresponse and inaccessible voices
Begin with the fictional consultation, keeping its four objects apart. The corpus consists of the collected texts; the contributors are the eight people who submitted them; the intended service population is the wider group of service users; and the nonparticipants are those in that population who did not submit. The invitation produces eight respondents from an unknown population: five request later hours and three do not. Those figures license a statement about the respondents. They do not license a claim that service users prefer later hours, and the diversity of topics among the eight supplies no representativeness. Because neither the number invited nor the nonresponse mechanism is given, no population estimate follows.
Absence from the corpus, moreover, has several routes that require different evidence. A person who never submitted a message is missing from the contributors. A concern that was never elicited may belong to someone who did respond: their message exists, but the topic does not. An entrance concern appearing in one collected message but not in a different summary is neither case; it was collected and then omitted from a derived artifact. Corpus inspection can settle the third dispute by comparing message and summary. The elicitation design bears on the second. The first cannot be settled from the corpus at all, because the relevant text was never submitted.
That is the local limit on perfect summarization. Even a system reproducing every supplied sentence exactly would leave an unsubmitted concern absent: completeness over the corpus is not completeness over the intended population. If later records established how many service users were invited and who they were, a response fraction could be calculated; that would still not reveal what nonparticipants thought. This does not make the consultation worthless. A project expressly gathering possible problems can legitimately surface candidates for action from its respondents. Its use is diagnostic rather than estimative, and its claims should stop at the boundary of the text actually gathered.
The fictional invitation produced eight respondents out of an unknown population of service users: five request later hours and three do not. Those counts describe the respondents and nothing more. Because the number invited is not given and the nonresponse mechanism is unknown, the five-to-three split cannot be converted into a response rate, nor can it reveal which service users stayed silent. This is where semantic accuracy reaches its limit. A compressor can be scored against the corpus for fidelity, and it can expose a concern that was collected but dropped — the entrance concern that appears in one message and disappears from a different summary. But the corpus contains no record of a concern that was never elicited, and still less of a participant who never replied. No degree of fidelity to supplied sentences can identify what was never supplied.
Distinguishing these three cases therefore requires evidence of different kinds. An omitted collected concern can be established by comparing the summary with the text actually gathered. An unelicited concern requires information about the questions posed and the opportunities respondents had to raise their own topics. An absent participant requires information about recruitment: who was invited, who could respond, and how silence arose. Semantic evaluation supplies evidence only for the first case; it cannot, by itself, compensate for the other two.
The stated purpose matters here. If the exercise is understood as gathering possible problems, the eight responses can be useful for exactly that purpose without licensing a population estimate. Diversity of topics among the respondents is not evidence that the respondents represent the wider service population. The qualification is one of scope rather than an impossible mandate: a voluntary consultation need not represent everyone to surface genuine difficulties. It must not, however, let success at the narrower task be reported as coverage of the wider one. If an invitation list and a documented nonresponse pattern were later available, coverage could be assessed; without them, the missing questions remain unknown before compression begins.
14.5Representativeness of issues versus people
Issue coverage and people coverage are different properties, and a corpus can achieve the first while failing the second. Suppose the collected text spans several distinct problems: scheduling, access, and, in one message, an entrance concern that appears nowhere in a separate summary. The corpus is thus varied in content. That variety does not establish that the contributors are varied in the way that matters, or that any group of service users is represented in anything like proportion.
Consider the consultation's eight respondents. Five request later hours and three do not. Those figures describe the respondents and nothing more: the number invited is not given, and the nonresponse mechanism is unknown. If the respondents happen to cluster in one segment of the service population, the frequency of the hours request reflects that clustering as much as it reflects the issue's importance. A corpus may therefore contain many issue types and still overrepresent a subset of people, because the subset speaks often, speaks early, or is the subset that was reachable.
The distinction between issue coverage and people coverage also separates three failures that require different evidence. A missing participant leaves no text to inspect. An unelicited concern may have been present in a participant but absent from the questions asked. An omitted collected concern is recoverable from the corpus itself. The entrance concern illustrates the third case only; it cannot settle either of the others.
The qualification matters. A project whose stated purpose is to gather possible problems can be genuinely useful on that basis, and it need not represent everyone to serve as a source of candidate concerns. But usefulness of that kind does not convert the corpus into a population estimate. Before any compression begins, the working question remains: whose questions are already missing from what was collected, and what evidence could show that they are missing?
The converse failure is equally consequential. A consultation may draw respondents from across the service-user population and still leave one live concern entirely unelicited. Broad participation widens the range of people heard; it does not, by itself, widen the range of issues that the procedure makes speakable. A topic can be absent because no participant raises it, because no prompt or exchange invites it, or because a submitted concern never reaches the summary. These are three different gaps, and they call for different evidence.
A missing participant is a recruitment question: who was invited, who answered, and how nonresponse occurred. An unelicited concern is a question about the consultation's design: whether any part of the encounter gave a reason to mention it. An omitted collected concern is a question about processing: whether the corpus and the summary agree. Comparing a summary with the messages it summarizes can establish only the third. It cannot show that a concern was never voiced, and it cannot show that a voice was never present.
The stipulated consultation makes the point concretely. Eight respondents are recorded, five requesting later hours and three not; these figures describe the respondents alone, since neither the number invited nor the mechanism of nonresponse is given. Nor does the spread of topics among them settle anything: diversity of subject matter in a small set is not evidence that the set represents a population, just as demographic breadth in a sample is not evidence that any one issue would have surfaced. The entrance concern appearing in a collected message but not in a different summary illustrates the last category, where the concern exists in the record but not in its account.
What the responses can support is narrower and still worthwhile. If the stated purpose is to gather possible problems, then a later-hours request and an entrance concern are candidate problems worth preserving, whatever their frequency. That usefulness does not license a population estimate, and it does not license a claim that the list of problems is complete. Coverage of people and coverage of issues must therefore be assessed separately, each against the evidence that would actually bear on it.
14.6A limited legitimate audience
A useful service-question archive can claim a limited audience without treating that limit as a defect, and without converting usefulness into authority. In the fictional consultation, the eight respondents are the contributors whose messages make up the corpus. The service users who might benefit from later hours form an intended service population, while those never reached or never replying remain nonparticipants. The corpus preserves what the contributors submitted; it does not count the nonparticipants and it is not a census of the intended population.
Five respondents request later hours and three do not. That split describes the respondents. Because the number invited and the nonresponse mechanism are not given, it cannot become a population estimate, and the range of topics among the eight does not make them representative of people. The stated purpose of gathering possible problems supplies the narrower warrant: these responses help enumerate concerns a service might investigate. Finding an entrance concern in one collected message that a different summary omitted shows that the archive can register a concern occurring among submissions. It does not show that entrance concerns are widespread or that they bear any known relation to the intended population.
Nor do the three gaps call for the same evidence. A missing participant must be identified as someone outside the contributor set; an unelicited concern requires evidence about the procedure that might have solicited it; an omitted collected concern requires comparing what the corpus contains with what a summary reports. Absence from one summary settles none of the three.
The resulting audience is limited and legitimate: people asking which possible service problems have been submitted and for what planning purpose, not people asking how common those problems are or whose views went unheard. A voluntary project can be useful without representing everyone, and it need not carry an impossible mandate; conceding that point narrows the warrant rather than the evidence. Within this audience the archive supports agenda-setting and design review. Beyond it, the respondents' split remains a property of the submissions, not a finding about service users at large.
A limited audience is workable only if its boundaries remain visible when the material is used. Four things must therefore stay distinct: the corpus of collected messages, the people who contributed them, the intended service population, and those who did not participate. In the fictional consultation, the eight respondents form the contributor set, and the service users they are meant to inform form a wider intended population. The five requests for later hours and the three responses without that request describe the contributors only. Because neither the number invited nor the mechanism of nonresponse is given, no prevalence estimate for the wider population follows.
The inferential step is narrow but decisive: an archive can establish which questions were asked by whom, without establishing which questions matter most across a population. Diversity of topics among the eight respondents does not repair this gap. It may show that several kinds of difficulty occurred in the collected material; it does not turn the contributors into a representative sample.
The distinction among the four categories also changes what counts as evidence of a missing question. A missing participant would require evidence about recruitment or nonresponse. An unelicited concern would require evidence about the consultation procedure and what it allowed people to raise. An omitted collected concern is different again: the concern exists in the corpus but is absent from a summary, as with the entrance concern. These three absences cannot be inferred from one another, and compression of the corpus will not by itself reveal which one is present.
Finally, limiting the legitimate audience does not settle whatever separate obligations an institution may have toward nonparticipants or toward the intended population. Those obligations may arise from other sources; this passage asserts no new legal obligation. The point is simply that the archive’s useful scope must not silently stand in for those broader questions.
14.7Stating the population claim
The report can make its population claim explicit by attaching each proportion to the set that actually generated it. In the fictional consultation, eight respondents produced the collected messages: five requested later hours and three did not. The defensible sentence is therefore that five of the eight respondents requested later hours, not that service users want later hours. The first wording names its numerator and denominator together; the second silently enlarges the denominator from eight respondents to an unspecified number of service users.
This is a matter of grammar as well as method. “Respondents,” “contributors,” and “intended service population” should appear as distinct terms in the report, with the corpus identified as the collection of messages rather than as the people themselves. A stated purpose of gathering possible problems can justify presenting the five and three requests as signals worth considering. It cannot justify treating the split as a population estimate, because the number invited is unknown and the nonresponse mechanism is unknown. The absence of additional topics from the three respondents who did not request later hours is not evidence that no such concerns exist.
The inferential step is modest but consequential: when the reported denominator equals the number of contributors, the claim is checkable against the collection, and readers can see exactly whose responses support it. A larger claim would require a larger denominator and a defensible link between respondents and the intended population. If the number invited were later supplied and the response mechanism documented, a broader estimate might become arguable; those facts are not available here, so the condition remains hypothetical.
The limit should also be stated plainly. Precision in wording does not convert an unknown-population consultation into a representative one. Semantic coverage of the collected messages—however varied the topics—cannot repair recruitment by itself. The report should therefore claim usefulness for problem-finding, name the eight contributors and their responses, and leave the intended service population as a separate object whose questions may remain unobserved.
Even when the report has named its intended service population and stated a denominator, the collection may not supply the information needed to fill it. In the fictional consultation, eight people responded: five asked for later hours and three did not. The number invited is not given, and nothing establishes why others did not answer. These two omissions block any proportion beyond the respondents themselves. Saying that five out of eight requested later hours is safe; saying that five out of eight service users want them is not, and neither is projecting the same fraction onto an unknown public.
The honest report therefore records the collection on its own terms and marks the missing denominator explicitly. It can say what eight respondents contributed, that the invited count and nonresponse mechanism are unavailable, and that the intended service population remains the population of interest without being the population observed. This is not a retreat from a population claim; it is the claim's correct scope: the evidence concerns a submitted corpus assembled from identifiable contributors, while nonparticipants are neither included nor represented by an assumption.
The stated purpose matters here. If the project set out to gather possible problems, then the eight responses can be useful for exactly that purpose—each concern is a candidate worth considering, and later hours has surfaced as one. Purpose cannot convert an open-ended collection into a representative sample. Diversity of topics among respondents, including any concern raised by only one collected message, likewise supplies no sampling warrant. A concern may be absent from the record because it was never elicited, because a participant was missing, or because a collected concern was omitted from a summary; distinguishing those cases requires evidence the fictional record does not provide.
Thus the limit of the local inference is precise: report the collection and its uncertainty, and refuse the universal public that the denominator would have to support.