32.1A participant adds an option
A wiki survey does more than record agreement with a fixed menu. Respondents can compare presented items and propose new ones, subject to creator approval (Salganik and Levy, 2015). That opening is the distinctive opportunity: a participant's contribution can become an item that later respondents encounter, so the choice set is partly shaped by the people answering it rather than fixed entirely in advance. Because later respondents draw on the same pool, an option contributed early can enter the comparisons of people who arrive after it.
The stages must stay separate. Submission puts a candidate into the queue; activation makes it presentable; comparison is what respondents actually do with displayed items. Submission is therefore necessary for a later encounter but not sufficient: the approval condition means a proposed item may never appear, and an activated item is merely available to later respondents, not guaranteed to reach each of them. What a tally of submissions counts is proposals made; it does not count activated items, and it certainly does not count the encounters those items receive. Naming which of these a figure compares is what gives the number meaning.
Applied to design, the opportunity buys openness to ideas the creator did not anticipate. It widens the range of candidate ideas without fixing who supplies the answers. It does not, by itself, buy representativeness. The two reported cases lacked probability sampling, so inference stays with respondents (Salganik and Levy, 2015). Item openness and sample coverage are separate properties: a menu that grows through proposals can still be answered by a self-selected group. Supposing, conditionally, that every useful proposal were activated, the resulting set would still describe what these respondents offered, not what all residents would think.
Continuing from the preceding passage, the distinctive opportunity is not simply that a participant may write an item; it is that the item may later appear to others. That statement needs its scope fixed. In this prospective design, submission and activation are separate stages: a participant proposes a hypothetical item, and the creator decides whether it is activated into the set shown to later respondents. Submission is therefore necessary but not sufficient for any later encounter; the encounter is possible after activation, not an inevitable consequence of writing something.
This division preserves the moderation condition rather than deleting it. Openness to new ideas sits at the proposal stage: anyone may contribute an option without prior permission. Structured response remains downstream. A participant who submits an item does not thereby determine how others answer, nor whether others see the item at all. What the participant controls is the addition itself; what the creator controls is entry into the presented set.
Consider the countercondition. If the creator does not activate a proposed item, later respondents never encounter it, and the survey they complete omits the suggestion entirely. If the creator activates it, the item becomes one option among others, subject to whatever response format the survey presents. Neither outcome implies that any respondent endorses the item: activation governs exposure, not agreement.
The resulting inference is narrower than saying that participants can change the questionnaire. Participants can propose additions to a questionnaire whose contents a separate activation decision governs. That distinction matters when contribution is credited: the proposal stage registers initiative, while the activation stage determines which suggestions ever become answerable options. The proposal stage therefore supplies candidate content without presuming that it will shape any later response.
32.2Pairwise comparisons
A pairwise response does one narrow thing well: it records a comparison between two presented items under the survey question. Because both items appear together and the question is held fixed, the response says which of those two proposals the respondent favours as presented — nothing broader. The comparison is bounded twice over, first by the question the creator posed and then by the items that were actually activated and shown. A submitted suggestion is not yet a term in any comparison; it enters the pairwise set only once it has passed activation.
What does this study license a reader to conclude? Aggregated pairwise responses show how items fared against the particular opponents with which they were matched. That is sufficient for the claim that item A was preferred to item B in responses where both were presented together. It is not sufficient for the claim that A was preferred to all other activated items, or that respondents would have selected A from the whole set, because many pairings may never have been offered. If a particular pairing was never presented, the resulting comparisons provide no evidence about it; the missing comparison is not a negative judgment on either item.
Applying the same care to the design itself, any comparison between this response format and another way of eliciting judgments must hold the question and the activated item set constant. Otherwise the difference being measured confounds the response encoding with different inputs: a survey asking one question about a fixed set of proposals is not the same task as one asking a different question about a larger or smaller set. Fixing the task first is what makes the encoding the variable under examination.
A pairwise response records a comparison between two presented items under the survey question, and its scope stops at that encounter. It does not place the preferred item on an absolute preference scale. Even a constructed ranking from many wins inherits its limits from which pairings were offered: an item that appears rarely, or only against weak opponents, can accumulate wins without having been compared with the strongest alternatives. A quantity such as win share therefore compares an item's performance against the particular opponents it happened to face; it is not a measure of how much anyone prefers that item in absolute terms, nor a comparison against all possible items. (Conditional supposition: if a fixed set of items were each paired against every other, a consistent ordering might be reconstructed, but the design would still yield orderings under the stated question rather than cardinal preference levels.)
The same boundedness governs representation. Only respondents who saw the presented pair can supply a recorded comparison, so the responses describe that sample under those presentations. People who were not sampled are not represented at all; extending the pattern to them is an inference the responses cannot carry, however large the recorded sample becomes. That limitation is not a defect of the format but a limit on what its records compare.
Absence of evidence in the pairwise record is also weaker than it first appears. In the prospective design already on the table, an item's name may be submitted as a new idea before the creator activates it. Submission and activation are separate stages. If a hypothetical item never wins a pairing, at least three conditions remain possible: it was never submitted, it was submitted but not activated, or it was activated yet never presented against a given opponent. No recorded loss is necessary in any of these cases, and lack of losses would not by itself establish support.
32.3An opinion map as a model
An opinion map is not a picture of opinions taken as they stand. It is a constructed representation of recorded relationships among responses: which responses were made, on which items, and how those responses relate to one another under a stated procedure. The object modeled is thus the set of records the procedure produced, together with the relations computed from them.
This distinction between the modeled object and the underlying opinions matters as soon as the records are incomplete. Return to the illustration: respondent A never encounters item X, respondent B encounters X but gives no response, and a display suppresses a response that respondent C recorded. These are three different record states — non-encounter, non-response, and recorded-but-not-shown — and each constrains a map differently. A's case concerns exposure, B's concerns what follows exposure, and C's concerns the gap between capture and display.
The unit modeled is accordingly a relation among recorded responses, not a disposition attributed to each respondent. A response must exist in the record layer before any relation involving it can be computed, so presence there is necessary. It is not sufficient for the map to preserve that response's contribution: the relations are computed, and the construction may merge, rank, or omit what it draws on.
Which records feed the construction also changes what it represents. Take C's suppressed response: a map computed from what the display retained cannot reflect it, although elicitation succeeded and the record exists; a map computed from the full capture can. The underlying responses are identical in the two cases, so any difference between the resulting maps belongs to the construction rather than to the respondents. Comparisons between maps — or between a map and a simpler tally of responses — must therefore fix the elicitation task and the input records. Otherwise variation in exposure, response capture, and display gets folded into the comparison and read as variation in opinion.
If the map records relationships among responses, its coordinates license claims about what was recorded, not about every position a respondent might hold. The distinction matters at the edge of the display. A position no respondent expressed, and an item no respondent encountered, can both fall outside the mapped set. Once absent, neither is thereby observed as a refusal, a neutral stance, or tacit agreement with what remains visible. Their status is unrecorded.
In the illustration, the item one respondent never saw, the item another saw but left unanswered, and the response a display suppressed are three different records, not three inferred attitudes. They bear respectively on exposure, elicitation and presentation. None is a directly observed position, and none supplies the missing cause: to read a blank or an absence as a stance is to supply a reason the record does not contain.
The countercondition is worth stating precisely. A coordinate is not sufficient for direct observation of a position, and expression is not necessary for holding one. A coordinate fixes scope: it covers relations recorded under the elicitation and display rules then in force. If those rules change — different items shown, different responses suppressed — the same underlying responses can yield a different map. Comparing two encodings therefore requires holding the task and the inputs fixed and asking what each preserves.
A summary quantity computed over the map accordingly compares the responses that were elicited, recorded and displayed. It does not compare the positions respondents would have taken on items they never saw; those might resemble the mapped ones, but resemblance is possible, not established. Nor does counting more submitted responses widen the comparison: submission volume measures participation in the elicitation, not coverage of items or of positions held.
32.4LLM assistance in reported pilots
Small and colleagues report pilot exploration of LLM assistance for Polis facilitation, moderation and summarization, including effects of context limits and associated risks (Small et al., 2023). These pilots motivate further investigation of assistance under specified conditions. They are more than a proposal to conduct research, but they do not certify general adequacy across models, populations and deliberative tasks. A careful application should retain both the reported promise and the limits of the setting studied.
The three tasks are worth separating because they bear differently on what participants have said. Facilitation shapes what is invited, moderation decides what is retained, and summarization compresses what remains. This ordering is an interpretation offered by this chapter rather than a claim the paper makes, but it clarifies why summarization deserves particular attention. Compression is often useful: a smaller set of themes can help readers navigate a large body of statements, and shared descriptions can make divergent positions discussable at all. Yet an operation that helps make material legible can itself be difficult for a reader to inspect, since a fluent summary can quietly merge positions, drop minority statements, or attribute to a group a theme it did not express.
What the pilots motivate, then, is a modest requirement of traceability rather than an assurance of accuracy. Summaries produced with model assistance should be reviewed against source-linked material: each summary claim should point back to the statements or subgroups it is drawn from, so that a reviewer can test whether the compression is warranted. Such links do not by themselves show that a summary is correct, nor that any group has been represented; they make the derivation inspectable so that misrepresentation can be detected. The case is procedural. It concerns how generated summaries ought to be checked before they are allowed to carry interpretive weight in an opinion map.
The favorable evidence preserved from the pilots is evidence of feasibility in the bounded settings examined: assisted functions were carried out and produced results that could be inspected. That supports proceeding with care. It does not supply a general accuracy rate, establish legitimacy for the resulting summaries, or demonstrate performance in any current platform or interface. Feasibility here is therefore a necessary step toward application, not a sufficient warrant for it.
A further distinction concerns what the assistance is being asked to do. Adding a previously unlisted item and navigating relationships among recorded responses are different purposes. The first extends the field of alternatives; the second organizes material already collected. Conditional on using model assistance for both, success at one purpose would not entail success at the other. An expansion task risks introducing an item whose relevance is unclear, while a navigation task risks overstating or flattening relations that the recorded responses may not support. The possible failure modes differ, so neither outcome is inevitable, and neither can be ruled out from the bounded pilot account alone.
This distinction also qualifies the traceability requirement. For a summary of recorded responses, each claim can be linked to the statements or subgroups from which it was drawn, allowing a reviewer to test the compression. For a newly added item, however, there is no recorded response to link to. Traceability would instead have to record how the item arose and why it entered the set, which is a different check from verifying a summary against source material. The same procedural discipline therefore applies to different derivations.
What the pilots warrant, then, is prospective and purpose-specific: model assistance may be treated as workable enough to design around, provided its outputs are matched to the task and checked in the manner that task requires. No claim about a current interface or deployed performance follows.
32.5Unobserved and difficult-to-map positions
In the illustration, respondent A never encounters item X; respondent B encounters it and remains silent; respondent C’s recorded response exists but the display suppresses it. These are three distinct absences in the record, not three inferred attitudes. The first is an absence of exposure, the second an absence of action after exposure, and the third an absence from presentation despite presence in the underlying record. Only the third concerns display logic; the first concerns assignment or opportunity, and the second concerns a decision not to respond. Because the records differ, so does the evidence that would locate each: exposure or assignment data for A, evidence of encounter without response for B, and a comparison of stored and rendered records for C. Calling all three a single “missing voice” would conflate opportunity, action, and presentation, and might tempt a reader to treat silence as opposition or support when the records permit neither inference.
The distinction matters for any summary statistic. A count of visible statements measures what the display presents; a count of recorded responses measures what was captured; neither by itself measures positions that were never elicited. Compare like with like: fix the task, the item set, and the participants when asking what a representation reveals, while allowing that encounter, response, and display may be encoded differently. Nor is suppression necessarily censorship. A display may compress, rank, or filter; whether that hides a position depends on what the display is asked to show. Conditional supposition: if the display aims to summarize salience, suppression may be appropriate; if it aims to preserve individual recorded responses, the same suppression is a mismatch. In either case, the label “unobserved” does not itself identify which record is missing.
Once the three absences in the illustration are separated, the diagnostic work begins, because each is supported by different records and admits a different repair. Respondent A's never encountering item X leaves no response to interpret: the relevant evidence is the assignment or exposure record, and the question is whether A had any opportunity to form a recorded position on X. A retained exposure log is necessary to establish that opportunity, but it is not sufficient to tell a reader what A would have said. Repair here is prospective — altering which items reach which respondents, or supplying later exposure — and retrospective display changes cannot create the missing encounter.
Respondent B's case is unlike A's in kind, not degree. B saw X and produced no response, so exposure is documented and what is missing is the response itself. That silence is consistent with several readings — no settled view, a declined judgment, or a failure of the response instrument — and the record alone does not choose among them. Repair may therefore lie in elicitation or in an explicit option for declining; assuming an attitude from the gap is not licensed.
Respondent C's case is the easiest to repair and the easiest to mislabel. The response exists in the record and is suppressed only in the display, so no participation or elicitation is missing; the fix belongs to filtering or summary logic. As a conditional supposition: if the display were rebuilt to include C's response, A and B would remain exactly as unobserved as before.
This is why a single representativeness label cannot locate the problem. Such a label compares the displayed set against some target set of participants or positions, but it does not compare opportunities to respond, elicitation failures, and suppression within one accounting. Each diagnosis requires its own inputs, and a repair aimed at one is not sufficient for the others.
32.6Two methods with different goals
Two methods can be compared once the design task is held fixed. Take the task to be eliciting and organizing views on an issue. One method treats the inventory of positions as open: a participant may add an item not previously listed, so the propositions available for agreement or disagreement grow. The other treats recorded responses as given and renders them navigable, showing which positions co-occur, which stand apart, and where an individual response falls. The first extends the inventory; the second clarifies relations within it.
The purposes warrant different inferences. Adding a previously unlisted item is necessary if the inventory may omit a live position, but not sufficient for coverage: an item can be proposed once yet remain marginal to others, and its appearance says nothing about how widely it is held. Navigable structure yields legibility, not completeness. It can make recorded responses easier to compare and locate without introducing any absent position, so a clear organization of an incomplete inventory still depicts only what happened to be submitted.
A comparison of encodings is informative only when the task and inputs are fixed. If one encoding receives an open inventory and the other a closed one, output differences reflect the inputs as well as the encoding. Named quantities also need checking: a count of proposed items measures additions to an inventory, not the share of people holding them. Conditional on an inventory that already covers the live positions, discovery adds little while navigation may still assist; conditional on sparse records, any relational structure is possible output, not an inevitable gain in understanding.
These are prospective design suppositions, not tests of a current platform, and the bounded favorable pilot account in 32.4.1 bears on its own setting. Both purposes concern elicitation and organization; neither turns a count of submitted messages into a sampling warrant.
The two purposes distinguished above settle nothing about which method is preferable, because a preference ranking needs a shared evaluation task. If the task is to recover a contribution that has not yet been listed, the relevant question is whether the procedure can accept it at all; if the task is to make relations among already recorded responses navigable, the relevant question is whether the summary exposes structure a reader can inspect. These are different denominators. Declaring a winner would require holding the task and the inputs fixed while varying only the encoding of a contribution, and even then the comparison would be about that encoding under that task, not about the methods at large.
The stronger point is negative: success on one purpose is neither necessary nor sufficient for success on the other. New-item uptake is not sufficient for navigable interpretation, since a system can accept contributions without rendering their relations legible; nor is it necessary, since a navigable account of recorded responses can be built from a closed set. Conversely, clean structure is not sufficient for openness to what the structure does not yet contain.
This leaves complementarity as a possible, not an inevitable, arrangement. A prospective design could feed contributions accepted under an open procedure into a second stage that organizes them for inspection. That supposition is worth stating precisely: it holds only if the output of the first stage remains usable as input to the second, and if the two stages stay distinguishable in analysis. Otherwise the joint pipeline blurs contribution with comparison, and one method's goal silently sets the other's success criterion.
No platform or interface is tested here; the argument concerns how goals are specified. A quantity that counts recorded responses compares volume, not whether any of those responses bear the relation a reader needs.
32.7Transferable principles and nontransferable claims
Because no winner can be declared without a shared evaluation task, the comparison can be carried forward as a distinction rather than a verdict. Adding a previously unlisted item answers a question about coverage: what have participants not yet been able to say? Navigating relationships among recorded responses answers another: how do the statements already in the record bear on one another? Treating one as a proxy for the other would let a quantity built for one purpose answer a question it never measured.
From this follows the design lesson this section contributes. Contribution, comparison, and interpretation should remain distinguishable operations in the proposed workflow. Contribution records what participants offer, including items the designers did not anticipate. Comparison arranges recorded material so that relations among statements can be inspected. Interpretation draws conclusions from those relations. Their separability is a design commitment, not a report about any existing platform; the favorable pilot account preserved earlier concerns a different, bounded setting and supplies no evidence here.
Two qualifications bound the lesson. Keeping the operations distinct is necessary for the accountability this chapter seeks, but not sufficient: an interface can separate all three and still elicit thin contributions or summarize relations misleadingly. Nor is failure inevitable; the case remains a prospective illustration, and no current platform or interface has been tested against these purposes.
A countercondition shows when the distinction matters. Fix the task as relating only the responses already recorded, and admitting an unlisted item silently changes the input set; any comparison of encodings that suppresses this change compares different tasks. Fix the task as admitting new contributions, and a summary over recorded relations cannot speak to coverage. Even supposing one encoding served both, its quantities would still need naming by purpose: a count of submitted items compares participation volume, while a measure of relational structure compares configuration within a fixed set. Neither number stands in for the other.
The distinction between contribution, comparison and interpretation is the part of the lesson that travels. Keeping those activities separate is a design principle: it states what a component is for, and it can be applied to a proposal without presuming that any existing system already implements it. What does not travel is the authority behind the original studies. Their sampling frames, participant pools, and institutional settings belong to their own contexts; none of them is inherited by the manuscript's untested proposal. A favorable pilot account, such as the bounded one recorded in 32.4.1, supports only the claims its bounds contain.
Two purposes must also be kept apart. Adding a previously unlisted item enlarges the set of things that can be considered. Navigating relationships among recorded responses organizes what is already there. Success at the first is not sufficient for success at the second, and the second is no evidence that the first was done well. If a proposal is said to surface new ideas, the claim needs to name which activity is meant.
A second qualification concerns what a quantity compares. A tally of submitted items counts messages produced under a particular elicitation and access arrangement; it does not measure the distribution of views in a population, and it does not substitute for sampling. Treating that tally as a proxy confuses the volume of contribution with the coverage of participants.
Finally, comparisons across encodings hold only when the task and inputs are fixed. If two representations of the same recorded responses are compared while their encodings differ, the observed difference may belong to the encoding rather than to the responses. Conditional on identical tasks and inputs, the encodings can be ranked; absent that, the comparison is unsettled.