33.1The baseline before discussion
Before participants learn anything from one another or from the briefing, a deliberative study should record where they stand. That pre-discussion baseline is the reference point against which later movement is read: it identifies what was measured before the deliberative event, and therefore what a post-event figure is a difference from. Without it, a reported shift in aggregate position or confidence describes only the state after the event. It leaves the starting distribution, its spread, and the direction of individual movement unspecified.
A baseline is necessary for that description, but not sufficient for attributing movement to deliberation. Consider a proposed comparison in which the same participants receive the same source packet in two encodings: direct reading, or a summary accompanied by access to the sources. The task and inputs are held fixed, and the encoding varies. Suppose reading time and discussion time are recorded. The resulting quantity then compares exposure under two formats. It does not compare comprehension, nor the quality of the judgments formed. Recorded minutes of reading say how long the session ran; they do not say how much of the packet was consulted, whether participants opened the sources beside the summary, or whether the summary's omissions affected their reasoning. Discussion time carries the same limitation. Longer discussion is possible under either encoding and is not, by itself, evidence that one produced better-informed positions.
In this manuscript, the comparison is a proposed design rather than a reported trial. No balanced briefing audit is reported here, so no baseline distribution, no observed reading time, and no outcome measure are available. What the proposal contributes is a discipline of measurement: settle in advance of the event what will be recorded, so that any later figure has a defined starting point and a stated comparison.
A later difference leaves alternative explanations standing unless the study design addresses them. If participants respond differently after discussing, the change might reflect the discussion itself, unequal time on task, repeated exposure to the material, or simple revision of an answer one has had longer to consider. None of these is ruled out by the existence of a before-and-after gap. The design therefore has to hold some of these factors fixed or vary them systematically; an imagined control group cannot stand in for a comparison that the study does not report.
The proposed comparison does part of this work. It keeps the source packet identical while varying the encoding: participants either read the packet directly or receive a summary with access to the sources. The response task is the same, and reading and discussion time are recorded. What varies is the presentation, not the underlying evidence, so any difference in outcome is more narrowly attributable than a difference across studies with different materials. It compares presentation formats, not the presence or absence of deliberation. This is a prospective illustration, however: no trial is reported here, and no audit of briefing balance has been run, so claims about actual effect sizes would be unsupported.
Two quantities require care. Reading time compares how long participants take under the two presentations; it does not compare comprehension, accuracy, or the quality of deliberation. Discussion time compares duration of talk, which may reflect efficiency, confusion, or engagement in either direction. Neither measure alone licenses a claim that one encoding is better.
Suppose, as a conditional supposition, that the summary arm reaches an answer faster. That outcome would establish possibility under these conditions, not that summarization inevitably shortens or improves discussion; the alternative explanations above remain live unless the design separately rules them out.
33.2Briefing and balanced material
Information supplied to participants before and during discussion is part of the intervention being evaluated, not a neutral backdrop to it. America in One Room illustrates the point. Fishkin and colleagues describe briefing materials, moderated small-group discussion and pre/post questionnaires (Fishkin, Siu, Diamond and Bradburn, 2021).
The materials, the moderation and the occasions of measurement enter together as a package. Any movement in responses is therefore movement after exposure to that package. A difference between questionnaires completed before and after the event compares the same respondents across occasions; on its own it is not a comparison of discussion against no discussion, and still less of deliberation against a quick summary of a text.
A conditional supposition shows what would separate the channels. If two comparable groups received identical materials and only one discussed, then the contribution of interaction could be distinguished from that of the information. Without such an arrangement, exposure to the material remains a rival explanation for whatever change appears. Random assignment to discussion groups does not provide it: the procedure distributes participants among small groups inside the treated set rather than assigning anyone to receive information without discussion or discussion without information. Differences among groups may bear on composition or on moderation, not on whether deliberation outperforms non-deliberation.
Timing qualifies the claim as well. Material read beforehand, material available in the room, and the conduct of the moderated discussion are distinct exposures, and folding them into one undifferentiated briefing variable hides which of them could carry an effect. Exposure to information makes changes of mind possible; it does not make them inevitable, and it does not by itself make the resulting answers better in any specified respect. How the materials themselves were described is the next question.
Calling a document a briefing tells a reader about its role in a procedure, not about its contents or their even-handedness. The relevant descriptive claim is narrower: what materials participants received, who prepared them, how competing positions were presented, and whether any review checked that presentation. Such a verified description supports statements about what was supplied. It does not by itself support the evaluative claim that the material was balanced, which requires a further assessment against stated criteria rather than acceptance of the label.
The distinction matters when the case is treated as a prospective design. Suppose a probability-based panel receives a briefing, meets in moderated small groups, and answers comparable questions before and after. Suppose further that small groups are formed at random. That randomization supports exchangeability across groups; it does not create randomly assigned treatment and control conditions, because every participant receives the briefing and the discussion format. A pre/post difference therefore compares the same respondents' answers at two times under several co-occurring conditions. It does not compare balanced against unbalanced material, nor does it isolate briefing from moderation, group exchange, or repetition of the questions. Change is possible under this design; it is not inevitable, and its direction cannot be read off the design's structure.
A second limit concerns sufficiency. Even a carefully verified account of balanced material would not show that participants weighed the alternatives evenly, while an uneven briefing would not make every resulting change spurious. What is necessary is an accurate description of the informational conditions; what remains to be established is how those conditions bear on the particular comparison being claimed. The defensible application is therefore modest: materials and their provenance belong inside the specification of what participants underwent, and any attribution of response change must rest on a comparison that actually separates briefing from the other elements of the occasion.
33.3Recruitment and the comparison group
Recruitment, participation, treatment assignment and comparison-group construction are four distinct operations, and a study can be transparent about one while silent about another. In America in One Room, the abstract-level report describes a national deliberative event with a pre/post control comparison (Fishkin, Siu, Diamond and Bradburn, 2021). Each element licenses a different claim.
Recruitment defines who could enter: a probability-based panel supplies a sampling frame and known selection probabilities. That makes population inference conceivable; it does not by itself deliver a representative realized sample, because who agrees, travels and attends is a later filter. Participation does not automatically change a predeclared estimand, but it does change the observed sample: a statement about attendees describes those who stayed through discussion, while a statement about the recruited panel describes everyone selected. When the two differ, the gap cannot automatically be attributed solely to attendance rather than the intervention itself.
Treatment assignment is the third operation. Here random assignment sorts participants among small discussion groups. That can balance who sits with whom and support comparisons across groups; it does not create an untreated arm. If (conditional supposition) the design had separately allocated people to deliberation versus no deliberation, that would require its own assignment record; group randomization does not supply one.
Comparison-group construction is the fourth: the pre/post control sample draws from the same probability panel, so treatment and comparison respondents share a sampling frame. A shared panel does not establish identical contact procedures, and a shared frame reduces one source of mismatch but is not sufficient for baseline exchangeability, since differential consent or response can still separate the samples. Also, a pre/post quantity compares the same respondents' answers across time; by itself it does not compare deliberators with non-deliberators. Keeping these four steps apart prevents a well-described recruitment procedure from being mistaken for the assignment mechanism a causal contrast would require.
The comparison group has to be specified at the level of the design, not inferred from the scale of the exercise or from the word "control." In the design sketched for this case, participants are drawn from a probability-based panel, receive a briefing, meet in moderated small groups, and are measured before and after; assignment to small groups is random. That last feature is narrower than it first sounds: it distributes people across discussion groups, and it does not by itself constitute random assignment to deliberation versus no deliberation. Without a separately constituted comparison group, the pre/post contrast estimates change over the study period for participants, not the effect of deliberation relative to what would have happened otherwise.
This distinction sets the scope of any claim the design can support. Probability sampling of the panel bears on who is represented; it does not make the treatment comparison randomized, and random group assignment bears on group composition, not on selection into the deliberative experience. Each feature is necessary for a particular inference but insufficient for a broader one. National scale enlarges coverage; it does not repair a missing counterfactual. A label such as "control group" likewise names a role rather than proving how people entered it: the relevant question is what process produced the two groups and whether that process is independent of expected outcomes.
Suppose, as a conditional supposition, that a comparison group were assembled by some nonrandom route—convenience recruitment, for instance, or self-selection. The difference between groups could then reflect pre-existing differences rather than the discussion experience. That is a possible distortion, not an inevitable one; whether it bites depends on how the groups compare at baseline and which alternative explanations remain viable. The quantity at issue is therefore specific: change in one group measured against change in another, or against no comparison at all. These are different comparisons. Fixing the task and inputs first—recruited participants, briefing, moderated discussion, timing of measurement—lets the encodings of the comparison vary without allowing one label to stand in for evidence about assignment.
33.4Time as a resource
Deliberation does not simply consume time; it distributes time among distinct activities. Participants must read what the issue supplies, listen to what others make of it, and compose responses that answer earlier contributions. Each activity supports a different part of the work. Reading establishes the evidential base participants share, listening exposes them to interpretations they had not reached alone, and responding allows objections to be answered in public. Altering the format of preparation can therefore alter how the same total duration is divided, even when the subject matter and the nominal time budget remain unchanged.
A summary plus access to the full source packet is a plausible case in which this reallocation occurs. The summary can compress initial orientation, while source access preserves the possibility of returning to the underlying material. Whether that possibility is used is an empirical question: access is not the same as consultation. The prospective comparison already noted—same source packet, direct reading versus summary with source access, recording reading and discussion time—could expose the redistribution, but no trial of that design is reported here. Its comparison, if run, would concern durations under fixed inputs and a fixed task; it would not by itself measure the quality of attention devoted to either activity.
This distinction matters when testing what time contributes. Sufficient time to read and discuss is necessary for some deliberative functions, yet time spent is not sufficient to produce shared understanding, accurate recall of the evidence, or reasoned agreement. Participants may read at length and still misunderstand; they may listen extensively without revising their view. Conversely, a shorter preparation period could release time for discussion that compensates, in part, for reduced reading—provided participants actually use it that way. The direction of the effect is therefore possible rather than inevitable, and the recorded durations should be read as evidence about allocation, not as a verdict on deliberative equivalence.
Time saved enters such a comparison as a benefit to be measured alongside understanding, not as evidence that equivalent deliberation occurred. In the proposed comparison, participants receive the same source packet and either read it directly or work from a summary with access to the sources, while reading and discussion time are recorded. Holding the task and inputs fixed while the presentation varies means that a difference in outcome is attributable to the summary-plus-access condition as a whole, not to a shorter assignment or a thinner packet.
The design records reading time and discussion time as separate quantities, so a reduction in one can be offset by growth in the other. A claim of time saved therefore compares total elapsed time, and that comparison should be stated before the reduction is treated as a benefit at all. Even a real reduction establishes only that the faster condition consumed less time; it does not show that participants understood the sources equally well or weighed the issue comparably.
Nor is the saving inevitable: as a supposition, if the summary leads participants to open the underlying sources repeatedly, their reading time could converge on or exceed direct reading. The same qualification applies to understanding were equivalence observed: it would hold for the study's stated measures and population, not for the two forms of engagement in general. Time saved and understanding are thus co-measures that can move independently, and each requires its own evidence. The practical application is accordingly narrow: the comparison can report where time was spent and what the stated measures registered, while leaving open whether the quicker condition reproduced the fuller process. Since no trial or balanced briefing audit is reported here, the comparison remains a prospective design: these are proposed tests, not findings.
33.5Attitude change and its interpretation
An attitude-change result is bounded by the measure that produced it. A study can support the claim that responses shifted on a particular item, scale, and occasion, among the people who supplied both measurements. That is enough to establish change in the recorded response, but it does not by itself establish a change in a broader disposition called attitude. Nor does a stable average show that no one changed: offsetting movements can leave a summary statistic unchanged. The quantity being compared is therefore the specified rating, not understanding, endorsement, or the reasons participants might give.
Population limits matter just as much. A result obtained from recruited participants who completed the follow-up question does not automatically extend to those who declined, dropped out, or were never eligible. Nor does it extend from the tested wording to attitude questions in general. When two representations of answers are compared, hold the task and the information presented fixed if the claim concerns the representation itself; otherwise a difference may reflect what respondents were asked to do rather than how their answers were recorded.
As a prospective design, suppose one arm receives more discussion time than another while recruitment also differs. The contrast is bundled: it compares packages, not discussion alone. If ratings rise in the longer arm, the design leaves open whether participants learned the issue, were persuaded, or followed the group. Each mechanism is possible; none is inevitable from the pattern. Nor would equal ratings show that discussion did nothing, since effects could cancel or fall outside the chosen scale. Time saved is therefore a proposed benefit to measure alongside understanding, not evidence that equivalent deliberation occurred.
A rating that moves between the first and second administration is an observed difference in responses. It is not, by itself, a diagnosis of what produced the difference. Learning, persuasion, conformity, fatigue, and demand cues are each sufficient to generate a shift in a rating under some conditions, so observing a shift cannot distinguish among them; conversely, a stable rating does not show that nothing was learned, because new considerations may cancel one another or the scale may fail to register them.
Consider a prospective design in which one arm receives an hour of moderated discussion and the other receives a brief summary, with participants recruited through different channels. The contrast is bundled. Any difference between the arms is compatible with the effect of discussion time, with the effect of who entered the study, or with their interaction. The study may establish that the two packages produce different response distributions in that population; it does not establish which component carried the effect, and the bundles are not necessarily separable after the fact. To narrow the account, hold recruitment fixed and vary only time; that removes one rival explanation but still leaves persuasion and learning as candidates, since both are compatible with more exposure to reasons.
What the named quantity compares matters. A mean difference on the rating scale compares average positions at two administrations for those who answered both. It compares neither the reasons participants endorsed, nor their comprehension, nor the durability of the change, and aggregation can conceal opposing movements within the group. Suppose the design added post-discussion reports of participants' reasons. Those reports would show what participants said moved them; they would not by themselves prove that persuasion rather than conformity to the discussion's framing did the work. Thus change is a possible outcome of structured discussion, not an inevitable one, and its interpretation depends on which rival explanations the design has actually ruled out.
33.6Participation costs
Participation costs are borne by participants as well as organizers. The relevant accounting begins with the minutes a person must give up and with the conditions under which those minutes can be spent: when sessions meet, whether the venue or platform is reachable, whether materials arrive in an accessible form, and whether someone can decline or interrupt without penalty. A software licence fee may fall on the host while none of these burdens do.
The proposed comparison makes the distinction concrete while holding the task fixed. In one arm participants read the same source packet directly; in the other they receive a summary with access to the same sources. Both groups discuss, and the design records reading time and discussion time. Because the assignment and inputs are constant, differences in recorded minutes can be attributed to the form of mediation rather than to a different question. The comparison is prospective, however: no trial or balanced briefing audit is reported here, so it shows what could be measured, not what an implementation will cost.
Even if run, the two timers would answer a narrow question. Reading time plus discussion time compares burdens under one schedule of access, not the full price of participating. A brief reading period can push work into preparation before the meeting, or leave participants without a route to raise questions later. A longer discussion may reflect richer engagement rather than waste. The recorded quantity therefore needs a definition: time to first response, active talk, waiting, or elapsed session time.
Access conditions are not incidental to the measurement. If a participant cannot reach the session, or cannot use the materials in the form offered, a low recorded time is uninformative about that person’s cost. Short measured time is thus a sufficient indication of low time burden only under stated conditions of access; it is not a sufficient indication of low participation cost overall, and high time remains possible even when the interface itself is easy.
Consider a proposed comparison, not yet reported as a trial: the same source packet is offered either for direct reading or as a summary with access to the sources, while reading and discussion time are recorded. As an illustration, it fixes the task and inputs and varies only the encoding. That isolates one question: how the two presentations allocate a participant's reading and discussion time.
The narrower result must not be read as the broader one. Less reading time would show that the summary lowers one burden for those who take part. It would not be sufficient to establish that participation is cheaper overall, since recruitment and conditions of access sit upstream of the measurement. If would-be participants never enter, or cannot use either presentation, their absence leaves no trace in the recorded times. An easier interface can therefore plausibly reduce time costs while recruitment and accessibility barriers persist.
Recording discussion as well as reading time also qualifies what lower cost means. The summary may shorten reading yet lengthen discussion, so the two quantities should be read together rather than merged into an unstated total. The comparison is between two encodings of a fixed packet for a fixed task; it does not compare overall participation costs across differently recruited populations.
Suppose, as a conditional assumption, that both arms draw on the same recruited group and neither changes the conditions of access. Even then, the recorded times would estimate a within-group difference in effort, not the cost of entering participation at all. Conditional on a trial finding less reading time, the warranted inference stays narrow: for this task and these participants, one encoding required less reading. Whether participation becomes genuinely easier for others depends on conditions the measurement does not reach.
33.7A fair comparison with an AI-assisted workflow
The comparison worth running next is narrow: hold the reading task fixed and vary only how participants meet the material. Both arms receive the same source packet; one reads it directly, the other reads a summary with source access. The proposed design records reading and discussion time, so an efficiency claim would name what it compares: time spent under one interface, not comprehension or agreement under the other. No trial and no balanced briefing audit is reported here, so this is a specification rather than a result.
Fixing the task and inputs separates two conditions that are easily confused. Comparable source access means the same packet is reachable in both arms, not that it is equally likely to be consulted. Availability is a weaker property than use. As a supposition, if the summary arm seldom opens the sources, a measured difference would partly describe that behavior rather than the summary's treatment of the evidence. Recording access use would therefore belong in the measurement, though it would still not show what participants understood. The converse is informative as well. If both arms consult the packet at similar rates, access can be treated as balanced, and a remaining difference is more narrowly about how the material was first presented.
Nor is saved reading time inevitable. A summary takes time to read and to check against the packet, and counting only the first pass would understate the total. Conversely, equal reading time would not establish that the assisted workflow achieved nothing: the same minutes might be spent differently.
Finally, the design should keep candidate explanations apart. Learning, persuasion and conformity remain distinct until something in the design identifies them. Faster reading, or similar answers, would not tell which operated. Comparable source access is necessary for calling the contrast one of workflow; it does not by itself separate those mechanisms.
The proposed comparison holds the source packet constant, but a constant packet is not a constant treatment. If one arm receives richer evidence, or more time in discussion, then any difference in responses belongs to the bundle of changes as a whole. The label "summary versus direct reading" names only the encoding of the source material; it does not establish that exposure, deliberation time, or recruitment were equal. Recording reading and discussion time is informative about what participants did, but recorded minutes do not by themselves allocate a causal role to any one of them.
Even if the arms were matched on those inputs, the inference would need narrowing. Matching is sufficient to attribute a difference to the encoding under this task and these sources; it is not sufficient to show that one workflow produces better judgments generally. A gap between the arms would still leave several candidate mechanisms open: learning from evidence, persuasion by an ordering or framing, and conformity during discussion. The design described here reports no trial and no balanced briefing audit, so nothing yet separates these possibilities.
Suppose, as a conditional supposition, that the summary arm reaches its response faster and endorses it more readily. Faster agreement is equally consistent with efficient uptake and with reduced scrutiny of the packet; tighter agreement among participants could reflect shared understanding or pressure toward a shared position. Distinguishing those accounts would require varying evidence richness and deliberation time independently while holding the task and participant pool fixed. Until such variation exists, the contrast should be reported as bundled, and its measured quantities—minutes, agreement rates, response choices—described as comparing what they actually compare.