data statements as a design solution and professional practice
27.1Who and what entered the corpus
The collection note for the fictional corpus answers a narrow question first: what entered the corpus, and under which conditions of entry. It records that the corpus accepts questions submitted during a specified consultation period. This identifies the material as question text and the contributors as people submitting questions during that period. It establishes no more. The description is therefore useful not because it settles every aspect of the corpus population, but because it marks the boundary within which later claims can be checked.
That boundary matters when the other records are consulted. The editing history indicates how submitted text was changed, but it does not enlarge the population described by the collection note. The model description states an intended summarization use, but intended use does not determine which texts or contributors were collected. The review record evaluates particular versions under a stated task; it does not retrospectively alter the corpus's collection scope. Each record answers a different question, and cross-referencing them can help reveal when a claim relies on the wrong kind of record.
A useful collection description consequently names the units admitted and the condition governing admission. Here those units are submitted questions, and that condition is arrival during the consultation period. Within this known scope, a reader can ask what the corpus contains and can distinguish it from what a later transformation produced. Anything outside that scope remains unaddressed rather than being reconstructed from adjacent documentation.
This resembles the discipline already required of a relation vocabulary: labels should contribute useful discrimination rather than ornamental completeness. A collection note earns its place by identifying the corpus's admitted material precisely enough to constrain subsequent documentation. The protocol here draws on Data Statements (Bender and Friedman, 2018) to ask which population a dataset describes and where generalization would exceed it. That proposal illustrates one documentation approach; it does not guarantee that completed documentation will itself remove bias or be read. The fictional corpus supplies only the stipulated collection facts above.
That bounded description leaves an important limit untouched. The collection note records no demographic attributes for the participants whose questions were accepted during the specified consultation period. Topic diversity among those questions cannot fill the gap: a corpus broad in subject matter still leaves those fields empty. An editor who treats the range of topics as evidence of demographic coverage has converted an absence of information into a positive claim. The consultation period bounds when submissions were accepted; it does not characterize who made them.
The attributes therefore remain unknown. That is not the same as saying participants lacked any demographic characteristic, or that they formed a representative population; each would assert more than the note establishes. A conditional route exists: if a later record supplies attributes tied to particular submissions, coverage could be described for that group and that period. Without such a record, no demographic summary follows from topical variety alone. An edit log cannot retrospectively supply attributes that were never collected.
The same discipline keeps the remaining documents from answering a question outside their scope. The editing history can show that a conditional phrase was removed from Q-v1; the model description states intended summarization use, but does not identify who removed the phrase in this case. The review record compares S-v1 with Q-v1 and Q-v2 under a stated task, and if S-v2 changes the conclusion, S-v1 remains the object of the older review. These records document collection, transformation, model and review conditions separately and cross-reference one another. Cross-referencing keeps the limits visible; it does not merge the records into a single answer. None supplies the participant attributes that the collection note never recorded, and assigning responsibility for a task does not stipulate that the task was carried out.
27.2Collection and exclusion
The collection note for this fictional corpus should state the criterion that admits a submission: a question is collected only if it was submitted during the specified consultation period. That temporal boundary belongs in the collection record, not in the editing history, the model description or the review record. Those documents address different objects—an edit made to a question, the intended use of a model, and a check of one summary against particular versions. Keeping the admission rule with the collection record keeps the boundary attached to the object it actually governs.
The note should record exclusion in terms of the same rule. A submission arriving outside the consultation period falls outside ordinary collection; if it is nevertheless accepted, the record should identify that action as an exception rather than let it appear to have been covered by the original criterion. The note should also avoid absorbing work that happened elsewhere: editing a question or reviewing a summary does not retroactively make a submission collectible.
Datasheets record dataset creation and intended use so producers and users can communicate assumptions (Gebru et al., 2021; first preprint 2018). That is the gain here: the collection record states the admission boundary in a form that can be compared with the corpus as delivered. The record alone, however, cannot verify that the criterion was applied correctly, or that the resulting corpus is of suitable quality for any particular reuse. It establishes what rule was said to govern entry and exclusion; whether that rule was a good one remains a separate matter. Recording the criterion precisely is therefore a prerequisite for evaluation, not a substitute for it.
Once the collection note states which submissions were accepted and which excluded, the reader gains a target for judgment, not a completed defense. In the fictional corpus, questions arrived during a specified consultation period, and submissions outside it fall outside the collection's declared scope. Recording that boundary allows a reader to ask whether the period was the right boundary, whether late submissions were alike or unlike included ones, and whether the exclusion served the corpus's intended use. None of those questions is answered merely by writing the criterion down.
Transparency has a limited role: it makes the criterion inspectable so that challenges can be precise. A documented exclusion can still be unjustified—for example, if the consultation period was chosen for convenience and systematically left out relevant questions. The note establishes what was done, and by what rule; it does not establish that the rule was proportionate, well matched to the research question, or applied consistently. The collection record therefore answers "which rule governed admission?" but not "was that rule defensible?" Nor does the model description, which reports intended summarization use, settle the question; a stated purpose is not evidence that the admission rule fit it.
Scope also limits what collection notes can repair. The note for the fictional corpus has no demographic attributes, so topic diversity among accepted questions cannot fill those missing fields. If a reader needs to assess whose questions were collected, that absence remains an absence; broadening the range of topics is not evidence of demographic coverage. The same boundary governs excluded submissions: without records of their features, the note cannot support claims about what was omitted.
These qualifications leave the criterion available for scrutiny rather than weakening the need to record it. A criterion that cannot be inspected cannot be assessed; one that is recorded can be challenged, refined, or defended. Explaining what changed in the material is a different record's task.
27.3Transformation records
Collection notes record how material entered the corpus and which material was left out. Transformation records answer a different question: what changed between source text and derived text? In the fictional corpus, the edit log records the removal of the condition if an extra staff member is available from Q-v1. The log therefore fixes a before and an after. Whatever else remains unclear, the reader can see that this condition appeared in the submitted wording and does not appear in the edited wording.
That is the work of transformation documentation, and it is enough to make a challenge precise. A reader who thinks the deletion matters can point to the exact revision and ask whether removing the condition was warranted for the intended summarization task. The record does not answer that question; it makes the question answerable in the first place. Without it, the disagreement would have to be reconstructed from the two versions, and a disputed edit could be mistaken for a difference that was present in the submitted material all along.
The limit follows from the record's scope. It states what changed, not why the editor acted, not whether the change improved the corpus, and not what any model subsequently did with the derived text. Those inquiries belong to collection notes, review records and model descriptions respectively. Keeping the documents separate matters here because the edit log can be complete while the reasons for the edit remain undocumented; assigning maintenance responsibility does not establish that anyone performed or examined that work.
Cross-referencing preserves the connection between the separate records. The edit log identifies the transition from Q-v1 to Q-v2, while the review record identifies which version was checked under the stated task. A conclusion then stays attached to the version actually assessed instead of silently carrying over to a later transformation that no review examined.
A model description can tell a reader what the model was built to do; it cannot answer who removed the condition that an extra staff member be available from Q-v1, or when, or under whose authority. That question belongs to the editing history. In the fictional corpus, the edit log records the removal while the model description states intended summarization use. Those two statements answer different questions, and the model description does not become evidence about the edit simply because the edit affects text the model may later process.
Cross-reference is therefore doing work of a specific kind: it points from the model description to the transformation record when the challenged change is at issue, without merging the two. Suppose the challenge is that the phrase removal distorted a question's meaning. The relevant comparison is between Q-v1 and the edited text recorded in the log, not between Q-v1 and the model's stated purpose. A general account of intended use may help explain why the model matters here, but it cannot identify the author, occasion, or rule of the particular change.
The same discipline applies to the review record. The review checks S-v1 against Q-v1 and Q-v2 under a stated task. If S-v2 later changes that conclusion, the review still has S-v1 as its object; it says nothing about S-v2 unless a new review is recorded. Assigning maintenance responsibility likewise states who is answerable for future work; it does not stipulate that any maintenance occurred. Each record therefore carries its own object and date, and cross-references should make those boundaries visible rather than blur them.
Scope has its own limit worth marking: the collection note records no demographic attributes. Topic diversity does not fill those missing fields, just as intended model use does not stand in for a record of the particular edit being challenged.
27.4Intended and unsuitable model uses
The model description in the fictional corpus reports intended summarization use. That statement is useful precisely because it is bounded: it names the work the model is meant to do, and by omission it leaves room to list the conditions under which the reader should not assume the same result. Mitchell et al. (2019) connect model reporting to evaluation conditions, intended uses and relevant groups (Mitchell et al., 2019). The pairing matters here. Intended use says what the description claims; the evaluation record says what was actually checked.
The review record checks S-v1 against Q-v1 and Q-v2 under a stated task. That is the evaluated condition, and it can be carried into a limitations statement almost literally: questions drawn from other periods, other task formulations, or outputs produced after the model was revised have not been covered by that review. If S-v2 changes the conclusion reached about S-v1, the old review's object remains S-v1; a reader who cites the review for S-v2 has moved outside its scope even though the model description has not changed.
The same discipline applies to the corpus side. The collection note records questions submitted during a specified consultation period but carries no demographic attributes. Topic diversity in those questions does not supply the missing fields, so the description cannot promise performance for particular groups, and the unsuitable-use list should say so rather than infer coverage from subject matter. Likewise, the edit log shows that an extra staff member clause was removed from Q-v1, but the model description does not record who removed it in this case; assigning maintenance responsibility does not stipulate that any maintenance work occurred. Intended-use language therefore draws two lines at once: one around the task and material evaluated, another around everything not evaluated, including populations whose attributes were never collected.
A statement of intended use answers a narrow question: what task a model is offered for. In the fictional corpus, the model description reports intended summarization use. That report tells a reader where relevant evidence should be sought; it does not supply any. Performance enters elsewhere, through the review record that checked S-v1 against Q-v1 and Q-v2 under a stated task. Treating the model description as if it carried that result silently merges two documents whose separate work the archive preserves.
The distinction matters precisely where the documents diverge. The edit log records removal from Q-v1 of the condition that an extra staff member be available, while the model description does not say who removed that phrase in this specific case. The two records therefore answer different questions about the same material, and neither absorbs the other. Likewise, assigning maintenance responsibility to a role does not stipulate that the maintenance work occurred. Responsibility is a forward-looking allocation; the record of performed work is absent unless it is recorded.
The inference is that intended-use language cannot be converted into evidence of performance, and still less into authorization for an institutional deployment decision. A deployment commitment rests on judgements about consequences, duties, and fitness for a particular setting that the corpus documents do not make. If an institution were to deploy the model on the strength of the wording in the model description, its decision would rest on a claim none of the four records demonstrates.
What the language does do is mark the boundary of evaluated relevance. A use falling outside the summarization task tested against Q-v1 and Q-v2 is unaddressed rather than prohibited, and any claim about it would need its own evidence. The cross-references keep that boundary checkable: the review naming S-v1 stays a review of S-v1 when S-v2 is produced. If S-v2 changes the conclusion, the old review's object is unchanged and the newer result requires its own record.
27.5Evaluation conditions
An evaluation result is not just a score; it is a score plus the conditions under which it was obtained. In the fictional corpus, the review checks S-v1 against Q-v1 and Q-v2 under a stated task. The result therefore names four things at once: the task, the two material versions, the model version S-v1, and the review record that made the comparison. Removing any one of these turns a bounded claim into an unattributed one.
The version labels do real work. The edit log records removal from Q-v1 of the condition that an extra staff member be available, while the model description does not say who removed that phrase in this specific case, and assigning maintenance responsibility does not stipulate that the work occurred. So the evaluation record may state that the comparison used Q-v1 and Q-v2; it cannot settle who edited them, and that unresolved question should not be smuggled into the score. The model description's intended-use language likewise remains a statement of purpose; it does not substitute for the task under which the review was actually run.
The material's limits travel with the result as well. The collection note has no demographic attributes, and topic diversity does not supply those missing fields. Accordingly, the reviewed comparison cannot support demographic characterizations of the questions it was drawn from. The absence is a condition of the evaluation, not a gap that some later document may silently fill.
This is what attachment means here: the conditions stay legible next to the claim, rather than being filed where a reader must reconstruct them. The qualification that matters is that attachment is only as good as its use. If the record is completed ceremonially and never consulted, the cross-references do no work, and the score will be quoted without its task, materials, model version or review conditions—exactly the free-floating result the documentation was meant to prevent.
A result travels farther than the conditions under which it was produced, and that gap is where an applicability argument belongs. Once the evaluated task, material, model version, and review conditions are attached to the score, a later reuse under different conditions needs its own argument. The earlier documentation supplies the premises for that argument; it does not license the extension by itself.
Consider the review record in view. S-v1 was checked against Q-v1 and Q-v2 under a stated task, and that review retains S-v1 as its object even if S-v2 would change the conclusion. Suppose, conditionally, that a later team reuses the reported score for a summarization deployment built on S-v2 and on material collected after the consultation period is reopened. The number may be reproduced exactly and still be the wrong evidence, because the evaluated version, the material, and the collection window have moved. The applicability argument must say what changed, which documented conditions still hold, and what the number can support under the new conditions. If it cannot make that case, the score has not been transferred.
That is why the four documents stay separate but cross-referenced. Collection notes bound the questions accepted during the stated consultation period and contain no demographic attributes; the editing history records removal of a conditional staffing phrase from Q-v1 without naming who removed it; the model description reports intended summarization use without filling that gap; the review record fixes its object. Topic diversity cannot supply fields that were never collected, and the model description cannot supply editorial authorship. Cross-reference lets a reader compare these limits directly rather than absorb them as one narrative. The applicability argument performs the connecting work, and it has force only when that comparison is actually consulted.
27.6Responsibility for maintenance
The separation of collection, transformation, model and review documentation remains useful only while each record still points to the things it describes. Once a link breaks, or a source's status changes, the cross-references that let a reader connect corpus conditions to model conditions decay without announcing themselves. Maintenance therefore needs a named owner, and ownership should be allocated by document rather than left as a general duty of whoever notices the problem.
For the fictional corpus, this yields four specific assignments. The collection note's owner maintains the link describing how questions were submitted during the specified consultation period and preserves the note's statement that no demographic attributes were collected, since topic diversity cannot supply those missing fields. The editing history's owner maintains the entry recording removal of the extra-staff condition from Q-v1 and any cross-reference from the model description to that entry. The model description's owner keeps its account of intended summarization use attached to the version it describes, rather than implying an editorial decision it does not record. The review record's owner keeps visible that the review checked S-v1 against Q-v1 and Q-v2 under the stated task.
Changed source status is a distinct maintenance event. If a cited source is later superseded or withdrawn, the owner of the document citing it updates that document's status note and repairs the links pointing to it, without retroactively rewriting what the earlier record asserted. The same discipline governs the review case: when S-v2 changes the conclusion, the old review's object remains S-v1, so the change appears as a new review record and the earlier one is annotated, not overwritten. This ordering is consistent with the problems of downstream propagation reported in an interview study of practitioners, where undervalued data work can compound into later difficulties (Sambasivan et al., 2021); the study concerns its own practitioner sample and supplies no rate for this workflow.
Assigning maintenance responsibility does not stipulate that maintenance has occurred. It names who owes the correction and what the correction covers; whether that owner can actually perform the work is a further question.
Naming a maintenance owner settles who is answerable; it does not settle whether the work can happen. Responsibility and capacity therefore need separate attention. An owner given no time, no authority to alter the relevant record, and no actionable queue of pending items may remain formally responsible while performing no maintenance at all. The assignment should be read as a locus of answerability, not as evidence of activity.
The fictional corpus makes the difference concrete. Its edit log shows that the extra-staff condition was removed from Q-v1, while the model description reports intended summarization use but does not identify who removed that phrase in this particular case. Assigning responsibility for future maintenance does not retroactively fill that gap, and it does not stipulate that any work has occurred. What helps is a queue that names the record to be examined, the condition to be addressed, and a point by which action is due: collection notes, editing history, model description, and review record should each be tracked as a distinct object and cross-referenced rather than merged into a single maintenance narrative.
Capacity still has limits of its own. Scheduled time and an actionable queue show that work is possible and planned; they do not show that a correction was made. Only the resulting record can support that claim. Thus if S-v2 changes the conclusion reached about S-v1 against Q-v1 and Q-v2, the earlier review continues to concern S-v1 unless a new review record is created, and a well-staffed maintenance arrangement does not alter that fact. Conditional on an owner receiving both time and a specific queue entry, one may say the relevant check is being managed; one may not yet say the documentation has been corrected or that the missing attribution has been recovered.
27.7Documentation tested by a concrete question
A direct test of the fictional corpus's documentation would ask who removed Q-v1's condition about an extra staff member being available, and which record supports the answer. The edit log is the appropriate record type to consult, because the removal is a transformation event. But unless the log names the responsible actor, it establishes only that the phrase was removed, not who removed it. The model description cannot fill the gap; its claim concerns intended summarization use, and it says nothing about this particular editing act. The answer, then, is not "consult the documentation" but a definite one: the available records support what changed, and they do not identify who changed it.
That negative result is useful precisely because the question forces each document to answer within its own scope. The collection note records how questions were admitted during the consultation period and what demographic fields are absent; topic diversity in the corpus does not supply those missing fields, and no cross-reference should let a collection-level descriptor stand in for transformation history. The review record is similarly object-bound: a review of S-v1 against Q-v1 and Q-v2 under the stated task retains that object even if a later S-v2 would change the conclusion. Asking who edited Q-v1 therefore requires a cross-reference that points from the question version to the transformation record, while leaving the review's conclusion attached to the summary version it actually examined. Responsibility for maintenance, likewise, is an assignment rather than evidence that the work occurred.
This method answers the ceremonial-documentation objection more concretely than a completeness checklist: a document is doing work when a stated question can be traced through it to a specific record and a specific object. The limit is that the test reaches only the path it queries; a corpus may satisfy the removal question while leaving unrelated transitions unsupported, so each new question requires its own trace.
Take the narrow question first: who removed the phrase about an extra staff member being available from Q-v1, and what record supports that answer? The edit log records the removal. That is its proper scope, and it answers the question directly. The model description does not: it reports intended summarization use but says nothing about who removed that phrase in this particular case. If a second documentation layer is to be kept alongside the edit log, then, it must answer something the log cannot. Here the model description earns its place by stating application scope—what the model was intended to do—not by restating the edit history in less precise terms.
The same division governs the review record. The review checks S-v1 against Q-v1 and Q-v2 under a stated task. Its conclusion is therefore bounded to S-v1. If a later S-v2 changes the conclusion, the old review's object has not moved; the review remains a record about S-v1, and any claim about S-v2 requires its own review or an explicit statement that the earlier review was reopened. Assigning maintenance responsibility does not stipulate that maintenance work occurred, just as a model description does not record an edit.
This yields the practical test for corpus-to-model connection: ask the concrete question, identify the single record that answers it, and then ask what each remaining layer adds. The collection note, for instance, records how questions entered the corpus during the specified consultation period and contains no demographic attributes; topic diversity in the collected material does not fill those missing fields, so no downstream model condition can be read as if it did.
The limit is that cross-reference keeps these answers adjacent without merging them. A chain of pointers across collection, transformation, model and review documents is useful for navigation, but it cannot supply an answer that none of the documents contains. When the edit log already answers the attribution question concisely, adding layers without distinct purposes creates ceremony, not coverage.