Chapter 1. Questions With Answers

The fifteen Dionyas questions on three treaties are designed so that each answer has an address. A question names the document in which its answer is to be read, or its list of needs names that document for it; the instrument is cut into five small facts, each of which a single line of a single document could hold; and a closing line says what the answer is not a judgment on, whether the law of today or the lawfulness of AI training or distribution. The chapter shows how, and adds to the two debates of the Introduction: on whether reference answers are found or made, and on what citing a document does.

1.1Data and method

The data are Dionyas's public list of inquiries, fetched with plain GET requests and kept as snapshots. The list holds 1,539 inquiries across 32 domains: 759 cases, 500 history questions, 256 general review designs, 18 chart questions and 6 signal questions. It describes its own type as editorial question design and gives its count of generated judgments as 0. Its summary of the history questions counts 100 events, 500 questions, 500 reference answers and 154 sources, and states that 0 automatic connector runs have been made and 0 judgments generated. Each event has exactly five questions. The reference answers themselves are not in the list, and Dionyas does not say who drafted these records.

A history record has the fields this study reads: a title in two parts, the event's name and then the question's own heading, separated by a dot; a question; a scope that gives a date, says what the date is, and names a place; a list of the documents it needs; a boundary that says what the answer may not be taken to mean; and a state.

The selection rule is one sentence: every history question in the list whose scope gives its place as international. Of the 500 history questions, 485 give one or more countries or regions as their place instead; the rule yields 15 records on 3 events, five on each. All fifteen lie in the rights domain, which holds 45 history questions on 9 events, and all fifteen lie in the evidence lane, like every history question. The rule was chosen because it isolates the only events in the history list that the list itself places above any one country; it was not chosen for what the questions ask. None of the fifteen titles or questions mentions music, a song or an artist, and none of the questions names a country.

The records are in Korean. Every quoted sentence below comes from one English translation made for this study, sentence by sentence, with nothing added; the names of the documents in the lists of needs are in Latin script in the original and stand as they are. The appendix gives the records read, word for word in that translation. Where a record says what a body publishes or what a treaty is called, that is the record's wording, which this study has not checked.

Two kinds of property are set aside before reading. The studio's form, a scope, a list of needs, a boundary and a state in every record, is not a finding. Nor is frame, wording copied from record to record. Frame is measured here by five-word runs on the Korean original, counting a run as frame when it recurs in at least 5% of the records that fill a field and in at least 3. Across the whole list the questions hold no frame, while the boundaries are 41% frame, the scopes 46% and the needs 30%; across the history questions alone the boundaries are 23% frame and the needs 2%. The state is the same short line in all 500 history questions, and in all fifteen:

Within the selection the copying is close to complete. Measured inside the fifteen, with a run counted as frame when it stands in at least 3 of them, the scopes and the boundaries are 100% frame, the needs 80% and the titles 56%, while the questions are 0%. There are 3 distinct scopes, 3 distinct boundaries and 1 state among fifteen records: each event's five questions share one scope and one boundary word for word, and the same holds nowhere else in the list for these three boundaries. The titles share their first part, the event's name, five times over, and differ in their second part: 3 distinct heads, 15 distinct subheads. The lists of needs fall between: 7 distinct lists, naming 6 distinct documents, and only 1 of the fifteen has a list that no other record in the selection shares.

So this study treats as frame the event's name in the title, the scope, the boundary, the state and the lists of needs taken one at a time. It quotes each frame line at most once in the body and draws no finding from what a frame line says on its own. What is each record's own is its question and the second part of its title: 15 distinct questions and 15 distinct subheads. These are the evidence. The lists of needs enter the evidence only in relation to a record's own question: whether the document a question names is the one its list names.

Every number in this paper comes from a script that reads only the snapshots, using simple word tests on the Korean original. The counts describe the records and nothing else; they say nothing about the treaties.

1.2Where the answer stands

The first finding is that these questions place their answers in named documents. Nine of the fifteen, 60%, name in the question itself the document or the part of a document that the answer is to come from, and 7 of the fifteen, 47%, ask what that document says, records, describes, lists or gives. In every one of the nine, the document the question names is also named in its list of needs, and in 8 of the nine it is the only one named there. The plainest instance hands the whole question to a treaty record.

The question does not ask when the convention entered into force. It asks what date a particular record gives. The difference is small in grammar and large in design: the answer it wants is a reading of a document, and a reader who knew the date from elsewhere, or who found two documents that disagreed, would still be sent back to the one named. "According to" is rare in Dionyas's history list. Only 5 of its 500 history questions, 1%, say it, and 3 of those five are among these fifteen.

The next question of the same event narrows the address further. It names the record and asks under which article that record says the convention entered into force.

Here the object of the question is a citation inside a document: the article a list gives as the ground for an event. The answer, whatever it is, is a pointer to yet another place in yet another text. Read through Thorne and colleagues, FEVER (2018), whose annotators recorded the sentences that form the evidence for a verdict, this question goes one step past their design: the evidence is itself the answer, and the question asks for the citation rather than for the claim it supports. That reading is this study's own.

In 11 of the fifteen, the first document the list of needs names is WIPO's, and the forms of those documents differ. The Geneva questions are the clearest case. Two of them ask about the convention itself and name the convention in their needs:

Three of them ask about a summary of it, and their needs name the summary instead:

The questions keep the two apart in their own wording. One asks whom the summary describes as protected; another asks whether a word, as the summary uses it, reaches a kind of sound that the event's title already marks as a boundary:

Both questions are built so that an answer can be right about the summary and still be silent about the convention. The second goes further: it takes a single English word inside the Korean question, phonogram, and asks about its reach as one document uses it. It is the only question of the fifteen that can be answered yes or no, and it is the only one that sets a definition against a nearby kind of thing. Lin, Hilton and Evans, TruthfulQA (2022), craft questions so that some people would answer them falsely from a misconception. This study reads the soundtrack question as built on a similar edge, a place where a plausible answer and the document's answer may part; but the record says nothing of misconceptions, and the fit is partial.

The WPPT questions draw the same line between an instrument and its text. Three of them name the treaty in their needs; two name the treaty text, and those two are the ones that ask about articles.

The question keeps an English phrase, rights management information, inside the Korean sentence, as the soundtrack question keeps phonogram. Eleven of the fifteen questions carry a word in Latin script. On this study's reading, the borrowed words mark where the question hands over to the document's own vocabulary: the answer is to be read in the document's terms, not translated into the questioner's.

The Rome questions name a third kind of document, a database entry.

The answer to this question would be a label that one database gives one text, and nothing else. It is the purest case of an answer that exists only in a document, and the one Rome record whose list of needs no other record of the selection shares. Buckland, What is a "document"? (1997), asks what should count as a document once access to growing numbers of them is at stake. The fifteen answer that question in practice by naming different kinds of thing: a convention, a summary of a convention, a treaty text, a treaty record kept by another body, and a database entry. Each question picks one, and the choice decides what its answer can be about.

Rajpurkar and colleagues, SQuAD (2016), design questions whose answers are segments of a passage that the dataset carries. The Dionyas questions point to passages in the same way, but the passages are not in the list, and neither are the answers; the list carries the address and counts the answers. On this study's reading, that leaves a reader of the list in an unusual position: able to see exactly where each answer is said to stand, and unable to see the answer.

The questions that name no document still have one. Six of the fifteen name none in the question; their lists of needs supply it, and 1 of the six names two documents there. The first Rome question, which asks when the convention was adopted, lists both:

Tuchman, Objectivity as Strategic Ritual (1972), finds that the reporters studied claim objectivity by citing procedures, quoting others instead of offering their own opinions. This study reads the fifteen as doing something formally similar: every question either says whose words its answer will be, or carries a list that says it, so that the reference answer, when written, is a report of what a document says. The state line that all fifteen share, that the reference answer was written from the documents, makes the same claim once more for the whole set. Whether this works as a ritual in Tuchman's sense, a protection against the risks of a trade, the records do not show; they show only the form.

1.3Five small facts and a fence

The second finding is that the questions cut each instrument into five narrow facts and then fence the answers with a line on what they are not. The fifteen ask when (4 of 15), in which city (2), who or whom (4), under or in which article (2), for an identifier (1), whether (1), and how many and how long (1). None asks why or how. Across the whole history list, 25% of the questions ask when, 6% ask where or in which city and 12% ask who, so the fifteen are not unusual in leaning on such words; what sets them apart is how tightly each answer is tied to a document. Their questions are also short, 7.6 words on average in the Korean original, against 9.2 for all history questions and 16.4 for the general review designs.

Each event opens with a title whose first part is its frame. For the Geneva event it reads:

Before the dot stands a short essay in a phrase: an instrument, an activity and a boundary. After it stands the narrow fact the question wants. The three event names are written in the same way, each with a theme: three parties between signature and entry into force for Rome; information that is part of the right too for the WPPT. The second parts are where the fifteen differ, and they are almost all nouns for a single fact: the date of adoption, the city of adoption, the role of those protected, what the definition leaves out, the body that receives the deposits.

The questions on dates show how far the cutting goes. Rome's event is split between adoption, entry into force and the article under which it entered into force; the WPPT's last question splits entry into force again, into a number and a length of time.

This is the only question of the fifteen that asks for two things at once, and it already names the article in which both are to be read. Uscinski and Butler, The Epistemology of Fact Checking (2013), object to treating a statement that contains several facts as a single fact. The fifteen run the other way: they take an instrument that could be described in a sentence and ask for its facts one by one, and where one question holds two, it ties both to a single named article. This study reads that as a design that leaves little room for an answer to be partly right, which is the problem Uscinski and Butler raise; it does not show that the facts asked for are unambiguous, which is the presupposition they criticise.

The cutting has a consequence the records make visible: one record's frame can hold words that another record of the same event asks for. All 3 of the questions that ask when an instrument was adopted share a scope that opens with a date and calls it the date of adoption. The Rome scope reads:

This study does not say whether that date is the reference answer; the list does not carry the answers, and the paper checks none. It notes only that the design places, in the line every question of the event shares, a date labelled with the very words the first question asks about. The same happens with names. The Geneva questions ask in which city the 1971 convention was adopted, and the event's own name, in every one of its titles, calls the convention by a city's name (this study does not check the name):

A convention's name need not record where it was adopted, and this study does not say whether it does here. The WPPT's city question has no such neighbour; of the 2 questions on cities, 1 has the name Geneva in its event's title. The protected parties repeat the pattern. Both questions that ask who the protected parties or beneficiaries are have a sibling in the same event whose own question names performers and phonograms. The Rome case is the plainest:

The second question asks what the official title lists; the first question of the event gives the convention a long name in which three kinds of party appear. The WPPT pair is looser: one question asks who the two groups of beneficiaries are, while the first question names the treaty for performances and phonograms.

Graves, Anatomy of a Fact Check (2016), argues that in moments of institutional unsettlement verification relies on factual coherence rather than straightforward correspondence. On this study's reading, the five questions of an event form such a coherent set before any document is opened: the scope, the event's name and a sibling question already hold words that bear on a question's answer. A reader who checks one answer against the others of its event is doing coherence work; a reader who opens the named document is doing correspondence work. The design invites both, and the list, which carries neither the documents nor the answers, cannot show which a reference answer used. Jaton (2017) describes ground truths as designed during a problematization that shapes what they can later evaluate. The overlap within each event can be read as a trace of such designing: the five questions were cut from one account of the instrument, and pieces of that account remain in the shared lines.

Each event then closes with a fence. The three boundaries differ, and each is shared word for word by the five questions of its event. For Rome:

For Geneva:

For the WPPT:

Each boundary first says what the question does and then what it does not. The first sentence ties the answer to a record, to documents or to an original text of the time; the second turns away from law as it stands today or, in the WIPO treaty's line, from the lawfulness of AI training or distribution. All 15 boundaries say what the question is not or does not do; 10 of 15, 67%, name law or lawfulness, and 10 of 15 name the present. Across all 500 history questions only 7% of boundaries name law or lawfulness, and only 2% name AI, against 5 of these fifteen. Because each boundary is frame, this study draws no finding from what any one of them claims. What it can say is narrower: the three events that the list places as international are fenced far more often than the history list at large by lines that turn to law.

The Geneva fence speaks of "this one definition", and the question it fences most closely is the soundtrack question, the one that asks about a definition's reach. The WPPT fence names AI training and distribution, which none of its five questions mentions. Raji and colleagues, AI and the Everything in the Whole Wide World Benchmark (2021), explore the limits of benchmarks set up as general measures of progress. This study reads the fences as the opposite gesture, a set of answers that announces its own narrowness before anyone can take it for a general measure. A correct date of adoption or a correct article number is, by the record's own statement, not a ruling on law as it stands today. The fence does not say what the answer is for; it says only what it is not.

The fifteen leave a great deal unasked. No question asks what the instruments meant for musicians, why they were made, how they were received, or what happened after them. The cutting into narrow, documented facts is what makes each question answerable; it is also what keeps the history out. Kang, Ground truth tracings (2023), argues that what can be learned bends to the practices by which a task is formalized. Read that way, the fifteen formalize the history of three treaties as the history of their documents: dates, places, articles, identifiers and depositaries, each with an address and a fence.