Chapter 1. What Is Fixed Before the Result

The thirteen Dionyas inquiries on the future of music agree on one rule above all: whatever is going to be measured must be fixed before the result is seen, a window of weeks, a reference date, a list of goals, the rival explanation, the event that would count as failure. What they do not fix is who would look at the result and say that a hypothesis has failed. This chapter shows how, and adds to the two debates of the Introduction: on prediction and postdiction, which has rarely studied preregistration as the shape of a published question, and on forecasting cultural success, which has asked how to predict better more often than how to make a prediction checkable.

1.1Data and method

The data are Dionyas's public list of inquiries, fetched by plain requests and kept as snapshots in the study's folder. The list holds 1,539 inquiries across 32 domains. They are of five kinds: 759 cases, 500 historical questions, 256 general designs, which the list calls foundations, 18 chart questions and 6 signal questions. Each sits in one of three lanes: 1,276 in the evidence lane, 250 in a qualitative lane and 13 in a symbolic lane. The list describes its own type as editorial question design and gives its count of generated judgments as 0. This study reads six parts of each record: title, question, scope, needs, boundary and state.

The selection rule is one sentence: every inquiry in the symbolic lane. The rule keeps 13, and all 13 belong to the domain the list calls hypotheses and future paths, which holds no other inquiry. The thirteen fall into three groups. Eight are general designs. They name no artist, work or market, and their scope says that these will be named when the inquiry is put to use. Four belong to a series of designed industry cases that runs across the whole list, 148 inquiries in all. Two of the four carry a title that names "The Elders–Palantir link"; the records do not say what these names refer to, and this study does not interpret them. Their identifiers mark them as part of a set on symbolic scenarios that has 4 members in the list; the rule leaves out the other 2, which sit in other lanes, and they were left out by the rule, not by reading them first. The last of the thirteen is an editorial case that names BTS and a tour, and its scope says that no actual figures have been entered. The appendix gives every record in full.

The records are in Korean; every quoted sentence below is from one English translation made for this study, sentence by sentence, with nothing added. Dionyas does not say who drafted the wording of these thirteen; it calls the list editorial question design. Across the whole list, 19 records, each marked as a synthetic question and a simulation, say that their source card was a model's draft that RISCON checked and registered; none of the thirteen is among them, and none says anything of the kind. Five of the thirteen name Skeptron, a name the records do not explain; across the whole list only 7 records do. What the records say about BTS, about RESCENE, about a tour or a date is how each inquiry frames its subject. This study has not checked any of it and draws no fact about any artist, tour or figure from the records.

Two properties are not findings. The first is the studio's form: every inquiry has a scope, needs and a boundary, and all thirteen questions end in a question mark. The second is the studio's frame, wording copied from record to record. The series this paper belongs to measures frame by five-word runs, counting a run as frame when it recurs across a site's records. By this study's count the questions of the whole list contain no frame at all, 0% of their runs, while the boundaries are 41% frame, the needs 30% and the scopes 46%. Inside the thirteen, counting the runs that recur in at least 3 of them, the boundaries are 84% frame and the needs 83%.

The domain table explains why. Every one of the 256 general designs in the list takes its boundary and first need word for word from its domain's caution and evidence lines. In the thirteen, all eight general designs and all four industry cases carry the domain's caution as their boundary, the industry cases followed by two sentences shared by the whole series, and all twelve carry the domain's evidence line as their first need. Only one record, the editorial case, has a boundary and needs of its own. This study therefore quotes the domain's lines to show the frame and does not read them as evidence about any one inquiry. It reads as evidence the questions, which are each record's own, the titles and scopes of the cases, and the boundary and needs of the editorial case.

Every number in this paper comes from a script that reads only the snapshots, by simple word tests on the Korean original. The counts describe the records and nothing else.

1.2The window before the wave

The first finding is that the thirteen inquiries are about the timing of their own questions. Six of the thirteen questions speak of something done before or first, against 52 of the 1,539 questions in the whole list, about 3%. Three speak of fixing something, against 9 in the whole list. A general design states the concern in its plainest form.

The question sets two procedures side by side and asks what separates them. In one, the start, the peak and the end of a period of interest are written down first, and only then is the period read as a wave. In the other, the window is chosen after the result is known. It does not call the second wrong; it asks what the difference is, the question Nosek, Ebersole, DeHaven and Mellor, The Preregistration Revolution (2018), answer for research: an analysis planned before the outcomes are seen can be told apart from one shaped by them, and credibility depends on telling them apart. This study reads the Dionyas question as that distinction applied to a single chart of fan interest, put as an open question rather than as a rule.

An industry case takes the same distinction to the choice of a single date.

The first sentence admits that the choice of a reference date is not neutral: two defensible dates may give different stories about what fans did next. The second draws the consequence. If both dates are defensible, the danger is not choosing the wrong one but choosing after looking. Simmons, Nelson and Simonsohn, False-Positive Psychology (2011), show that flexibility in collecting, analysing and reporting data dramatically increases false findings. A reference date is one such flexibility. The Dionyas case asks the question that leads to their remedy of disclosure: what has to be on the record before the data arrive.

The editorial case is the only one of the thirteen that commits to a date, and it does so in the form of a premise:

The tour, its leg and its date are the record's framing, and this study treats them only as the record's. What matters for the design is the grammar of the question. The reference point is fixed and named T0. The window is fixed at 12 weeks. The measures are named. The scenario is described as set in advance, and it is set against a rival that needs no symbol at all, a simple trend. The question asks which of two descriptions the flow will be closer to, and the needs of the record ask for what that comparison would require:

The second need is, in this study's reading, the nearest thing in the thirteen records to a full plan registered in advance: an assumption, a comparison, a date on which the inquiry would be reviewed and the conditions under which the scenario would be declared void. The scope of the same record adds that no actual figures have been entered. The plan is published with no figures entered, as an empty form; this study reads that as the order Nosek and his co-authors recommend, a plan defined before the outcomes are observed.

The boundary of the editorial case, the only boundary among the thirteen written for its own record, names the two practices the plan is meant to block:

Kerr, HARKing (1998), defines his subject as presenting a hypothesis formed after the results as if it had been formed before them. The Dionyas boundary does not forbid either of the two practices it names. It requires that a result reached by them be recorded separately. Kerr's own position is that the costs and benefits of the practice are a complex question for research and debate. This study reads the boundary as taking a similar line in editorial form: an analysis made after the fact is not banned but labelled, so that it cannot pass for a prediction. One of the industry cases states the same aim as a question:

The verb is about display: the concern is the label a reading carries, not the reading itself.

1.3A symbol under a rule

The second finding concerns the ratios. Five records in the whole list of 1,539 mention a wave, Elliott, Fibonacci, a retracement or the ratio 0.382, and all five are among the thirteen. The symbolic lane is, in that sense, the place where the list keeps its symbol. Every record that holds it also holds a restriction on it. The plainest is a general design whose title says what it wants:

The question takes the use of the ratios as given; it calls the use exploratory, and what it asks is how to write down three things the ratios are not. They are not a rate at which fans are lost, not a probability, not a verified rule. Each denial closes off a reading that a line drawn at a fixed fraction of a rise invites. Thibodeau and Boroditsky (2011), in the paper on metaphor in reasoning cited in the Introduction, find that a metaphor can steer how people reason while they point to numerical information as their reason. A ratio laid over a chart of fans is a metaphor that arrives already dressed as a number. This study reads the Dionyas question as addressing exactly that risk: it asks for the limit to be recorded because the numbers make the metaphor easy to forget.

The domain's caution line, which every general design and industry case in the thirteen carries as its boundary, says the same thing in general terms, and is quoted here once as the studio's frame:

It is frame, not a finding about any one inquiry, and it is read here only for what it shows of the studio's design. The line limits claims about the future, and it puts scenarios beside symbols and metaphors, as things an inquiry may use and may not promote.

The industry case on the ratios puts the restriction into the form of a comparison:

Its title states the point of the comparison: the ratios are to be "set against an explanation without ratios". The question asks for an identical window for the ratio scenario and for two plain rivals, persistence and trend. The second sentence names the danger the comparison guards against. Sullivan, Timmermann and White, Data-Snooping, Technical Trading Rule Performance, and the Bootstrap (1999), evaluate trading rules while adjusting for the bias of judging a rule chosen from a larger universe of rules. Choosing only the cases that fit is a small version of the same bias, applied to cases rather than rules. The Dionyas question does not propose a statistical correction; this study reads it as asking the prior question their method presupposes, whether the comparison is made on the same footing for every candidate explanation.

Lo, Mamaysky and Wang, Foundations of Technical Analysis (2000), locate the main obstacle to charting in the eye of the beholder and replace the eye with an automatic procedure. The Dionyas inquiries do not automate the reading of the wave; they fix its window, name its rivals and require it to be labelled exploratory. Lo and his co-authors find that some indicators may have practical value; the Dionyas records make no claim either way, and their own boundary forbids one. The thirteen records are a design for testing a method, not a test of it.

1.4What would count as wrong, and for whom

The third finding concerns what a decision about the future needs beyond a well-timed question: an account of what would show the hypothesis wrong, and someone to act on it. The thirteen inquiries are generous with the first and silent on the second.

Four of the thirteen questions name revision, refutation, a change of judgment or failure. The general designs set this up from the start of a career, a release and an expansion abroad.

Each divides a single idea of success into several and asks for the evidence that would bear on each before the event. The first asks for events that would refute as well as support; the second asks what would revise the hypothesis, not what would confirm it; the third puts the signs of being wrong ahead of the most visible sign of being right, a rise in the rankings. This study reads the division into several futures as scenario-like, in its own sense of the word and not as a claim about any school. Bradfield, Wright and others, The Origins and Evolution of Scenario Techniques in Long Range Business Planning (2005), describe the scenario literature as a methodological chaos of definitions and methods and classify it into three main schools; the Dionyas designs do not say which school, if any, they follow.

Salganik, Dodds and Watts, Experimental Study of Inequality and Unpredictability in an Artificial Cultural Market (2006), give a reason for asking these questions this way. In their artificial market, social influence made success both more unequal and less predictable, and quality only partly determined it. This study reads that as one reason a record might break success into goals, each with its own sign of failure; the paper does not claim that the experiment holds outside its artificial market. The Dionyas general design on a future success that came true for the wrong reason makes this explicit:

Salganik and Watts, Leading the Herd Astray (2008), found that for most songs a falsely inverted popularity became real over time, results they call partial and speculative; this study reads them as one way in which an outcome can follow a forecast for a reason other than the one given. The Dionyas question asks for the two to be scored apart, the hit and the explanation, and its title says why: "A prediction can come true and its process still be wrong."

Mellers, Ungar and others, Psychological Strategies for Winning a Geopolitical Forecasting Tournament (2014), report that training, teaming and tracking improved the calibration and resolution of forecasters in a tournament. This study reads all three as needing a forecaster, a set of forecasts made over time and a way of scoring them. The thirteen Dionyas inquiries supply the questions a tournament would need, with windows, rivals and conditions of failure, and none of the rest. No record names a forecaster. No record names who would score a hypothesis or declare it void. The editorial case asks for a date of review, and no record gives one. The states of the records are consistent with this: 8 of the thirteen have their question designed and await inquiry, 4 have their question registered and await judgment, and the editorial case has its sources under review with interpretation and hypothesis kept apart. Across the whole list, 715 of the 1,539 records, 46%, say that they await judgment.

The general design on colliding futures shows that the missing decider is not only a gap in the procedure but a choice about whose success counts.

The question gives three parties each its own measure of success and asks how the difference can be preserved, not resolved. A forecasting tournament scores every forecaster against one outcome; this question declines to say which outcome counts. In this study's reading, the decision about who decides is not overlooked here but kept open, in favour of a record in which the three readings stand side by side.

The last two records take the same care to the border between the studio and another. The records speak of Skeptron's creative context and its intuitive inquiry, and the two records that face it ask what must survive the crossing.

The first asks, in this study's reading, whether three things can be kept distinct as a market observation becomes a creative question: what was observed, how it was read and what it was taken to symbolise. The second names the decision it fears, a market response leading straight to the approval of a production, and asks for a boundary against it. Here at last a decision appears in the thirteen records, the approval of a production, and it appears as the thing an inquiry must not turn into. The design that elsewhere leaves the decider unnamed here names a decision only in order to keep the inquiry from making it.

The three findings fit together: the inquiries fix their windows, dates and rivals before the result, admit the ratios only as an exploratory scenario recorded with its limits, and spell out what would count as being wrong while naming no one who would say so. For the debate on prediction and postdiction they add a case in which preregistration is the shape of a published question rather than a step in a study. For the work on forecasting cultural success they show a design that makes a forecast checkable and leaves the checking, the scoring and the decision to whoever takes the inquiry up.