Once a song has climbed, a tour has ended or a comeback has failed, an explanation can be found for almost any outcome, and it can be made to look as if it had been there all along. The difficulty is not a lack of theories about success in music. It is that the moment of looking at the result is also the moment when the question is most easily rewritten to fit it. This study reads a set of public records that try to write the question down first.
The records come from Dionyas, the music studio of the RISCON family. Dionyas does not publish judgments. Its list of 1,539 inquiries describes itself as editorial question design and states that it has generated 0 judgments. Each inquiry is a question with a stated scope, the material it would need, a boundary that says what it may not claim, and a state such as "awaiting judgment". The list places 13 inquiries in a lane of their own, which it calls symbolic, all in the domain the studio calls hypotheses and future paths. They ask about what might happen next to an artist, a video or a tour, and several of them carry a pair of numbers, 0.382 and 0.618, which the records themselves tie to Fibonacci, to retracement and to an Elliott-style path, that is, to the reading of waves in a chart. How a studio that publishes no answers designs such questions is the subject here.
The research question and the contribution
The research question is this: how does an inquiry about the future of an artist or a release frame a forecast before any judgment is made, that is, what does it fix in advance, what would it count as being wrong, what does it allow a symbol to claim, and who would decide?
The study reads the thirteen inquiries chosen by one rule, set out in the method section, and it makes two contributions. The first is to the debate on prediction and postdiction in research practice. Its remedy, writing down hypotheses and analyses before the data are seen, is usually studied in laboratories and journals; here it appears as the shape of a published question about music, and its choices can be read line by line. The second contribution is to the work on how cultural success is forecast. The work on cultural markets has shown how hard success is to predict; a separate work, on a forecasting tournament, asks how forecasters can do better. The Dionyas inquiries do not try to forecast better; they try to make a forecast checkable, and so show what checking requires and what it leaves to someone else.
Prediction, postdiction and the time of a hypothesis
The first body of work concerns the order in which a hypothesis and its evidence meet. Brian A. Nosek, Charles R. Ebersole, Alexander C. DeHaven and David T. Mellor, in The Preregistration Revolution (2018), distinguish generating hypotheses from existing observations, postdiction, from testing them with new ones, prediction, and argue that the distinction is appreciated in concept but not respected in practice. Mistaking a postdiction for a tested prediction lowers the credibility of a finding, and ordinary biases such as hindsight make the mistake hard to avoid. Their remedy is preregistration: the research questions and the plan of analysis are defined before the outcomes are observed, so that what came from a prediction can be told from what came from a postdiction.
Norbert L. Kerr, in HARKing: Hypothesizing After the Results are Known (1998), names the practice the remedy is aimed at. He defines it as presenting a hypothesis that was based on or informed by one's results as if it had been an a priori hypothesis, and reports survey data suggesting that at least some forms of it are widely practised and widely seen as inappropriate. He concedes that whether its costs exceed its benefits is a complex question for research and debate. Joseph P. Simmons, Leif D. Nelson and Uri Simonsohn, in False-Positive Psychology (2011), show that flexibility in how data are collected, analysed and reported dramatically increases false findings, and propose a disclosure-based solution: requirements for authors and guidelines for reviewers.
A similar worry appears in the study of charts. Ryan Sullivan, Allan Timmermann and Halbert White, in Data-Snooping, Technical Trading Rule Performance, and the Bootstrap (1999), evaluate simple technical trading rules while quantifying the data-snooping bias, the bias of judging a rule chosen from a larger universe of rules, and adjust for it in the context of the full universe from which the rules were drawn. Andrew W. Lo, Harry Mamaysky and Jiang Wang, in Foundations of Technical Analysis (2000), note that charting has long been part of financial practice but that one main obstacle to its acceptance is its subjectivity: the shapes seen in a price chart are often in the eye of the beholder. They make the recognition of patterns systematic and automatic, and find that several indicators do provide incremental information and may have some practical value. This study reads the two papers as marking two ways of disciplining a chart: correcting for the choice among rules, or replacing the eye with a procedure.
Forecasting cultural success
The second body of work concerns the thing the inquiries ask about. Matthew J. Salganik, Peter Sheridan Dodds and Duncan J. Watts, in Experimental Study of Inequality and Unpredictability in an Artificial Cultural Market (2006), start from a paradox: hit songs, books and films are many times more successful than average, which suggests that the best are qualitatively different from the rest, yet experts routinely fail to predict which will succeed. In an artificial music market, stronger social influence increased both the inequality and the unpredictability of success, and quality only partly determined it: the best songs rarely did poorly and the worst rarely did well, but any other result was possible. Matthew J. Salganik and Duncan J. Watts, in Leading the Herd Astray (2008), inverted the apparent popularity of songs in such a market and found that most songs experienced self-fulfilling prophecies, in which perceived but initially false popularity became real over time, though not for the market as a whole, partly because the very best songs recovered in the long run. They call these results partial and speculative.
One response is to forecast better. Barbara Mellers, Lyle Ungar and others, in Psychological Strategies for Winning a Geopolitical Forecasting Tournament (2014), report three drivers of accuracy in a forecasting tournament: training in probability, which corrected biases and encouraged the use of reference classes; teaming, which let forecasters share information and discuss the reasons for their beliefs; and tracking, which placed the best performers together. They hold that forecasting, often treated as a statistical problem, can be improved by behavioural interventions. Another body of work concerns scenarios. Ron Bradfield, George Wright and others, in The Origins and Evolution of Scenario Techniques in Long Range Business Planning (2005), describe the scenario literature as a methodological chaos of different and at times contradictory definitions and methods, and classify its techniques into three main schools.
A last work gives this study its terms for the symbol itself. Paul H. Thibodeau and Lera Boroditsky, in Metaphors We Think With (2011), find in five experiments that even a metaphor carried by a single word can powerfully influence how people try to solve a social problem and how they gather information about it. The influence is covert: people do not recognise the metaphor as influential and point instead to more substantive, often numerical, information as their reason. This study reads a wave drawn over a chart of fan activity as a metaphor of this kind, which the Dionyas inquiries attach numbers to; the paper says nothing about whether the metaphor is apt.
What follows
The first chapter gives the data and method and reads the inquiries in three sections: on fixing before the result, on the ratios held under a rule, and on what would count as being wrong, and for whom. An appendix gives the thirteen records; the conclusion returns to the two debates.
The contribution, stated at the start so that it can be checked at the end, is this. The Dionyas inquiries on the future of music turn the discipline of preregistration into the shape of a published question: they ask for the window, the criterion and the rival explanation before the result, and they let a symbol in only as a hypothesis under a rule. In doing so they make a forecast checkable without making it anyone's to check, which is both their discipline and their limit.