Introduction

The question

A drug candidate reaches people only after a long chain of evidence about what it does and what harm it may cause. For most of the last century, an early link in that chain was a test on animals. Computers now offer to take over some of those links: a model can screen a library of molecules, predict how toxic a compound will be, or simulate how an antibody spreads through the human body. Such work is called in silico, done in silicon, as a test in a dish is done in vitro and a test in a living animal in vivo. The claim made for it is easy to state and hard to judge: that a computed result can tell a regulator what an animal study or a clinical trial would otherwise have told it.

This paper does not ask whether that claim is true. It asks the question Meridian asks of any decision: who must decide, by when, and what would make them decide again? Put in full, the question is this. When should a regulator accept in-silico evidence in place of animal or clinical evidence, who decides, by when, and what would make the decider revisit the decision? The question has a subject, an object, a time and a condition, and a decision about evidence can fail on any of them. A model may be accepted by someone who had no authority to accept it, for a use it was never tested in, with no date by which anyone must look at it again.

The paper reads eleven public texts on this question; four further works enter only through their abstracts. They are United States texts, because the United States has written parts of the question into statute in recent years and its documents are in the public domain; and they are scholarly articles that their authors released under an open licence. The paper reads them as a set of answers, each partial, to the same question. It reports where each text places the decision and where each leaves it open. It does not take a side on any drug, method or policy, and nothing in it is medical, clinical, regulatory or investment advice.

The debate

Four lines of argument meet in the texts. They disagree less about the facts than about where the difficulty lies.

The first holds that a computational model can be evidence if its credibility is shown in proportion to what rests on it. Marco Viceconti and others, In silico trials: Verification, validation and uncertainty quantification of predictive models used in the regulatory evaluation of biomedical products (2021), set out this view through a technical standard first written for medical devices. Their abstract opens with a statement of fact about regulators:

Their method ties the trust a model must earn to the weight it carries in a decision and the harm a wrong decision would do. On this view the problem is not whether models may be used but how much checking each use requires. Rajanikanth Madabushi and others, Review: Role of Model-Informed Drug Development Approaches in the Lifecycle of Drug Development and Regulatory Decision-Making (2022), a review by FDA authors, supports the same picture from the agency's side: by their abstract, model-informed drug development has a long history of use at FDA across the drug lifecycle and informs dose selection and review questions. This study reads the line as holding that in-silico evidence is not a newcomer asking to be let in but an old resident whose rooms are being enlarged.

The second line holds that the difficulty lies in the data from which drug discovery models learn. Andreas Bender and Isidro Cortes-Ciriano, Artificial intelligence in drug discovery: what is realistic, what are illusions? Part 2: a discussion of chemical and biological data (2021), argue that chemistry and biology do not give a learning method what images and speech give it: plenty of examples, an agreed way to describe them, and labels that mean one thing. Their sharpest sentence is about labels:

A compound is toxic or safe at a dose, in a species, in an assay; the same molecule carries different labels in different settings. If the labels depend on context, a model trained on them inherits that dependence, and a regulator who accepts its output accepts the context with it. Jessica Vamathevan and others, Applications of machine learning in drug discovery and development (2019), make a related point in their abstract: machine learning helps with well-specified questions backed by abundant high-quality data, and the main obstacle is the lack of interpretability and repeatability of results. Rizwan Qureshi and others, AI in drug discovery and its clinical relevance (2023), extend the doubt from data to problems:

For this line the question of acceptance is premature for much of the field. A model cannot be credible for a use whose answer no one can check.

The third line turns the comparison around. It asks not whether a model is as good as an animal, but whether the animal was ever as good as it was taken to be. Peter-James H. Zushin and others, FDA Modernization Act 2.0: transitioning beyond animal models with human cells, organoids, and AI/ML-based approaches (2023), argue that differences between species, and the inbred uniformity of laboratory rodents, help explain why promising findings so often fail in people. They describe the requirement the new law addressed in these words:

On this view the burden of proof falls on the old method as much as on the new one. Jack W. Scannell and others, Diagnosing the decline in pharmaceutical R&D efficiency (2012), give the background against which such arguments are heard: by their abstract, the number of new drugs approved per billion dollars of research spending has fallen steadily for decades, and among the causes they name is a 'cautious regulator' problem. This study reads that phrase, on its own account and beyond what the abstract says, as a reminder that a regulator's caution is itself one of the costs to be counted.

The fourth line declines to choose. Petra Schneider and others, Rethinking drug design in the artificial intelligence era (2019), an expert panel, say in their abstract that the reality lies between enthusiasm and scepticism. That is a fair description of the texts as a whole, but it does not say who decides where on that line a given model sits.

The gap and the contribution

The four lines argue about what is true of models, data and animals. None of them asks, in the same terms, who holds the decision, what time limit binds it, and what is written in advance as the reason to reopen it. Those are the questions of a decision record rather than of a scientific review, and they can be put to a statute and a press announcement as well as to an article.

The paper makes three narrow contributions. First, it shows that the statutes answer the question of who decides precisely and the question of whether to accept not at all: the law widens what may count as a nonclinical test, keeps animal tests on the same list, has the Cures Act declare that the standards of evidence are not altered, and gives the decision to the agency for one context of use at a time. Second, it shows that the deadlines in the texts fall on the agency's procedures, its guidance, framework, taxonomy and reports, and that no text sets a date by which any model must be accepted or refused. Third, it collects the conditions for revisiting a decision that the texts write down, and finds three kinds: new information that calls a qualification's basis into question, the model risk of a given use and the range of inputs a model was tested on (the engineer's), and the gap between past and future performance on proxy data (the data scientist's). Each is a condition someone could observe; none says who must look, or how often.

The rest of the paper is short. Chapter 1 says how the texts were chosen and read, and then takes the three questions in turn. An appendix gives longer passages of every text, word for word. The conclusion returns to the debate, states the contributions again, and names the limits of reading eleven texts from one country.