The question
A protein is a chain of amino acids that folds into a three-dimensional shape, and the shape goes a long way towards explaining what the protein does. Until recently the shape could be known only by measuring it: by crystallising the protein and scattering X-rays off the crystal, by freezing it and imaging it under an electron microscope (cryo-electron microscopy, or cryo-EM), or by reading its atoms' signals in a magnet (nuclear magnetic resonance, or NMR). The results are deposited in the Protein Data Bank, the PDB, a single public archive. The AlphaFold paper of 2021 states the cost of that route plainly:
Structural coverage is bottlenecked by the months to years of painstaking effort required to determine a single protein structure.
The same paper then reports that a computer can do much of this from the sequence alone, and in the blind assessment the field uses to test such claims:
We validated an entirely redesigned version of our neural network-based model, AlphaFold, in the challenging 14th Critical Assessment of protein Structure Prediction (CASP14)15, demonstrating accuracy competitive with experimental structures in a majority of cases and greatly outperforming other methods.
Two words in that sentence carry the whole problem this paper takes up: "competitive" and "majority". If a prediction is competitive with an experiment in a majority of cases, then in some cases it is not, and someone has to tell which. Structural biology done without an experiment, which laboratories sometimes call dry-lab work, turns on that act of telling. The independent assessment that this paper reads most closely puts the question in a single line:
A question that immediately arises is to what extent these predictions can substitute for experimental structure determinations11,12.
This paper asks the question in the form Meridian gives to every decision. When may a predicted structure stand in for an experimental one? Who decides: a journal that accepts a paper built on a prediction, the database that publishes the prediction, the laboratory that uses it, or the funder that pays for an experiment or declines to? By when must the decision be made, and what would make the decider look at it again? The paper does not answer the question for any protein or project. It asks how eight public texts allocate the decision: to whom they give it, what clock they put on it, and what they say would reopen it.
What the texts already say about substitution
The texts do not agree on a single answer, and the disagreement is the starting point. Read side by side, they give two framings of the same predictions.
The first framing measures a prediction against experiment and reports how close it comes. The assessments of the Critical Assessment of protein Structure Prediction, or CASP, a blind contest held every two years in which predictors model structures that have been solved but not yet released, take this line. Andriy Kryshtafovych and others, Critical assessment of methods of protein structure prediction (CASP)—Round XIV (2021), summarise the round in which AlphaFold took part; their abstract states that one group's deep-learning structures rivalled experimental ones for single proteins, so that the classical problem of protein folding is in this sense solved. Minkyung Baek and others, Accurate prediction of protein structures and interactions using a three-track neural network (2021), describe a second network, RoseTTAFold, that approaches AlphaFold's accuracy in the same round and is released to the community, and their abstract says it helps solve hard crystallography and cryo-EM problems. A community assessment of the predictions across many tasks of structural biology reached a guarded version of the same verdict:
AF2-based predictions of protein disorder and complexes surpass dedicated tools, and AF2 models can be used across diverse applications equally well compared with experimentally determined structures, when the confidence metrics are critically considered.
The second framing asks what a prediction is for and finds that closeness is not the same as fitness for use. Kiersten M. Ruff and Rohit V. Pappu, AlphaFold and Implications for Intrinsically Disordered Proteins (2021), note in their abstract that many regions AlphaFold predicts with low confidence overlap regions of proteins that have no fixed shape, and warn against reading structures or interactions into them. The dispute is sharpest over docking, the computation that places a small molecule such as a drug candidate into a pocket of a protein. A study of receptors that are frequent drug targets reports the gap between accuracy and use:
This has left the large community of academic and industrial researchers who wish to use protein structures to design drugs and other ligands unsure of whether, when, and how to use the new structural models.
On the other side of that dispute, Jiankun Lyu and others, AlphaFold2 structures guide prospective ligand discovery (2024), docked large libraries of molecules against unrefined predicted models of two receptors and report, in their abstract, hit rates and affinities similar to docking against experimental structures, with a cryo-EM structure that resembled the prediction. The CASP round after AlphaFold's, summarised by Andriy Kryshtafovych and others, Critical assessment of methods of protein structure prediction (CASP)—Round XV (2023), adds a qualification on the side of the method: its abstract says AlphaFold2 still leads for single chains but that its default settings give the best result for only about two thirds of targets, while the main advance of that round lies in complexes.
This paper reads the two framings as answers to different questions rather than as a contest one of them must win. The first asks whether the method works; the second asks whether a given prediction may be used for a given task. The first can be settled by a blind test. The second is the decision this paper studies, and no blind test settles it once for every use.
The contribution
The paper makes three claims, each drawn from what the texts say rather than from any judgment of its own on the science.
First, the decision whether a prediction may stand in for an experiment is not made once, by an institution, for the method. The texts make it many times, by the user, for a residue and a purpose. The developers supply instruments for that decision, chiefly a per-residue confidence score and a predicted error between parts of the structure; the database's stewards publish the instruments with guidance and say, in so many words, that the interpretation is the user's. None of the eight texts gives the decision to a journal or a funder.
Second, the texts put dates on the evidence and not on the decision. A predictor is trained on structures released before a cut-off and tested on structures released after it; a database is released in versions and corrected; an assessment covers a window of months. What the texts never set is a date by which anyone must decide. The cost of waiting appears instead as the time an experiment takes.
Third, the texts themselves write the conditions under which a use of a prediction should be looked at again. They converge on four: new experimental data that disagree with the prediction; a change in what the coordinates are used for; a partner, such as a bound molecule, or a state, such as a second fold, that the prediction left out; and a new version of the predictor or of the database.
The rest of the paper sets out the evidence. Chapter 1 states how the texts were chosen and read, then takes the three parts of the question in turn: who decides, by when, and what reopens a decision. An appendix quotes the eight texts at length, so that a reader can check each reading against the words it rests on. The conclusion discusses what the allocation leaves open and states the paper's limits.