This chapter answers the paper's question from the eight texts. Its first section says what the texts are and how they were read. The second shows that the texts hand the decision to the user and equip the user with confidence measures, while no text gives it to a journal or a funder. The third shows that the texts date their evidence rather than the decision. The fourth gathers what the texts say would reopen a use of a prediction.
1.1Data and method
The study read every text in its saved set on AlphaFold and dry-lab structural biology that is stored in full and published under a licence that allows quotation. The rule takes all eight such texts, so no text was chosen for what it says. Four come from the makers of the method and of its database: the AlphaFold paper of 2021 (Jumper and others), the AlphaFold 3 paper of 2024 (Abramson and others), and the two papers on the AlphaFold Protein Structure Database (Varadi and others, 2022 and 2024), which DeepMind built in partnership with the EMBL-European Bioinformatics Institute. Four come from outside: a comparison with crystallographic density maps (Terwilliger and others, 2023), a community assessment across applications (Akdel and others, 2022), a test of docking drug-like molecules to predicted receptors (Karelina and others, 2023), and a method that coaxes the predictor into showing more than one shape of a protein (Wayment-Steele and others, 2023). Each is published under the Creative Commons Attribution 4.0 licence and was read in full from PubMed Central; the study's manifest records the licence of each file and the evidence for it. The manifest also lists three web pages it holds only as bibliographic records, without their text; the paper quotes none of them.
These are published journal articles, not synthetic records. The paper does not judge whether their results are right. It reads what each says about four things: the confidence measures it offers or tests, the limits it states, how it validates predictions against experiment, and who it says should interpret a prediction. Every passage set as a block quote is copied word for word from one of the eight texts and carries the name of the saved file, so that a reader can find it. The appendix quotes the texts at greater length, grouped by text. Every number in the prose stands in one of the texts and is listed, with its place, in the study's numbers file. Nothing was coded or counted by a model. The works by other authors that the introduction discusses are drawn on only through short reading notes made from their abstracts.
Two measures recur, and they are defined here once. The predicted local distance difference test, pLDDT, is a score from 0 to 100 that AlphaFold attaches to each residue (each amino acid in the chain) as its estimate of how well that part of the structure is placed relative to its neighbours. The predicted aligned error, PAE, estimates for each pair of residues how far off one would be if the structure were aligned on the other; it speaks to whether two parts of a protein, such as two domains, sit correctly relative to each other.
1.2Who decides
The texts give the decision to the person who uses a prediction, and they give that person instruments rather than a verdict. The allocation runs in three steps: the developers make the instruments, the database's stewards publish them with guidance, and the user decides.
The developers' first claim for their method is about its reliability as a whole, and they rest it on a blind test run by others. The AlphaFold paper describes CASP as carried out on recently solved structures that have been neither deposited nor disclosed, so that it is a blind test for the methods taking part, and calls it the gold-standard assessment of accuracy in structure prediction. A blind test answers whether the method works on average. It cannot answer whether a particular prediction may be used, and the same paper does not pretend that it does. Its claim for the particular case is of another kind:
Finally, the model is able to provide precise, per-residue estimates of its reliability that should enable the confident use of these predictions.
The sentence is careful about its subject. The model provides the estimates; the estimates should enable a confident use; the use is someone else's. The developers do not say who that someone is or what threshold they should apply. The AlphaFold 3 paper keeps the same shape for the newer model, which predicts complexes of proteins with other molecules: it states that its confidence measures are well calibrated with accuracy and leaves their use to the reader.
The database's stewards turn the estimates into bands. Their first paper sets four:
Residues with pLDDT ≥ 90 have very high model confidence, while residues with 90 > pLDDT ≥ 70 are classified as confident. Residues with 70 > pLDDT ≥ 50 have low confidence, and residues with pLDDT < 50 correspond to very low confidence (13).
The bands describe confidence; they do not say what may be done at each level. On one point only does the first paper give an instruction, and it concerns the relative placement of domains:
If the PAE values are high, then the relative position and orientation of the two domains are unreliable, and users should not attach biological or structural relevance to these.
The second database paper, written when the archive had grown to over 214 million predictions, goes further in two directions at once. It attaches a use to each band, saying that regions above 90 suit applications that depend on high accuracy, such as characterising binding sites, and that regions between 50 and 70 should be used cautiously. And it says, more plainly than any other text in the set, whose decision the use is:
Regardless of the absolute pLDDT score, the PDB and mmCIF files provide coordinates for all regions, and it is incumbent upon the user to interpret the model prudently, in line with the provided guidance.
The sentence holds the whole allocation. The files give coordinates for every region, including those the stewards' own guidance marks as probably disordered; the database does not withhold what it does not trust. The duty to interpret falls on the user, and the guidance is a frame for that duty, not a substitute for it. The same paper then concedes that many users may not be ready for it:
While the availability of millions of predicted protein structures promises valuable insights into molecular biology, they might be behind a barrier for many researchers who may lack familiarity with handling macromolecular structure data and may not sufficiently understand the strengths and limitations inherent in predicted structures.
The stewards' answer is a training platform, not a gate. Read together, the two sentences describe a decision handed to people whom the hand-over itself expects to be partly unprepared.
The independent assessments accept the allocation and add a condition to it. The community assessment states the condition as a matter of workflow:
Importantly, in all cases, we find that it is critical to take into account the confidence metrics provided, and that these should be incorporated into the corresponding workflows.
The comparison with density maps states it as a matter of verification, and it names the one thing a user should not take on trust:
We further suggest that it is important to consider the confidence in prediction when interpreting AlphaFold predictions and to carry out experimental structure determination to verify structural details, particularly those that involve interactions not included in the prediction.
Here the decider is still the user, but the decision has changed shape. In the database's guidance, the user decides whether a region is good enough to use. In the comparison with density maps, the user decides which details to send back to the laboratory. The prediction stands in for the experiment only until the experiment can be run on the detail that matters.
Two kinds of decider the question named are missing. None of the eight texts states a rule of a journal on when a paper may rest on a predicted structure, and none states a rule of a funder on when an experiment need not be paid for. The texts describe the stewards of archives, and one boundary between archives: the first database paper calls the PDB the single worldwide archive of experimental structure data and presents the prediction database as a new resource beside it, while the second reports that predictions have been integrated into primary resources including the PDB and UniProt. The texts do not say how that integration keeps a prediction from being read as a measurement; this study notes the question and does not answer it. Within the eight texts, the decision whether a prediction may stand in for an experiment belongs to the user, residue by residue, with the developers' measures and the stewards' guidance as instruments.
1.3By when
The texts are full of dates, and almost none of them is a deadline. They date the evidence a decision would rest on, not the decision.
The first kind of date separates what a predictor could have learned from what it is tested on. The AlphaFold paper trains on structures from the PDB released no later than 30 April 2018, and tests the method on structures released after that date. The assessment of docking takes that date and builds its whole design on it, so that no structure the network might have seen can flatter the result:
The AF2 neural network was trained on structures in the Protein Data Bank (PDB) (Berman et al., 2000) as of April 30, 2018 so we evaluated structural models of proteins for which no experimentally determined structure was available as of that date.
The AlphaFold 3 paper does the same with an earlier date for one benchmark, training a separate model on data up to 30 September 2019 so that it could not have seen the test structures. A cut-off of this kind tells a user which evidence about the method is clean. It also bounds what the published tests can show: they were run on structures released after the cut-off, and some of them were further filtered to remove proteins closely similar to structures in the training data.
The second kind of date belongs to the database. Predictions are released, revised and corrected in versions, and the stewards say which version a reader sees:
The data files in the archive are versioned, and previous snapshots of the data will be available via FTP, but the web pages will always display the latest version.
A version is not a formality. The second database paper records one correction that changed the evidence under a large number of users:
A November 2022 update rectified structures affected by a temporary numerical bug presented in the July release. This bug led to low accuracy predictions with correspondingly low pLDDT for ∼4% of the total structure predictions in the database.
The confidence scores did their work here: the faulty predictions came with low scores. But a user who had read a structure from the July release and decided on it had decided on a version that was later replaced. The same paper adds that some sequences in the database may be outdated compared with UniProt, the reference catalogue of protein sequences, because the database is released less often. The date of a decision made from the database is therefore the date of the version it read, and the texts leave the user to record it.
The third kind of date bounds an assessment. The comparison with density maps took every unique structure containing protein, of the kind it studied, deposited in a 6-month period from December 2021 to May 2022, 215 structures in all. A window makes a test fair and dates its findings. A finding about how often a prediction disagrees with experiment holds for the version of the method and the kinds of proteins in that window; it is not a constant.
What the texts do not date is the decision. No text says by when a laboratory must choose between using a prediction and running an experiment. The clock the texts do supply is the cost of waiting, and they state it in time. Against the months to years the AlphaFold paper gives for one experimental structure, the community assessment reports what a prediction can do inside an experiment: rebuilding a published crystal structure from predicted models raised the number of modelled residues from 600 to 818, and the work took about one working day. The deadline, in these texts, is not set by any decider; it is set by the price of the alternative.
1.4What reopens a decision
The texts write their own conditions for looking again. Four recur.
The first is new experimental data. The comparison with density maps is built on this condition: it holds each prediction against a map computed without reference to the deposited model and asks where they disagree. Its summary is the most direct statement in the set that confidence does not close the question:
In other cases, even very high-confidence predictions differed from experimental maps on a global scale through distortion and domain orientation, and on a local scale in backbone and side-chain conformation.
The paper then puts numbers on how often confidence and experiment part ways, band by band:
For the structures analyzed here, about 10% of Cα atoms with pLDDT over 90 are found to be in error by over 2 Å, along with 22% of those with pLDDT between 80 and 90, 33% of those between 70 and 80, and 77% of those with pLDDT under 70 (Table 1).
The sentence supports the database's bands and qualifies them in the same breath. Errors rise as confidence falls, as the bands say they should. Yet even in the highest band about one atom in ten is more than 2 Å from where the experiment puts it, a distance larger than the width of a carbon atom, which the AlphaFold paper gives as approximately 1.4 Å. The community assessment reaches the same point from the work of model building, where a high-confidence side chain turned out to be built backwards:
We emphasize that detailed manual inspection remains necessary to find and correct larger errors in the experimental model, sites of disagreement arising from conformational variability and sites where high-confidence predictions are in fact incorrect.
The second condition is a change of purpose, and it is the one the texts state with most care. The comparison with density maps refuses to say whether its errors are large or small until it knows what the coordinates are for:
If coordinates are intended for use in comparing distantly related structures to infer evolutionary and structural relationships, where typical differences among structures may be large (for example, 2–3 Å), median coordinate errors of 1.1 Å may have little effect on the analysis. On the other hand, the same coordinate errors might substantially affect an analysis involving docking of a ligand to identify specific protein–ligand interactions.
The passage turns a fixed error into two different answers. The same structure, with the same errors, may stand in for an experiment when the question is how proteins are related and may not when the question is how a small molecule binds. A decision to use a prediction is therefore a decision for a purpose, and a new purpose reopens it. The docking study supplies the evidence for the second half of that sentence. It found that predicted receptor pockets were nearly as close to experimental pockets as two experimental structures of the same receptor are to each other, and yet:
Strikingly, however, the accuracy of ligand-binding poses predicted by computational docking to AF2 models is not significantly higher than when docking to traditional homology models and is much lower than when docking to structures determined experimentally without these ligands bound.
In its tests, docking placed ligands correctly in 15% of cases with the predicted models, in 9% with older template-based models, and in 44% with experimental structures solved with other ligands. The study draws the general lesson from the gap:
We find that the RMSD of a model—a classic structural accuracy metric—is not a good predictor of the accuracy that model yields for pose prediction.
This is the sharpest finding in the set for the question of who decides. If the measure of closeness to experiment does not predict fitness for a task, then no single score, whether set by a developer or a database, can settle every use in advance. The study's own caveats keep the finding in bounds: it examined one class of receptors and three docking methods. Jiankun Lyu and others, AlphaFold2 structures guide prospective ligand discovery (2024), report by their abstract's account hit rates and affinities on two receptors similar to those from experimental structures; this paper reads the two results as answers for different designs and targets, and does not decide between them.
The third condition is a partner or a state that the prediction left out. A protein in a cell binds other molecules, carries chemical modifications and changes shape. The comparison with density maps names what the earlier predictor could not see:
We note that as AlphaFold prediction does not take into account the presence of ligands, ions, covalent modifications or environmental conditions, it cannot be expected to correctly represent the many details of protein structures that depend on these factors.
AlphaFold 3 was built to take ligands, ions, nucleic acids and modified residues as inputs, and its paper reports large gains on complexes. It also lists, under its own heading of limitations, what remains:
A key limitation of protein structure prediction models is that they typically predict static structures as seen in the PDB, not the dynamical behaviour of biomolecular systems in solution.
The paper's example is concrete. E3 ubiquitin ligases, a class of enzymes, adopt an open shape when nothing is bound and have been observed closed only when a ligand is bound, yet the newer model predicts the closed shape in both cases. The same paper names a new risk that came with the newer design. Because it is a generative model, built on a diffusion process, it can produce order where there is none:
Although hallucinated regions are typically marked as very low confidence, they can lack the distinctive ribbon-like appearance that AF2 produces in disordered regions.
The sentence matters for the allocation described in section 1.2. A user who had learned to recognise low confidence by its look, a loose ribbon, has to read the score instead; a new version changes what the user must check.
The study of fold-switching proteins takes the third condition furthest. Some proteins switch between two different folds as part of their work, and the default predictor tends to return one of them. By clustering the related sequences the predictor reads, the authors obtained both, and confirmed predictions by NMR. They then warn against reading the confidence score as more than it is:
First, the pLDDT metric itself cannot be used as a measure of free energy.
A score that is high for a shape does not say that the protein spends most of its time in that shape. In their own system, the state the protein disfavours still scored higher. The authors also say how far the problem might reach, and the estimate is theirs:
this would correspond to approximately 1–8 million fold-switching proteins with possible alternative states that would not be predicted by the default AF2 method.
They add a limit on the experiment as well: a new fold found by computation may be hard to confirm in the laboratory if the trigger of the switch, whether pH, redox reaction or a binding partner, is unknown. Here the reopening condition runs both ways. A prediction can reopen a settled experimental picture, and an experiment may not be able to settle the prediction.
The fourth condition is a new version, of the predictor or of the database, and the sections above have already shown it at work: a corrected release, a newer model with different failure modes, a database that shows only its latest files. The AlphaFold paper adds one more boundary that a user can check before deciding, the depth of the evidence the predictor reads. A multiple sequence alignment, or MSA, lines up the protein's sequence with those of its evolutionary relatives. The developers report that accuracy decreases substantially when the median depth of that alignment is less than around 30 sequences. For a protein whose alignment is that shallow, the developers report lower accuracy, so this paper reads the depth of the alignment as a fact a user can look up, which a later version or a larger sequence database could change.
Taken together, the four conditions describe a decision that is never closed by the prediction alone. The texts that make and publish predictions supply the measures; the texts that test them supply the occasions on which a use should be checked again. None of them claims that the substitution, once made, stands for good.