Conclusion

Discussion

This paper asked when a result produced by an AI-scientist system may stand as a scientific finding, who decides, by when, and what would make the decider revisit the decision. The ten texts it read do not answer the question in general, and they do not claim to. What they do, read together, is distribute the decision among people at points they name, and leave the rest open.

Who decides is answered in the plural. A scientist sets the objective and prepares the data, as Kosmos describes; a person selects which outputs go forward, as the second AI Scientist's authors did when they chose among runs; a person reviews each stage, as in Agent Laboratory's co-pilot mode; researchers perform the experiments a system proposes, as with Robin; a venue reviews; and named authors sign, as the Coscientist paper's disclosure shows. The systems' own reviewers are evaluated, and in one case found to overrate their own outputs, but none is given the power to admit a result to the record. The one time a system's output did pass a human peer review, its authors had already arranged to withdraw it, and they gave the reason as the absence of a community consensus. The executive order adds a funder that measures progress partly by publications, which returns the decision to the venues. Tang and others propose committees that would review risk before research. No text assigns anyone the task of judging, after the fact, whether a given output is true.

By when is answered for institutions only. The order gives the Secretary of Energy days to identify resources, assets, laboratories and challenges, and a year to report. The system papers report single runs that experts estimate as months of their own work. Neither sets a time by which a result must be checked, and the Kosmos authors state that checking remains slow, expert work that no automated method yet does reliably.

What reopens a decision is answered most fully, because the authors wrote down what went wrong in their own systems: tables of results that were never run, test data that overlapped training data, runs that do not converge, interpretations that mistake statistical significance for value, and hypotheses built on literature that may itself be wrong. Each failure implies a condition for looking again, and each condition requires someone with the code, the data and the expertise to compare a claim with its record.

The laboratory of one is the case in which every human point but the venue falls to one person. The texts do not say that this is impossible. The first AI Scientist paper looks ahead to a world in which even the reviewing is automated:

The second paper offers a forecast in its authors' words:

These are the claimants' expectations, and this paper does not weigh them. It notes only that, even in the first passage, a human scientist remains, and that neither passage says who would then decide that an output is a finding. If the reviewers and the conferences are also automated, the question this paper asks moves from the venue back to whoever runs the ecosystem. For the present, the texts leave a single researcher who directs such a system in a definite position: answerable, as an author, for what the system produced; holding, as selector, the first decision about what goes forward; and dependent, for the final decision, on readers who have the time and the access to check.

Contributions

The paper makes three contributions. It sets out, from the systems' own papers, the human points at which an output of an AI-scientist system becomes eligible to be a finding, and shows that each paper names its own set: objective and data, selection, stage review, experiment, venue and signature. It shows that the deadlines in the federal texts bind the building of platforms and the writing of reports, not the judging of results, so that the question "by when" has no answer in the texts for any single result. And it reads the authors' sections on limitations as the clearest statement anywhere in the texts of what should make a decider look again, while showing that these conditions are written as admissions, not assigned as duties. A laboratory of one, on this reading, is a single person who holds every one of the points the papers spread across a group.

Limits

The paper reads ten texts and nothing else. The system papers are their authors' accounts of their own work; their evaluations are not independent, and the one independent evaluation in the bibliography is read only through its abstract. The systems were tested mostly in machine learning and biomedical research, and nothing here speaks for other fields. The government texts are United States texts of 2025 and are read only for the responsibilities they assign; other countries and later texts are not covered. The paper does not test any system and does not judge any result the papers report. It is itself AI writing from public texts, and its readings of the works named are its interpretation.

Further research

Four questions follow. First, how venues that accept or reject research will state who may submit work generated by such systems and when the generation must be disclosed, a question the second AI Scientist's authors leave to the community. Second, whether the automated verification that the first AI Scientist paper proposes, a check of reported results against the code and experiments that produced them, can be built and evaluated independently of the systems it checks. Third, how the annual reports that the executive order requires will count results produced by the platform's agents, and whether they will distinguish results that others have checked from results that have only been produced. Fourth, how much checking one person can do: a study that measured the time a single researcher needs to verify a system's report, against the time the system needs to produce it, would turn the central constraint this paper finds into a number that others could test.