The texts give three answers to the question this paper asks, and they give them in different places. The system papers answer "who decides" in their descriptions of how the systems are run and evaluated, and they answer "what reopens it" in their sections on limitations. The government texts answer "by when", but for institutions rather than findings. This chapter first states how the texts were chosen and read, then takes the three parts of the question in turn.
1.1Data and method
The texts were chosen by one rule: every openly licensed paper that presents a system its authors describe as automating more than one stage of scientific research, from hypothesis to analysis or report, and whose full text could be saved, together with the openly licensed perspective on the risks of such systems and the two United States government texts that assign federal responsibilities for AI-driven science. Seven system papers met the rule: Chris Lu and others on the AI Scientist; Yutaro Yamada and others on its second version; Samuel Schmidgall and others on Agent Laboratory; Juraj Gottweis and others on Co-Scientist; Ludovico Mitchener and others on Kosmos; Ali Essam Ghareeb and others on Robin; and Daniil A. Boiko and others on Coscientist, a chemistry system. The perspective is that of Xiangru Tang and others. The government texts are Executive Order 14363, Launching the Genesis Mission, and America's AI Action Plan. The system papers and the perspective are published under the Creative Commons Attribution 4.0 licence; the government texts are in the public domain. Each was saved in full, and sources/manifest.json records its licence and address.
Other works enter the argument only through short reading notes, and only as their notes allow: an independent evaluation of the first AI Scientist, two autonomous laboratories, and three studies of discovery and research productivity. Their texts are not quoted, because their licences do not permit it or because only their abstracts were read.
The reading asked of each text the three questions of Meridian's form. For "who decides", it looked for every point at which a person acts on an output: sets a goal, prepares data, selects, reviews, signs, withdraws. For "by when", it looked for every stated date or interval and asked what it binds. For "what reopens", it looked for every failure the authors report in their own system and every check they propose. The paper quotes what the argument needs in the body; the remaining passages read are given word for word in the appendix.
Three limits follow from the method. The system papers are their authors' own accounts, and their evaluations are the authors' evaluations; where a capability is claimed, this paper gives it as a claim. Most of the systems were tested in machine learning or in biomedical research, and nothing here is evidence for other fields. And the government texts are read only as they assign responsibility; this paper does not judge whether the federal programme will succeed. None of the biomedical results the papers report is read as medical evidence, and nothing here is medical, regulatory or investment advice.
1.2Who decides
No system paper lets its system decide that its own output is a finding. The clearest case is also the one in which a system came closest to deciding. Yamada and others submitted three manuscripts written entirely by the second AI Scientist to a workshop at a major machine learning conference, with the organisers' agreement. Reviewers knew that some of the 43 submissions might be AI-generated, but not which. One manuscript scored 6.33 out of 10, above the workshop's threshold. By the workshop's own process, it had become a paper. The authors had already decided otherwise:
Post-Review Withdrawal: Prior to the workshop submission, we arranged with the workshop organizers and ICLR leadership that any accepted AI-generated manuscripts would be withdrawn after the review process. This decision was made to avoid prematurely setting a precedent for purely AI-generated research entering the official scientific record without broader community discussion and consensus.
The passage separates two acts that peer review usually joins: judging a paper and admitting it to the record. The reviewers judged; the authors, the organisers and the conference leadership, by an agreement made before any review, kept the result out. The decision whether such a result may stand was given to a body that does not yet exist, "the community", whose discussion and consensus the passage names as the missing condition. This paper reads the withdrawal as the most precise answer any of the texts gives to its question: at present, the people who decide are those who can keep a result out, and they chose to.
The same paper is candid about how much of the work people did before the review. Humans chose the workshop's theme, picked three of the ideas the system generated, ran each through the system several times, and chose the best manuscript of each set. The authors insist that this is not intervention:
The selection of initial ideas from the AI’s output, the execution of multiple seeds, the subsequent selection of the best complete run, and the automated handling of length constraints represent high-level experimental setup and process management (meta-selection from fully autonomous outputs), not human-in-the-loop intervention in the scientific content generation of the chosen manuscript.
The distinction is exact, and it matters for a laboratory of one. The authors say that no person edited the code, results, figures or text. But selection is a decision about which outputs reach a reviewer at all, and it is the decision a single researcher would make every day. The paper compares it to a professor choosing which student's work to submit. In a laboratory of one, there are no students and no professor apart from the researcher; the selector and the person answerable for the selection are the same.
The first AI Scientist paper goes further, and tells its readers how to treat what the system produces:
More generally, we do not recommend taking the scientific content of this version of The AI Scientist at face value. Instead, we advise treating generated papers as hints of promising ideas for practitioners to follow up on.
Here the decision is handed to "practitioners", the people who follow up. A generated paper, on this account, is not yet a finding of any kind; it becomes one only when a person takes up the idea and tests it. The same paper also reports an automated reviewer that reached 70% accuracy on 500 past conference decisions, against 73% for human reviewers in an earlier experiment, and the authors use it to score their system's papers. They do not, however, let it admit anything to the record, and they warn that its wide adoption by reviewers could lower the quality of reviews.
Agent Laboratory tested the automated reviewer's standing directly, by comparing its scores with those of people:
However, a clear gap emerged between human and automated evaluations, with automated scores significantly overestimating quality (6.1/10 vs. 3.8/10 overall).
Read with the first AI Scientist's figures, the result shows why neither paper gives the machine reviewer the last word. A reviewer that agrees with human decisions on average can still score a system's own outputs far above what people give them. Agent Laboratory's answer is a co-pilot mode, in which a person reviews the work at the end of each stage and gives feedback. The paper reports that this mode produced higher overall scores than the autonomous one. For a laboratory of one, the co-pilot is the researcher; the paper's design assumes a human checkpoint and supplies none.
Kosmos, which reads data and literature over many cycles and writes reports in which every statement cites its code or source, describes the human role as two ends of a pipeline:
A Kosmos-integrated pipeline begins with human-generated and -curated high-quality dataset and ends with human interpretation and critical evaluation of the results.
Following a Kosmos run, the scientist’s role shifts to evaluation and interpretation of results. This human oversight is crucial, as independent evaluators noted that Kosmos tends to make excessively strong claims and can sometimes veer in unexpected trajectories ( Figure 1c ).
The authors place the decision at the end, with the scientist, and they give the reason: the system overstates. Their own evaluation shows where. Expert evaluators, who were not given the system's code or citations, judged 79.4% of 102 statements accurate, by the authors' report: 85.5% of the statements based on data analysis were reproducible and 82.1% of those based on the literature were validated with primary sources, against 57.9% of the synthesis statements, which the authors' discussion describes as the statements that required interpretation of results. The part of a report that most resembles a finding, the interpretation, is the part the system gets wrong most often. That is the part left to the scientist.
Robin and Coscientist place people in the middle of the loop and at its end. Robin proposes the experiments and the drugs to try, but researchers carry the experiments out in the laboratory and return the data to the system, which analyses them and proposes the next round. Its authors call the result semi-autonomous, and the word is exact: the system does the reasoning between experiments, and people do the experiments and decide to run them. The paper's own list of open problems confirms where people still stand. The system writes outlines of experiments, not protocols a laboratory can follow; its analysis agent depends on prompts written by domain experts; and the authors suggest that its ranking of hypotheses would gain from closer alignment with human scientific judgement.
The Coscientist paper closes with a statement of responsibility that is easy to pass over:
All authors have read, corrected and verified all information presented in this manuscript and Supplementary Information.
The sentence concerns a language model's help with the manuscript's grammar, not the system's experiments. Yet its form is the form every finding takes in these texts: named people read, corrected and verified, and answer for the result. Robin's paper says that all its hypotheses, plans, analyses and figures were produced by the system; it is still signed by its authors.
The government texts assign responsibility of another kind. The executive order names who runs the programme:
(b) The Secretary of Energy (Secretary) shall be responsible for implementing the Mission within DOE, consistent with the provisions of this order, including, as appropriate and authorized by law, setting priorities and ensuring that all DOE resources used for elements of the Mission are integrated into a secure, unified platform.
The Secretary decides priorities and the platform; the Assistant to the President for Science and Technology leads and coordinates. Nothing in the order assigns anyone to judge whether an output of the platform's agents is a finding. The order measures success, in its annual report, partly by what others have already judged:
(iv) updates on research efforts and outcomes achieved, including measurable scientific advances, publications, and prototype technologies;
A publication is a finding that a venue has accepted. The funder's measure therefore returns the decision to the venues, which is where Yamada and others found it unsettled. The order also asks for policies on ownership of intellectual property, "including innovations arising from AI-directed experiments", and for vetting of users who seek access to the platform. Vetting decides who may run the systems, not what their outputs are worth.
Tang and others propose to fill the gap with an institution:
Similar to clinical studies requiring Institutional Review Board (IRB) approval, autonomous scientific research needs institutional oversight.
Their committees would assess risk before research, not the truth of results after it, and they would sit in institutions. A laboratory of one may have none. Read together, the texts answer "who decides" in the plural: a person who sets the objective, a person who selects, a person who verifies, a venue that accepts, and authors who sign. A laboratory of one is the case in which all of these but the venue are the same person.
1.3By when
The texts set many deadlines, and none of them falls on a finding. The executive order is written in days. Within 60 days the Secretary must list at least 20 science and technology challenges; within 90 days, identify computing resources; within 120 days, identify initial data and model assets; within 240 days, review robotic laboratories able to engage in AI-directed experimentation. Then:
(f) Within 270 days of the date of this order, the Secretary shall, consistent with applicable law and subject to available appropriations, seek to demonstrate an initial operating capability of the Platform for at least one of the national science and technology challenges identified pursuant to section 4 of this order.
The deadline binds the Secretary to demonstrate a capability, and even that obligation is to "seek to" demonstrate it, subject to law and appropriations. Within 1 year comes the first annual report. These dates measure the building of a platform. They say nothing about how long an output of that platform must wait, or be checked, before it may be called a result. The action plan, in its section on AI evaluations, likewise sets a pace only for meetings: it asks that federal agencies and the research community convene at least twice a year to share practice in building evaluations.
The system papers set no deadline for judgement either, but they reveal a gap in time. The systems work quickly. Kosmos's authors estimate that each run performs about 4.1 expert-months of research, and academic groups they consulted put the findings of a 20-cycle run at 6.14 months of their own work; by the authors' count, an average run reads 1,500 papers and executes 42,000 lines of code. Judgement does not speed up with them:
Second, identifying valuable discoveries Kosmos made is a time-intensive process that relies on human scientists with significant domain expertise. While the provenance of each claim is traceable, there exists no automated method to reliably evaluate if a claim is accurate, novel, and significant.
An average report, by the same section, holds three or four discovery narratives of 25 claims each. Production is measured in machine time and verification in expert time, and the second does not shrink when the first does. This paper reads this as the hardest constraint the texts reveal for a laboratory of one. The researcher can launch more runs than the researcher can check, and the texts give no date by which the checking must be done. Tang and others name the same trade-off from the side of safety: in their account, more thorough checks inevitably make the response slower, so that higher standards of safety come at a cost in speed. Their proposal accepts the cost. A laboratory of one meets it in its plainest form, because the person who would bear the delay is the same person who would gain from skipping the check.
The first AI Scientist paper names the consequence for the venues. If papers can be generated and submitted automatically, its authors warn, the workload of reviewers could grow until peer review is overwhelmed and its quality control compromised. Agent Laboratory's authors make the same warning in their section on ethics: lower barriers to producing research could also lower the barriers to substandard or misleading work, and that work too would arrive at the reviewers.
If the decision rests finally with the venue, and the venue's time is fixed while the volume of submissions is not, then "by when" is answered by whoever is slowest. The texts do not assign that wait to anyone.
The one place a text speaks of timing in a finding's life is Yamada and others, and they speak of it as an open question:
As a community, we need to establish norms for AI-generated science–including disclosure requirements and timing. We advocate for transparency about AI-generated content, though questions remain about whether work should first be judged on merit to avoid bias.
The question of timing here is not how long a judgement may take but when a reader should learn that a machine wrote the work: before reviewing it, which may bias the review, or after, which may mislead the reviewer. The authors chose to tell reviewers in advance that some submissions might be AI-generated and to reveal which only after review. They present the choice as one experiment's, not as a rule. The first AI Scientist paper and Agent Laboratory both call for disclosure of AI involvement; neither says when.
1.4What would reopen the decision
The texts are most specific about what would make a decider look again, because their authors looked again at their own systems and wrote down what they found. Four conditions recur.
The first is a result that was never observed. The first AI Scientist paper reports it plainly:
Rarely, The AI Scientist can hallucinate entire results. For example, an early version of our writing prompt told it to always include confidence intervals and ablation studies. Due to computational constraints, The AI Scientist did not always collect additional results; however, in these cases, it would sometimes hallucinate an entire ablations table.
An ablation table reports what happens when parts of a method are removed; this one reported experiments the system had not run. The authors fixed it by instructing the system to include only results it observed, and they keep copies of the files executed so that results can be reproduced. Agent Laboratory reports the same failure in some of its papers, with an example of training details that did not occur. The condition for reopening is therefore a mismatch between a paper and the record of what was run, and it can be found only by someone who compares the two.
The second is a test that was not a test. Reviewing the one manuscript the workshop accepted, the second AI Scientist's authors examined its code:
Our examination of the code revealed potential issues with dataset overlap—approximately 57% overlap between training and test sets—which could significantly affect the reliability of the results.
The workshop's reviewers, by the paper's account, asked for clearer justification, other architectures and other tasks; the overlap was found by the authors, in the code. The case shows which reader could reopen this result: one with access to the code and the time to read it. The workshop's review did not need that access to accept the paper, and so could not have reopened it on this ground.
The third is a run that does not repeat:
Furthermore, as Kosmos is stochastic, multiple independent runs may not consistently converge on the same discoveries. The system’s research directions are currently sensitive to the phrasing of the research objectives.
The authors do not say what a reader should make of a result that one run produces and another does not. The passage gives the reopening condition in operational terms: a second run, or the same objective phrased another way. It also explains why the authors had each discovery corroborated by independent analysis from a domain expert. The same section names the interpretive weakness that makes such checks necessary. The authors report that the system was only 57% accurate in statements that required interpretation of results (the 57.9% above, as their discussion rounds it), and they attribute this to a tendency to treat a statistically significant result as a scientifically valuable one. The distinction is old in science, but here it marks a division of labour. Significance can be computed; value has to be judged, and the authors leave the judgement to the scientist. They also say that the system tends to invent unusual quantitative measures that are often statistically sound but hard to interpret. A reader who cannot say what a measure means cannot say whether a finding built on it should stand.
The fourth is an error inherited from the literature. Co-Scientist builds hypotheses from published work, and its authors name the risk:
Furthermore, the quality of generated hypotheses relies on the mixed and contradictory quality of the source literature; thus, there is a risk of propagating erroneous or irreproducible findings.
Here the reopening condition lies outside the system: a source it relied on is retracted or fails to replicate, and every hypothesis built on it must be looked at again. The authors propose agents that trace claims to specific figures or data within a source. Kosmos claims to provide such tracing for its own reports, linking every statement to the code or paper behind it. Traceability does not reopen a decision, but it makes reopening possible, because it tells a reader where to look.
The first AI Scientist paper proposes to automate the check itself:
Crucially, future work should address the reliability and hallucination concerns, potentially through a more in-depth automatic verification of the reported results. This could be done by directly linking code and experiments, or by seeing if an automated verifier can independently reproduce the results.
This paper reads an automated verifier as the kind of tool that would take on work the texts otherwise leave to people; the authors offer it as future work. The texts as they stand give the reopening conditions to whoever has the code, the data, the time and the expertise to compare a claim with its record.
The executive order contains one reopening clause, and it concerns the agenda, not results:
(d) On an annual basis thereafter, the Secretary shall review and update the list of challenges in consultation with the APST and the NSTC to reflect progress achieved, emerging national needs, and alignment with my Administration's research and development priorities.
Each year the list of problems is revisited, by named officials, on named grounds. No clause of comparable precision exists for the findings the programme's agents may produce. Across the texts, the conditions for looking again at a result are written by the system builders about their own systems, as limitations; they are not assigned to anyone as a duty.