Chapter 2. Appendix: The sources read

Each idea is implemented and developed into a full paper at a meager cost of less than $15 per paper, illustrating the potential for our framework to democratize research and significantly accelerate scientific progress.

Unlike standard reviewers, the automated reviewer is unable to ask questions to the authors in a rebuttal phase, although this could readily be incorporated into our framework.

The AI Scientist may incorrectly implement an idea, which can be difficult to catch. An adversarial code-checking reviewer may partially address this. As-is, one should manually check the implementation before trusting the reported results.

The ability to automatically generate and submit papers to academic venues could greatly increase the workload for reviewers, potentially overwhelming the peer review process and compromising scientific quality control.

Because of this, we believe that papers or reviews that are substantially AI-generated must be marked as such for full transparency.

Reviewers were informed in advance that some submissions might be AI-generated, but were not told which submissions were produced by The AI Scientist-v2 . Reviewers could also opt out of reviewing potentially AI-generated manuscripts.

We concluded that none of the manuscripts met the quality standards typical of top-tier main-track conferences.

We emphasize that the community has not yet reached a consensus on integrating AI-generated research into formal scientific publications, making careful and transparent experimentation essential at this preliminary stage.

Thus, the current version of The AI Scientist-v2 does not yet consistently reach the rigorous standard required for top-tier conference publications, nor does it even reach workshop-level consistently.

In co-pilot mode, in addition to providing the research idea, there is also a checkpoint at the end of each subtask, where a human is involved in reviewing the work produced by agents in that phase (e.g., the literature review summary or generated report).

The ability to autonomously generate research code, reports, and experiment plans may inadvertently lower the barriers to producing substandard or misleading scientific outputs. This could overwhelm peer review systems and jeopardize the integrity of academic discourse.

Transparent disclosure of AI involvement in research outputs is important in order to mitigate such risks and maintain accountability.

We also find that automated reviewer scores do not predict human reviewer scores demonstrating the importance of human evaluations in automated research.

Co-Scientist is purpose-built for a “scientist-in-the-loop” collaborative paradigm.

Additionally, the validation of Co-Scientist’s hypotheses, while successful, remains preliminary.

Improper use of such AI systems without rigorous peer-review and guardrails could also lead to worsening of the scientific reproducibility crisis through production of low quality scientific artifacts.

However, limitations remain; human expert involvement is crucial, as LLMs may not align with human judgment in specialized domains

The original code or cited paper(s) supporting the statements were not made available to the evaluating scientist during this process.

Kosmos is designed not to replace human scientists, but to augment and accelerate their work.

Similarly, Kosmos was found to be only 57% accurate in statements that required interpretation of results, likely due to its propensity to conflate statistically significant results with scientifically valuable ones.

Given these limitations, the central value proposition is therefore not that Kosmos is always correct, but that its extensive, unbiased exploration can reliably uncover true and interesting phenomena.

All hypotheses, experimental plans, data analyses, and data figures in the main text of this report were produced by Robin.

Researchers next conduct the experiments and provide the resulting data to Robin for autonomous analysis.

For example, while Robin generates experimental outlines, it does not yet produce precise, executable protocols—future iterations aim to provide detailed methodologies that require minimal human translation for laboratory execution.

Finally, while Robin uses an LLM-judged tournament to nominate therapeutic hypotheses, future work on better aligning hypothesis generation and evaluation with human scientific judgment may be helpful in more reliably producing high-quality hypotheses.

Second, we advocate restricting autonomous operations to well-characterized, lower-risk scenarios where safety parameters and operational boundaries have been thoroughly validated through empirical testing and expert review.

First, potential users should obtain a license to access AI scientists, analogous to how drivers must be licensed before operating vehicles.

However, more thorough safety checks inevitably increase response latency.

Another possible solution is to require autonomous agents to obtain approval from a committee consisting of human experts before each query involving critical tools and APIs that may lead to potential safety concerns.

What are the capabilities of LLMs in the scientific process? What degree of autonomy can we achieve? How can we understand the decisions made by autonomous agents?

In this paper, we presented a proof of concept for an artificial intelligent agent system capable of (semi-)autonomously designing, planning and multistep executing scientific experiments.

The development of new intelligent agent systems and automated methods for conducting scientific experiments raises potential concerns about the safety and potential dual-use consequences, particularly in relation to the proliferation of illicit activities and security threats.

(ii) AI modeling and analysis frameworks, including AI agents to explore design spaces, evaluate experimental outcomes, and automate workflows;

(iv) establish procedures to ensure the highest standards of vetting and authorization of users and collaborators seeking access to the resources of the Mission and associated research activities, including the Platform and associated Federal research resources.

(ii) establish clear policies for ownership, licensing, trade-secret protections, and commercialization of intellectual property developed under the Mission, including innovations arising from AI-directed experiments;

(a) Nothing in this order shall be construed to impair or otherwise affect: (i) the authority granted by law to an executive department or agency, or the head thereof; or

Like many other domains, science itself will be transformed by AI.

AI-enabled predictions are of little use if scientists cannot also increase the scale of experimentation.

Require federally funded researchers to disclose non-proprietary, non-sensitive datasets that are used by AI models during the course of research and experimentation.

Over time, regulators should explore the use of evaluations in their application of existing law to AI systems.