Using agents in scientific research

Isidora Araya Day

6 September 2026

individual submissions

arxiv submissions

Source: tecunningham/ai-discovery-data · output-arxiv

physics submissions

Source: tecunningham/ai-discovery-data · output-arxiv

prompting 1

prompting 2

arxiv rejections

Source: Nature

How do we continue to produce high quality research?

Four ways to work with AI

  1. Chatbot — Ask a question and use the answer.
  2. Copilot — Write a function together.
  3. Agent — Give it a task.
  4. Agentic workflow — Let it run full research loops.

Comparison between a chatbot and an agent

Chatbot

A person exchanges messages with a chatbot.

“How can I reproduce this figure?”

  1. You provide context and ask a question.
  2. The chatbot suggests methods or code.
  3. You run the code and bring back results.

You carry out each step.

Agent

An agent connected to files, code execution, and result checks.

“Reproduce this figure in this project.”

  1. You define the goal and checks.
  2. The agent reads files, writes and runs code.
  3. It checks results and revises its work.

You review the evidence and decisions.

What turns an LLM into an agent?

What it needs

  • Prompt — Your question/instruction.
  • Context — Papers, data, notes and code.
  • Memory — Private notes, decisions, and progress.
  • Skills — Instructions for recurring tasks.
  • Tools — Access to files, code execution, and tests.

How it works

  1. Reason and inspect the project.
  2. Act by using tools to carry out the work.
  3. Observe the results and revise as needed.
  4. Record the result and how to reproduce it.

A prompt guides the LLM, which works with a harness that manages context, memory, skills, tools, and the inspect, act, check, record loop.

OpenAI paper, 2022.

Demo

Tam & Kane · Probing Fermi sea topology by Andreev state transport

arXiv:2210.08048 · Reproduce and extend a paper.

Create a subfolder and work there. Your job is to read arxiv: 2210.08048, write a
research project codebase that reproduces the results using tight-binding simulations,
 and extends the conductance simulation for a normal hamiltonian with random onsite disorder.
Report back with an html summary.

How I structure my work

ALL my projects live in Projects/ 

Projects/
├── spin-qubit/
├── another-project/
└── ...

My workflow for every project

  1. Create a project from my template, with its own Git repository and environment.
  2. Start Codex inside it, with the README, notes, and a concrete task.
  3. Review the outputs, scientific checks, and Git diff; commit the changes I accept.
  4. Resume from saved decisions and next steps in notes/.

↓ Explore the project: project structure → working session

1. Organize the project

Projects / spin-qubit 

spin-qubit/
├── .git/
├── README.md
├── AGENTS.md
├── pyproject.toml
├── pixi.lock
├── code/
│   ├── spin_qubit_package/
│   │   ├── reusable_modules/
│   │   └── tests/
│   ├── simulations/
│   ├── results/
├── data/
├── manuscript/
├── notes/
└── slides/

One reproducible research workspace

  • Package — spin_qubit_package/ holds reusable models, solvers, and plotting.
  • Scripts — simulations/ runs calculations; results/ analyses saved data. Both import the package.
  • Tests — Check known limits and catch regressions.
  • Environment — pyproject.toml defines dependencies and tasks; pixi.lock pins versions.
  • Research record — Save data, manuscript, notes, and slides together; document commands in the README and track changes with Git.

Repetitive task → Use a skill to set up a project structure!

2.Start, check, and leave a record

Start inside the project 

cd ~/Projects/spin-qubit
codex
Inspect this repo, focus on the
tight-binding implementation.
Generalize the code to ...
Verify the result by ...

Process

  1. Choose a model
  2. Give a task — Define deliverable and checks.
  3. Ask questions — Ask for figures, explanations.
  4. Prompt benchmarks — Ask for tests.
  5. Ask for a record — Write a summary.
  6. Check the diff — If possible, inspect the Git diff.
  7. Commit — Save the changes for later.

Prefer focused sessions, each with a clear goal.

3. Finalize the project

What I do:

  • Discuss results with collaborators.
  • Clean up the code and notes.
  • Have my collaborators prompt the project.
  • Disclose the use of AI.

The questions I still have:

  • Should I read every line of code?
  • Can another agent review it for me?
  • How many checks are enough to trust the result?
  • What results are worth publishing?

A researcher and an agent scrutinize research plots, with unresolved questions prompting further review before documenting results.

Producing outputs is easier than knowing whether to trust them.

What can become an agent project?

  1. Creating a website
  2. Filling out reimbursement forms
  3. Scraping a website for data
  4. Collecting literature from a chat
  5. Reviewing someone else’s code
  6. Launching cluster simulations
  7. Reproducing a figure from a paper
  8. Generalizing an algorithm
  9. Testing an idea

A project can be anything with inputs and a deliverable.

How to get started

Install and connect

  1. Choose an interface — An agent app or terminal CLI.
  2. Sign in — Pay a subscription.
  3. Prepare your tools — Ask the agent to install Git, Python, Pixi, LaTeX, etc.
  4. Open a project folder — Use a skill to organize it.
Frozen screenshot of ChatGPT documentation.
OpenAI documentation ↗
Frozen screenshot of Claude Code quickstart documentation.
Claude Code quickstart ↗

Terminal-Bench Science 0.1: resolution rate versus API cost, including GPT-6 Astra.

Terminal-Bench Science 0.1 tests whether agents can complete scientific research workflows using code and terminal tools, including analyzing data, running simulations, and fitting models.

Source: OpenAI, GPT-6 Astra launch results

FrontierMath Tier 4 (v2): accuracy versus API cost, including GPT-6 Astra.

FrontierMath Tier 4 tests advanced mathematical reasoning on exceptionally difficult problems.

Source: OpenAI, GPT-6 Astra launch results

Things I do not have time to talk about

  • How to deal with data privacy?
  • How to create skills?
  • How to set up remote control and launch agents on a server/cluster?
  • How to document the use of AI in research?
  • What are the costs of using agents in research?
  • How to stay up to date with all this?

Questions?