Memory, research and references¶

Every climb has a Memory tab: what the climb remembers. You tell AutoLab things worth knowing, the agent keeps a short summary of what it has learned, and research reports land there too. The agent reads its memory every time it plans new jobs.

autolab open      # then Memory in the climb's sidebar

The Memory page¶

The page has tabs. From people opens first:

From people. A box to tell AutoLab something to remember (what you tried in another project and how it went, a constraint like "never use more than 2 GPUs", a paper worth reading; Markdown and links work), then the memories people added, word for word. Memories the climb chat saved for someone show as "name, via chat".
What AutoLab knows. One short, readable summary that the agent writes and keeps current: the best result so far and how it got there, what worked, what didn't, what the literature says, and what comes next. Nobody can edit it by hand. To change it, tell the agent: add a memory, or say it in the climb chat.
Deep research. Deep research reports, each linking to its deep research job page.
AutoLab's notes. The agent's detailed memories behind the summary.

You can edit or delete memories you added. Climb admins can delete any memory. If someone changes a memory while you are editing it, your edit is refused instead of overwriting theirs: reload it and edit again.

How the agent uses memory¶

Every time the agent plans (new jobs or a queue update) it reads its summary, every memory from people, and the titles of its own memories. Memories from people are operator knowledge: the agent weighs them heavily, won't re-run what you say was already tried unless something concrete has changed, and can't edit or delete them.

Tell the agent what you already tried¶

The most useful memory is a summary of experiments you ran before this climb: in another repo, another climb, a notebook, or a paper's ablations. Without it, the agent may spend compute rediscovering what you already know.

Write one line per attempt with what changed, the result with numbers, and whether it helped:

- Cosine LR with 500-step warmup: val loss 3.41 vs 3.38 baseline. No gain.
- Batch size 1024: diverged after 2k steps at lr 3e-4. Fine at 1.5e-4, but slower.
- Muon optimizer: about 8% faster to target loss. Worth exploring further.

autolab memory add --title "Prior experiments" -f summary.md

A memory can be up to 20,000 characters. Split a long history into a few memories by theme.

Memory from the CLI¶

autolab memory                        # What AutoLab knows, then a table of memories
autolab memory --by people            # only memories from people (or: autolab, research)
autolab memory --search warmup
autolab memory show <id>              # one memory's full text and sources

autolab memory add "Data loader is the bottleneck above 4 GPUs"
autolab memory add -f idea.md --ref https://arxiv.org/abs/XXXX.XXXXX
cat results.md | autolab memory add -f -     # text from stdin

autolab memory edit <id> --title "..."       # your own memories only
autolab memory rm <id>

--ref URL cites a source (repeat it for several), --exp ID links a memory to a job, and --json prints machine-readable output. Outside the climb's workspace, add --project owner/slug.

Deep research jobs¶

A deep research job sends AI researchers to search the web and papers on a question, read what they find, and write a report with citations. It runs in the background on AutoLab, not on your machines, so experiments keep running while it works.

Deep research jobs live in the climb's Jobs list next to experiments:

While it runs, a deep research job sits under Active with live progress. You can stop it there.
When it finishes, it moves into Completed, in time order with your experiments, showing its title, a one-paragraph summary and how many sources it cites.

Click one to open its page: the question, status, progress, cost and the full cited report. The report is also saved to Memory, where the agent learns from it.

Start a deep research job¶

Add Job on the Jobs page: switch from Experiment to Deep research, type the question, and pick a depth (Quick, Standard or Deep).
The climb chat: "do deep research on what speeds up small GPT training".
The CLI:

autolab research "what speeds up convergence for small GPT models?"
autolab research --depth deep --context "we train on 8xH100 for 10 minutes" "..."
autolab research --wait "..."        # wait and print the report when it is ready
autolab research list                # recent deep research jobs (also: bare `autolab research`)
autolab research show <id>           # status, progress, and the report once done
autolab research cancel <id>

Depth	Researchers
`quick`	1
`standard` (default)	3
`deep`	5

Deeper research takes longer and costs more. Research uses your AutoLab credit, like the agent's own model usage.

The agent researches too¶

The agent can search the web, search papers and read pages whenever it is useful, in any step, without starting a deep research job. It can also start a deep research job by itself when a question deserves one. It decides when; it is never forced to.

If your climb's instructions say something like "no deep research" or "don't use the web", the agent follows them.

Limits per climb:

At most 2 deep research jobs at a time.
People can start 30 deep research jobs per 24 hours; the agent has its own budget of 8 per 24 hours.

A deep research job that can't start (internet research is off, you are out of credit, or a limit is reached) is refused with the reason.

References¶

Every job explains itself¶

Each job opens with an Idea: one to three plain sentences on what it tries and why, written so someone who has seen no other job can follow it. Sources are cited as [1], [2], which link to the job's references, with the exact spot (for example "Appendix B, p. 14") when known.

After the run, the job gets a Result: one to three plain sentences on what happened, the score compared with something named (such as "the best so far"), and what it means. For a failed run, it says what broke and whether a retry can fix it. It is the only analysis shown to people; the agent keeps its own technical notes for planning.

Job lists show the idea (queued jobs) or the result (completed jobs) under each title, and autolab status shows the idea, its numbered sources and the result of the experiment you are in.

References are checked¶

The agent can only cite sources it actually opened or found with its web tools on this climb, or that a research report on this climb cites. AutoLab checks every reference the agent attaches to a job or a memory: the link must exist and the page must match the claimed title, so a wrong arXiv id that points at another paper fails.

When a reference can't be confirmed, AutoLab sends the submission back to the agent with the reason, so it can look the source up or drop it. After two tries, unconfirmed references are dropped and the rest is kept. Stored references show the source's real title.

References you add yourself (on the site, with the CLI or the API) are stored as you wrote them, without a check. With internet research off, the agent can only cite sources that were already confirmed on this climb.

Every source in one place¶

autolab refs                     # every source cited by memories and jobs, and what cites it
autolab refs --search muon

Attach a source to an idea when you queue it, so the agent can read it and it shows up in references:

autolab submit --no-code -m "try the Muon optimizer" --ref https://kellerjordan.github.io/posts/muon/

Internet research setting¶

Each climb has an Internet research setting, on by default. It controls whether AutoLab's agents can use web tools: web search, paper search, reading web pages, and deep research jobs. When it is off, the agent doesn't get the web tools at all, and deep research jobs can't be started.

Set it when you create a climb, with the Internet research switch under Evaluation version and advanced settings on the Start a climb form (or autolab init --no-internet to start with it off).
Change it any time in the climb's Settings, under Limits > Advanced, or from the CLI:

autolab settings --internet off
autolab settings --internet on

The setting covers AutoLab's own web tools only. It does not firewall your machines: experiments running on your nodes keep whatever network access those machines have.

Paper search always works while the setting is on. Whether general web search is available depends on the server you use; autolab research list tells you when it isn't.

See also: Drive the agent for other ways to steer a climb, and Climb settings for the rest of a climb's settings.