Agent Memory Tuning Adds 16 Points for 5% More Tokens

Give an AI agent memory of its past runs and it should get better. That is the intuition behind most agentic memory work, and a study IBM Research published on August 18, 2026 suggests the intuition is only about a third right. Memory helped enormously on one model, moderately on another, and not measurably at all on a third — and the deciding factor was the capability of the model itself.

• On gpt-oss-120b, a 117B mixture-of-experts model, selective retrieval added 16.1 percentage points of task completion at only 5% more tokens
• On DeepSeek-V3.2, a 671B MoE model, feeding the full guideline set was optimal and produced a 9.5 point gain
• On GLM-5, a 745B MoE model, no configuration produced a measurable improvement
• Testing used AppWorld, a benchmark of 585 multi-step tasks across nine simulated applications

How ALTK-Evolve Builds Memory Without Retraining

The method IBM calls ALTK-Evolve works entirely at the prompt layer. It reads an agent's past trajectories, extracts behavioral guidelines from what worked and what did not, consolidates overlapping guidelines into a coherent set, and reinjects the relevant ones at inference time. No model weights are updated and no human annotation is required.

That design choice is what makes the result actionable for teams who do not fine-tune. The whole intervention is retrieval and prompt construction — the layer nearly every agent framework already has. It also means the cost is measured in tokens rather than GPU hours, which is why the 5% token overhead figure matters as much as the 16.1 point accuracy figure.

The benchmark tracks two metrics: task goal completion, which asks whether an individual task succeeded, and scenario goal completion, which asks whether every variant of a scenario succeeded. The second is the stricter and more honest measure of reliability.

Why Does More Memory Help Some Models and Not Others?

The pattern IBM found is a capability ladder. A weaker model benefits from being told, specifically and at the right moment, what to do — but it also gets confused by a large undifferentiated pile of guidance, so curated retrieval wins on both accuracy and cost. A strong model with headroom can absorb the entire guideline set and pick what it needs, so the full-context approach wins, and prompt caching keeps that affordable.

A saturated model has already internalized the behaviors the guidelines encode. There is nothing left for the memory system to teach it on this benchmark, so the extra tokens buy nothing. The study's own framing is the useful takeaway: agentic memory is not a feature you switch on, it is a dose you calibrate to the model.

What to Do With This in Practice

The practical instruction is to measure before you architect. If your agent runs on a mid-sized open-weight model, a selective retrieval layer is likely the highest-return change available and it will lower your token bill rather than raise it. If it runs on a frontier model, test whether the memory layer is earning its keep before you build infrastructure around it.

That cost-first framing has been a recurring theme this month, from open agent harnesses cutting run costs by 75% to retrieval work on dense versus multi-vector embedding models. The common thread is that the cheapest improvements in agent systems are increasingly at the context layer, not the model layer. More on this and other research is on our artificial intelligence page.

Sources: IBM Research on Hugging Face — August 18, 2026; Hugging Face blog index — August 2026.