Agent Memory Leaderboard (@AgentMemoryL) header
Agent Memory Leaderboard (@AgentMemoryL) profile picture

Agent Memory Leaderboard Official. First open championship for agent memory systems. Neutral · Reproducible · Fair. mailto: [email protected]

Joined July 2026
Photo by Agent Memory Leaderboard (@AgentMemoryL)Photo by Agent Memory Leaderboard (@AgentMemoryL)Photo by Agent Memory Leaderboard (@AgentMemoryL)Photo by Agent Memory Leaderboard (@AgentMemoryL)Photo by Agent Memory Leaderboard (@AgentMemoryL)Photo by Agent Memory Leaderboard (@AgentMemoryL)Photo by Agent Memory Leaderboard (@AgentMemoryL)Photo by Agent Memory Leaderboard (@AgentMemoryL)Photo by Agent Memory Leaderboard (@AgentMemoryL)Photo by Agent Memory Leaderboard (@AgentMemoryL)
Agent Memory Challenge 2026 Cycle 2 is now open. Long-term memory is not just about storing more history. It is about retrieving the right evidence, recognizing what has changed, and avoiding stale context when an agent needs to act. Three tracks: Textual · Coding · Multimodal Open-source Methods · Commercial Products Over USD 22,000 prize pool for eligible open-source teams. A shared Add/Search interface. Standardized Answer/Eval. Public, comparable results. Join: agentmemoryleaderboard.ai/ev…
19
7
10
36
57,076
A total of 312 memory systems have been submitted for evaluation in this edition. The full evaluation and result verification are now complete. On November 19, 2026, AML will publish the full rankings, track-level performance analysis, and selected technical approaches. We welcome researchers and developers to explore the results and share their feedback. agentmemories.ai/home
37
Beyond Information Storage What makes a memory system useful for an AI agent? It's not just about retrieving stored information. A capable system should handle changing facts, distinguish outdated information, resolve conflicts, and retrieve relevant experience when needed. AML's Text Memory track includes a new Streaming Memory evaluation to examine how systems handle information updates and changing states over time.
1
1
52
How do we evaluate long-term memory in AI agents? Storing information is one thing. Remembering the right information, adapting to updates, resolving conflicts, and applying past experience to new tasks is another. The 2nd Agent Memory Leaderboard (AML) aims to provide a systematic evaluation of these capabilities. This edition has received submissions from 312 memory systems across three evaluation tracks: 🧠 Text Memory 6,000+ evaluation instances, including a new Streaming Memory evaluation that tests how systems handle continuously evolving information. 💻 Code Memory 300 evaluation units across 150 base tasks, testing the use of relevant and noisy memory in bug fixing and feature development. 🌐 Multimodal Memory Nearly 5,000 evaluation instances, focusing on image-text memory, cross-modal retrieval, and evidence use. The full evaluation and result verification process has been completed. The full rankings, track-level performance analysis, and selected technical approaches will be published on November 19, 2026. We hope AML can help make long-term agent memory capabilities more measurable and comparable. Explore AML: [agentmemories.ai/home]
3
4
110
A customer explains a business rule once. Will an AI agent still remember it when making a decision several rounds later? FDE-Bench highlights a subtle warehouse challenge: two batches of the same product may still count as mixed goods under the warehouse's rules. An agent that groups only by product ID could get the entire plan wrong. Remembering a rule isn't enough. The agent must retrieve and apply it when the decision actually depends on it. This is exactly the kind of capability we evaluate in the Textual Memory track of Agent Memory Leaderboard (AML). AML Cycle 2 results are expected in mid-November. Stay tuned to see how participating memory systems perform! Research: arxiv.org/html/2610.14539v3#… AML: github.com/AML-memory/agent-…
6
81
Replying to @AidenParking
Thanks for your question. The Cycle 2 leaderboard is scheduled to be released on November 19. A formal official announcement with full details will follow afterwards. Please stay tuned.
11
That is the question behind Coding Memory evaluation: Can a system retrieve, filter, and reuse historical engineering experience under both relevant and noisy history? At AML, Coding Memory evaluates this across bug-fix and feature tasks. Finding a similar record is only the start; using it correctly is what changes the task outcome. agentmemoryleaderboard.ai/
18
Imagine a team fixed a stale-page bug by adding cache invalidation after data updates. Later, an agent builds a batch-update feature. That past fix should be a useful clue — not a patch to copy blindly. The agent still needs to check the current code, version, and constraints.
1
65
Coding agents are getting faster. But speed is only one part of maintaining a real project. The harder question: after fixing a bug today, can an agent use that experience when a related task appears next week? Zed’s Agent Metrics makes responsiveness visible. Project memory asks what happens over time.
1
2
84
“The wiki stays right. The person moved on.” That’s a great test case. What would you count as enough evidence for the agent to stop applying an old preference without an explicit correction? A few choices that go against it, or would you want the agent to check in?
1
44
Replying to @mem0ai
A case we’d like to see more systems tested on: the shared wiki stays the same, but a user changes their preference. Does the agent follow the update in a new session? That’s the kind of distinction we care about in AML’s textual memory evals.
1
1
10
Replying to @omarsar0
One thing we’re watching in AML Cycle 2: which memory systems help coding agents solve tasks when repo history gets noisy? For self-improving agents, we’d also want to know how much each extra solved task costs once memory-building is included.
1
FrontierMemory asks whether past experience helps agents solve new tasks. One thing we’re looking forward to seeing in the AML Cycle 2 results: which memory systems still help coding agents when useful repo history is buried among unrelated records?
14
Replying to @amayahfenox
That’s a great distinction. The failure isn’t always missing retrieval — it’s missing precedence. An explicit invalidation record gives the agent a state transition, rather than asking it to infer one from two competing facts. Have you found that works best when invalidations are tied to a source and timestamp as well?
2
40
Replying to @embercox
Exactly — retrieval gives you the file, not the meaning. The harder benchmark question is whether the agent can decide which memory is current, relevant, and actionable. Would you prioritize temporal validity, source reliability, or downstream task success when evaluating that?
20
How does AML evaluate Textual Memory? Participants provide two standardized interfaces: Add & Search. AML controls downstream Answer and Eval under a shared protocol. This isolates the memory layer from the LLM. Explore AML: agentmemoryleaderboard.ai/do…
18
To understand where failures happen, examine the full pipeline: Add — Was the update stored? Search — Was the right evidence retrieved? Answer — Was the evidence used correctly? Eval — Was the answer judged consistently? A final score alone cannot tell us where the failure occurred.
1
27
Consider a simple case: A policy has been updated, but the retrieved memories contain both old and new versions. The correct update is right there. Yet the Agent may still apply the outdated rule. Finding evidence and using it correctly are two different capabilities.
🤖 Made with AI
1
77
Finding the right memory doesn't guarantee the right answer. An October 8 analysis by Engram tested 6 memory systems on 1,536 LoCoMo questions, evaluating retrieval and answer accuracy separately. The takeaway: retrieval success ≠ answer correctness. Report: engram.tools/en/insights/ai-…
4
4
169
How does AML evaluate Textual Memory? Participants provide two standardized interfaces: Add & Search. AML controls downstream Answer and Eval under a shared protocol. This isolates the memory layer from the LLM. Explore AML: agentmemoryleaderboard.ai/do…
1
18
To understand where failures happen, examine the full pipeline: Add — Was the update stored? Search — Was the right evidence retrieved? Answer — Was the evidence used correctly? Eval — Was the answer judged consistently? A final score alone cannot tell us where the failure occurred.
1
15