Agent Memory Leaderboard publishes first public results for AI text memory systems

We just released the first public results from the Agent Memory Leaderboard (AML).The first evaluation focuses on Text Memory across two tracks:- Open-source Methods - Commercial Products136 teams registered, and 69 representative memory frameworks completed the first evaluation.Commercial Products — Text Memory:1. MemoraX — 58.02 2. MemOS — 45.89 3. NTES-MEMORY-SMART — 44.21The benchmark uses a common evaluation framework and a clearer system boundary:Memory system: Add → Search Benchmark: Answer → EvalThe goal is to make memory-system results more comparable and reproducible. Comments URL: https://news.ycombinator.com/item?id=49281370 Points: 2 # Comments: 0

Everyone is building AI memory systems. But how do we know which ones actually work? As AI agents move from one-off interactions toward long-term collaboration, memory is becoming a core capability. Yet evaluating memory systems fairly is surprisingly difficult. Different systems often use different datasets, answer models, prompts, and evaluation methods. When the final score changes, it can be hard to tell whether the difference comes from the memory system itself or from the evaluation setup. That's why we built Agent Memory Leaderboard (AML). Why Do We Need a Memory Benchmark? Memory is more than storing conversation history. A useful memory system needs to retrieve relevant information, connect information across time, handle changing states, and provide useful context for an agent's current task. But there hasn't been a common evaluation environment where different memory approaches can be compared under the same conditions. AML was created to provide that common ground. The first edition was jointly initiated by nearly 30 universities and research institutions and covers two evaluation tracks: Open-source Methods — Text Memory Commercial Products — Text Memory As of August 12, 2026: 136 teams registered for the competition 67 representative memory frameworks successfully completed the first evaluation The AML website surpassed 200,000 clicks The website passed 100,000 clicks within the first 10 days The first leaderboard results are now live. Making Memory Systems More Comparable One of the main challenges we wanted to address was evaluation consistency. In a typical setup, a memory system may be evaluated together with a particular answer model, prompt, or judge. That makes direct comparison difficult. A higher score could come from a better memory system — but it could also come from a stronger downstream model or a different evaluation setup. AML tries to separate these components. The core interface for participating memory systems is: Memory System Add → Search The memory system receives long-term history through Add, and returns relevant memories through Search. Then AML handles: AML Evaluation Platform Answer → Eval Answer generation and evaluation are completed by the benchmark platform under the same evaluation process. This helps reduce the impact of different answer models, prompts, judges, and scoring conventions. The goal is simple: Compare memory systems under the same conditions as much as possible. Memory Is More Than Retrieval A memory system shouldn't be judged only by whether it can retrieve something that looks similar. Memory quality is not only about retrieving similar information, but about understanding relevance, context, time, and task requirements. For text memory, AML evaluates multiple capabilities, including: Factual recall Relational and multi-hop reasoning Temporal and event understanding Memory governance Personalization and care Rules and workflow execution Epistemic safety and privacy This matters because real-world agent memory is not just a search problem. An agent may need to understand: What happened before? When did it happen? What changed afterward? Which previous experience is relevant now? Which information should be trusted? How should that memory affect the current task? A useful memory system needs to handle these questions together. The First Results The first AML evaluation has now been completed. 67 representative memory frameworks successfully completed the evaluation across two tracks covering text memory for both open-source methods and commercial products. The complete rankings, scores, and system versions are available on the leaderboard. Leaderboard: [https://agentmemoryleaderboard.ai/leaderboard/industry/textual ] What Comes Next? The first leaderboard is not the finish line. We plan to keep AML running as a long-term evaluation and public leaderboard for agent memory systems. Going forward, we will publish deeper technical analyses of the first-round results, including: Technical architectures that performed well Interesting evaluation patterns Performance across individual memory capabilities Differences between text and code memory What current results tell us about the evolution of agent memory We also want the benchmark itself to evolve. If you are working on AI agents, memory systems, evaluation, or related research, we would love to hear what you think a useful memory benchmark should measure next. Four Evaluation Tracks To better reflect different types of memory systems, AML organizes evaluation into two tracks: Open-source Methods Text Memory Commercial Products Text Memory Each track evaluates systems under the same benchmark framework. Explore AML Leaderboard: [https://agentmemoryleaderboard.ai/leaderboard/industry/textual] GitHub: [https://github.com/AML-memory/agent-memory-leaderboard] Hugging Face Space: [https://huggingface.co/agent-memory-leaderboard] X: [https://x.com/AgentMemoryL/status/2087544165433590240] Contact: contactus@agentmemoryleaderboard.ai The first results are out. Now the real work begins: Make memory measurable. Make progress verifiable.