Build vs. buy · Agent memory
Build your own retriever.
Or don't.
On the standard benchmark for agent memory, we score 83.0%— graded by the benchmark's own official judge, in a sealed bundle you can re-verify yourself. And 76.6% out of the box, with no LLM calls at ingest at all. Below: that result, then the controlled test that shows why your agent needs memory in the first place.
LongMemEval-S — the number you can check
500 questions over long multi-session conversations. Graded by the benchmark's own official GPT-4o judge — not by us.
No extraction pipeline. No model API key. No prompts to tune. You point agents at it and it remembers — deterministically, and without your conversations ever leaving your infrastructure to be “understood” by a third-party model. This is the default, not a trial tier.
Switch it on and the same system gains +6.4 points, for roughly a tenth of a cent per conversation session, once, at ingest. An explicit trade you make per workspace — not a hidden dependency you discover in a security review.
The run, the hypotheses, the official judge's transcript, and the code that produced them, under a single Merkle root hash. Most numbers in this category cannot be checked: vendors self-report, on different readers, on different releases of the dataset, with no artifacts attached. Ours is not the highest figure you will see quoted. It is the one you can verify. Check the bundle.
Retrieval: 94.0% hit@10 on a static, CPU-only embedder — no model call per query. Weakest axis, stated plainly: preference-style recall (inferring unstated taste from chit-chat) at 30%. Dataset: LongMemEval-S, 500 questions, 2025/09 cleaned release.
Where MMPM sits: the L2 cache for AI
Your vector database is main memory. MMPM is the L2 cache in front of it — the fast, predictive, verifiable tier that keeps the right context warm before your agent asks. Verify a memory yourself →
Build your own RAG, or use MMPM
| Capability | Build-your-own RAG you own it | MMPM managed |
|---|---|---|
| Simple keyword lookup | Yes | Yes — 100% |
| Multi-hop recall (answer shares no words with the question) | No — 0% in test | Only arm that answered any |
| Verifiable provenance (Merkle proofs) | No | Every atom |
| Knowledge-graph edges (relationships, not chunks) | No | Built in |
| Conflict detection (stale facts flagged) | No | Built in |
| Cross-session persistence | You build & host it | Built in |
| MCP-native — drops into your agent | You wire it | One endpoint |
| Who builds, tunes, hosts & maintains it | You | Managed for you |
| Cost | Engineering time + infra | From $5/mo |
The honest row is the first one: on simple keyword lookups, a RAG you build can match MMPM. Every row below it is what you'd still be missing — or still be maintaining.
What you're paying for
Managed
The retriever you don't build
No chunking, embeddings, vector database, or ops to run. MMPM drops into your agent over a single MCP endpoint — the memory layer is someone else's problem to keep alive.
Verifiable
Every memory, provable
Each atom is sealed in an RFC 6962 Merkle tree — tamper-evident and auditable. You can prove what your agent knew, and when. Keyword retrieval can't offer that.
Predictive
The right context before you ask
Markov spreading activation surfaces facts that share no words with your query — the one capability that beat keyword retrieval in our benchmark, and the reason memory is more than search.
One managed layer, priced to scale
Every tier ships Merkle proofs, Markov prediction, knowledge-graph edges, and MCP-native access.
And separately: why your agent needs memory at all
LongMemEval measures us against the field. This measures whether a memory layer is needed in the first place — a controlled retrieval + answer test (Opus 4.8) on our real 3,716-fact production substrate. Deterministic and reproducible.
Answer accuracy with no memory (or a recency prompt) vs. with MMPM-retrieved context. The no-context control scored 0%, proving answers come from retrieval, not the model.
On questions whose answer shares no words with the query, keyword RAG scored 0/18. MMPM was the only method to answer any (directional; small sample).
On direct keyword lookups (n=48), MMPM and keyword RAG both answered 100%. We report where the baseline wins — it's what makes the rest credible.
Retrieval-side: a recency-maintained prompt surfaced the needed fact 0 of 48 times even at a 32,000-token budget; MMPM surfaced it using about 500 tokens — the same answer on roughly 0.2% of the tokens.
Questions people ask
How does Parametric Memory score on LongMemEval?+
83.0% on LongMemEval-S with typed ingest, and 76.6% out of the box with zero LLM calls at ingest — both graded by the benchmark's own official GPT-4o judge, not by us. Retrieval is 94.0% hit@10 on a static, CPU-only embedder. Every run ships as a sealed bundle containing the run, the hypotheses, the official judge's transcript and the code, under one Merkle root hash, so the result can be independently re-verified.
Why is your number lower than some vendors claim?+
Because ours is checkable. Most figures in this category are self-reported, on different readers, on different releases of the dataset, with no artifacts attached — Mem0's own write-up on memory benchmarks concedes that cross-vendor comparison is 'nearly impossible'. 83.0% is not the highest number you will see quoted; as far as we know, it is the one you can verify. Ask any vendor for the same sealed bundle.
What do I get without configuring anything?+
76.6% on LongMemEval-S, with no extraction pipeline, no model API key and no prompts to tune. Ingest performs zero LLM calls, so it is deterministic and replayable, and your conversations never leave your infrastructure to be processed by a third-party model. Typed ingest — one extraction pass, about a tenth of a cent per session — raises that to 83.0%.
What is Parametric Memory bad at?+
Preference-style recall — inferring unstated user taste from chit-chat — is our weakest axis at 30%; the evidence shares neither keywords nor surface semantics with the question. We have no image support. And we are not an environment or UI indexer: we piloted a web-agent trajectory benchmark, measured that our substrate stores claims rather than inventories of screen contents, and published the negative result rather than chase the number.
Can't I just build this with a vector database?+
For simple keyword lookups, yes — in our benchmark, keyword retrieval tied MMPM at 100%. But that's a retriever you build, tune, host, and keep alive, and it still can't do multi-hop recall, give you Merkle-verifiable provenance, a knowledge graph, or conflict detection. MMPM is all of that, managed, from $5/mo.
So does MMPM actually beat RAG?+
On simple keyword lookups, no — it's a tie (both answered 100%). We say that plainly. MMPM's edge is threefold: multi-hop recall (it was the only method to answer any multi-hop question in our test), cryptographic verifiability, and the fact that it isn't your team's problem to operate.
Is the benchmark run on a real system?+
Yes — on our own production substrate, the same one we run our SaaS on, hardened across many revisions. Not a toy corpus. The numbers are deterministic and reproducible; the harness, probes, and seeds are in the repo.
Why did the no-memory baseline score 0%?+
The facts are private to the substrate, so the model can't know them from training. With no retrieval it correctly refuses rather than guessing — which is exactly why any score above zero is attributable to the memory layer, not the model.
What do I actually get at each price?+
Every tier ships the differentiators — Merkle proofs, Markov prediction, knowledge-graph edges, MCP-native — and scales on atoms and infrastructure: Starter ($5) and Solo ($9) on shared infra, Professional ($29, most popular) and Team ($79) on dedicated infrastructure, Enterprise on custom or self-hosted.
Skip the retriever. Keep the memory.
Verifiable, connected, and predictive memory behind one MCP endpoint — from $5/mo.