The last post ended on a claim: memory for a coding agent is not a search problem, it’s a compilation problem. This post is what that actually means in practice.
Search takes a query and returns the things most similar to it. Compilation takes a query and builds the context an agent needs to do the task, which is a different job with different steps. In Borg, every memory request runs through the same five stages. None of them are hidden. You can see every candidate that was considered and why it was kept or dropped.
Here is the pipeline, start to finish.
Stage 1: Classify the intent
Before retrieving anything, Borg asks what kind of task this is. A debugging query needs different memory than an architecture question, which needs different memory than a compliance check. Debug work wants past episodes and procedures: what happened last time, what fixed it. Architecture work wants semantic facts: the decisions that are still in force. Compliance work excludes procedural memory entirely.
This classification is the lever that makes everything downstream task-shaped instead of generic. A search engine treats every query the same way. Borg decides what category of memory matters before it spends a single retrieval.
Stage 2: Retrieve across profiles
With the intent known, Borg retrieves across more than one memory profile at once: the typed-fact graph, the episodic record of past sessions, and stored procedures. These are not separate databases. They all live in one Postgres, reachable through pgvector for similarity and recursive CTEs for graph traversal.
The point is breadth before narrowing. A debug query pulls graph neighbors and prior episodes together rather than only the chunks that textually resemble the question.
Stage 3: Rank on four dimensions
This is where compilation diverges hardest from search. Instead of one similarity score, every candidate is scored on four separate, orthogonal dimensions:
Relevance, how well it matches the task. Recency, how current it is. Stability, how settled the fact is versus how often it has churned. Provenance, where it came from and how trustworthy that source is.
These four scores are surfaced individually, never mashed into one opaque number. Borg then fuses the candidate sets using reciprocal rank fusion and selects the final set. In an illustrative debug trace, that means going from fourteen candidates down to the six that actually earn a place.
The reason for keeping the dimensions separate is debuggability. When the wrong thing gets compiled, you can see whether it was ranked up on recency, on relevance, or on something else, and tune accordingly.
Stage 4: Compile for the target model
The selected memory is then assembled into a context package shaped for the specific model that asked, fitted to a per-namespace token budget. The output format is chosen by the target model, and the reason is concrete: Claude reasons over structured XML attributes, while GPT prefers compact JSON. Sending XML to GPT wastes tokens on tags it ignores; sending flat JSON to Claude strips metadata Claude would use. So the model parameter picks the format. In the trace, six selected items compile down to roughly 1,840 tokens for Claude.
This is the step RAG skips. RAG hands over raw matches and makes the agent sort them out. Borg hands over a finished, budgeted package, ordered so the most important material is not buried where models tend to ignore it.
Stage 5: Log the trace
Every compilation writes an audit record: what was selected, what was rejected, and why, plus access tracking. The full trace in the example completes in around twelve milliseconds for the logging step.
This matters beyond compliance. The audit log is the debugger. When an agent gets bad context, you do not guess, you open the trace and see exactly which candidate ranked where and what the four scores were.
Why five stages instead of one lookup
A vector lookup is one operation pretending to be a memory system. The five stages exist because each solves a problem the others can’t. Classification makes context task-specific. Multi-profile retrieval makes it complete. Four-dimension ranking makes it current and trustworthy. Compilation makes it usable. Logging makes it inspectable.
Take any stage out and you slide back toward returning results instead of returning context.
The next post steps away from the pipeline to a question that shapes the whole system: why all of this runs on one Postgres instead of the multi-database stacks everyone else assembles.
Part 2 of a 6-part series on building memory for AI coding agents. Next: The case for one Postgres.