Why we made LLM memory a URL, not an SDK

Every team building on top of a large language model eventually hits the same wall: the model forgets. Session context evaporates when the connection closes. Long conversations get truncated. Users repeat themselves. And the engineering team starts researching vector databases.

That research path is well worn. Pick Pinecone or pgvector. Design an ingestion pipeline. Chunk documents, embed them, store them. Write retrieval logic. Tune similarity thresholds. Debug why the model retrieved the wrong paragraph from last Tuesday's conversation. Ship it three weeks later and hope it works at scale.

We built Kortexio because we think that path is the wrong default for most conversational products.

The hidden cost of "just add a vector store"

Vector search is a powerful primitive. It is not a memory system. When you embed raw conversation turns and retrieve by cosine similarity, you get chunks — not context. The model receives a bag of loosely related snippets with no narrative structure, no notion of what changed since the last turn, and no guarantee that the most important fact from turn four is still present in turn forty.

The integration cost goes deeper than retrieval quality. You need:

• A new client SDK or HTTP layer in your application
• Embedding model selection and cost management
• Schema design for metadata filters
• Background jobs to re-embed when content changes
• Monitoring for index drift, latency spikes, and empty retrievals

Each of these is a product decision disguised as infrastructure. Teams spend sprint cycles on plumbing before they ship the feature users asked for: an assistant that remembers.

Proxy-shaped architecture

Kortexio sits between your application and your LLM provider as a drop-in replacement for your chat completion endpoint. Same HTTP method. Same request body schema. Same response format. You change one URL.

Behind that URL, Kortexio maintains a compiled session memory — a structured wiki updated after each turn — and injects it into every request before forwarding to your model. Your application code does not call a memory API. It does not manage embeddings. It does not pass retrieval parameters. Memory is ambient.

This is deliberate. The best infrastructure disappears. If integrating memory requires importing a new SDK, refactoring your chat handler, and learning a retrieval DSL, adoption friction kills the feature before it ships.

What you gain by not building

When memory is a URL rather than a subsystem, several things become simpler:

Time to first working prototype. Point your existing OpenAI-compatible client at api.kortexio.io/v1 (chat completions). Send the same messages you already send. Memory works on the second turn without additional code.

Operational surface area. No vector index to size, shard, or back up separately from your application database. No embedding pipeline to monitor. Kortexio compacts session memory automatically as conversations grow.

Contract stability. Your frontend and backend continue to speak the chat completion protocol your team already understands. Swapping providers or models does not require rewriting memory integration.

Agentic tools in the same hop. Because Kortexio already proxies your chat traffic, agentic tool execution and human-in-the-loop confirmation live in the same request path — not in a separate orchestration layer you bolt on later.

When a vector store still makes sense

We are not arguing that embeddings are useless. Document search over a static corpus — support articles, internal wikis, product manuals — is a strong fit for vector retrieval. Kortexio can complement that pattern via MCP integrations and BYOK setups.

But conversational session memory — what the user said five minutes ago, what the assistant promised, what changed in this specific thread — behaves differently from document search. It needs compilation, not chunking. It needs recency and salience, not top-k similarity.

If your product's core loop is multi-turn dialogue, rebuilding a vector pipeline per app is expensive overkill. If your product's core loop is search over millions of documents, you probably need both — and they should not be the same system.

The integration cost in minutes, not weeks

Here is the entire integration for a typical Node.js backend:

const response = await fetch("https://api.kortexio.io/v1/chat/completions", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.KORTEXIO_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    model: "gpt-4o-mini",
    user: userId,
    messages: [{ role: "user", content: userMessage }],
  }),
});

Save the X-Session-Id header from the response. Send it on the next request. That is session memory.

Compare that to standing up pgvector, choosing chunk sizes, writing ingestion workers, and tuning retrieval — before you have validated whether users even want persistent memory in your product.

Conclusion

We made LLM memory a URL because the teams we talk to do not want to become database operators. They want to ship assistants that remember, act, and stay inside their compliance boundaries — without rewriting their stack.

If you are evaluating memory for your next LLM feature, try changing the endpoint first. Build the product. Add a vector store later if you still need one.

Read the quickstart →