Butter

What it does

Butter is an LLM proxy that caches and replays language model responses. It sits between an agent system and an LLM API, intercepting requests and storing responses. When similar requests come in, it returns cached answers instead of calling the API again. The proxy is compatible with the standard chat completions endpoint, so it can replace an existing API base URL with minimal code changes.

The caching mechanism is template-aware, meaning it can treat dynamic content like names and addresses as variables. This allows the cache to match similar requests even when they contain different specific values, and return appropriately customized responses.

Who it is for

The product targets developers building autonomous agent systems that need consistent, repeatable behavior. Agents that call LLMs multiple times—whether orchestrating workflows, running multi-step tasks, or debugging behavior—can waste API calls and produce varying results with each run. Butter appeals to teams building and testing agent systems where determinism matters, and where reducing API costs through deduplication has value.

Pricing

The site does not show prices.

How it stands out

Most LLM caching focuses on token-level savings or simple request matching. Butter's template-aware caching is more sophisticated: it treats variable parts of prompts as slots, so a request about Alice's address and one about Bob's address can share the same cached logic. This reduces redundant API calls when agents repeatedly interact with similar entities or follow the same decision paths.

The proxy approach also reduces friction. Rather than rewriting agent code to handle caching, developers point their LLM calls to Butter's endpoint instead. It works with any LLM framework that accepts a custom base URL.

What a founder should check

A competitor considering this space should verify three things. First, how much API cost and latency savings actually matter to real agent builders. If most users run agents infrequently or don't care about small cost reductions, demand may be limited. Second, whether the template-aware matching really works well across diverse agent workloads. If the cache requires manual tuning per use case, adoption friction rises. Third, whether existing LLM providers will build caching directly into their APIs, making a standalone proxy less valuable. If OpenAI or Claude add sophisticated response caching themselves, Butter's moat narrows quickly.