Building TokenGate: a meter that would rather overcharge…

Bias declared up front, same as last time: TokenGate is mine. I designed it, I wrote it, I run it in production, and I have every commercial incentive to tell you it is wonderful. So this piece is specific about what it does, and equally specific about what it still cannot do — there is a section near the end listing the gaps, including one a client walked straight into last month while I watched.

It is a quota management service. Applications call it to define metered resources — LLM tokens, API calls, storage bytes, credits, order placements — attach quota and rate-limit policies to subjects, and atomically check-and-consume against them. That is the whole product. It sounds small. It was not small.

1. The problem: everybody writes the same broken counter#

Every service we run needed limits. Every one of them grew its own. And they grew the same counter, with the same four bugs, independently, in four codebases:

# variant one — you have already counted it before you knew whether to
n = redis.incr(key)
if n > limit:
    raise QuotaExceeded

# variant two — two callers both read 99, both pass, both write 100
n = int(redis.get(key) or 0)
if n >= limit:
    raise QuotaExceeded
redis.incr(key)

Then the subtler ones. TTL-based windows that reset at whatever instant the first request of the month happened to land. No idempotency, so a client retry after a timeout charged twice. No ledger, so when finance asked what a customer used in March there were two answers and no way to choose. And the one that actually costs money: when Redis restarted, the counters came back empty and every customer on the platform got a free month.

A limiter that loses its count on restart is not a limiter. It is a suggestion.

I got tired of reviewing the same pull request. The fifth time, I extracted it.

2. The invariant that decided every other decision#

There is exactly one rule at the centre of TokenGate, and everything else is downstream of it:

Every failure direction may over-count. No failure direction may under-count.

Over-counting denies somebody who should have been allowed. That failure is loud. The customer notices immediately, complains, and you fix it within the hour. Under-counting lets somebody through who should have been stopped. That failure is silent, and if the quota represents money it is revenue you will never bill, discovered — if at all — in a quarterly reconciliation nobody enjoys.

So: when in doubt, deny. Redis unreachable and the policy is money-shaped? Deny. Ambiguous commit where the database may or may not have recorded the write? Assume it did, and deny. Reconciler unsure whether a counter has an uncommitted increment in flight? Leave the counter high and walk away.

Once you commit to that asymmetry, most of the hard design questions answer themselves. It is the single most useful thing I did on this project, and I did it on day one, mostly by accident.

3. Two enforcement modes, because one of them is always a lie#

There is no single correct trade-off between “exact” and “fast”, so TokenGate has both, chosen per policy.

strict — a PostgreSQL transaction with ordered row locks. Exact, durable, survives anything the database survives. This is for money-like budgets, where being wrong by one is being wrong.

fast — Redis-authoritative, with a write-behind ledger and background reconciliation. Microseconds. This is for rate limits, where being wrong by one for a few milliseconds is irrelevant and being slow is fatal.

One detail from the strict path that I am unreasonably fond of:

SELECT ... FROM counters c
WHERE ...
ORDER BY c.policy_id, c.window_id
FOR UPDATE

ORDER BY before FOR UPDATE means rows are locked in sorted order, which makes concurrent batches touching overlapping policy sets deadlock-free by construction. One clause of SQL. An entire class of 3 a.m. incident that now cannot happen.

4. The hot path is a Lua script, and it deliberately runs twice#

A single consume call can carry several resources and hit several policies at once — a daily quota, a monthly quota, a rate limit, all in one request. Which raises the obvious question: what happens when the first two policies allow and the third denies?

If you apply as you go, you have just charged the customer for two resources and refused the request. Every naive limiter does this. It is invisible until someone reconciles.

TokenGate’s fast path is a Lua script that runs two passes:

Evaluate every policy. Apply nothing. If any policy denies, return the denial with zero mutations performed.
Only if everything passed, apply all mutations.

All-or-nothing across every applicable policy, inside a single Redis round trip, using Redis’s single-threaded execution as the atomicity guarantee. No distributed lock. No transaction. No optimistic-retry loop.

The other detail in that script that took an outage to learn: time is an argument.

local now = tonumber(ARGV[2])   -- passed in by the service, never read inside the script

Read the clock inside the script and two API replicas will disagree about which window a request belongs to, right at the boundary. The bug appears for a few milliseconds at midnight UTC and nowhere else. There is exactly one time authority per request, and it is the caller.

5. Reserve and commit, because an LLM will not tell you the price in advance#

This is the case that generic limiter libraries simply do not have an answer for.

You cannot meter an LLM call up front. You do not know the token count until the response comes back. So you get two bad options: charge your estimate and be wrong in both directions forever, or charge after the fact — at which point you have already spent money you may not have had.

TokenGate does neither:

with tg.meter("user_42", {"llm.tokens": 4000}) as m:   # reserve an estimate
    response = call_the_model()
    m.record("llm.tokens", response.usage.total_tokens)   # commit the actual

The reservation is a real hold. It counts against the budget while the work is in flight, so a hundred concurrent callers cannot all reserve the same last 4,000 tokens. On commit, the actual is charged and the unused remainder is released. If the process dies mid-call, a sweeper expires the hold and the budget comes back.

The invariant survives here too: a reservation that cannot be settled fails in the direction of the hold staying put. Better a customer waits for a sweeper than gets budget they did not have.

6. The ledger is append-only, and that is not an aesthetic preference#

usage_ledger is append-only, monthly-partitioned, and exactly-once. Corrections are new rows. Nothing is ever mutated or deleted.

The reason is not purity. It is that “what did we charge this customer in March” must have exactly one answer, and it must still have that answer in a year, after the policy has been rewritten four times.

Partitions are pre-created three months ahead by a daily job. And — this is the part worth copying — that same job shouts if the DEFAULT partition is non-empty. A row landing in the default partition means a partition was missing, which means the automation that pre-creates them failed silently some time ago.

Automating a task and alarming when the automation did not happen are two different pieces of work. Most teams do the first and call it done. The second one is what actually saves you, because silent automation failure is indistinguishable from success right up until the query that needed it.

7. What breaks, and what heals it#

None of this section is anticipated design. Every item is a bug I shipped and then fixed.

Redis loses its counters. Reconciliation pulls Redis back down to ledger truth — the database is the authority, Redis is a fast cache of a derived number.

But reconciliation must never lower a counter with an increment in flight. Otherwise the reconciler reads the ledger, a consume lands, the reconciler writes its stale total, and you have manufactured free quota out of a repair job. There is an in-flight guard — a token set written before every Redis apply and cleared after the PostgreSQL phase — plus a per-key reconciler lock. A counter that is mid-flight is skipped, not corrected. Skipping leaves it too high, which is the safe direction. See §2; it keeps paying out.

The outbox is trimmed by the consumer group’s ack frontier, not by length. Trim by length and a slow consumer silently loses events, and you will find out from a customer. Poison entries are acked and logged individually rather than wedging the whole batch.

Redelivery cannot double-count. The flusher applies deferred counters only for ledger rows it actually inserted, using ON CONFLICT ... RETURNING. An ambiguous commit — the write succeeded but the acknowledgement was lost — replays harmlessly.

I want to be plain about the shape of that list: it is a list of things that were wrong in production. The design did not anticipate them. Injecting failures found them.

8. Scopes, and why no service will ever hold `admin:write`#

API keys carry capability scopes: consume, catalog:*, assignments:*, overrides:*, alerts:*, webhooks:*, keys:*, reporting:read, tenant:read, ops:jobs. A key can never mint a scope it does not itself hold.

admin:write is a superset, and it is worth naming what it actually is: tenant root. It mints keys with any scope — including scopes it does not hold itself — revokes every other key in the tenant, and rewrites every policy the tenant has.

Which is why no service gets one. A metering key sits on the hot path of a busy application: it is in an environment file, in a container image, in a crash dump, in a log line if somebody was careless with a debug statement. Scoped to consume with a subject prefix, a leak is an irritation. Scoped to admin:write, a leak is the end of the tenant.

Admin mutations are additionally authorised per-actor against our house RBAC service, and recorded in an append-only audit trail you can actually read over the API. Which brings me to the rule I would most like other people to steal:

The data plane never calls the authorisation service.

Control-plane operations — create a policy, mint a key — check RBAC. Consume calls do not. An outage in the auth service must never be able to stop metering. Your control plane may depend on other services. Your data plane may not.

9. The part I am most pleased with: the API documents itself to agents#

Two endpoints, both deliberately unauthenticated, both docs-only:

GET /llms.txt — the entire integration contract as plain text.
/mcp/ — the same contract over the Model Context Protocol, streamable HTTP.

claude mcp add --transport http tokengate https://tokengate.rodmena.co.uk/mcp/

One source of truth, rendered twice. The MCP server and the text file cannot drift, because they are the same document.

The reasoning is unsentimental. Most integration work against this API is now performed by coding agents rather than humans reading a docs site. An agent that has to guess your API shape will guess plausibly and wrongly, and you will receive the bug report. An agent that can fetch the contract writes correct code on the first attempt.

OpenAPI is not sufficient for this. OpenAPI tells an agent the shape of POST /v1/consume. It does not tell it that a plan enforces nothing until a subject is assigned, or that assignment is exact-match, or which of two enforcement modes it should pick. That knowledge used to live in my head and get dispensed over chat. Now it is a file, and the file is the same one I would have read from.

10. The stack, and why almost none of it came off a shelf#

PostgreSQL, Python, Redis. Beyond that, the dependency list is mostly our own tools, and each one replaced something generic for a stated reason:

migretti for migrations, never Alembic. SQL-first, with real down sections that actually get exercised. Migrations are the one place I want to read exactly what will run, not a Python DSL’s opinion of it.
supervice supervises the API and worker processes. In the bundle deployment it runs both in one container — no systemd, no supervisord, no shell script with a while true in it.
bulkman for bulkheads, with the circuit breaker explicitly disabled, and resilient-circuit for circuit breaking and retries. Two libraries, one concern each. A bulkhead that also breaks circuits gives you two failure policies interacting in ways neither author tested.
RunFlow runs the periodic jobs — rollups, drift reconciliation, maintenance — as container workflows. The invoker calls back into the service over HTTPS to trigger a job by name, which means database credentials never leave the service. The job runner does not get a connection string; it gets permission to ask. And if RunFlow is unavailable, jobs fall back to running in-process, so the dependency degrades rather than stops.
Our own auth service for per-actor RBAC, subject to §8’s rule about the data plane.

The through-line: every one of these is a thing I can open, read, and fix at 2 a.m. That is not ideology. It is the difference between an outage I can end and an outage I can only report.

11. What it still cannot do — the ugly#

Four real gaps. All four share a shape, and the shape is the interesting part.

There is no aggregate or cross-subject cap. Policy resolution is exact-match on a single subject. There is no wildcard, no prefix, no hierarchy, and — critically — no way to express “and the total across all of these must not exceed X”.

This bit a client last month, and it is worth spelling out because the failure looked like success. They were bounding a third-party API limit that applies per account, by summing per-host limits by hand. Two hosts, two independent policy resolutions, one upstream account. Each host individually compliant. Nothing anywhere bounding the pair. The arithmetic on the whiteboard was correct; the model it described did not correspond to the thing it claimed to bound. That is why it survived review — not because nobody checked, but because checking the numbers is not the same as checking the model.

A plan enforces nothing until a subject is assigned, and assignment is exact-match. So a dynamic subject space — one subject per host, per session, per anything generated at runtime — goes silently unmetered for every subject nobody remembered to assign.

A resource with no policy route is silently unmetered. Same client, same month: real upstream traffic flowing through resources that had never been added to the catalogue. Nothing surfaced it. TokenGate today cannot tell you “here is traffic you are not metering” — it will report perfectly healthy while counting nothing, because from the inside those two states are identical.

Rolling-window quotas do not report their reset time. You discover when capacity returns by being denied and reading the Retry-After. A limit whose reset instant you cannot read without tripping it cannot be planned around.

Now the shape. Every one of those is a failure of absence. The service is loud when it blocks you and silent when it is not watching. Loud failures get fixed within a day, because somebody is shouting. Silent ones survive audits, because the check that would catch them returns an empty result, and an empty result looks exactly like a clean bill of health.

I have adopted one rule out of this, and it is now the first thing I ask of any check I write: a check that has never produced a positive cannot be trusted to produce a negative. Point your query at a case you know should match. If it cannot say yes, it cannot say no. Three of the four gaps above hid behind checks that had never once gone green.

12. What I am not claiming#

Not that it is the fastest. Any latency figure I quoted would be from my own load tests, on my own hardware, with my own access patterns. Benchmark it against yours.
Not that exactly-once is unconditional. It is exactly-once under the failure modes I have injected. I inject the ones I have thought of. §7 is a list of the ones I had not.
Not that it is battle-hardened by time. It has been in production for months, not years, across a modest number of tenants. Some classes of bug only arrive with scale I have not reached.
Not that §12 is complete. Those are the gaps I know about. The whole point of §12 is that the dangerous ones are quiet, and I have no principled reason to believe I have found the last quiet one.

What I will claim is narrower, and I think it is the only claim worth making about a meter: when it is uncertain, it errs in the direction that costs a customer a retry rather than the direction that costs me a number I can never reconstruct. Everything above is just the machinery for keeping that promise under failure.

The Python client is on PyPI as tokengate — sync and async, typed, Apache-2.0. The contract an agent needs lives at /llms.txt. If you point one at it and it writes something wrong, I would genuinely like to know, because that is a documentation bug and I can fix it in an afternoon.