Turning Your Blockchain Indexer Into a Paid API Without Slowing Down Your Explorer

Your indexer has done the hard part. Every block, transaction, event and balance change on the chain is decoded and stored in Postgres, with indexes that make your explorer fast.

Sooner or later, someone asks the obvious question: can we get this data through an API? Wallets want balances. Analytics teams want transaction history. Trading bots want events the moment they land. Some will pay for it.

The first version is usually easy: point a few REST endpoints at the same database the explorer uses, hand out API keys, done.

Then a single customer writes a loop that pages through every transaction for a busy contract address, ten requests at a time, all day. The explorer's p99 latency climbs. The indexer's writes start competing with those reads for disk and memory, and the indexer falls behind the chain. Now your explorer shows stale blocks, every API customer gets stale data, and none of it shows up as an error anywhere. (It's the same slow-burn pattern I described in Your Blockchain Indexer Is Fine Until It Isn't, this time triggered from outside.)

Exposing indexer data is not an endpoint problem. It's an isolation problem. This post walks through a design that serves blockchain data to anyone, metered by credits and plans, at low latency, without letting that traffic touch the two things you can't afford to slow down: ingestion and your explorer.

The Kitchen and the Counter

A restaurant doesn't let customers walk into the kitchen and cook whatever they like. Customers order at the counter, from a menu, and every dish on the menu has a price that reflects how much work it takes.

Map that onto your system:

• The kitchen is your primary database and indexer. It has one job: keep up with the chain. Nobody outside eats here.
• The counter is your API gateway. It checks who you are, what plan you're on, and whether you've got credits left. It takes orders. It doesn't cook. (If you're choosing what that gateway should be, I compared the options in Demystifying API Gateway Patterns.)
• The menu is a fixed set of endpoints, served by a query service that knows how to cook each dish one way. No arbitrary queries. If it isn't on the menu, you can't order it.
• The prices are credits. A balance lookup is cheap. A page of event history costs more. The price reflects what the request costs you, not just that it happened.

Every section below is one part of that restaurant.

The Architecture at a Glance

chain ──▶ [indexer] ──writes──▶ [primary DB]
                                     │ streaming replication
                  ┌──────────────────┼──────────────────────┐
                  ▼                  ▼                      ▼
        [explorer replicas]  [paid API replica pool]  [free API replica pool]
                  ▲                  ▲                      ▲
             [explorer]              └──────┬───────────────┘
                                            │
                                   [API query service]
                                   endpoint + cursor validation,
                                   query building, per-plan pools,
                                   timeouts, retries, response shaping
                                            ▲
                                      [API gateway] ◀──── clients + API keys
                                   auth, rate limit, credits,
                                   cache, routing
                                            │
                                 [Redis: limits, credits, cache]

Four rules hold this together:

1. The primary is reserved for ingestion and tightly controlled internal workloads. External API and explorer traffic never reads it.
2. The explorer and the API never share a replica. An API spike can't slow the explorer, and vice versa.
3. Every request passes the gateway first. Auth, rate limit, credit check and cache happen before a single query runs.
4. The gateway never talks to the database. A separate query service owns every query: it validates the endpoint and cursor, builds the query, picks the replica pool for the caller's plan, enforces timeouts and shapes the response. The gateway decides whether a request runs. The query service decides how.

That last split keeps the gateway small, fast and stateless, and puts everything that touches SQL in one place you can reason about.

Layer 1: Isolate Reads on Dedicated Replica Pools

Postgres streaming replication gives you read-only copies of the primary. Under healthy conditions they may be only milliseconds or seconds behind, but that lag depends on the workload: how much WAL the indexer generates, the network, and how busy the replica is replaying it while also serving queries. Give the API its own replica pools, separate from the explorer's.

Why this matters in production: read traffic from the API now can't evict the explorer's hot pages from memory or compete with the indexer's writes for disk on the primary. A badly behaved customer can only hurt the replica pool for their plan.

Two settings decide how that replica behaves under pressure, and they pull in opposite directions:

• hot_standby_feedback = on makes the replica tell the primary which row versions its running queries still need. Your API queries stop getting cancelled, but long-running queries on the replica can now prevent vacuum on the primary from cleaning up those row versions. Dead rows pile up and the tables your indexer writes to start to bloat. That's exactly the coupling you were trying to remove.
• hot_standby_feedback = off with a bounded max_standby_streaming_delay keeps the primary clean. The cost: when replaying WAL conflicts with a query on the replica, Postgres waits up to that delay and then cancels the query with canceling statement due to conflict with recovery (SQLSTATE 40001).

For a public API, I'd generally prefer the second option, provided the query service is prepared to handle cancelled reads. Catch that error, retry once on another replica in the same pool, and if that also fails, return a clean 503 with Retry-After instead of an unexplained 500. Pair it with short statement timeouts (Layer 5): a public API shouldn't be running queries long enough to conflict often in the first place. If it is, that's a menu problem, not a replica setting problem.

Replicas also lag. A replica serving a heavy query load can fall behind the primary, and an API that silently serves old data is worse than one that admits it. I've written about replication lag in blockchain indexers separately. For the API, the rule is simple: tell the client how fresh the data is. Return the latest indexed block height with every response. (The Replication & High Availability module in PostgreSQL In-Depth covers how streaming replication, lag and standby conflicts work underneath.)

X-Indexed-Height: 18452231
X-Indexed-At: 2026-10-08T09:14:02Z

Clients can then decide for themselves whether data that's two blocks old is fine for their use case.

Layer 2: Serve From Read Models, Not Raw Tables

The tables your indexer writes are shaped for ingestion. The API needs tables shaped for the questions customers ask. The most common one by far is "show me the transactions for this address, newest first."

Have the indexer maintain a read model built for exactly that question, as part of ingestion:

CREATE TABLE address_transactions (
  address         bytea  NOT NULL,
  ledger_version  bigint NOT NULL,
  tx_hash         bytea  NOT NULL,
  PRIMARY KEY (address, ledger_version)
);

Here ledger_version assumes the chain gives every transaction its own unique, increasing version number, as Aptos-style ledgers do. Block height is not a substitute: a block holds many transactions, so (address, block_height) would collide as soon as an address appears twice in one block. If your chain has no per-transaction version, use a composite position such as (block_height, tx_index) in the key instead. And since one transaction can touch the same address more than once (sender and receiver, or several events), have the indexer insert with ON CONFLICT DO NOTHING, so repeated observations of the same (address, ledger_version) pair collapse into one read-model row.

The primary key index on (address, ledger_version) keeps each address's history in index order, so one address's newest transactions are always a short index range scan, whether the address has 10 transactions or 10 million. The Indexes module in PostgreSQL In-Depth explains why composite key order decides which queries an index can serve.

Then serve history with keyset pagination, never OFFSET:

SELECT ledger_version, tx_hash
FROM address_transactions
WHERE address = $1
  AND ledger_version < $2   -- the cursor from the previous page
ORDER BY ledger_version DESC
LIMIT $3;

OFFSET 50000 makes Postgres read and throw away 50,000 rows before returning the page you asked for, so every page gets slower than the last. A cursor jumps straight to the right spot in the index, so page 1,000 costs the same as page 1. (The pagination post goes deeper on this.) Return the cursor as an opaque string, so clients can't build their own and you can change the format later.

Rule for the whole menu: every endpoint must map to an index-backed access path with a bounded result size. No free-form filters, no "sort by any column", no unbounded block ranges. An events endpoint takes a maximum window of blocks per call. If a customer needs more, they page.

Layer 3: Cache by Finality

Blockchain data has a property most application data doesn't: once a block is considered final by the chain's consensus and finality rules, its canonical data can be treated as immutable for caching. A finalized transaction from last year can be treated as immutable forever.

What "final" means differs a lot between chains. Some finalize a block within seconds and never revert it. Others only become safe after a number of confirmations, and recent blocks can still be reorganized. Define finality per chain, and expose it: the indexer should track the latest finalized height separately from the latest indexed height, and the API should only treat data at or below the finalized height as immutable.

That splits your caching strategy cleanly in two:

That last row matters for transaction-by-hash: a lookup for a transaction that's still pending, or that hasn't been indexed yet, must not be cached as "not found" for an hour.

Why this matters in production: most traffic on a blockchain API is for things that already happened. If finalized responses are served from a CDN or Redis, a large share of requests never reach a replica at all, and those cache hits are where the low latency comes from.

Watch out for one failure mode. When a popular address's cached balance expires, a thousand clients can miss the cache at the same moment and send a thousand identical queries to the replica. Use request coalescing (one in-flight query per key, everyone else waits for its result). The cache stampede module in Redis In-Depth covers the patterns.

Layer 4: Plans, Credits and the Gateway

Now the part that makes this a product: who can call what, how often, and how much.

Two limits, not one

A plan needs two separate limits, because they protect against two different problems:

• Rate limit (requests per second): protects your infrastructure from bursts. A customer with plenty of monthly credits still can't send 5,000 requests in one second.
• Credit quota (per month): protects your business. It's what the customer is paying for.

Credits priced by cost

Charging "one request = one credit" is unfair to you. A balance lookup and a 100-row event scan are not the same amount of work. Price each endpoint by what it actually costs to serve:

These numbers are illustrative. Derive your own from measured query costs, and keep the price fixed per endpoint and page size, decided before the query runs. Customers can then predict their bill, and you never have to run a query just to find out what to charge for it.

Plans

Again, these numbers are for illustration. The structure is what matters: plans differ in more than just credits. Page size, history depth and which replica pool you land on are all levers.

Checking limits without touching the database

The gateway runs this check on every request, so it can't afford a database write per call. Keep both counters in Redis and make the check atomic with a Lua script, so that two simultaneous requests can't both spend the last credit:

-- KEYS[1] = per-second window key, e.g. rl:{keyId}:{unixSecond}
-- KEYS[2] = billing-period credits key, e.g. cr:{keyId}:{yyyymm}
-- ARGV[1] = requests/sec limit,  ARGV[2] = cost of this request
-- ARGV[3] = period quota,        ARGV[4] = unix time the key may expire
local reqs = redis.call('INCR', KEYS[1])
if reqs == 1 then redis.call('EXPIRE', KEYS[1], 2) end
if reqs > tonumber(ARGV[1]) then
  return {0, 'rate_limited'}
end
 
local used = tonumber(redis.call('GET', KEYS[2]) or '0')
if used + tonumber(ARGV[2]) > tonumber(ARGV[3]) then
  return {0, 'quota_exhausted'}
end
 
used = redis.call('INCRBY', KEYS[2], ARGV[2])
if used == tonumber(ARGV[2]) then
  redis.call('EXPIREAT', KEYS[2], ARGV[4])
end
return {1, tonumber(ARGV[3]) - used}

The whole script runs as one atomic step inside Redis, so there's no race between "check the balance" and "spend the credits." The atomic counters and rate limiters module explains why that matters.

Some honest notes about this script, because each one is a product decision, not just an implementation detail:

• Rejected requests count toward the rate window. The counter is incremented before the limit check, so a client hammering at 10x its limit keeps its own window full, and a request that passes the rate check but fails the quota check has still used a rate slot. That's deliberate here: it makes ignoring 429s pointless. If your product should count only admitted requests, decrement on rejection or use a different limiter design.
• The credits key expires at an absolute time, not after a duration. The gateway passes the end of the billing period plus your reconciliation grace window (say, a few days) to EXPIREAT. A relative TTL set on the first request of the month would expire at a different moment for every customer, possibly before billing has read the final count.
• It's a fixed window per second, so a client can fit up to twice its limit into the instant where one second ends and the next begins. If that matters for your infrastructure, use a token bucket instead. The rate limiting and quotas module compares the algorithms.

Redis admits, Postgres bills

Credits represent money, so be precise about who owns what. Redis is the authoritative low-latency counter for admission: it decides, in microseconds, whether this request runs. The durable billing system in Postgres remains the source of truth for what a customer used and owes.

Every minute or so, a background job reads the Redis counters and records usage in a Postgres billing table. Make that sync idempotent (keyed by API key and time window), so a retried sync never double-charges. Same idea as idempotency keys, applied to your own billing.

Then plan for the obvious question: what happens if Redis loses the counters? A failover or restart without persistence resets usage to zero, and every customer suddenly has a full quota. Two safeguards:

• When a credits key is missing mid-period, seed it from the last usage recorded in Postgres before admitting the request, rather than starting from zero.
• Reconcile the two regularly. The reconciliation gap is bounded by the sync interval, assuming the billing sync itself is durable and complete. A failed or delayed sync widens it, so alert when a sync is late. Don't refund or bill from Redis directly.

The API key's plan itself (limits, quota, replica pool) should be cached in the gateway's memory for a short time, so authenticating a request costs zero database queries. Store only a hash of each API key, the same way you'd store a password, and log the key's ID, never the key.

Tell clients where they stand

Every response should include the limits, and every rejection should say why and when to retry:

RateLimit-Policy: "per-second";q=50;w=1
RateLimit: "per-second";r=37;t=1
X-Credits-Remaining: 8421337
X-Credits-Cost: 5

And when a request is rejected:

HTTP/1.1 429 Too Many Requests
RateLimit-Policy: "per-second";q=50;w=1
RateLimit: "per-second";r=0;t=1
Retry-After: 1
 
{"error": "rate_limited"}

RateLimit-Policy describes the limit (a quota q of 50 per window w of 1 second). RateLimit reports the current state (r requests remaining, resetting in t seconds). These come from the IETF HTTPAPI working group's RateLimit header fields draft. It's still a draft, not a published RFC, and earlier draft versions used separate RateLimit-Limit, RateLimit-Remaining and RateLimit-Reset fields, so you'll see both those and the older X-RateLimit-* convention in real APIs. Pick one, document it, and keep it stable. Credits are your own application-level concept, so they keep their own X-Credits-* headers.

Clients that can see their limits back off on their own. Clients that can't just retry harder.

Layer 5: Bulkheads So Free Users Can't Hurt Paying Ones

Plans aren't just billing. They're also a way to keep different kinds of traffic from interfering with each other.

The query service routes each tier to its own replica pool, and connects to each with that tier's own database role and timeout:

CREATE ROLE api_free LOGIN;
ALTER ROLE api_free SET statement_timeout = '2s';
 
CREATE ROLE api_paid LOGIN;
ALTER ROLE api_paid SET statement_timeout = '5s';

Role-level settings apply when a connection logs in. If you use PgBouncer, give each tier its own pool that logs in as that tier's role, and keep each pool small. A pool's configured connection limit is a hard ceiling on how many database sessions that tier can have executing work at the same time, no matter how many requests arrive. Pool sizing has its own traps, covered in Connection Pooling in the Serverless Era and the Connection Pooling Failure Modes module.

Why this matters in production: a viral free-tier app that suddenly sends 100x traffic fills the free pool and starts getting 429s. Paid customers, on a different pool and replica, don't notice. This is the bulkhead pattern from the three layers of failure isolation, applied to pricing tiers.

When the system as a whole is under pressure (replica lag above a threshold, CPU pinned), shed load in plan order: free tier first, then lower paid tiers. Return 503 with Retry-After, and keep the explorer and the indexer out of it entirely. The backpressure and load shedding module covers how to pick those thresholds.

Layer 6: Don't Make Clients Poll for New Blocks

A trading bot that wants every new event will poll /events?from=latest once per block time. A thousand bots doing that means a thousand identical queries per block, all returning the same rows.

Flip it around. The indexer already knows the moment a block is indexed. Have it publish new blocks and events once, to a fan-out layer (Redis Streams, a message broker, or your own WebSocket service), and push them to subscribers. The indexed data is fetched once for the fan-out path, instead of once per subscriber.

Two things to get right:

• Charge for streams per message or per subscription, not per request, or a busy contract's event stream becomes free unlimited data.
• Let clients resume. Every message carries its block height. When a client reconnects, it sends the last height it saw and gets the gap filled from the API, so a dropped connection never means missed events.

Layer 7: Bulk Data Doesn't Belong in the API

Some customers don't want an API. They want everything: every transaction since genesis, to load into their own warehouse. If the only tool you give them is a paginated endpoint, they will page through your entire chain, one request at a time, for days.

Give them a better door: periodic exports of finalized data (for example, daily Parquet files per table) in object storage, sold as their own plan. Exports are produced once, off a replica, during quiet hours, and downloaded any number of times at no cost to your databases.

Observability: Know Who Is Expensive

You can't price or protect what you can't see. Track, per API key:

• Requests and credits spent, by endpoint
• Cache hit rate
• p50/p99 latency and error rate
• Which keys are hitting rate limits or timeouts

Tag queries so a slow one on the replica can be traced back to a customer. Postgres's application_name connection setting, or a comment in the SQL text, can carry the key ID (never the key). When a replica slows down, you want to know in seconds whether it's one customer or everyone.

A Checklist Before You Launch

• Does any external API or explorer traffic still read the primary, or share a replica pool across the two?
• Does the gateway stay out of the database, with every query owned by the query service?
• Does every endpoint map to an index, with a bounded result size and cursor pagination?
• Is "final" defined per chain, and is only finalized data cached as immutable?
• Are replica query cancellations retried and turned into clean 503s?
• Are rate limits and credits checked in Redis, atomically, with zero database queries per request?
• Is the billing sync idempotent, and can lost Redis counters be re-seeded from Postgres?
• Does each tier have its own pool, role and statement timeout?
• Does every response include the indexed height and remaining credits?
• Is there a plan for real-time data that isn't polling, and for bulk data that isn't pagination?

The Takeaway

A blockchain indexer that serves only your explorer is a database. One that serves anyone, on plans, is a product, and a product needs a counter in front of the kitchen.

The design comes down to one idea: external traffic should only ever reach things that are built to absorb it. Replicas instead of the primary. A query service instead of SQL in the gateway. Read models instead of raw tables. Caches instead of replicas, wherever the data is final. Redis instead of the database for every limit check, with Postgres still keeping the books. Separate pools so one tier's spike stays its own problem.

Get that right, and selling your data stops being a risk to the explorer that made it worth buying.