Most backend systems get a quiet window somewhere. A maintenance mode, a deploy freeze, a Sunday night when traffic drops and you can safely change something load-bearing. A blockchain indexer doesn't get that window. The chain produces a new block whether your system is ready for it or not, and every second you're not caught up is a second of history you have to make up later, under load, while more blocks keep arriving.
That's the actual engineering problem behind SupraScan, the block explorer and indexing layer for the Supra network. Its job sounds simple: consume every block, parse every transaction, write structured data to PostgreSQL, with no missed blocks, no duplicate processing, and no gaps in the indexed history. The interesting part isn't any single piece of that sentence. It's that all of it has to happen continuously, at production throughput, forever, with the failure modes only showing up once you're running at real scale.
What "no pause button" actually costs you
Three constraints shape everything else about this system.
Throughput. The indexer has processed at 2,000+ transactions per second at peak. Even sustained, non-peak load is enough to make naive designs fall over: a benchmark run measured 110-118 TPS sustained across 10 running instances, and at that load Postgres itself sat at 50% CPU idle. The database wasn't the bottleneck. The RPC layer was, throwing socket-hangup errors and climbing past 600ms latency under the same load. That single number reframes the whole architecture problem: you're not primarily fighting the database, you're fighting how fast you can safely pull data out of the chain node and fan it out to workers.
Ordering, without the safety net most indexers get. Blocks have to land in strict sequence, block N fully processed before block N+1 is trusted. Most chain indexers also have to handle reorgs, the chain deciding retroactively that a block it already gave you doesn't count anymore, and rolling back whatever you'd already written. Supra has instant finality, so that entire class of complexity doesn't exist here. It's a real simplification, and it's worth naming, because it's easy to assume every blockchain indexer needs rollback logic. This one doesn't.
No central authority. Every worker instance is equal. Any worker can crash at any moment, mid-block, mid-write, without warning. Nothing about the design can assume a single instance stays alive to coordinate the rest. That constraint is what makes the rest of this interesting.
The shape of the system
Redis is the coordination layer: lock state, progress tracking, health signals. Postgres is the store of record. The blockchain's RPC endpoint is the source of truth for block data itself.
On startup, each pod goes through a fixed sequence: connect to the database, start the partition-management service (creates the next several days of table partitions automatically, ahead of when the data lands), run any pending one-time jobs, and start polling for the current chain height. Several of these services, and a few others, only run on one pod at a time. Leader election over a Redis heartbeat decides who does TPS calculation, fee calculation, and periodic stats. Every pod, leader or not, does the actual work: pulling blocks, indexing transactions, indexing wallets. The leader role exists to stop duplicate work on shared aggregates, not to gate the core pipeline behind a single instance.
Once running, the main loop is intentionally simple: check whether processing is paused, claim the next batch of block numbers, process them, log throughput, and loop again with almost no delay between iterations. The interesting decisions all live one level down, in how "process them" adapts to how far behind the system currently is.
A system that changes its own strategy under load
The indexer runs in one of two modes, and it switches automatically based on measured lag. When it's within a small number of blocks of the chain tip, it processes one block at a time: every block in the current range gets handled concurrently, one RPC call per block, full write pipeline per block, no batching overhead. Once it falls further behind than that, it switches into a batch mode, pulling a larger group of blocks at once and firing all their RPC calls concurrently, trading a bit of per-block latency for a lot more throughput. If a batch RPC call fails, it doesn't just retry the batch. After a couple of failed attempts it falls back to processing that range one block at a time, trading speed for reliability once concurrency itself looks like the problem.
This is a small design decision that pays for itself constantly: a system that's usually keeping up gets the latency profile of single-block processing, and a system that's falling behind (a slow node, an RPC hiccup, a burst of transaction volume) automatically shifts into a mode built for catching up, without anyone paging on-call to flip a setting.
One transaction, a dozen tables
Here's where the fan-out really shows up. A single indexed transaction isn't one row. Inside the write path, one transaction batch triggers somewhere between 15 and 20 sequential database operations: the core transaction record, an "advanced information" record (payload detail), deletion of any stale associated data from a prior attempt, event records, sender/receiver/fee-payer records, then separate tables for coin transfers, fungible asset transfers, NFT transfers, and automation registration, cancellation, execution, and gas-assessment records where relevant. It's a full relational decomposition of one on-chain event, not a JSON blob dropped into a single column.
That decomposition is what makes the system queryable at all, but it's also where a genuinely subtle constraint shows up. Senders, receivers, and fee-payer tables carry database triggers that decrement a wallet's transaction count on delete, and the code that manages this is explicit about why it can't be parallelized: bulk-deleting across multiple transactions concurrently causes concurrent trigger executions on the same wallet row, which either deadlocks or silently miscounts. A comment in the transaction repository says it plainly: deletions happen one at a time specifically to avoid that. It's not an oversight or unfinished optimization. It's a constraint the team hit, understood, and left documented in the code, because the "obvious" faster version is the one that corrupts data under load.
Failures get quarantined, not swallowed
Any block that fails processing three times doesn't get retried forever inline. It drops into a dead-letter topic, out of the main pipeline's way, and a separate reprocessing service picks it back up in chunks. That service is deliberately run on a single instance in production, not because it wouldn't be nice to parallelize, but because its own claim function has no row-locking mechanism to prevent two pods from grabbing the same message. Rather than build that locking immediately, the honest tradeoff was made to run it single-instance and accept the smaller throughput ceiling on the recovery path, which handles a small fraction of total volume, in exchange for correctness without added complexity.
This is the same instinct that shows up in the leader-election pattern for aggregate services: constrain concurrency exactly where correctness demands it, and leave everything else free to scale horizontally.
A concurrency bug the design didn't originally account for, and a different one still open
The main loop used to claim its next batch of blocks by writing a control key to Redis, then reading it back on the next iteration to know where to pick up. That write-then-read pattern looks harmless in isolation. Under real multi-pod load during a catch-up run, it wasn't. A measured benchmark across 10 pods under lag showed a 4.01x duplication ratio: pods were collectively doing four times the useful work in raw completions, because the shared coordination key was being read after a multi-second processing window had already passed, not claimed atomically at the start of it.
It's worth being precise about what this bug did and didn't do. It didn't corrupt data. Every write in the pipeline is idempotent by design, so overlapping work from two pods processing the same range produced the same end state, not conflicting ones. What it cost was infrastructure efficiency: real compute and real RPC calls spent on collisions instead of throughput, at exactly the moments, catch-up under lag, when throughput matters most. It's also specifically a multi-pod problem: a live check of production mainnet's actual deployment configuration confirmed it currently runs as a single instance, which means single-instance coordination didn't strictly need this fix to be correct in that specific environment. The fix shipped anyway, because any environment or future scale-out that does run multiple instances would otherwise inherit the race immediately: the write-then-read claim was replaced with a single atomic INCRBY-based claim, so two pods reading and writing at once now get distinct, non-overlapping ranges by construction instead of racing through a window between two separate calls.
A different, smaller-scope version of the same class of bug is still open, not fixed. The leader-election check that decides which single pod runs the aggregate services, TPS calculation, fee calculation, periodic stats, uses a check-then-set pattern rather than a single atomic operation, and that carries its own theoretical race during the moment two pods could both see no current leader and both try to claim the role. A proposed fix exists, replacing it with a single atomic conditional set, but it hasn't shipped. It's a smaller blast radius than the block-range race was, duplicate leadership would mean redundant aggregate computation, not duplicate block processing, but it's the same category of gap: understood, scoped, and honestly still sitting there unfixed.
It's also worth naming an alternative that was tried at the transport layer and abandoned, not just tuned. Block delivery in this system doesn't run through a message-streaming platform; an earlier version of the architecture did use one, and it was removed in favor of a simpler, database-backed work queue paired with the same Redis coordination layer described above. That's a real, lived example of the opposite failure mode from the one usually worried about in architecture discussions, not under-engineering, but a genuine willingness to walk back a heavier piece of infrastructure once a simpler one covered the same need with less to operate.
What this actually enables
The dead-letter and reprocess design isn't just a safety net for today's failure modes. It's the same mechanism that absorbed a real production incident when the chain started emitting a new transaction type the indexer's release branch hadn't been updated to handle safely, crashing on unguarded property access exactly on that reprocessing path. Because the recovery path already existed, the fix was a scoped hotfix, not an emergency rebuild.
Having closed the multi-pod race mattered for a specific reason beyond the efficiency it recovered at the time. Supra's MultiVM work means blocks will start carrying transactions from multiple virtual machines simultaneously, which increases both volume and per-transaction complexity. A coordination layer that quietly wasted 2-4x its capacity to collisions would have been a much worse foundation for that next phase than one that doesn't. Fixing it ahead of that was directly in service of scaling for what's coming, not abstract cleanup, and the still-open leader-election race is worth closing for the same forward-looking reason, before a busier, multi-VM future makes any coordination gap more expensive to leave sitting.
What this demonstrates
None of the individual pieces here are exotic. Leader election, adaptive batching, dead-letter queues, and idempotent writes are all known patterns. What's harder to fake is the discipline underneath them: a system that adjusts its own throughput strategy based on measured lag instead of a fixed setting, a data model that decomposes correctly under a documented, hard-won concurrency constraint, and a team that can point to a real, unfixed inefficiency in its own coordination layer and explain exactly why it hasn't caused a correctness problem yet. Production systems that run continuously, with no maintenance window and no single point of coordination, get built by making these tradeoffs explicitly and writing down the ones you haven't closed yet, not by pretending you've already closed all of them.