Why your indexer resumes from the same block

Restart a consumer on a quiet feed and it logs the cursor it logged yesterday:

The Stacks chain tip is forty thousand blocks past it. The consumer is caught up anyway. Its cursor marks the last row it stored, and on a quiet feed that can sit far behind the last block it checked.

A consumer's health endpoint shows all three, and the gaps between them explain the log line:

On the chain, they show a marketplace that went quiet:

On a busy feed the three heights move together, so the difference never shows. It only appears when a filter goes quiet, and the backlog is zero either way.

The server already read every block, decoded every Clarity event and filed the results in a sorted index. The consumer asks that index one question every 500 milliseconds:

When the answer is "no rows," the consumer hasn't skipped forty thousand blocks of work. There was no work to skip. A cursor an hour old and one forty thousand blocks old cost the same single seek^¹1. B-tree index on (filter, cursor); the seek is O(log n) over the event count, not the block count, so cursor age never enters the cost., so restarting from 8,637,064 costs what polling from the tip costs.

A checkpoint could record one of two things. A bookmark says where I was looking. A receipt says what I have safely stored. On a busy feed they behave the same; they differ in what each costs when the process dies.

So only the receipt is written durably, in the same transaction as the rows, which is the correctness story of a sink. Persisting the bookmark would mean a disk write every 500 milliseconds on every idle consumer, to remember something the first poll after a restart recomputes for free.

The receipt has a second property: it points at a row you hold. The sink contract spells out what that buys during a reorg:

Rollback deletes above the fork and the receipt rewinds with it, in one transaction. A bookmark parked at the unfinalized tip can point at a block that stops existing.

If you expected the checkpoint to advance through empty ranges, that expectation comes from systems where it should. Subsquid processors and graph-node subgraphs checkpoint the last processed block. Kafka consumers commit an offset for every message. In those systems the client does the walking, and losing your place means redoing real work.

When everything is delivered, the receipt and the bookmark are the same pointer. They split only when a server-side filter makes delivery sparse, which is when the index is doing the walking for you.

The split gives two ways to measure distance from the tip. Event age (tip_height − last_delivered_height) is how old the newest matching event is. On our quiet marketplace it climbs all day , which is a fact about the contract. Backlog (tip_height − scanned_height) is how much ground the consumer still has to check. It falls through a backfill and holds at zero once caught up, and it's what blocks_behind reports.

A dashboard that alerts on event age will page someone about a consumer doing its job. Gate liveness on ok, capacity on blocks_behind, and read last_delivered_height as the age of your data^²2. scanned_height is conservative on purpose: a capped scan claims only the cursor it returned, finalizedOnly consumers stop at the finalized boundary, and a reorg rolls it back below the fork..

• Restart prints the same cursor when the feed is quiet. The cursor moves when delivery moves. Resuming re-checks the empty range in one query.
• A caught-up consumer is silent. No matching events means no output and no checkpoint writes. idle_s and ok on the health endpoint show it's alive.
• blocks_behind falls during a backfill and reads about 0 at the tip, even while event age grows.
• Checkpoint writes share a transaction with your rows. A crash mid-batch leaves neither, and the batch is read again. The sink interface shows how to implement that for your own store, and rollback covers what a reorg deletes.