Streams

The raw firehose, for when you want to decode it yourself. Most people want Index; this is what Index is built on.

events.consume is the checkpointed consumer with automatic reorg rewind; see Index. Reorgs rewind to the fork point and re-deliver, inclusive of the fork block; delivery is at-least-once, so keep writes idempotent. Only forks at or below your checkpoint roll back (a fork above it has nothing to undo), a rewind deeper than maxRollbackDepth (default 1000 blocks) is refused before anything is deleted, and on empty pages the loop polls reorgs.list so a fork that lands while idle at the tip is caught too, at the cost of one extra request per idle poll. A fork at genesis hands onReorg a null cursor.

For cold history, events.replay backfills from the signed bulk dumps, then tails live from the manifest's latest_finalized_cursor, no gap or dupe at the seam:

await streams.events.replay({
  from: "genesis",
  async onDumpFile(file, { from }) {
    const bytes = await streams.dumps.download(file); // sha256-verified
    await ingestParquet(bytes, { after: from }); // file-granular, at-least-once: skip rows at or below `from`
  },
  async onBatch(events, envelope) {
    await db.insertEvents(events);
    return envelope.next_cursor;
  },
});

from: "genesis" covers every window the dumps hold; the manifest's coverage reports that range, which starts where the dump program began, not chain block 1. Pass a cursor for a partial backfill.

Dumps + DuckDB

Plain parquet. Pull a verified range and query it locally:

secondlayer streams dumps --to ./dump   # sha256-verified against the signed manifest

duckdb -c "SELECT event_type, count(*) AS events
           FROM read_parquet('./dump/**/*.parquet')
           GROUP BY 1 ORDER BY 2 DESC;"

┌──────────────┬────────┐
│  event_type  │ events │
├──────────────┼────────┤
│ print        │ 610417 │
│ nft_mint     │ 107479 │
│ stx_transfer │ 104690 │
│ ft_transfer  │  40294 │
│ …            │      … │
└──────────────┴────────┘

Where to run it: Deploy.

curl "http://127.0.0.1:3800/v1/streams/events?types=ft_transfer&limit=100"

Pass the response's next_cursor back as cursor. Narrow with types, not_types, contract_id, sender, recipient, asset_identifier, filters, and from_height/to_height: exact-match, and event types without a given field don't match. Full grammar: API reference.

Two concerns, one loop

A flat filter is one AND-ed set, so tracking sBTC mints and treasury spends meant two consume loops, two cursors, two checkpoints, or fetching the union and throwing half of it away. filters names each concern and the server ORs them into a single scan:

await sl.streams.events.consume({
  filters: {
    peg: { types: ["ft_transfer"], assetIdentifier: `${SBTC}::sbtc-token` },
    treasury: { types: ["stx_transfer"], sender: TREASURY },
  },
  on: {
    peg: (events) => events.map((e) => e.payload.asset_identifier),
    treasury: (events) => events.map((e) => e.payload.sender),
  },
});

Each label's declared types narrows its handler, so event.payload is typed without an event_type guard, and a label added to filters is a compile error until on handles it. A label's own events arrive in cursor order; events of different labels don't interleave within a page. Add onBatch for the whole page when you need strict global order.

One cursor covers all of it. The position is computed over the block's full event set, not the filtered one, so you can add or drop labels between restarts and keep the checkpoint.

Over REST it's one JSON query param, and each event echoes the labels it hit:

curl -G --data-urlencode 'filters={"peg":{"types":["ft_transfer"]}}' \
  "http://127.0.0.1:3800/v1/streams/events"

{ "cursor": "8054704:12", "event_type": "ft_transfer", "matched": ["peg"] }

SSE takes the same param, so subscribe narrows the same way.

Deliveries are idempotent, so retries never double-count. Fully-finalized pages are served Cache-Control: public, immutable with an ETag (304 on If-None-Match); the moving tip stays short-lived.

curl -N "http://127.0.0.1:3800/v1/streams/events/stream?types=ft_transfer"

GET /v1/streams/events/stream takes the same filters as /events, a start position via cursor (or from_cursor) or from_height, and a ping keepalive every 20s. With none it live-tails from the reorg-clamped tip. Wire contract: REST API.

client.events.subscribe(...) auto-reconnects from the last delivered cursor and verifies per-frame signatures; see the SDK. Fan out to your backend instead with Subscriptions.

Per-frame signatures

Each frame body is { event, sig, key_id }; sig is an ed25519 signature over JSON.stringify(event). The SDK verifies every frame by default against the key at /public/streams/signing-key. Lenient, strict, skip, and pinned-key modes: SDK.

Dated changes land in the changelog.