secondlayer
.md

Streams

The raw firehose, for when you want to decode it yourself. Most people want Index; this is what Index is built on.

Hosted reads are keyed (Keys and billing); on your own box see Self-host. Cold history beyond what your instance holds restores from the signed archive (credits).

You getRaw signed events, replayable, reorgs rewound for you
You writeA consume() loop and your own decoder

Build an indexer on it

events.consume is the checkpointed consumer with automatic reorg rewind; the loop is the one on Index.

indexer.tsts
1await sl.streams.events.consume({
2 types: ["ft_transfer"],
3 fromCursor: await db.loadCheckpoint(),
4 onBatch: async (events, envelope, ctx) => {
5 await db.transaction(async (tx) => {
6 await tx.insertEvents(events);
7 await tx.saveCheckpoint(ctx.cursor);
8 });
9 return ctx.cursor;
10 },
11 onReorg: (reorg) => db.deleteFromHeight(reorg.fork_point_height),
12});

3Null on first run.

7Commits with the rows.

11Delete from the fork height, inclusive.

A consumer , , and . Each page ends with ; hand it back as and the next page starts right after it.

Where to run it: Deploy. Dated changes land in the changelog.

Read it directly

The loop is this call on repeat. A live page from mainnet, trimmed:

ft_transfer
$curl "$SECONDLAYER_API_URL/v1/streams/events?types=ft_transfer&limit=100" \
-H "Authorization: Bearer $SECONDLAYER_API_KEY"

2

Pass the response's next_cursor back as cursor. Filters are exact-match, and event types without a given field don't match.

FilterNarrows to
types, not_typesEvent types, included or excluded
contract_idOne contract
sender, recipientThe two sides of a transfer
asset_identifierOne asset
filtersSeveral named concerns on one cursor (Filters)
from_height, to_heightA block-height range

Full grammar: API reference.

200response.jsonjson
{
"events": [
{
"cursor": "9210757:0",
"block_height": 9210757,
"tx_id": "0x0d70db6001a4f477fa032622a6aa51e570874a1e1586987dae5533aab71b13bc",
"event_type": "ft_transfer",
"contract_id": "SM3VDXK3WZZSA84XXFKAFAF15NNZX32CTSG82JFQ4.sbtc-token",
"payload": {
"amount": "12354",
"sender": "SP4SZE494VC2YC5JYG7AYFQ44F5Q4PYV7DVMDPBG.rewards-pox5-v1",
"recipient": "SP4SZE494VC2YC5JYG7AYFQ44F5Q4PYV7DVMDPBG.stbtc-reserve"
},
"canonical": true,
"finalized": true
}
],
"next_cursor": "9210757:0",
"tip": { "block_height": 9211740, "finalized_height": 9211449, "lag_seconds": 11 },
"reorgs": []
}

Check it

The SDK checks every frame's ed25519 signature before onBatch sees it; pin the key and you trust only the key (Per-frame signatures). To check what you stored, compare it with the signed archive. A divergence reads as it does on Archive.

~/secondlayerexit 0
$secondlayer verify raw --against https://archive.secondlayer.tools/latest.json
✓ Local data matches the archive across 183 ranges.
reference signature verified

Reorg behavior

Reorgs rewind to the fork point and re-deliver, inclusive of the fork block. Delivery is at-least-once, so keep writes idempotent.

  • Only forks at or below your checkpoint roll back; a fork above it has nothing to undo.
  • A rewind deeper than maxRollbackDepth (default 1000 blocks) is refused before anything is deleted.
  • On empty pages the loop polls reorgs.list, so a fork that lands while idle at the tip is caught too. The cost is one extra request per idle poll.
  • A fork at genesis hands onReorg a null cursor.

Cold history

events.replay backfills from the signed bulk dumps, then tails live from the manifest's latest_finalized_cursor, no gap or dupe at the seam:

backfill.tsts
await streams.events.replay({
from: "genesis",
^Covers every window the dumps hold.
async onDumpFile(file, { from }) {
const bytes = await streams.dumps.download(file);
^sha256-verified.
await ingestParquet(bytes, { after: from }); // file-granular, at-least-once: skip rows at or below `from`
},
async onBatch(events, envelope) {
await db.insertEvents(events);
return envelope.next_cursor;
},
});

from: "genesis" covers every window the dumps hold; the manifest's coverage reports that range, which starts where the dump program began, not chain block 1. Pass a cursor for a partial backfill.

Dumps and DuckDB

Plain parquet. Pull a verified range and query it locally:

~/indexer
$secondlayer streams dumps --to ./dump # sha256-verified against the signed manifest
$duckdb -c "SELECT event_type, count(*) AS events
FROM read_parquet('./dump/**/*.parquet')
GROUP BY 1 ORDER BY 2 DESC;"
┌──────────────┬────────┐
│ event_type │ events │
├──────────────┼────────┤
│ print │ 610417 │
│ nft_mint │ 107479 │
│ stx_transfer │ 104690 │
│ ft_transfer │ 40294 │
│ … │ … │
└──────────────┴────────┘

Manifest signature is the root of trust

The bulk manifest is itself ed25519-signed and verified before any per-file hash is trusted. Listing dumps and events.replay() both enforce it; opt out with verifyDumpsManifest: false.

From the CLI

Every read has a secondlayer streams subcommand that prints JSON.

CommandDoes
tipCurrent canonical tip.
eventsOne page of events; takes the REST filters as flags.
by-tx <txId>Every event a transaction emitted.
block-events <heightOrHash>Every event in one block.
canonical <height>Canonical block metadata at a height.
consumeLong-running pull from a cursor, one event per line, until SIGINT or --max-pages.
reorgsRecent reorgs, cursor-paginated.
dumpsDownload finalized parquet dumps to a local dir, sha256-verified.

Bitcoin events (Runes)

chain=bitcoin serves Runes events on their own cursor space.

Event types
rune_etch
rune_mint
rune_transfer
rune_burn
$curl "$SECONDLAYER_API_URL/v1/streams/events?chain=bitcoin&rune=840000:3" \
-H "Authorization: Bearer $SECONDLAYER_API_KEY"

2

  • A bitcoin cursor only resumes with chain=bitcoin set again.
  • Stacks-only filters (contract_id, sender, recipient, asset_identifier, filters, clock) answer 400 here.
  • rune and address are bitcoin-only.
  • The tip and canonical block drop the burn-chain fields Stacks carries, since Bitcoin has no burn chain of its own.
  • Events carry ts, the block's time.

The rune catalog, per-rune activity feed, and balances live on Index; this is the raw event log underneath them.

Index Runes into your own database

events.consume takes chain: "bitcoin" and runs the same checkpointed loop as Build an indexer on it. Rows and cursor commit together, and a Bitcoin reorg deletes from the fork height.

runes-indexer.tsts
await sl.streams.events.consume({
chain: "bitcoin",
types: ["rune_etch", "rune_mint", "rune_transfer", "rune_burn"],
fromCursor: await db.loadCheckpoint(),
onBatch: async (events, envelope, ctx) => {
await db.transaction(async (tx) => {
await tx.insertRuneEvents(events);
await tx.saveCheckpoint(ctx.cursor);
});
return ctx.cursor;
},
onReorg: (reorg) => db.deleteFromHeight(reorg.fork_point_height),
});
  • rune (an id or name) or address narrows the feed.
  • finalizedOnly: true waits out the 6 confirmations instead of handling reorgs.

Filters

One filters object can follow several concerns on one cursor.

Two concerns, one loop

A flat filter is one AND-ed set, so tracking sBTC mints and treasury spends meant two consume loops, two cursors, two checkpoints, or fetching the union and throwing half of it away. filters names each concern and the server ORs them into a single scan:

await sl.streams.events.consume({
filters: {
peg: { types: ["ft_transfer"], assetIdentifier: `${SBTC}::sbtc-token` },
treasury: { types: ["stx_transfer"], sender: TREASURY },
},
on: {
peg: (events) => events.map((event) => event.payload.asset_identifier),
treasury: (events) => events.map((event) => event.payload.sender),
},
});

Each label's declared types narrow its handler, so event.payload is typed without an event_type guard.

A label added to filters is a compile error until on handles it.

  • A label's own events arrive in cursor order.
  • Events of different labels don't interleave within a page.
  • Add onBatch for the whole page when you need strict global order.

One cursor covers all of it. The position is computed over the block's full event set, not the filtered one, so you can add or drop labels between restarts and keep the checkpoint.

Over REST it's one JSON query param, and each event echoes the labels it hit:

$curl -G --data-urlencode 'filters={"peg":{"types":["ft_transfer"]}}' \
-H "Authorization: Bearer $SECONDLAYER_API_KEY" \
"$SECONDLAYER_API_URL/v1/streams/events"

2

200response.jsonjson
{ "cursor": "8054704:12", "event_type": "ft_transfer", "matched": ["peg"] }

SSE takes the same param, so subscribe narrows the same way.

Deliveries are idempotent, so retries never double-count.

  • Fully-finalized pages are served Cache-Control: public, immutable with an ETag (304 on If-None-Match).
  • The moving tip stays short-lived.

Real-time firehose (SSE)

$curl -N "$SECONDLAYER_API_URL/v1/streams/events/stream?types=ft_transfer" \
-H "Authorization: Bearer $SECONDLAYER_API_KEY"

2

GET /v1/streams/events/stream takes the same filters as /events.

ParamDoes
cursor (or from_cursor), from_heightStart position. With none it live-tails from the reorg-clamped tip
pingKeepalive, every 20s

Wire contract: REST API.

Near-real-time, not instant

The server polls roughly every 1.5s from a tip held a couple of blocks back for reorg safety, so events land seconds after they appear on chain.

client.events.subscribe(...) auto-reconnects from the last delivered cursor and verifies per-frame signatures; verification modes are under per-frame signatures. Fan out to your backend instead with Webhooks.

Per-frame signatures

Each frame body is { event, sig, key_id }. The signature is ed25519 over JSON.stringify(event).

The SDK verifies every frame by default, against the key at /public/streams/signing-key. Lenient, strict, skip, and pinned-key modes: SDK.

Edit on GitHub