.md

Self-host

The same product on your Postgres: one container, no cluster, no queue. Bring your own node, or bundle one.

You getEvery product on your box, no meter
You bringDocker, 4 GB RAM, the CLI; a 600 GB SSD for mainnet history

Install the CLI with bun add -g @secondlayer/cli. Sizing: Hardware; Runes adds a Bitcoin node.

Configure

$git clone https://github.com/ryanwaits/secondlayer
$cd secondlayer/docker/oss
$cp .env.example .env
^Set POSTGRES_PASSWORD in .env.
$secondlayer init --network mainnet
^Writes the instance token and signing keys.
$cat .env.local >> .env

Start

$docker compose pull
$docker compose up -d
$curl http://127.0.0.1:3800/health
  • init generates the instance token and signing keys. Appending them puts everything compose reads in one .env.
  • pull fetches the published images. Without it, compose reuses any older copy already on the box.
  • If /health never answers, docker compose logs secondlayer says why, most often a floor.

Use these docs on your box

Every example in these docs reads SECONDLAYER_API_URL (and SECONDLAYER_API_KEY where a header is shown). On your box, point the URL at your own API.

$export SECONDLAYER_API_URL=http://127.0.0.1:3800
  • Reads on a loopback bind need no key. A header carrying an unknown or empty key is ignored there, so the same curl lines work unchanged.
  • Writes over curl, and any read past loopback, send Authorization: Bearer $INSTANCE_TOKEN.
  • The CLI, SDK and MCP send INSTANCE_TOKEN for you on any URL that is not the hosted API.
  • MCP needs the token in the client config's env, not just your shell.

So secondlayer subgraphs deploy needs only the token in your env:

$export INSTANCE_TOKEN=… # from .env.local
$secondlayer subgraphs deploy ./my-subgraph.ts

Leave SECONDLAYER_API_KEY for the hosted account: archive quote, fetch and credits read it as the account key, so setting it to the instance token breaks them.

Keep POSTGRES_PORT and INDEXER_PORT on loopback.

Fill it with history

A new box holds only what your node sends from now on. Restore verified history from the archive:

$export DATABASE_URL=postgres://secondlayer:<POSTGRES_PASSWORD>@127.0.0.1:5432/secondlayer
$secondlayer bootstrap --against https://archive.secondlayer.tools/latest.json
~/secondlayer
$secondlayer bootstrap --against <archive>
coverage genesis → 8,745,422
signature verified
sync from genesis ~19.4 days (estimate)
restore from archive ~2.3 hours

The official archive quotes a price in credits before it fetches. Verify is free.

Check it

Compare what you restored with the signed archive. Free, and read-only.

~/secondlayerexit 0
$secondlayer verify raw --against https://archive.secondlayer.tools/latest.json
✓ Local data matches the archive across 183 ranges.
reference signature verified

Stop it

$docker compose down

The index, chainstate and keys survive. Never add -v: it deletes the index volume.

Ready for real traffic: your node, a pinned version, backups, upgrades. Starting out? Run it locally first with Start it.

Profiles

ProfileWhat runs
defaultpostgres + secondlayer (API, ingest, decoder, subgraph, webhooks)
full-node+ bundled Stacks node and bitcoind
bitcoin+ runes, the Runes indexer

Set them in .env so every docker compose command picks them up:

.envsh
$COMPOSE_PROFILES=full-node
NODE_MODEYou bringAlso set
external (default)Your Stacks node
stacksYour Stacks node, same as external
fullNothing: both nodes run in the boxBITCOIN_RPC_PASSWORD, the full-node profile

There is no bundled Stacks-only mode: a bundled node needs Bitcoin data, and there is no public RPC default for it. Sizing for each: Hardware.

Pin a version

docker compose up runs latest. Pin a commit so a restart never changes what you run:

.envsh
$SECONDLAYER_VERSION=975e0b9
  • Images are ghcr.io/ryanwaits/secondlayer-runtime and secondlayer-bitcoin, tagged by short commit.
  • docker compose up -d --build builds your checkout instead.

Connect your node

With NODE_MODE=external, this entry in your node's Config.toml sends Secondlayer every block.

Config.tomltoml
[[events_observer]]
endpoint = "10.0.0.5:3700"
events_keys = ["*"]
timeout_ms = 500
disable_retries = true

secondlayer observer --mode signer-shared --endpoint 10.0.0.5:3700 prints it.

Fieldindexersigner-sharedEffect
endpointhost:porthost:portWhere the node posts blocks. No scheme.
events_keys["*"]["*"]Every standard event. Unknown keys stop a stock node from starting.
timeout_ms2000500How long the node waits on each delivery.
disable_retriesfalsetruefalse retries until delivered, holding up the node meanwhile.

Node on another host:

SettingValueWhy
INDEXER_PORT in .env10.0.0.5:3700The default 127.0.0.1:3700 is unreachable off-box.
STACKS_NODE_RPC_URL in .envhttp://<node-ip>:20443Polled for the tip when pushes stop.
Firewall3700 open to the node's IP onlyThe observer port takes unauthenticated posts.

Signer on the same node

Use signer-shared, so a stalled Secondlayer can't hold up the signer. Tradeoff: a missed block stays a gap until secondlayer repair pays credits to fetch it.

Config

Six required non-secrets, plus one compose derives for you. Secrets come from secondlayer init.

KeyDefaultNotes
NETWORKmainnetmainnet, testnet, devnet
DATABASE_URLcomposeInjected; one Postgres
NODE_MODEexternalexternal, stacks, full
DATA_DIR/dataHandler + runtime files
API_PORT127.0.0.1:3800Publish spec for the API port
INDEXER_PORT127.0.0.1:3700Publish spec for the observer port
API_PUBLISH_ADDRcomposeInjected from the same API_PORT value the port is published with

Unknown keys and contradictions fail before ingest.

Instance token

The instance token opens your box. init writes it to .env.local.

  • The variable is INSTANCE_TOKEN. It is not the account key from Keys and billing.
  • Never put sk-sl_* in INSTANCE_TOKEN, or the hex token in SECONDLAYER_API_KEY.
SurfaceCredentialBehavior
/v1 on 127.0.0.1noneyour Postgres, no meter
/v1 past loopbackINSTANCE_TOKENsame header on Index, Streams, subgraphs
Writes (/api/subgraphs, /api/webhooks, /api/node)INSTANCE_TOKENloopback included
# Local read: no key
$curl "http://127.0.0.1:3800/v1/index/events?limit=5"
# Write: instance token, from 127.0.0.1 too
$curl -X POST http://127.0.0.1:3800/api/subgraphs/my-balances/reindex \
-H "Authorization: Bearer $INSTANCE_TOKEN"

Loopback is the publish spec (API_PUBLISH_ADDR), not the caller's address. On a keyless read, a wrong key is ignored; a matching token is used.

Compose requires the token, for two reasons:

  • The container process has to bind 0.0.0.0 or the published port never reaches it.
  • An instance that binds past loopback with no token refuses to start.

API_PUBLISH_ADDR is how the API learns which of those two facts to serve on. It carries the publish spec:

  • With the default API_PORT of 127.0.0.1:3800, the /v1 reads stay keyless.
  • Changing API_PORT to 0.0.0.0:3800 makes every /v1 request carry the token.
  • Writes take the token either way.

What differs on your box

AreaOn your box
MeteringNone. Your Postgres, no meter, no rate limit
Subgraph sourcesAny. Hosted takes event filters, contract_call and contract_deploy
Typed ORM schemassecondlayer codegen subgraph needs direct Postgres, so it is self-host only
Handler RPC callsSet STACKS_NODE_RPC_URL on the subgraph processor, pointing at the node you already run
History below your bootstrap floorA metered archive backfill
Webhook limitsAny max_retries and timeout_ms, no rate limits, egress as your network allows
Proof routesWitness, burn and Bitcoin-header routes answer 503 PROOFS_UNAVAILABLE until PROOF_SIDECAR_URL points at a proof sidecar. The Bitcoin transaction route reads the bundled bitcoind (full-node profile, txindex=1) or the node at BITCOIN_RPC_URL, and answers 502 PROOF_SOURCE_ERROR when it can't reach one
Bitcoin output spends/v1/bitcoin/outspend reads the same node and needs Bitcoin Core 24 or later. It answers 503 BITCOIN_UNAVAILABLE until BITCOIN_RPC_URL is set, and 502 BITCOIN_SOURCE_ERROR when it can't read the node
Error reportsNo /v1/feedback. Error bodies link to a GitHub issue instead

Consumers running on another box: Deploy your app.

Verify and repair

$secondlayer verify all --against <manifest>
$secondlayer repair --against <manifest> # plan
$secondlayer repair --against <manifest> --apply

Verified archive owns flags and exit codes.

Archive trust root

Every archive manifest is checked against one key before the CLI acts on it. The key comes from, in order:

  1. --public-key
  2. ARCHIVE_SIGNING_PUBLIC_KEY in your env
  3. The key compiled into the CLI release
  • It applies to bootstrap, verify and repair.
  • init writes the resolved key into .env.local, so the instance verifies with no path to any Secondlayer host.
  • Plaintext http:// key sources are never consulted.
  • A signed latest.json pointer has to verify against the same key before its snapshot is followed.

The tradeoff: rotating the archive key means a CLI release until a key ceremony lands. To pin a different archive:

  • Replace ARCHIVE_SIGNING_PUBLIC_KEY in .env.
  • A value you set there survives init re-runs.

Running other teams' subgraphs

A subgraph handler is code you deploy into the instance, so it runs with the instance's trust. That is fine for your own subgraphs.

To run another team's on one box, give each team its own stack instead of sharing one:

  • Its own Postgres and its own network.
  • Its api and subgraph-processor containers under gVisor (runtime: runsc), as a non-root user.
  • A read-only root filesystem, with bounded memory, CPU and process count.

docker/workload/tenant.compose.yml is the working shape. gVisor can't reach Docker's embedded DNS, so give each stack a static address for Postgres and a plain resolv.conf. The file explains both.

Not in the box

  • Multi-tenant catalog
  • A managed stack: hosted subgraphs and webhooks run on api.secondlayer.tools with an account key
  • Archive publisher (separate image)
  • Pay-per-call metering on your instance (the x402 rail was removed)

Support is community: github.com/ryanwaits/secondlayer.

Index every rune etch, mint, transfer and burn since block 840,000 on your own box, read through the same Runes routes as hosted. Add these lines to .env, pointing at a Bitcoin node with txindex=1:

.envsh
$COMPOSE_PROFILES=bitcoin
$BITCOIN_RPC_URL=http://10.0.0.6:8332
$BITCOIN_RPC_USER=bitcoin
$BITCOIN_RPC_PASSWORD=…
$docker compose pull
$docker compose up -d
$curl "http://127.0.0.1:3800/v1/index/runes/doggotothemoon"

The first sync reads every block from 840,000 and takes about half a day. tip.block_height in each response shows how far it has read. The tradeoff: Runes has no archive yet, so a new box rebuilds from your node instead of downloading, and secondlayer verify covers Stacks only.

Bitcoin node

Node.envNotes
YoursBITCOIN_RPC_URL, BITCOIN_RPC_USER, BITCOIN_RPC_PASSWORDNeeds txindex=1, so not pruned
BundledCOMPOSE_PROFILES=full-node,bitcoin, no BITCOIN_RPC_URLAlso runs the Stacks node; bitcoind syncs first, which takes days

Runes only

No Stacks node? Skip Fill it with history and connect nothing. The Stacks side still runs and stays empty; only runes reads a chain.

Upgrades with Runes

A schema migration on a full Runes database can run for hours on upgrade. The Runes tip holds still until it finishes; that is expected, not a hang.

What a self-hosted box needs, so you size it before the sync starts rather than after.

Mainnet full history is large: the reference index measures ~500 GB.

  • About 250 GB is blocks, transactions, and events. The rest is the decoded plane.
  • Budget roughly 3 GB per 100k blocks for core datasets, or 6 GB with a broad decoder set.
  • Those are averages over history that is much denser at the tip.
SetupRAMDisk
App services, external node, mainnet4 GB600 GB
App services, external node, testnet/devnet4 GB80 GB
Bundled node, mainnet (floor)96 GB2.5 TB
Bundled node, mainnet (recommended)128 GB3 TB NVMe

Postgres

Postgres max_connections ≥ 50.

  • Compose sets the Postgres cache to 1 GB (POSTGRES_SHARED_BUFFERS).
  • For a full-chain sync, give it about 25% of RAM. At the stock 128 MB, inserts wait on index reads from disk and the sync runs slower.
  • The bundled-node figure is dominated by the chain itself: Stacks chainstate alone measures 1.3 TB, and bitcoind is additional.

With Runes

Runes adds to the rows above:

Bitcoin nodeRAMDisk
Yours, elsewhere+22 GB+100 GB
bitcoind on the same box+22 GB+1.1 TB

runes keeps Runes state in memory: 7 to 12 GB while following the tip, up to 17 GB during the first sync, capped at 22 GB.

Floors enforced at start

The runtime checks RAM, disk, and max_connections at start and refuses to boot below these floors, so an undersized box finds out before it spends hours acquiring an index it cannot hold. Disk is total filesystem capacity, not free space, since a healthy instance legitimately fills most of its disk. An unmeasurable dimension is reported and skipped rather than assumed bad.

To run below a floor anyway:

$SECONDLAYER_ALLOW_UNDERSIZED=true

That downgrades the refusal to a warning at every start. It exists so a box that was already undersized can still come back up after an upgrade: you keep the ability to run your own hardware, you just do it knowingly.

Back up first, then move to a newer release.

Backup

secondlayer backup writes one bundle: the index, plus the keys encrypted under a passphrase you supply.

$secondlayer backup --out ./backups/$(date +%F)
$secondlayer restore --from ./backups/2026-08-16 # dry run
$secondlayer restore --from ./backups/2026-08-16 --apply
Flag or envEffect
--passphrase or SECONDLAYER_BACKUP_PASSPHRASEThe passphrase the keys are encrypted under
--no-secretsWrites a data-only bundle
--force (restore)Lets a restore write over a database that already holds chain data

A restore refuses before writing anything if the secrets key the instance will use is not the key the data was encrypted with.

A restore is all or nothing:

  • pg_restore runs in one transaction and stops on the first error, so a disk that fills mid-load leaves the target as it was rather than half replaced.
  • The tradeoff is no parallel restore.
  • After the load the canonical block range is compared with the manifest's scope. A dump that stops short of what it promised exits 1 instead of reporting success.
  • The dump digest is streamed, so a multi-gigabyte bundle hashes without being read into memory.

The bundle carries three keys:

  • INSTANCE_TOKEN
  • SECONDLAYER_SECRETS_KEY
  • STREAMS_SIGNING_PRIVATE_KEY

Losing those costs you more than losing the index. The index can be rebuilt from the archive; the keys cannot. Store the bundle somewhere your Postgres backups are not.

Confirm any restore with secondlayer verify all --against <manifest> before trusting the result. Take a backup before you upgrade.

Upgrade

Move from the commit you run (X) to a later one (Y), keeping the database and the keys.

$curl -s "$SECONDLAYER_API_URL/health"
$docker compose images
$secondlayer --version

One jump to Y; read every changelog entry dated after X first, since that page owns what broke. The CLI is a separate npm version: bump it to match the instance you land on.

The index can be rebuilt from the archive. The keys cannot.

KeepWhy
postgres_data volumechain, subgraph tables, webhook rows
subgraphs_data volumecompiled handlers
SECONDLAYER_SECRETS_KEYdecrypts webhook signing secrets in Postgres
INSTANCE_TOKENcallers; required at boot
STREAMS_SIGNING_PRIVATE_KEY, SECONDLAYER_WEBHOOK_SIGNING_PRIVATE_KEYplatform signature on deliveries
POSTGRES_PASSWORDthe volume already uses it

Losing SECONDLAYER_SECRETS_KEY makes every existing whsec_ unreadable; rotate each webhook rather than minting a new master key (signing secrets). A full dump of a mainnet index takes hours, so a filesystem snapshot of the volume is the faster backup if the host can take one.

Do not wipe the volume

  • docker compose down -v deletes postgres_data.
  • secondlayer init --force mints a new secrets key that leaves existing webhook secrets unreadable.
  • Stop with docker compose down (no -v).
  • Bootstrap is not an upgrade: it refuses a database that already has blocks.

Check out Y for its compose file, pin the same commit, then restart:

$git fetch && git checkout <Y>
# .env: SECONDLAYER_VERSION=<Y>
$docker compose pull
$docker compose up -d --remove-orphans

--remove-orphans stops services Y's compose file no longer lists. Migrations run at boot under an advisory lock, forward-only. Then confirm /health is ok and verify against the archive:

$curl "$SECONDLAYER_API_URL/health"
$secondlayer verify all --against <manifest>

To roll back, pin X again. A migrated schema stays migrated, so roll back the image, not the database.

Edit on GitHub