Run Secondlayer

Two processes: Postgres and one container. No cluster, no queue. Bring your own node, or bundle one.

secondlayer setup

One guided command: generates secrets, writes docker-compose.yml + .env into a target directory (--dir, default cwd — no manual copy-paste), brings the stack up, prints the observer stanza for an external node, restores verified history from the archive, and verifies the result. Without a TTY (or with --yes), it skips the prompts and runs from flags instead:

secondlayer setup --yes \
  --network mainnet --node-mode external \
  --against https://archive.secondlayer.tools/latest.json

--network and --node-mode are always required in that mode — there's no safe default for either. --skip-bootstrap syncs from genesis instead of restoring an archive; --skip-verify skips the post-bootstrap check.

API: http://127.0.0.1:3800. Health: curl http://localhost:3800/health.

Advanced / manual setup

The same five steps setup runs, one at a time — useful for scripting a custom flow or debugging a step in isolation:

secondlayer init --network mainnet
# Copy .env.local secrets into docker/oss/.env

cd docker/oss
docker compose up -d
secondlayer observer --mode indexer --endpoint secondlayer:3700
secondlayer bootstrap --against <manifest>
secondlayer verify all --against <manifest>

docker compose --profile full-node up -d

NODE_MODE is external | stacks | full. full needs BITCOIN_RPC_PASSWORD and runs the full-node profile above. stacks behaves like external — bring your own Stacks node — there's no bundled-stacks-only profile, since a bundled node needs Bitcoin data from somewhere and we don't wire a public RPC default for it.

Six required non-secrets, plus one compose derives for you. Secrets come from secondlayer init.

Unknown keys and contradictions fail before ingest. From a checkout, docker compose -f docker/oss/docker-compose.yml --env-file .env.local up -d brings the stack up; secondlayer setup writes its own compose file and .env and runs the same command for you.

INSTANCE_TOKEN is required in compose: the container process binds 0.0.0.0 — it has to, or the published port never reaches it — and an instance that binds past loopback with no token refuses to start. API_PUBLISH_ADDR is how the API learns which of those two facts to serve on: it carries the publish spec, so with the default 127.0.0.1:3800 the /v1 reads stay keyless, and changing API_PORT to 0.0.0.0:3800 makes every /v1 request carry the token. Writes take the token either way; see Authentication.

secondlayer verify all --against <manifest>
secondlayer repair --against <manifest>          # plan
secondlayer repair --against <manifest> --apply

Verified archive owns flags and exit codes.

Archive trust root

Every archive manifest is checked against one key before bootstrap, verify, or repair act on it. The key comes from, in order: --public-key, ARCHIVE_SIGNING_PUBLIC_KEY in your env, and the key compiled into the CLI release. setup and init write that resolved key into .env so the instance verifies with no path to any Secondlayer host. Plaintext http:// key sources are never consulted, and a signed latest.json pointer has to verify against the same key before its snapshot is followed.

The tradeoff: rotating the archive key means a CLI release until a key ceremony lands. To pin a different archive, replace ARCHIVE_SIGNING_PUBLIC_KEY in .env; a value you set there survives setup re-runs.

docker compose pull && docker compose up -d
curl http://localhost:3800/health

Migrations run at start under an advisory lock, and are forward-only. To roll back, pin the previous image tag: a migrated schema stays migrated, so roll back the image, not the database.

secondlayer backup writes one bundle: the index, plus the keys encrypted under a passphrase you supply.

secondlayer backup --out ./backups/$(date +%F)
secondlayer restore --from ./backups/2026-08-16            # dry run
secondlayer restore --from ./backups/2026-08-16 --apply

Pass --passphrase, or set SECONDLAYER_BACKUP_PASSPHRASE. --no-secrets writes a data-only bundle. A restore refuses before writing anything if the secrets key the instance will use is not the key the data was encrypted with, and --force is what lets it write over a database that already holds chain data.

A restore is all or nothing: pg_restore runs in one transaction and stops on the first error, so a disk that fills mid-load leaves the target as it was rather than half replaced. The tradeoff is no parallel restore. After the load the canonical block range is compared with the manifest's scope; a dump that stops short of what it promised exits 1 instead of reporting success. The dump digest is streamed, so a multi-gigabyte bundle hashes without being read into memory.

The bundle carries INSTANCE_TOKEN, SECONDLAYER_SECRETS_KEY, and STREAMS_SIGNING_PRIVATE_KEY. Losing those costs you more than losing the index, because the index can be rebuilt from the archive, the keys cannot. Store the bundle somewhere your Postgres backups are not.

Confirm any restore with secondlayer verify all --against <manifest> before trusting the result.

Mainnet full history is large: the reference index measures ~500 GB, of which ~250 GB is blocks, transactions, and events and the rest is the decoded plane. Budget roughly 3 GB per 100k blocks for core datasets, or 6 GB with a broad decoder set, and remember those are averages over history that is much denser at the tip.

Postgres max_connections ≥ 50. The bundled-node figure is dominated by the chain itself: Stacks chainstate alone measures 1.3 TB, and bitcoind is additional.

The runtime checks RAM, disk, and max_connections at start and refuses to boot below these floors, so an undersized box finds out before it spends hours acquiring an index it cannot hold. Disk is total filesystem capacity, not free space, since a healthy instance legitimately fills most of its disk. An unmeasurable dimension is reported and skipped rather than assumed bad.

To run below a floor anyway:

SECONDLAYER_ALLOW_UNDERSIZED=true

That downgrades the refusal to a warning at every start. It exists so a box that was already undersized can still come back up after an upgrade: you keep the ability to run your own hardware, you just do it knowingly.

export SECONDLAYER_API_URL=http://localhost:3800
export INSTANCE_TOKEN=…            # from .env.local
secondlayer subgraphs deploy ./my-subgraph.ts

subgraphs deploy is a write, so it sends INSTANCE_TOKEN even here on loopback. Keep POSTGRES_PORT / INDEXER_PORT on loopback. Consumers running elsewhere: Deploy.

• Multi-tenant catalog
• Archive publisher (separate image)
• Pay-per-call metering on your instance (the x402 rail was removed)

Support is community: github.com/ryanwaits/secondlayer.