Self-host
The same product on your Postgres: one container, no cluster, no queue. Bring your own node, or bundle one.
Install the CLI with bun add -g @secondlayer/cli. Sizing: Hardware; Runes adds a Bitcoin node.
Configure
Start
initgenerates the instance token and signing keys. Appending them puts everything compose reads in one.env.pullfetches the published images. Without it, compose reuses any older copy already on the box.- If
/healthnever answers,docker compose logs secondlayersays why, most often a floor.
Use these docs on your box
Every example in these docs reads SECONDLAYER_API_URL (and SECONDLAYER_API_KEY where a header is shown). On your box, point the URL at your own API.
- Reads on a loopback bind need no key. A header carrying an unknown or empty key is ignored there, so the same curl lines work unchanged.
- Writes over curl, and any read past loopback, send
Authorization: Bearer $INSTANCE_TOKEN. - The CLI, SDK and MCP send
INSTANCE_TOKENfor you on any URL that is not the hosted API. - MCP needs the token in the client config's
env, not just your shell.
So secondlayer subgraphs deploy needs only the token in your env:
Leave SECONDLAYER_API_KEY for the hosted account: archive quote, fetch and credits read it as the account key, so setting it to the instance token breaks them.
Keep POSTGRES_PORT and INDEXER_PORT on loopback.
Fill it with history
A new box holds only what your node sends from now on. Restore verified history from the archive:
The official archive quotes a price in credits before it fetches. Verify is free.
Check it
Compare what you restored with the signed archive. Free, and read-only.
Stop it
The index, chainstate and keys survive. Never add -v: it deletes the index volume.
Ready for real traffic: your node, a pinned version, backups, upgrades. Starting out? Run it locally first with Start it.
Profiles
| Profile | What runs |
|---|---|
| default | postgres + secondlayer (API, ingest, decoder, subgraph, webhooks) |
full-node | + bundled Stacks node and bitcoind |
bitcoin | + runes, the Runes indexer |
Set them in .env so every docker compose command picks them up:
NODE_MODE | You bring | Also set |
|---|---|---|
external (default) | Your Stacks node | |
stacks | Your Stacks node, same as external | |
full | Nothing: both nodes run in the box | BITCOIN_RPC_PASSWORD, the full-node profile |
There is no bundled Stacks-only mode: a bundled node needs Bitcoin data, and there is no public RPC default for it. Sizing for each: Hardware.
Pin a version
docker compose up runs latest. Pin a commit so a restart never changes what you run:
- Images are
ghcr.io/ryanwaits/secondlayer-runtimeandsecondlayer-bitcoin, tagged by short commit. docker compose up -d --buildbuilds your checkout instead.
Connect your node
With NODE_MODE=external, this entry in your node's Config.toml sends Secondlayer every block.
secondlayer observer --mode signer-shared --endpoint 10.0.0.5:3700 prints it.
| Field | indexer | signer-shared | Effect |
|---|---|---|---|
endpoint | host:port | host:port | Where the node posts blocks. No scheme. |
events_keys | ["*"] | ["*"] | Every standard event. Unknown keys stop a stock node from starting. |
timeout_ms | 2000 | 500 | How long the node waits on each delivery. |
disable_retries | false | true | false retries until delivered, holding up the node meanwhile. |
Node on another host:
| Setting | Value | Why |
|---|---|---|
INDEXER_PORT in .env | 10.0.0.5:3700 | The default 127.0.0.1:3700 is unreachable off-box. |
STACKS_NODE_RPC_URL in .env | http://<node-ip>:20443 | Polled for the tip when pushes stop. |
| Firewall | 3700 open to the node's IP only | The observer port takes unauthenticated posts. |
Signer on the same node
Use signer-shared, so a stalled Secondlayer can't hold up the signer. Tradeoff: a missed block stays a gap until secondlayer repair pays credits to fetch it.
Config
Six required non-secrets, plus one compose derives for you. Secrets come from secondlayer init.
| Key | Default | Notes |
|---|---|---|
NETWORK | mainnet | mainnet, testnet, devnet |
DATABASE_URL | compose | Injected; one Postgres |
NODE_MODE | external | external, stacks, full |
DATA_DIR | /data | Handler + runtime files |
API_PORT | 127.0.0.1:3800 | Publish spec for the API port |
INDEXER_PORT | 127.0.0.1:3700 | Publish spec for the observer port |
API_PUBLISH_ADDR | compose | Injected from the same API_PORT value the port is published with |
Unknown keys and contradictions fail before ingest.
Instance token
The instance token opens your box. init writes it to .env.local.
- The variable is
INSTANCE_TOKEN. It is not the account key from Keys and billing. - Never put
sk-sl_*inINSTANCE_TOKEN, or the hex token inSECONDLAYER_API_KEY.
| Surface | Credential | Behavior |
|---|---|---|
/v1 on 127.0.0.1 | none | your Postgres, no meter |
/v1 past loopback | INSTANCE_TOKEN | same header on Index, Streams, subgraphs |
Writes (/api/subgraphs, /api/webhooks, /api/node) | INSTANCE_TOKEN | loopback included |
Loopback is the publish spec (API_PUBLISH_ADDR), not the caller's address. On a keyless read, a wrong key is ignored; a matching token is used.
Compose requires the token, for two reasons:
- The container process has to bind
0.0.0.0or the published port never reaches it. - An instance that binds past loopback with no token refuses to start.
API_PUBLISH_ADDR is how the API learns which of those two facts to serve on. It carries the publish spec:
- With the default
API_PORTof127.0.0.1:3800, the/v1reads stay keyless. - Changing
API_PORTto0.0.0.0:3800makes every/v1request carry the token. - Writes take the token either way.
What differs on your box
| Area | On your box |
|---|---|
| Metering | None. Your Postgres, no meter, no rate limit |
| Subgraph sources | Any. Hosted takes event filters, contract_call and contract_deploy |
| Typed ORM schemas | secondlayer codegen subgraph needs direct Postgres, so it is self-host only |
| Handler RPC calls | Set STACKS_NODE_RPC_URL on the subgraph processor, pointing at the node you already run |
| History below your bootstrap floor | A metered archive backfill |
| Webhook limits | Any max_retries and timeout_ms, no rate limits, egress as your network allows |
| Proof routes | Witness, burn and Bitcoin-header routes answer 503 PROOFS_UNAVAILABLE until PROOF_SIDECAR_URL points at a proof sidecar. The Bitcoin transaction route reads the bundled bitcoind (full-node profile, txindex=1) or the node at BITCOIN_RPC_URL, and answers 502 PROOF_SOURCE_ERROR when it can't reach one |
| Bitcoin output spends | /v1/bitcoin/outspend reads the same node and needs Bitcoin Core 24 or later. It answers 503 BITCOIN_UNAVAILABLE until BITCOIN_RPC_URL is set, and 502 BITCOIN_SOURCE_ERROR when it can't read the node |
| Error reports | No /v1/feedback. Error bodies link to a GitHub issue instead |
Consumers running on another box: Deploy your app.
Verify and repair
Verified archive owns flags and exit codes.
Archive trust root
Every archive manifest is checked against one key before the CLI acts on it. The key comes from, in order:
--public-keyARCHIVE_SIGNING_PUBLIC_KEYin your env- The key compiled into the CLI release
- It applies to
bootstrap,verifyandrepair. initwrites the resolved key into.env.local, so the instance verifies with no path to any Secondlayer host.- Plaintext
http://key sources are never consulted. - A signed
latest.jsonpointer has to verify against the same key before its snapshot is followed.
The tradeoff: rotating the archive key means a CLI release until a key ceremony lands. To pin a different archive:
- Replace
ARCHIVE_SIGNING_PUBLIC_KEYin.env. - A value you set there survives
initre-runs.
Running other teams' subgraphs
A subgraph handler is code you deploy into the instance, so it runs with the instance's trust. That is fine for your own subgraphs.
To run another team's on one box, give each team its own stack instead of sharing one:
- Its own Postgres and its own network.
- Its
apiandsubgraph-processorcontainers under gVisor (runtime: runsc), as a non-root user. - A read-only root filesystem, with bounded memory, CPU and process count.
docker/workload/tenant.compose.yml is the working shape. gVisor can't reach Docker's embedded DNS, so give each stack a static address for Postgres and a plain resolv.conf. The file explains both.
Not in the box
- Multi-tenant catalog
- A managed stack: hosted subgraphs and webhooks run on
api.secondlayer.toolswith an account key - Archive publisher (separate image)
- Pay-per-call metering on your instance (the x402 rail was removed)
Support is community: github.com/ryanwaits/secondlayer.
Index every rune etch, mint, transfer and burn since block 840,000 on your own box, read through the same Runes routes as hosted. Add these lines to .env, pointing at a Bitcoin node with txindex=1:
The first sync reads every block from 840,000 and takes about half a day. tip.block_height in each response shows how far it has read. The tradeoff: Runes has no archive yet, so a new box rebuilds from your node instead of downloading, and secondlayer verify covers Stacks only.
Bitcoin node
| Node | .env | Notes |
|---|---|---|
| Yours | BITCOIN_RPC_URL, BITCOIN_RPC_USER, BITCOIN_RPC_PASSWORD | Needs txindex=1, so not pruned |
| Bundled | COMPOSE_PROFILES=full-node,bitcoin, no BITCOIN_RPC_URL | Also runs the Stacks node; bitcoind syncs first, which takes days |
Runes only
No Stacks node? Skip Fill it with history and connect nothing. The Stacks side still runs and stays empty; only runes reads a chain.
Upgrades with Runes
A schema migration on a full Runes database can run for hours on upgrade. The Runes tip holds still until it finishes; that is expected, not a hang.
What a self-hosted box needs, so you size it before the sync starts rather than after.
Mainnet full history is large: the reference index measures ~500 GB.
- About 250 GB is
blocks,transactions, andevents. The rest is the decoded plane. - Budget roughly 3 GB per 100k blocks for core datasets, or 6 GB with a broad decoder set.
- Those are averages over history that is much denser at the tip.
| Setup | RAM | Disk |
|---|---|---|
| App services, external node, mainnet | 4 GB | 600 GB |
| App services, external node, testnet/devnet | 4 GB | 80 GB |
| Bundled node, mainnet (floor) | 96 GB | 2.5 TB |
| Bundled node, mainnet (recommended) | 128 GB | 3 TB NVMe |
Postgres
Postgres max_connections ≥ 50.
- Compose sets the Postgres cache to 1 GB (
POSTGRES_SHARED_BUFFERS). - For a full-chain sync, give it about 25% of RAM. At the stock 128 MB, inserts wait on index reads from disk and the sync runs slower.
- The bundled-node figure is dominated by the chain itself: Stacks chainstate alone measures 1.3 TB, and bitcoind is additional.
With Runes
Runes adds to the rows above:
| Bitcoin node | RAM | Disk |
|---|---|---|
| Yours, elsewhere | +22 GB | +100 GB |
| bitcoind on the same box | +22 GB | +1.1 TB |
runes keeps Runes state in memory: 7 to 12 GB while following the tip, up to 17 GB during the first sync, capped at 22 GB.
Floors enforced at start
The runtime checks RAM, disk, and max_connections at start and refuses to boot below these floors, so an undersized box finds out before it spends hours acquiring an index it cannot hold. Disk is total filesystem capacity, not free space, since a healthy instance legitimately fills most of its disk. An unmeasurable dimension is reported and skipped rather than assumed bad.
To run below a floor anyway:
That downgrades the refusal to a warning at every start. It exists so a box that was already undersized can still come back up after an upgrade: you keep the ability to run your own hardware, you just do it knowingly.
Back up first, then move to a newer release.
Backup
secondlayer backup writes one bundle: the index, plus the keys encrypted under a passphrase you supply.
| Flag or env | Effect |
|---|---|
--passphrase or SECONDLAYER_BACKUP_PASSPHRASE | The passphrase the keys are encrypted under |
--no-secrets | Writes a data-only bundle |
--force (restore) | Lets a restore write over a database that already holds chain data |
A restore refuses before writing anything if the secrets key the instance will use is not the key the data was encrypted with.
A restore is all or nothing:
pg_restoreruns in one transaction and stops on the first error, so a disk that fills mid-load leaves the target as it was rather than half replaced.- The tradeoff is no parallel restore.
- After the load the canonical block range is compared with the manifest's scope. A dump that stops short of what it promised exits 1 instead of reporting success.
- The dump digest is streamed, so a multi-gigabyte bundle hashes without being read into memory.
The bundle carries three keys:
INSTANCE_TOKENSECONDLAYER_SECRETS_KEYSTREAMS_SIGNING_PRIVATE_KEY
Losing those costs you more than losing the index. The index can be rebuilt from the archive; the keys cannot. Store the bundle somewhere your Postgres backups are not.
Confirm any restore with secondlayer verify all --against <manifest> before trusting the result. Take a backup before you upgrade.
Upgrade
Move from the commit you run (X) to a later one (Y), keeping the database and the keys.
One jump to Y; read every changelog entry dated after X first, since that page owns what broke. The CLI is a separate npm version: bump it to match the instance you land on.
The index can be rebuilt from the archive. The keys cannot.
| Keep | Why |
|---|---|
postgres_data volume | chain, subgraph tables, webhook rows |
subgraphs_data volume | compiled handlers |
SECONDLAYER_SECRETS_KEY | decrypts webhook signing secrets in Postgres |
INSTANCE_TOKEN | callers; required at boot |
STREAMS_SIGNING_PRIVATE_KEY, SECONDLAYER_WEBHOOK_SIGNING_PRIVATE_KEY | platform signature on deliveries |
POSTGRES_PASSWORD | the volume already uses it |
Losing SECONDLAYER_SECRETS_KEY makes every existing whsec_ unreadable; rotate each webhook rather than minting a new master key (signing secrets). A full dump of a mainnet index takes hours, so a filesystem snapshot of the volume is the faster backup if the host can take one.
Do not wipe the volume
docker compose down -vdeletespostgres_data.secondlayer init --forcemints a new secrets key that leaves existing webhook secrets unreadable.- Stop with
docker compose down(no-v). - Bootstrap is not an upgrade: it refuses a database that already has blocks.
Check out Y for its compose file, pin the same commit, then restart:
--remove-orphans stops services Y's compose file no longer lists. Migrations run at boot under an advisory lock, forward-only. Then confirm /health is ok and verify against the archive:
To roll back, pin X again. A migrated schema stays migrated, so roll back the image, not the database.
Edit on GitHub