Self-Hosted BSC Full Node (Internal RPC)

In-house BNB Smart Chain RPC — cheaper than paid providers, keyed, rate-limited, and actually monitored.

Internal RPC

paid BSC providers off the hot path

Per-app keys

401 / 429 at the gateway

~1.5TB

official PBSS snapshot

Out-of-sync alerts

snap-sync aware, not false pages

Overview

I deployed and stabilized a BNB Smart Chain full node for internal RPC. Web3 backends used to call paid public RPC — rate limits, per-request cost, noisy-neighbor latency. The node is the in-house eth/net/web3 endpoint those services talk to.

This was not a one-shot compose up. The box was about 1.7 million blocks behind and lying about being synced. I restored an official ~1.5TB PBSS pruneancient snapshot, fixed Geth’s ancient-path semantics, survived a 6.9TB disk at 100%, put a keyed rate-limited gateway in front of unauthenticated Geth HTTP, and rewrote alerts so snap sync did not page as an outage — and a node that actually fell off tip did.

The problem

Backends depended on third-party BSC RPC. Cost and limits scaled with traffic. A self-hosted node existed but was stale, OOM’d, and paging falsely. Geth HTTP has no API keys — a naked port would let one app starve the node.

Per-request billing and shared rate limits on public RPC
Vendor outage stalled every dapp on that endpoint
eth_syncing said false while eth_blockNumber was stale — snap sync frozen after swap/OOM
Ancient history heading for a full 6.9TB disk
Telegram using eth_blockNumber vs public tip looked like tens of millions of blocks of lag during normal snap sync
Unauthenticated shared RPC: one chatty indexer or a leaked URL could lock tip-follow

The solution

Treat it as production storage and protocol ops, not a Compose file. Confirm sync with the right RPC. Restore the official PBSS snapshot. Cap history with BNB’s prune flags. Keep Geth on loopback. Put a gateway in front with per-app keys and per-key limits. Alert on real lag vs chain tip, not snap-sync artifacts.

Host

Bare metal, Ubuntu, 64GB RAM, 6.9TB RAID

Client

BSC Geth, snap sync, PBSS path scheme, pruneancient so ancient history does not grow without bound

RPC

Geth HTTP only on loopback — eth, net, web3 for internal consumers

Gateway

Reverse proxy with a key per service, per-key rate + burst, 401 / 429 before traffic hits Geth

P2P

Host networking for the chain’s peer port

Alerts

Check eth_syncing first; page when the node is out of sync with public tip after it should be following

Architecture

rpc.txt

Web3 app  →  RPC gateway (API key + rate limit)  →  BSC Geth (localhost)
                 │
                 ├── key per service / team
                 ├── req/s + burst per key
                 └── 401 / 429 instead of melting the node

Alerts: eth_syncing first
  → syncing  → “sync in progress (current/highest)” — not an outage
  → synced   → compare to public tip; page if actually out of sync

What I implemented

Diagnosed a node that reported synced while ~1.7M blocks behind: eth_syncing, eth_blockNumber, peer count, logs, dmesg — snap sync frozen after swap exhaustion
Restored the official BNB Chain PBSS pruneancient snapshot (~1.5TB): resume-safe download, checksum, lz4→tar stream, data owned by the container user
Fixed Geth ancient-path semantics: the client appends /chain to --datadir.ancient, so the flag must be the parent of chain/ — plus empty index stubs and stale trie.journal after a chaindata wipe
Monitored snap sync with eth_syncing, not eth_blockNumber (stays at genesis until the pivot)
Disk at 100%: ancient segments ~4.2TB of 6.9TB; Geth’s low-disk exit looked like a healthy restart loop. --history.blocks is the wrong prune on BSC — used --pruneancient plus transaction/log history flags, then deleted snapshot staging
RPC gateway: Geth HTTP on loopback only; per-app API keys, per-key rate limits, 401/429, rotatable without restarting the node; optional IP allowlist on VPN/VPC
Out-of-sync alerting: cron checks eth_syncing first so snap sync is “in progress,” not a false outage; once following tip, lag vs a public reference pages as actually out of sync

Before / after

Before

Apps on paid / public BSC RPC (one vendor quota)
Or a raw RPC port with no auth
Node 1.7M behind, reporting synced
Swap exhausted, snap sync frozen
Alerts screaming during normal snap sync
Ancient data heading for a full 6.9TB disk

After

Backends call gateway + API key
401 / 429 at the edge; Geth only sees allowed traffic
Rotate or revoke a key without restarting the node
Snapshot + snap sync with the correct APIs
--datadir.ancient parent path (no chain/chain)
--pruneancient + history flags
Alerts: syncing vs actually out of sync vs tip

Lessons

Problem	Cause	Fix
Sync stalled	Swap / OOM	Lower cache and maxpeers; snapshot if the pivot is dead
Ancient path error	Geth appends /chain	--datadir.ancient is the parent of chain/
Disk full (~4TB ancient)	History not pruned	--pruneancient (~90k blocks), not --history.blocks
False out-of-sync pages	eth_blockNumber is 0 in snap sync	Check eth_syncing first
Real lag missed	Alerts treated all lag the same	Once synced, page when behind public tip
Node overloaded by one app	Unauthenticated shared RPC	API key per consumer + per-key 429
Clean-exit restart loop	Disk full or stale trie.journal	Free disk; delete stale journals — don’t blindly wipe chaindata

Results

Internal RPC for Web3 backends — paid public RPC off the hot path
API keys per app — no shared open endpoint; noisy clients isolated with 429
Node operable and disk-stable (pruneancient, not a 4TB ancient grow)
Alerts match real lag vs chain tip: snap-sync progress vs actually out of sync
Runbook: snapshot layout, ancient flag, what not to wipe

Stack

UbuntuDocker ComposeBSC GethPBSSJSON-RPCHAProxyAPI keysRAIDBash

Have a project in mind?
Let's build something together.

Contact me