Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 3 additions & 5 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
## How the upstream version is pulled
- dockerTags in `startos/manifest/index.ts`:
- `mempool/frontend:v<version>`
- `mempool/backend:v<version>`
- Both must be updated together. Sidecar `mariadb` has its own version.
# CLAUDE.md

See [CONTRIBUTING.md](CONTRIBUTING.md) for the doc map and contribution workflow.
43 changes: 31 additions & 12 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,21 +1,40 @@
# Contributing

## Building and Development
This repo packages [Mempool](https://github.com/mempool/mempool) for StartOS.

See the [StartOS Packaging Guide](https://docs.start9.com/packaging/) for complete environment setup and build instructions.
## Documentation — keep it in sync

### Quick Start
- **`README.md`** — what this package is and how it's built (image, volumes, interfaces). For developers and AI assistants.
- **`instructions.md`** — the user-facing instructions packed into the `.s9pk` and shown on the **Instructions** tab in StartOS, for the person running the service.
- **`CONTRIBUTING.md`** — this file.
- **`CLAUDE.md`** — operating rules for AI developers working in this repo.

```bash
# Install dependencies
npm ci
**Any code change that warrants it must update `README.md` and `instructions.md` in the same change** — a new or renamed action, an added or removed volume / port / interface / dependency, a changed default, a new limitation, any altered user-visible behavior. Don't defer: a package that ships with a stale README or stale instructions is not done, even if the code is perfect. Content rules live in the packaging guide: [Writing READMEs](https://docs.start9.com/packaging/writing-readmes.html) and [Writing Service Instructions](https://docs.start9.com/packaging/writing-instructions.html).

## Building

# Build universal package
make
See the [StartOS Packaging Guide](https://docs.start9.com/packaging/) for environment setup, then:

```bash
npm ci # install dependencies
make # build the universal .s9pk
```

## How to Contribute
## Updating the upstream version

Mempool ships as two upstream containers that must move together, plus a MariaDB sidecar that tracks its own release line.

1. Bump both `dockerTag` values in `startos/manifest/index.ts` to the new Mempool release:
- `images.frontend.source.dockerTag` → `mempool/frontend:v<new version>`
- `images.backend.source.dockerTag` → `mempool/backend:v<new version>`
Both must be bumped together; the frontend and backend share a release tag.
2. If MariaDB itself needs a bump, update `images.mariadb.source.dockerTag` independently. MariaDB has its own version cadence and should usually only move on a deliberate decision.
3. Update `version` and `releaseNotes` in the file under `startos/versions/`, renaming it to the new version string. A *new* version file is only needed when the bump carries an `up`/`down` migration, or when you want the old release notes preserved in git history — see [Versions](https://docs.start9.com/packaging/versions.html).
4. Rebuild (`make`), sideload the `.s9pk`, and confirm it starts.
5. Review `README.md` and `instructions.md` for anything the bump changed.

## How to contribute

1. Fork the repository and create a branch from `master`
2. Make your changes
3. Open a pull request to `master`
1. Fork the repository and create a branch from `master`.
2. Make your changes — including the doc updates above.
3. Open a pull request to `master`.
60 changes: 41 additions & 19 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ StartOS-specific files:
3. On install, StartOS creates a **critical task** to select an Electrum indexer for address lookups
4. Mempool will not start until Bitcoin Core (and the selected indexer, and the Lightning backend if configured) report healthy via their StartOS health checks — see [Health Checks](#health-checks)
5. Optionally run "Enable Lightning" for Lightning network data
6. Optionally run "Configure Indexing" to opt in to block-summary, goggles, audit, and/or CPFP indexing
6. Optionally run "Indexing and Performance" to change the performance profile, toggle statistics, or opt in to block-summary, goggles, audit, and/or CPFP indexing

On first install, StartOS auto-generates a 22-character MariaDB password and writes it to `mempool-config.json`. The database is localhost-only.

Expand Down Expand Up @@ -106,16 +106,19 @@ Mempool is configured via `mempool-config.json`, managed by StartOS.

### Written by Actions

| Setting | Action | Notes |
| ------------------------------------------ | ------------------ | ------------------------------------------------------------- |
| `ELECTRUM.HOST` / `.PORT` / `.TLS_ENABLED` | Select Indexer | `fulcrum.startos` or `electrs.startos`, port `50001`, TLS off |
| `LIGHTNING.ENABLED` / `.BACKEND` | Enable Lightning | Backend is `lnd` or `cln` |
| `LND.TLS_CERT_PATH` / `.MACAROON_PATH` | Enable Lightning | Paths under the LND mount |
| `CLIGHTNING.SOCKET` | Enable Lightning | `lightning-rpc` socket under the CLN mount |
| `MEMPOOL.BLOCKS_SUMMARIES_INDEXING` | Configure Indexing | Default off |
| `MEMPOOL.GOGGLES_INDEXING` | Configure Indexing | Default off |
| `MEMPOOL.AUDIT` | Configure Indexing | Default off; requires `BLOCKS_SUMMARIES_INDEXING` |
| `MEMPOOL.CPFP_INDEXING` | Configure Indexing | Default off |
| Setting | Action | Notes |
| ------------------------------------------ | ------------------------ | ------------------------------------------------------------- |
| `ELECTRUM.HOST` / `.PORT` / `.TLS_ENABLED` | Select Indexer | `fulcrum.startos` or `electrs.startos`, port `50001`, TLS off |
| `LIGHTNING.ENABLED` / `.BACKEND` | Enable Lightning | Backend is `lnd` or `cln` |
| `LND.TLS_CERT_PATH` / `.MACAROON_PATH` | Enable Lightning | Paths under the LND mount |
| `CLIGHTNING.SOCKET` | Enable Lightning | `lightning-rpc` socket under the CLN mount |
| `MEMPOOL.POLL_RATE_MS` | Indexing and Performance | `8000` (Low-CPU) / `4000` (Balanced) / `2000` (Responsive) |
| `MEMPOOL.MEMPOOL_BLOCKS_AMOUNT` | Indexing and Performance | `4` (Low-CPU) / `6` (Balanced) / `8` (Responsive) |
| `STATISTICS.ENABLED` | Indexing and Performance | Default on |
| `MEMPOOL.BLOCKS_SUMMARIES_INDEXING` | Indexing and Performance | Default off |
| `MEMPOOL.GOGGLES_INDEXING` | Indexing and Performance | Default off |
| `MEMPOOL.AUDIT` | Indexing and Performance | Default off; requires `BLOCKS_SUMMARIES_INDEXING` |
| `MEMPOOL.CPFP_INDEXING` | Indexing and Performance | Default off |

### Bitcoin Core Requirements

Expand Down Expand Up @@ -158,20 +161,39 @@ Selecting an indexer enables address search and transaction history features. Se

When enabled, configures the `LIGHTNING` and `LND`/`CLIGHTNING` sections of the configuration and mounts the selected Lightning node's volume.

### Configure Indexing
### Indexing and Performance

- **Name:** Configure Indexing
- **Purpose:** Opt in to richer block visualizations and audit data by enabling backend indexing features
- **Name:** Indexing and Performance
- **Purpose:** Tune backend CPU/responsiveness, toggle the statistics service, and opt in to optional indexing features
- **Visibility:** Enabled
- **Availability:** Any status
- **Inputs:** Four independent toggles — Block Summaries Indexing, Goggles Indexing, Block Audit (requires Block Summaries Indexing), CPFP Indexing
- **Inputs:**
- **Performance Profile** — one of `low-cpu` / `balanced` / `responsive`
- **Enable Statistics** — toggle (default on)
- **Block Summaries Indexing** — toggle (default off)
- **Goggles Indexing** — toggle (default off)
- **Block Audit** — toggle (default off; requires Block Summaries Indexing)
- **CPFP Indexing** — toggle (default off)
- **Outputs:** None

All four toggles are off by default, matching upstream. Enabling any of them triggers a historical backfill on the next service restart, which can take several hours and consume significant disk space. Sets the `MEMPOOL.BLOCKS_SUMMARIES_INDEXING`, `MEMPOOL.GOGGLES_INDEXING`, `MEMPOOL.AUDIT`, and `MEMPOOL.CPFP_INDEXING` fields in the configuration.
Sets `MEMPOOL.POLL_RATE_MS`, `MEMPOOL.MEMPOOL_BLOCKS_AMOUNT`, `STATISTICS.ENABLED`, `MEMPOOL.BLOCKS_SUMMARIES_INDEXING`, `MEMPOOL.GOGGLES_INDEXING`, `MEMPOOL.AUDIT`, and `MEMPOOL.CPFP_INDEXING` on the configuration. Changes apply on the next service restart.

**RAM requirement:** The action rejects any combination that enables at least one toggle when the host has less than ~16 GB of total RAM (threshold: 15 GiB). Backend indexing competes with Bitcoin Core's dbcache, the selected Electrum indexer, and any Lightning node, so enabling it on a low-memory device is likely to OOM one of the services in the stack.
**Performance profile.** The Mempool backend recomputes a Rust-based block-template projection on every poll; the cost scales with poll frequency and projection depth, and on healthy nodes this loop is the dominant background CPU consumer. The profile picks both together:

**Heap behavior:** When any toggle is on, the backend's V8 heap is raised on the next restart so indexing has room to work. The formula subtracts a 6 GB reserve for the co-resident stack (Bitcoin Core, the selected indexer, any Lightning node, StartOS) and then takes 1/4 of the remainder, clamped 4–8 GB. A 16 GB box gets a 4 GB heap; a 32 GB box gets ~6.5 GB; ≥40 GB gets the 8 GB max.
| Preset | `POLL_RATE_MS` | `MEMPOOL_BLOCKS_AMOUNT` | Notes |
| ------------------ | -------------- | ----------------------- | ------------------------------------------------- |
| Low-CPU (default) | 8000 | 4 | Recommended for low-power devices |
| Balanced | 4000 | 6 | |
| Responsive | 2000 | 8 | Matches upstream in-source default; highest CPU |

New installs land on Low-CPU. Existing installs are migrated to Low-CPU on upgrade; run the action afterwards to pick Balanced or Responsive if you prefer.

**Statistics.** When on (default, matching upstream), the backend samples mempool throughput at 1 Hz and writes periodic statistics rows to MariaDB to power the dashboard charts. Turning it off stops the sampler and the writes; saves background CPU and disk I/O at the cost of the tx/s + vbytes/s charts.

**Indexing.** All four indexing toggles are off by default, matching upstream. Enabling any of them triggers a historical backfill on the next service restart, which can take several hours and consume significant disk space.

- **RAM requirement:** The action rejects any submission with at least one indexing toggle on when the host has less than ~16 GB of total RAM (threshold: 15 GiB). Backend indexing competes with Bitcoin Core's dbcache, the selected Electrum indexer, and any Lightning node, so enabling it on a low-memory device is likely to OOM one of the services in the stack.
- **Heap behavior:** When any indexing toggle is on, the backend's V8 heap is raised on the next restart so indexing has room to work. The formula subtracts a 6 GB reserve for the co-resident stack (Bitcoin Core, the selected indexer, any Lightning node, StartOS) and then takes 1/4 of the remainder, clamped 4–8 GB. A 16 GB box gets a 4 GB heap; a 32 GB box gets ~6.5 GB; ≥40 GB gets the 8 GB max.

## Backups and Restore

Expand Down Expand Up @@ -291,7 +313,7 @@ startos_managed_env_vars:
actions:
- select-indexer
- enable-lightning
- configure-indexing
- indexing-and-performance
health_checks:
- mariadb: healthcheck.sh (120s grace)
- api: port_listening 8999 (45s grace)
Expand Down
44 changes: 44 additions & 0 deletions instructions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# Mempool

## Documentation

- [Mempool documentation](https://mempool.space/docs/) — upstream API reference, configuration, and feature documentation.

## What you get on StartOS

- A **Web UI** interface — the Mempool block explorer, visualizer, fee estimator, and REST/WebSocket API in one site.
- Address lookup powered by a separate StartOS Electrum indexer (Fulcrum or Electrs).
- An optional **Lightning** explorer that pulls network data from a local LND or Core Lightning node.
- A bundled MariaDB sidecar; you do not configure a database.

## Getting set up

Mempool needs Bitcoin Core, an Electrum-style indexer, and (optionally) a Lightning node to be useful. Install dependencies before or alongside Mempool.

1. Install **Bitcoin Core** if you don't have it. Mempool posts a critical task on Bitcoin Core requiring `txindex` enabled and pruning disabled, with an autoconfig action attached — accept it. The task re-appears any time those conditions stop being met.
2. Install **Fulcrum** (recommended) or **Electrs**.
3. After installing Mempool, run the **Select Indexer** task that appears for Mempool and pick **Fulcrum** or **Electrs**.
4. Optionally install **LND** or **Core Lightning**, then run **Enable Lightning** and pick the backend you want feeding the Lightning tab.
5. Start Mempool. It will wait until Bitcoin Core, the selected indexer, and (if enabled) the Lightning backend are healthy and synced.

## Using Mempool

### Web UI

Open the **Web UI** interface to reach Mempool. The home page shows the live mempool, recent blocks, and fee estimates; use the search bar for transactions, blocks, and (once an indexer is selected) addresses. The **Lightning** tab appears when Enable Lightning is configured against a running LND or Core Lightning node. WebSocket and REST API consumers use the same hostname as the Web UI.

### Actions

- **Select Indexer** — switch the Electrum backend between Fulcrum and Electrs. Mempool's dependency set updates accordingly.
- **Enable Lightning** — choose LND, Core Lightning, or none for the Lightning tab's data source. The selected node is mounted read-only.
- **Indexing and Performance** — tune backend behavior on a single form:
- **Performance Profile** — pick **Low-CPU** (default; polls bitcoind every 8s, projects 4 future blocks), **Balanced** (4s / 6 blocks), or **Responsive** (2s / 8 blocks; highest CPU). The Mempool backend rebuilds its block projection on every poll, so this is the main lever for CPU usage on low-power devices.
- **Enable Statistics** — leave on (default) for the tx/s and vbytes/s dashboard charts; turn off to skip the 1 Hz sampler and periodic MariaDB writes.
- **Indexing toggles** — **Block Summaries Indexing**, **Goggles Indexing**, **Block Audit** (requires Block Summaries), and **CPFP Indexing**. Each trades disk and CPU for richer block visualizations. Enabling any toggle triggers a historical backfill on the next start that can take several hours. The action rejects any submission with an indexing toggle on when the device has less than 16 GB of system RAM.

## Limitations

- **Mainnet only.** Testnet, testnet4, signet, regtest, and Liquid are not available.
- **Electrum backend only.** The Esplora backend is not used.
- **One indexer and one Lightning node at a time.** You cannot run Fulcrum and Electrs, or LND and CLN, simultaneously against Mempool.
- **No paid acceleration, no MaxMind GeoIP, no Redis, no Stratum, no replication** — these upstream features are deliberately disabled.
Loading
Loading