Skip to content

chore: replace docker-compose with nix process-compose for dev deps#725

Open
vindard wants to merge 3 commits into
mainfrom
chore/nix-process-compose-dev-deps
Open

chore: replace docker-compose with nix process-compose for dev deps#725
vindard wants to merge 3 commits into
mainfrom
chore/nix-process-compose-dev-deps

Conversation

@vindard
Copy link
Copy Markdown
Contributor

@vindard vindard commented May 27, 2026

Summary

Aligns cala's dev tooling with lana-bank's docker-free pattern: contributors no longer need Docker (or podman) installed locally. start-deps now runs Postgres (with pg_stat_statements preloaded) plus the OTel collector directly via process-compose-flake, with data stored under .nix-deps/.

  • Adds process-compose-flake flake input
  • Defines nix-deps-base (process-compose stack: server-pg + setup-db + otel-agent) and setup-db-dev packages
  • Updates nextest-runner and perf-runner to use the new stack instead of podman-compose-runner
  • Drops docker-compose.yml, dev/bin/{clean-deps,docker-compose-up}.sh, and nix/podman-runner.nix
  • Drops docker-compose, podman, podman-compose, wait4x from nativeBuildInputs
  • Updates Makefile start-deps / clean-deps / setup-db to call nix run .#...
  • Switches dev/otel-agent-config.yaml's deprecated logging exporter to debug for compat with otelcol-contrib 0.144+ (the nixpkgs version — the old docker image pinned 0.57.2)
  • Removes Docker from README install instructions

CI is unchanged: both .github/workflows/test-integration.yml / perf-report.yml and ci/pipeline.yml already invoke nix run .#nextest / .#perf, and the runners' internals are what we updated.

Refs

Test plan

  • make start-deps brings up server-pg + otel-agent + auto-applies migrations under .nix-deps/server-pg/
  • cargo nextest run --workspace --locked — all 97 tests pass against the new PG
  • pg_stat_statements extension available (shared_preload_libraries set in postgresql.conf)
  • make clean-deps stops the stack and clears .nix-deps/
  • CI green (test-integration + check-code + perf-report workflows)

🤖 Generated with Claude Code


Note

Medium Risk
es-entity 0.10.37 changes SQL for job polling and entity list pagination, which can affect runtime DB behavior beyond the dev-tooling swap.

Overview
Replaces Docker/podman-based dev dependencies with a Nix + process-compose stack: local Postgres (with pg_stat_statements), automatic sqlx migrations, and the OTel collector run from .nix-deps/. make start-deps, clean-deps, and setup-db now call nix run .#nix-deps-base and .#setup-db-dev; nextest-runner and perf-runner start the same stack instead of compose/podman.

Removes docker-compose.yml, dev/bin/clean-deps.sh, docker-compose-up.sh, and nix/podman-runner.nix, and drops Docker from the README. Updates dev/otel-agent-config.yaml to use the debug exporter (replacing deprecated logging) for current otelcol-contrib.

Bumps es-entity to 0.10.37 and refreshes .sqlx offline metadata for generated job-poller and account/account-set list queries (extra timestamptz bind parameters and revised cursor/pagination predicates).

Reviewed by Cursor Bugbot for commit 31460dc. Bugbot is set up for automated code reviews on this repo. Configure here.

vindard and others added 2 commits May 26, 2026 21:34
es-entity 0.10.37 contains the direction-aware NULL fallback in cursor
WHERE clauses for nullable sort columns (GaloyMoney/es-entity#137) and
the opt-in `nullable` column attribute for non-Option<T> Rust types
(GaloyMoney/es-entity#138).

The macro change in #137 alters the DESC SQL emitted for `Option<...>`
columns — for cala-ledger this affects `Account.external_id` and
`AccountSet.external_id` pagination. New fingerprints replace the old
ones in `cala-ledger/.sqlx/`; semantics are unchanged for non-NULL rows
and now also include NULL rows on page 2+ for DESC (previously dropped).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Align with lana-bank's docker-free dev tooling: contributors no longer need
Docker (or podman) installed. start-deps now runs Postgres (with
pg_stat_statements preloaded) plus the OTel collector directly via
process-compose-flake, with data stored under .nix-deps/.

Drops docker-compose.yml, dev/bin/{clean-deps,docker-compose-up}.sh, and
nix/podman-runner.nix. The otel-agent config switches the deprecated
`logging` exporter to `debug` for compatibility with otelcol-contrib 0.144+
(nixpkgs default).

CI surfaces (.github workflows + Concourse pipeline) are unchanged — both
already invoke nix run .#nextest / .#perf, whose internals now use the
process-compose path.

Refs GaloyMoney/volcano-wip#772.
Comment thread flake.nix
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 27, 2026

📊 Performance Report

Commit: 31460dc
Updated: 2026-05-27 14:17:31 UTC

Cala Performance Benchmark Results (non-representative)

Criterion Benchmark Results (single-threaded)

Benchmark Time per Run Throughput % vs Baseline
post_simple_transaction 4.292ms 232 tx/s 0 (baseline)
post_and_recalculate_ec_account_set 11.299ms 88 tx/s -163.0%
post_and_batch_recalculate_ec_account_set 9.612ms 104 tx/s -123.0%
post_multi_layer_transaction 5.249ms 190 tx/s -22.0%
post_simple_transaction_with_effective_balances 6.583ms 151 tx/s -53.0%
post_simple_transaction_with_skipped_velocity 4.109ms 243 tx/s +4.0%
post_simple_transaction_with_velocity 4.999ms 200 tx/s -16.0%
post_simple_transaction_with_hit_velocity 2.253ms 443 tx/s +47.0%
post_simple_transaction_with_one_account_set 4.827ms 207 tx/s -12.0%
post_simple_transaction_with_five_account_sets 5.431ms 184 tx/s -26.0%
post_simple_transaction_with_ec_account_set 4.088ms 244 tx/s +4.0%

Load Testing Results (parallel-execution)

Scenario tx/s
1 parallel 144.27
2 parallel 233.08
5 parallel 311.14
10 parallel 364.42
20 parallel 345.91
2 contention 197.67
5 contention 252.78
2 acct_sets 179.81
5 acct_sets 214.81

Note: Performance results may vary based on system resources and database state.

Last updated by commit 31460dc

Closes the only gap in per-service readiness coverage: without a probe,
process-compose marks otel-agent as Running but never Ready, so
`is-ready --wait` hangs forever if the collector fails to start (e.g. a
deprecated config option that decoded fine until otelcol 0.144). With the
probe, PC fails the process after 60s and the wait exits non-zero —
fast, self-describing failure instead of CI timeout.

The probe targets the health_check extension already enabled in
dev/otel-agent-config.yaml (default :13133/).
@vindard vindard requested review from a team and nicolasburtey May 27, 2026 14:03
Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 31460dc. Configure here.

Comment thread flake.nix
signal = 2;
timeout_seconds = 10;
};
};
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Race condition: tests may start before migrations complete

Medium Severity

The setup-db process has no readiness_probe, so project is-ready --wait considers it "ready" the moment it enters the Running state — even while migrations are still executing. Both nextest-runner and perf-runner immediately proceed to run tests/queries after is-ready returns, creating a race where the database schema may not yet exist. In practice the otel-agent probe's 2-second initial delay provides a timing buffer, but this is fragile and could cause flaky failures on slow CI machines or as migrations grow.

Additional Locations (2)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 31460dc. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant