build(nix): add flake for reproducible dev shell and build - #8
Merged
Conversation
Mirrors the nim-libp2p flake setup so contributors can `nix develop` into a pinned Nim/nimble shell and `nix build` to type-check the package against locked deps. - `flake.nix` exposes `packages.default` and `devShells.default` across linux/darwin/windows architectures. - `nix/default.nix` runs `nim c --compileOnly libp2p_mix.nim` with every dep wired in via `--path:` (no install — the build verifies the package compiles). - `nix/deps.nix` is autogenerated from `nimble.lock` by `tools/gen-deps.sh` (ported from libp2p's tooling). - `nimble.lock` is checked in. It is currently generated with `nimble --solver:legacy lock` because the default SAT solver can't resolve the transitive git pins libp2p brings in (boringssl, websock, jwt). The Makefile and CI bake in the legacy flag; drop it once libp2p_mix is pinned by version. - `Makefile` exposes `make deps` / `make build` / `make clean`. - `config.nims` gains `--noNimblePath` (added by `nimble lock`), so direct `nim c` invocations now rely on `nimble.paths`. README has a callout for this. Verified locally: `nix build .#default` succeeds, `nix develop` provides nim 2.2.4 + nimble 0.18.2. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
gmelodie
approved these changes
May 13, 2026
This was referenced Jun 4, 2026
chaitanyaprem
added a commit
that referenced
this pull request
Jun 4, 2026
See #13 — isolating the hypothesis that removing the lockfile alone fixes the Nim source-tree checksum mismatch that's been failing CI since #8 (most recently on chore/bump-libp2p-v2.0.0). Without `nimble.lock`, `nimble setup --localdeps -y` resolves directly from `libp2p_mix.nimble`'s `requires` clauses — no Nim source-tree sha1 to re-validate against a lockfile entry written by a different nimble version. If CI goes green, this validates Option B in #13 and we can land it on the bump branch (and cascade to master + the nix/doc cleanups).
jm-clius
pushed a commit
that referenced
this pull request
Jun 12, 2026
* chore: bump nim-libp2p pin to master HEAD Bumps the nim-libp2p pin from `#d4cd68b91` to master HEAD `#069a05bde492d717fce26e7df8b5b0c4221664f0` so downstream consumers (waku) can pick up the service-discovery propagation fixes (PR #2434 table insertion check, PR #2502 build advert once byte-identical, plus the empty service-rtable fallback) that landed after the prior pin. libp2p_mix itself does not need any of those fixes — the bump exists to let downstreams have a single consistent libp2p version across waku + libp2p_mix + mix-rln-spam-protection-plugin. * Revert "chore: bump nim-libp2p pin to master HEAD" This reverts commit a173abe. * chore: bump nim-libp2p to release/v2.0.0 tip + adapt to v2.0 API - libp2p_mix.nimble: pin libp2p to release/v2.0.0 tip (c43199378). vacp2p/nim-libp2p has not yet published a v2.0.0 git tag, so we SHA-pin to the release branch tip (3 patch commits past the v2.0.0 bump commit). Bump nim minimum to >= 2.2.4 to match v2.0.0's transitive requirement. - libp2p_mix/{entry,exit,reply}_connection.nim: method write* override parameter switched to `sink seq[byte]` to match libp2p v2.0.0's LPStream base method signature. - libp2p_mix/pool.nim: write peer addresses with AddressConfidence.Infinite. libp2p v2.0.0 introduced AddressBook auto-prune (Medium TTL = 1h, the default for legacy [] = assignment) which would silently evict our curated mix-pool addresses after 1h. - .github/workflows/{ci,examples}.yml: drop --solver:legacy. vnext SAT now resolves cleanly with v2.0.0's URL+range transitive deps. - nimble.lock: regenerated against v2.0.0; jwt entry removed (libp2p v2.0.0 replaces nim-jwt with an internal JWS signer); websock bumped to a clean v0.4.0+ commit (no longer URL+SHA-pinned by libp2p). * ci: align install recipe with logos-delivery + drop incompatible Nim 2.0.16 `nimble install -dy` strictly re-validates every lockfile entry, which trips on the `nim` package's source-tarball checksum drifting between the machine that ran `nimble lock` and CI runners. Switch to `nimble setup --localdeps -y`, the recipe used by logos-delivery (waku) — it provisions into a project-local nimbledeps/ and is tolerant of that drift. Also drop the Nim 2.0.16 matrix row: libp2p v2.0.0 transitively requires nim >= 2.2.4, so 2.0.16 jobs can no longer resolve. See #13 for the broader nimble.lock decision. * fix: align with new move semantics for libp2p write Propagate `sink seq[byte]` + `move()` through the mix send path so the hot loop avoids a copy on every message. Without this, the libp2p v2.0.0 `Connection.write*` method takes `sink seq[byte]` at the override boundary, but every internal proc the body forwards to (`MixDialer`, `mixDialer` lambda, `prepareMsgWithSurbs`, `buildMessage`, `MixMessage. init`, `anonymizeLocalProtocolSend`, `MixReplyDialer`) still takes a plain `seq[byte]`, forcing Nim to copy the message back into copyable storage at the very next call. Files touched: - entry_connection.nim — `MixDialer` proc type, `write` call, `mixDialer` lambda + its call into `anonymizeLocalProtocolSend` - exit_layer.nim — `replyDialerCbFactory` lambda's `msg` param - mix_message.nim — add sink-taking `init` overload alongside the existing openArray one - mix_protocol.nim — `prepareMsgWithSurbs`, `buildMessage` (incl. the `MixMessage.init` call), `anonymizeLocalProtocolSend`, and the `msgWithSurbs`/`buildMessage` call sites (`let` → `var` so the `move()` is well-formed) - reply_connection.nim — `MixReplyDialer` proc type, `write` call Validated by rebuilding wakunode2 + chat2mix against the patched mix under logos-delivery's `nimbledeps/`. Both link cleanly. Mirrors the move-semantics portion of #11's commit fb2c675, with the readSpec-feature changes from cf38f48/853e80d intentionally left out (those belong to a separate feature PR). * experiment: drop nimble.lock to test no-lockfile install path See #13 — isolating the hypothesis that removing the lockfile alone fixes the Nim source-tree checksum mismatch that's been failing CI since #8 (most recently on chore/bump-libp2p-v2.0.0). Without `nimble.lock`, `nimble setup --localdeps -y` resolves directly from `libp2p_mix.nimble`'s `requires` clauses — no Nim source-tree sha1 to re-validate against a lockfile entry written by a different nimble version. If CI goes green, this validates Option B in #13 and we can land it on the bump branch (and cascade to master + the nix/doc cleanups). * chore: document nimble.lock as a build artefact Follow-up to the lockfile deletion: add `nimble.lock` to `.gitignore`, explain the lockfile/nix/deps.nix relationship in the Makefile target comment, and rewrite the README "Nix" section so devs understand the new flow: libp2p_mix.nimble → nimble.lock (gitignored, ephemeral) → nix/deps.nix (committed, long-lived pin) CI is not affected — it never consumed nimble.lock once we switched to `nimble setup --localdeps -y`. Only the maintainer flow for refreshing nix/deps.nix touches nimble.lock now, and `make deps` regenerates it on demand from libp2p_mix.nimble. Closes #13 * Build/add missing nix inputs (#15) * build: adds missing nix build inputs * chore: updates deps.nix * chore: updates README * docs(makefile): clarify ./tools/gen-deps.sh path in comment Addresses Copilot review nit on PR #14 — the comment referenced `gen-deps.sh` while the actual invoked path is `./tools/gen-deps.sh`. * ci: matrix to Nim 2.2.4 + 2.2.10 Replaces the single 2.2.6 entry with two: 2.2.4 (the libp2p v2.0.0 transitive floor; what this PR was validated against locally) and 2.2.10 (current 2.2.x, ships nimble 0.22.3 which is what most contributors have on their host machines). Net job count unchanged (3 platforms × 2 nim versions = 6), but coverage now spans both the floor and the nimble-0.22 line — which is precisely the gap that produced the original `nimble.lock` checksum mismatch in #13. * ci: add nix-build job (paths-filtered + weekly cron) Adds `.github/workflows/ci-nix.yml` to validate the nix build path without taxing every code-only PR. Triggers: - pull_request, but only when nix-relevant files change (libp2p_mix.nimble, nix/**, flake.nix, flake.lock, Makefile, tools/gen-deps.sh, this workflow itself); - push to master as a post-merge smoke test; - weekly cron (Mon 06:00 UTC) to surface external drift (nimble releases, nixpkgs bumps, upstream rev moves) that no in-repo change would otherwise reveal; - workflow_dispatch for ad-hoc validation after libp2p pin bumps. Steps: 1. Install Nix via DeterminateSystems installer + magic cache. 2. Run `make deps` inside `nix develop`, then `git diff --exit-code nix/deps.nix` — fails the job if the committed snapshot is stale (catches a libp2p_mix.nimble bump that forgot to refresh deps.nix). 3. `nix build -L` to validate the snapshot actually resolves. Net per-PR cost: ~0 min for typical *.nim PRs (path filter skips the job entirely), ~5 min for the rare PR touching nix infra. * chore: pin libp2p v2.0.0 + review fixes --------- Co-authored-by: Marcin Czenko <marcin.czenko@pm.me>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Mirrors the nim-libp2p flake setup so contributors can
nix developinto a pinned Nim/nimble shell andnix buildto type-check the package against locked deps. Addresses Gabriel's feedback that nix flake was not migrated.flake.nixexposespackages.defaultanddevShells.defaultacross linux/darwin/windows architectures.nix/default.nixrunsnim c --compileOnly libp2p_mix.nimwith all deps wired via--path:.nix/deps.nixis autogenerated fromnimble.lockbytools/gen-deps.sh(ported from libp2p's tooling).nimble.lockis checked in, generated vianimble --solver:legacy lock(default SAT solver can't resolve the transitive git pins libp2p currently brings in).Makefileexposesmake deps/make build/make clean.config.nimsgains--noNimblePath(added bynimble lock); README callout explains.Test plan
nix build .#defaultsucceeds locally (257875 lines; 8.058s; SuccessX)nix developdrops into a shell with Nim 2.2.4 + nimble 0.18.2nix flake checkpasses./tools/gen-deps.sh -hruns (shebang on line 1)Followups
--solver:legacyflag and switch the libp2p pin to a version constraint.🤖 Generated with Claude Code