Library overhaul - #1062
Conversation
dabe947 to
fb0096c
Compare
Restores a valid merge-base with dev so GitHub/Ultrareview can diff this branch again. The previous filter-repo history rewrite (blob cleanup) severed the shared ancestry with dev; content is unchanged, only the commit history is squashed onto dev's current tip. Reconciled with the 7 commits dev gained after the previous fix (mostly the "13-job sweep" adding artist_id to repair-job findings). Ported that change into the 6 repair-job files this branch still has (acoustid_scanner, dead_file_cleaner, lossy_converter, metadata_gap_filler, short_preview_track + its test); the other 6 touched files (album_completeness, duplicate_detector, mbid_mismatch_detector, quality_upgrade_scanner, single_album_dedup, test_album_mbid_consistency) are jobs already retired in RETIRED_JOB_IDS (function moved to native Library-v2 engines), so upstream's edit to their now-dead code doesn't apply here. Also fixed two more test fixtures (test_acoustid_scanner.py, test_lossy_converter_scan.py) whose row tuples needed the same artist_id-width extension but weren't touched by the auto-merge since they're unique to this branch. Single parent, not a merge commit: content is fully reconciled so the tree matches what a merge would produce, but the PR's commit list stays at 1.
Preserve the latest dev fixes while resolving the Library v2 migration, enrichment, import, repair, media mapping, async bridge, and UI test conflicts.
|
@Nezreka, i worked through all your points now and honestly this became alot rougher than i expected xD.. you were right to keep asking which database we actually want to maintain in the future. keeping legacy and lib2 alive as two equal libraries makes no sense.. every import, metadata edit, enrichment result and repair job would need to be synced forever, and sooner or later both databases would disagree again. so i stopped treating lib2 as an optional second library. lib2 is now the main Library and the source of truth for normal runtime. the old tables are only kept for the first upgrade import and as a temporary safety/rollback boundary. i want to remove them in a later update, but only after this survived another real test on your 9gb database. i also thought alot about why not just put everything back into legacy.. but the old model simply does not fit anymore. one legacy track has one about the problems you found:
i added a ratchet test for this too.. normal production Library code is not allowed to add new reads or writes to the legacy artist/album/track tables. the one-time upgrade reader is counted separately, so we cannot quietly start depending on legacy again later. the media-server behaviour is also clearer now:
i merged the newest dev too and checked the conflicts one by one so we keep the fixes from there. manual artwork now has a real lock in lib2 and survives provider/server refreshes, the new thumbnail cache is connected to the active lib2 page, genre enrichment scans lib2, reorganize keeps repair findings on the new path and i fixed a crash in the automatic native enrichment sweep that only showed up when starting the real app. the final merge is if you have the time i would realy like you to test these painful cases again:
i ran everything before pushing:
the tests are clean now, but your database is still the test that matters most because it is on a completely different scale than mine.. so pls tell me anything that looks stuck, missing, duplicated or just behaves differently. even one weird provider or badge is useful and i will dig into it.. thx again for pushing on this.. the rewrite was rough, but your question about which database we actually maintain was the right one and the architecture is alot clearer now.. |
# Conflicts: # core/listening_stats_worker.py
A review of the 59 migration commits since 7aacec8 asked one question of each ported worker and read site: does it still do what the code it replaced did? Ten places did not. - media_mappings: the startup backfill's batch predicate selected rows upsert_mapping refuses to write, so one blank server_id spun the loop at 100% CPU until shutdown. Selection and write now share the condition. - media_server_sync: the ownership branch wrote track_number/disc_number outright while the mapping branch beside it used COALESCE, so every import stamped disc 1 over a real disc. New rows keep their default in the INSERT. - imports: the strict lib2 gate ran before the second catalogue writer, so a correctly tagged and placed file whose identity the autolink could not derive was reported as failed and skipped the automation event, the history row and the wishlist check. The gate now asks the catalogue, after both writers. - imports: the redownload hook guarded on the track id but repoints by path, and logged success when it had moved nothing. - worker_support: provider_id_conflict read the whole artist table on every artist match in twelve workers. Narrowed in SQL; _ids still decides. - worker_queue: the pending query's ORDER BY could not use an index, so every worker tick sorted the entire entity table for one row. Split into its two disjoint halves, both of which stop early. - music_database: enrichment coverage divided by every lib2 artist including provider-only and discography rows, diluting all nine percentages. - search: a natively imported track could come back with track_id null and record the play against an empty id. - workers: eight _mark_status docstrings still claimed `error` never retries. The auto-retry is the intended contract; the docstrings now say so. - provider_attempts: the ledger had no delete trigger, and orphans count as processed — the progress bar sat at a clamped 100% with work outstanding. 14 regression tests, six verified red against the previous state. Full suite: 15403 passed, 3 skipped. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Q9Rk5htjywarUZeqLyWUsF
The LV2-MIG-10 orphan purge ran the first time the new code started, including when a bootstrap import was still in flight. Half the catalogue is imported at that point, so "no entity row" does not yet mean "no entity" — and the rows it would delete are exactly the history `backfill_from_legacy` seeds to stop every worker re-asking every provider about the whole library. Trigger and purge now wait for the import to settle. Deferring costs nothing: an import creates entities, it does not delete them, so no orphan can appear in the window where the trigger is still missing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Q9Rk5htjywarUZeqLyWUsF
Second review pass over the lib2 cutover, this time every worker and tool individually plus the upgrade import as its own question. The import itself came back clean — data-faithful, resumable, and `backfill_from_legacy` does stop the workers re-asking every provider after an upgrade. Five other things did not. - music_database: `get_all_track_ids_for_server` changed from returning primary keys to returning server ids but still only guarded IS NOT NULL. A server never reports an empty id, so one row stamped `server_id=''` was always stale — and detaching by server id turned that single entry into every blank-id track of the source. The 50% safety net could not see it: it counts distinct ids while the detach removes catalogue rows. - worker_queue: legacy's tables *were* the owned library. lib2 keeps discography, wishlist and provider-only rows beside it, and nothing filtered them, so twelve workers enriched — and counted — work legacy never did. Same cause in the genre-enrichment repair job. `sql_util.owned_sql()` is now the one definition of ownership; it had been re-derived in four places and was simply missing from the queue. - worker_queue: the phases were nested inside the entity types, so a fresh album queued behind a backlog of artist retries. Legacy ran priorities 1-3 before 4-6; phase is now the outer loop again. - listening_stats / get_track_by_server_id: the authoritative mapping table was merged with the compatibility snapshot by UNION+setdefault and by OR. Neither has a defined row order, so after a re-match the stale row won and play counts landed on the wrong track. Mapping first, snapshot for the gaps. Also closes LV2-MIG-04 properly. Narrowing it in SQL was not enough: the non-indexable json_extract collapsed the whole predicate to a table scan (measured, not assumed). Per-service expression indexes make it a MULTI-INDEX OR; predicate and index are generated from one function because SQLite only uses an expression index when the text matches. Queue fixtures had seeded catalogue rows without files, which under the restored ownership rule correctly means nothing — `tests/lib2_ownership` gives them the file evidence a real library has. Full suite: 15410 passed, 3 skipped. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Q9Rk5htjywarUZeqLyWUsF
Five conflicts. Three were mechanical: the two dashboard bands kept upstream's live-refresh behaviour and our import ordering, and `album-meta-row.tsx` was two blank lines against a file this branch had already deleted. The other two were real. `core/repair_worker.py` — upstream's fix targets `_track_identity_for_redownload` / `_fix_quality_upgrade`, neither of which survives here: the native scan raises `quality_below_cutoff` against `lib2:<id>`, and lib2 rows survive the full refresh that invalidated the legacy ids in the first place. What DOES survive is migrated pre-V2 `quality_upgrade` findings, which we were refusing outright whenever they carried no pre-searched match. `_legacy_quality_track_data` now rebuilds the payload from the finding's own details, reads both key vocabularies, refuses rather than queueing "Unknown - Unknown", and gives an entity-id-less finding a stable hashed wishlist id so two of them cannot dedupe each other away. Upstream's test file was rewritten onto that path rather than deleted. Reassign an album to a different artist — upstream hung it off the artist detail page this branch removed, so it had to move, and moving it fixed a data-loss bug on the way. The service read `tracks`/`albums` and put the legacy track id into the hint's `replace_track_id`; on this branch `delete_replaced_track` resolves that id against `lib2_track_files.track_id`. Both id spaces are bare integers, so "replace the original" would have deleted a DIFFERENT track's file, silently. The subject is now `lib2:<id>` and a bare id is refused in preview and apply alike; the local side reads `lib2_tracks` joined to its active files (one row per track, primary file first) and resolves the stored path, because staging opens the file for real. The same-release guard finally covers the source the request names instead of only Spotify. The UI moved with it: modal, client half and test now live under `webui/src/routes/library/-ui/`, reached from "Reassign to another artist…" in the album overflow menu. The `.reid-*` chassis and upstream's new `.reassign-*` rules were already in style.css and are reused unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Found by driving the merged flow in the browser. The album overflow menu offered "Reassign to another artist…" on a pure discography row — a release that is in the artist's list but owns no files. The preview can only answer "That album has no files on disk to reassign" there, so the menu was walking the user into a guaranteed dead end, and the first thing they see is an error toast for something that was never possible. The entry is now disabled when the album owns no files, and its tooltip says why rather than leaving the user to guess which of the two "Memory Reboot" rows was the wrong one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Until now there were two ways a file left the disk. The delete dialog went through `delete_entity_files`: preview token, root containment, an operation row, a per-file result, crash recovery, an entry in the album's History. The maintenance worker went through a bare `os.remove` behind its own guards — nothing recorded anywhere, and a process that died mid-run left no state to recover from. That gap is survivable only while a human confirms every single deletion. It stops being survivable the moment a job deletes unattended, which is what this is preparing for, so it goes first rather than after. `delete_files_journaled` is the shared primitive: same containment rule, same journal, same statuses, same recovery, callable with a list of paths instead of an entity preview. The execution loop is now written once and used by both entry points. The maintenance fixes (corrupt audio, preview clips, unwanted content, AcoustID mismatch), the orphan-file delete and the lossy converter's "delete the lossless original" all route through it, each with its own actor so the History can say who did it. What deliberately did NOT change is WHICH files a job may delete. The worker still decides containment with `fuzzy_resolved_path_is_deletable` — stricter for a path the resolver guessed, laxer for one the catalogue names exactly — and calls the journal with require_library_root=False. Enforcing the dialog's stricter rule here would have silently stopped deletions for every library whose folders were never listed in `library.music_paths`: a behaviour change hiding inside a bookkeeping change. Three existing tests caught exactly that and were right to. Still on raw unlink, with reasons: `delete_replaced_track` deletes inside the caller's write transaction, so journalling it needs the unlink hoisted out first (a second connection there is the self-deadlock from the production DB-lock investigation); the expired-download cleaner deletes downloads, not library files; the empty-folder cleaner removes directories. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Reported: the AcoustID checker processed everything, and Michael Jackson's tracks still showed "Not scanned" under Check — after a refresh, and after a re-scan. They always would have. The Check column renders `lib2_track_files.acoustid_status`. The scanner never wrote it. It recorded its verdict in `verification_status` and in library_history, and left the fingerprint column NULL — so every file it agreed with fell through to the unscanned branch. In the reporter's database that is 164 files; the 57 rows that DO carry a status all came from the import pipeline, never from a scan. Worse at the sharp end: a FAIL on an untagged file moved no verification status at all, so nothing was written, and a file whose fingerprint says it is a different recording rendered exactly like one nothing had ever looked at. The two columns answer different questions — "does this file stand verified" and "what did the fingerprint check conclude" — so the scan writes both now, independently, including `fail`. No AcoustID answer at all still writes nothing: no verdict is not a verdict. The files verified before this fix carry no fingerprint status and cannot honestly be given one, so the badge falls back to `verification_status` = verified rather than calling them unchecked, and `fail` gets a red "Mismatch" of its own instead of hiding among the unscanned. Second half of the report — "it should update by itself". It could not: the library invalidated its cache on exactly one event, dispatched by Watch All. A maintenance job that rewrites verification, retags a track or deletes a file changes what the open page is showing and nothing told it. The signal already existed app-wide (core.js re-broadcasts the worker's socket frames as `ss:repair-progress` on every page); `useMaintenanceChanged` listens for a job LEAVING the running state and invalidates then — not on every frame, which would refetch the artist view once a second for the length of a scan. Verified against the real library: "Scream" now shows 12 Verified where it showed 12 Not scanned, no console errors. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…, and a full UI pass on track detail/History Deletion was effectively unusable and its data stale: - Permanent delete was blocked for any library without library.music_paths configured, and one already-missing file blocked deleting the rest. Already-gone files are now journaled as such instead of blocking anything. - The AcoustID checker finished a full run but files still read "Not scanned": the scanner never wrote acoustid_status independently of verification_status, and the UI never invalidated its cache when a job completed. Both fixed; inconclusive scan outcomes (rate limits, missing API key) are now distinguished from a genuine no-match instead of both silently no-op'ing. - A complete provider tracklist could only ever raise expected_track_count, never correct a stale inflated one downward (DAISIES read "1 missing" with nothing missing) — now safety-gated to complete, own-release-id fetches only. - "Missing" counted every absent track regardless of whether the user still wanted it; unmonitoring a track you don't want no longer keeps it counted forever. Un-materialized provider-promised slots still always count. - A track's own History tab never showed that its own file had been deleted (only the album/artist timelines did) — now scoped correctly. A follow-up critical UI pass on the same modal: - The duplicate Check/Verification columns are one column now; its "Skipped" verdict splits into "Skipped" (an actual bypass) vs "Unverified" (AcoustID ran, found nothing confident) — collapsing both lost exactly the distinction a reader needs. - History gets its own tab (ahead of Metadata, Lyrics demoted), renders as a real table like the album/artist History, and its Status column shows the same colored verdict badge as the Check column instead of a bare "—", with the reasoning in Detail instead of a raw field-name list. The legacy track_downloads feed no longer duplicates an acquisition-tracked completion as a second "Downloaded" row, and a manual grab now says which automatic-gate rejections it overrode. - Retag/Reorganize preview and Interactive Search: the checkbox/monitor/ actions columns were `width: auto` under a fixed table layout, which grabs an equal share of whatever's left instead of shrinking to content — with few other pixel-width columns in those tables, the checkbox column ate roughly a third of the width. Pinned to real widths. - disc/duration/BPM/file-size aligned consistently; the shared dialog close button is a thin SVG X instead of a bare "×" in an oversized box; the track detail modal is bigger and holds one FIXED size across every tab (content-sized height made it visibly jump between tabs); a release with everything unmonitored no longer claims "complete" — that is nothing being wanted, not completion.
# Conflicts: # core/downloads/candidates.py # web_server.py
|
@Nezreka Quick update since the last message:
No urgency on a reply, just wanted to keep you posted that I'm tracking what you implement upstream, and making sure it lands in Library V2 too... |
43 upstream commits (YouTube Premium quality via cookies.txt, MusicBrainz artist-alias resolution, collapsible sidebar, wishlist Wing-It guesses, deezer throttle/genre/sync fixes, soulseek stall-guard, quarantine-wedge fix, search-punctuation fix, and more). Conflicts (8, all resolved): - core/imports/pipeline.py: kept our journal call AND adopted upstream's wrapper-level quarantine retry/fail fix (real "wedged in Processing" bug upstream root-caused independently of our own downloads work). - database/music_database.py: adopted upstream's fix removing a fabricated-filename fallback that could corrupt file_path on transient Subsonic/Navidrome API gaps; kept our lib2-native track upsert over upstream's legacy `tracks`-table INSERT/UPDATE (dead path on this branch). - pyproject.toml: merged both branches' pytest markers. - 5 modify/delete (upstream modified, we'd deliberately deleted): the legacy quality_upgrade test (superseded by test_lib2_upgrade_scan) and 4 files under the unreachable legacy artist-detail page (§46) — kept deleted, consistent with that standing decision. Post-merge fixes for upstream tests that assumed the legacy artists/albums/tracks schema instead of this branch's lib2_* catalogue (runtime legacy SQL is 0 reads/0 writes per the ratchet test — nothing reads those tables anymore): - tests/imports/test_import_side_effects.py: read the new opus-bitrate assertion through the file's own lib2 _track_row() helper. - tests/library/test_library_bitrate_estimate.py: reseed via seed_artist/seed_album/seed_track instead of raw legacy INSERTs. - tests/test_fuzzy_search_punctuation.py: reseed both fixtures through ensure_library_v2_schema + the lib2 seed helpers, one album per artist so the search join (via albums.primary_artist_id) resolves correctly. - database/music_database.py get_artist_full_detail: the enhanced-view track query joined lib2_track_files but never selected its `size` column, so opus bitrate estimation silently fell back to a disk stat() call instead of the already-fetched value (worked in production by luck, broke in tests against a fake path). - webui account-tabs.test.tsx: added a test for the new DEEZER_PLAYLIST_PROGRESS_EVENT export (our own export-coverage guard caught it unmentioned). Verified: full pnpm vitest suite green (6906 tests). Full pytest suite green when chunked per-directory/batch (a serial full run can wedge on a pre-existing, unrelated race in utils/async_helpers.py's shared cross-thread event loop — reproduced identically pre- and post-merge in isolation; not introduced by this merge, just exposed more often by the added test volume). Three remaining failures under serial batching are also pre-existing/artifacts of the batching itself, confirmed by running each in isolation on both branches. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
A five-agent review of `library-overhaul` produced 3,148 lines of findings
(docs/audits/library-overhaul-review-2026-08-20/). This closes all 14 backend
correctness bugs, the two Critical and both High performance findings, nine of
the ten frontend findings, and restores the two features the cutover silently
dropped. `06-remediation.md` records what was measured, what was refuted, and
what was deliberately left.
Correctness — the three that could destroy files
------------------------------------------------
* The ADR-05 delete dialog acted on a resolver GUESS. `resolve_lib2_path`
suffix-walks the transfer folder FIRST, so a stored path that no longer
exists (share remounted, folder renamed) resolved onto a freshly downloaded
replacement awaiting import — and confirming the dialog unlinked that.
`fuzzy_resolved_path_is_deletable` was written for exactly this hazard and
applied only by the maintenance path. Now applied here too.
* `move_track_file` read `deleted` tombstones as real files, both directions:
the target check refused a move forever with "remove or dedup it first" when
there was nothing left to remove, and a source whose only row was a tombstone
demonitored the track with USER provenance on the strength of a file that is
not there.
* Per-artist repair scope was built, passed, logged — and read by nobody.
`get_scope_file_paths`/`file_path_in_scope` had zero production callers, so
"run Library Reorganize for this artist" moved the whole library while the API
answered `scope_files: 180`. Twelve jobs now honour the allowlist; the rest
declare `supports_file_scope = False` and the endpoint REFUSES a file scope
rather than silently widening it.
Correctness — the rest
----------------------
* Delete-journal crash recovery was never wired to startup, so a container
restart mid-delete wedged the operation in `executing` forever while the
catalogue asserted a file that may already be gone.
* Library Reorganize looked up lib2 track ids in a legacy-keyed map (zero
findings, one error per track) and refused every album created after the
cutover — `legacy_album_id` is written only by the one-shot importer, so on a
fresh install the job emitted one permanently unfixable warning per album.
* `_fix_path_mismatch` read a bare integer as a lib2 id while the sync layer
read the same id as a legacy back-reference: one track's row repointed at
another track's file, with a third track's history recording it.
* Every repair fix ran an UNSCOPED full-library `recompute_wanted` — "Fix All"
over 500 findings held the single SQLite writer for hours.
* One failing `sync_repair_change` discarded the rest of the flush batch: files
already mutated on disk, nothing rescanned, one error reported for 23 losses.
* `fix_finding` had no atomic claim, so a bulk run and a user's click could both
execute the same fix. Claimed via a new `fix_claimed_at` column — `status`
deliberately unchanged, because a finding vanishing from the list mid-fix
would be the worse bug.
* Batch enrichment dropped the ownership filter and spent API budget on whole
discographies while the UI read 0 pending / 100%.
* `prune_done` could erase the row that supersedes a stuck outbox op, re-arming
the dd28-13 wishlist resurrection.
* `repoint_file_path` rewrote deleted tombstones too and returned a rowcount
that lied to the one caller that reads it.
Performance (measured: 12k artists / 288k tracks / 264k files)
--------------------------------------------------------------
list_artists(sort="albums") 11,469 ms -> 46 ms
list_artists(search=...) 21,686 ms -> 10 ms
list_artists(sort="tracks") ~5,000 ms -> 47 ms
list_artists + size roll-up 168 ms -> 44 ms
list_cutoff_unmet HTTP 500 above ~33k tracks -> works
Three separate defects, only the first of which the audit found:
* The count sorts were a correlated `ORDER BY` subquery run per artist row.
Hoisting it into a CTE only reached 4.9 s, because SQLite will not build an
automatic index on the right-hand side of a `LEFT JOIN` against a CTE — it
scanned the 12,000-row aggregate once per artist. The same aggregate in an
indexed table, joined on its primary key, is 3 ms. Hence
`core/library2/artist_rollup.py`: a SORT KEY only, rebuilt in 374 ms on
staleness or artist-count drift. The numbers rendered beside each artist
still come from the exact per-page CTEs, and a test pins that the two agree.
* `list_artists`' size CTE had PERF-03's exact shape in the function PERF-03
said was already fixed: `EXISTS (...)` over a bare `FROM lib2_track_files`
scanned all 264k file rows.
* The `iss29-D04` search clause is 21.7 s at 12k artists (the audit measured
258 ms and passed it): an `OR` across two tables plus a correlated `EXISTS`
whose index ANALYZE cannot judge selective on a library with few aliases.
Also: `list_cutoff_unmet` materialised the whole library and passed one bind
variable per track (a hard, permanent 500 above SQLite's 32,766 default — which
is what a stock `python:3.x` image ships); the scan/retag loops opened a fresh
connection per file (~3.8 ms of schema parse each, ~19 minutes at 300k files);
`synchronous` was never set, leaving an fsync per commit; the metadata precache
shared the DOWNLOAD concurrency knob at a default of 3.
Frontend
--------
* Export Artists and Watch All Unwatched shipped on upstream/dev and were
deleted with `library-page.tsx`. Both components survived, imported only by
their own test — which is why CI stayed green. Rewired: "Monitor All" beside
Automatic Search, "Export Artists…" under Files/Tools. The monitor action is
irreversible, library-wide and one pixel from a button people click routinely,
so it ARMS on the first click and fires on the second. On this page the label
is Monitor; the API, tables and identifiers stay `watchlist`.
* The blocklist 409 branch was unreachable — ky threw on the status before the
body was read, so the user got a bare HTTP status instead of being told which
artist is blocked.
* The monitor-failure message rendered and was clipped out of the viewport by
an ancestor's `overflow-x: hidden` (which makes BOTH axes clip).
* Ten modals: eight promised `aria-modal="true"` with no focus trap, no Escape
and no focus restore, and two had no `role="dialog"` at all. One shared hook.
* Two globals named `_escAttr` — the HTML-escaping one won, so the onclick
builders that needed JS-string escaping got `&Nezreka#39;`, which the attribute
parser decoded back into a quote that killed the handler.
* `bitrate > 5000`, inlined six times, rendered every hi-res lossless file as
"9 kbps".
Docker
------
`build_webui_vite_assets` returned "" when the bundle manifest was missing or
its entry key had drifted: the page rendered with no module script, every
vanilla feature kept working, and nothing was logged — so the entire React half
could disappear from an image with no diagnosis available. It now says so, once,
with the fix. (Verified separately that the builder stage's output is
byte-identical to a local build and contains the current UI.)
Coverage and what is left
-------------------------
Backend correctness 14/14. Performance 14/16 (PERF-09 and PERF-13 are both
structural and neither fails). Frontend 9/10 — FE-06 is left: nothing in the app
is virtualized, so a very large discography renders every row, which needs a
windowing library rather than a fix in passing. Dead code: the hazards only
(the tracked MP3 and config backup, the shadowing `docker_resolve_path`
duplicate, the hot-path no-op config reads); the rest costs nothing per run and
would bury this diff.
The `api/library_v2.py` blueprint rebuild and the `library-v2-page.tsx` split
are both correct and both days of pure motion — folding either in would make
this unreviewable. Deleting `legacy_mirror.py` states "there is no rollback
path", which is a product decision; its docstring, which asserted the opposite
of the truth, is fixed instead.
PERF-05 is recorded as REFUTED: the pragma it blamed was simply the first
statement that opens the database file, so it was charged for the schema parse
every caller pays anyway. Measurements are in 06-remediation.md.
Verification
------------
Green: library2 1,760 · repair 213 · downloads 929 · webui 6,914 (334 files),
plus tsc, ruff, oxlint and oxfmt. The perf figures come from a synthetic
12k-artist / 288k-track catalogue driven through the real query functions.
The full `pytest tests/` sweep in deterministic order (`-p no:randomly`) stalls
at 51%, inside `tests/test_bandcamp_client.py`'s threading tests. That is
PRE-EXISTING: a worktree at unmodified HEAD, same command, stalls at the same
point. The file passes in 0.29 s on its own.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
It was given `.btnGhost` alone. The size comes from `.automaticSearchButton`, which constrains the icon to 15x15 and makes the button an inline-flex row — without it the SVG rendered at its natural size and the button was visibly taller than the Automatic Search button it sits beside. Same class pair as that neighbour now, so both share one vertical height. The class name is historical; the rule itself is the generic "ghost button with a leading icon" used in this header. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
21 upstream commits (611b64f..250f7f9): curation retention for the Expired Download Cleaner, live download narration (Nezreka#1156), the MetaSync export endpoint, atomic staging that works on Docker, Deezer share links, and the Nezreka#1164 sync matching fixes. Six conflicts, three of them substantive — all three because upstream hung a new feature on the legacy tables this branch no longer has. MetaSync export ported to Library v2 ------------------------------------ `api_export_entities` walked `artists`/`albums`/`tracks`. The SoulID worker writes lib2 here, so the endpoint would have returned zero rows on every install — silently, with HTTP 200. The legacy-usage ratchet caught it: 0 -> 6 reads, all six in that one function. Rewritten as `_EXPORT_SPECS`: one projection per entity that serves the lib2 row under the OLD field names, because those are the wire format the sidecar and its peers already speak. Two provider ids come from dedicated columns and the rest from `external_ids`; `record_type` is `album_type`, a track's `year` comes from its release. The artist name comes from `lib2_albums.primary_artist_id`, never `lib2_track_artists` — the SoulID worker hashes the album artist, so exporting a featured credit would describe a row under a name that derives a different key than the `soul_id` shipped beside it. lib2 has no `*_match_status` columns. The replacement is the rule the provider chips already use: a stored id IS the match. Only when there is no id does `lib2_provider_attempts` answer (`not_found`/`error`/`skipped`); never tried reads as `pending`, the legacy vocabulary. The provider list is derived from `match_status.SERVICES` so a new provider cannot go missing here. lib2 ids are INTEGER, so the keyset cursor now orders numerically and is parsed as an int on the way in — a malformed token is a 400, not "return the whole library from row one". Expired Download Cleaner ------------------------ Upstream's round is a data-loss fix: `play_count = None` means UNKNOWN rather than zero, a path-suffix fallback covers the Docker/NAS namespace difference, media-server curation signals protect a track, and anything downloaded before the last library rebuild is permanently out of scope. Our copy of the file was a rewrite for the lib2 delete. Resolved by taking upstream's file and putting our `delete_origin_download` back into it. That surfaced a real gap: our version REPORTS a lib2 failure in the return value instead of raising, and upstream's auto-delete branch only counts `except` — so a run would have claimed a clean sweep while nothing was deleted. It now checks `res['error']`. The candidate query and the suffix map read `lib2_track_files.path` + `lib2_tracks.play_count`, and `COALESCE(MAX(...), 0)` became `MAX(...)` so "unknown" cannot read as "never played" again. soul_id_path ------------ New column on `lib2_artists` (CREATE + migration); the worker writes it and the legacy importer carries it across. Only `canonical` is reproducible on another install — the album fallback depends on what this library owned at the time — so it cannot be recomputed and has to travel with the id. Staging guard ------------- Upstream moved the atomic-publish staging tree to `<transfer>/.soulsync_atomic_staging` and generalized `skip_deleted_quarantine` into `is_internal_transfer_dir`, with a suite pinning every transfer walker. Three entries on that list no longer walk here: `fake_lossless_detector` and `track_number_repair` take their subjects from the catalogue or from `filesystem_subjects`, and `quality_upgrade_scanner` is deleted (`lib2_upgrade_scan` is DB-driven). The list now names `filesystem_subjects.py` — the one shared walk — instead of the three. Anything staged is by definition not in the catalogue. Fifteen failing tests, all ours, fixed here too ----------------------------------------------- The first sweep after the merge failed 15. All 15 failed identically on the pre-merge tip, and all three affected upstream files pass on upstream/dev — so none of it came from the merge. It is fallout from 05f4597 (the five-agent audit remediation), i.e. our debt, and it is closed in this commit. * `test_repair_stale_findings` (4) — the fixture builds its own repair_findings DDL, and the audit added `fix_claimed_at` to production but not to the copy. fix_finding's claiming UPDATE raised "no such column", the outer handler swallowed it, and every assertion read the pre-fix row. Column added. * `test_repair_worker_path_mismatch` (4) — the audit put `_fix_path_mismatch` behind `_stale_legacy_subject`: a bare integer subject is a LEGACY back-reference by contract (T-12), and applying it would move a file on behalf of a different track. The test still passed '10'/'20'/'11'/'12'. Now `lib2:<id>`, plus a new test pinning the refusal itself. * `test_script_split_integrity` (1) — `_escAttr` was deduped by the frontend audit (FE-07) but still listed in KNOWN_CROSS_FILE_DUPES, and the test fails on a STALE entry by design. Moved to the resolved block with its cause. * `test_cross_batch_dedup` (6) — the file landed as 191 new lines without the feature. Implemented: - `_find_owning_sibling` finds the task in ANOTHER batch that already owns this recording. Only completed/post_processing/already_owned count — searching/downloading is a promise, not a file, and standing down against a peer that later fails strands the track. Cross-batch only: a task with no batch is a hand-started download (a re-rip, say), and answering that with "you already have it" would undo a deliberate action. - `download_track_worker` short-circuits before searching: status already_owned, inherits the owner's verification/quality/path/source (a blank badge reads as a failure for a track that is present and fine). - lifecycle counts `already_owned` as finished — without it a deduped task held its batch in 'downloading' forever. Both completion checks now read one `_FINISHED_TASK_STATUSES`. - Identity comes from `status.download_identity` / `status.normalize_track_fields`, the same triple the downloads view dedups on, so the UI and the dedup cannot disagree about what "the same song" is. That normalization was inline in build_unified_downloads_response; it is a shared function now. Verification ------------ Legacy ratchet back to 0 reads / 0 writes. Full sweep GREEN: 16,711 passed, 3 skipped, 0 failed. webui vitest 6946 across 336 files, oxlint type-check clean, oxfmt clean, ruff clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
@Nezreka synced with dev again (up to 250f7f9), conflicts all resolved on my side. also ported the new work over to library v2 here, metasync export and the expired cleaner queries and soul_id_path. i'll keep doing that with every sync, it just means the PR keeps growing a bit each round. i'll keep doing it, but the PR gets bigger the longer this goes on.. splitting it isn't really an option anymore either, the cutover is one piece since i ported every worker...da |
…ogue Two halves. First, everything in LIBRARY_V2_FINDINGS.md that exists because of the Library v2 overhaul: the ten findings it caused outright, plus the lib2 halves of five findings whose dev-side fix cannot reach this branch. Each carries its regression test. The six dev-side fixes themselves live on `fix/dev-review-findings-2026-08-21` so they can be merged upstream first, independently of this branch. Second, the legacy catalogue schema goes — the last section. == the ten overhaul findings == L2-003 cross-batch dedup: post_processing no longer counts as ownership (its import gates can still fail the owner), the requesting side builds its identity through track_info_identity like the sibling side does (collaborations were a false negative), provider ids decide in both directions inside one namespace with duration/disc/track backing the metadata fallback, and an owner must have a file and the same quality profile before a task stands down against it. L2-004 manual match: set_library_v2_match syncs lib2_provider_attempts in the same transaction. A PUT settles the queue entry, a DELETE removes it so the entity is selectable again without waiting out a retry window. L2-006 watchlist import: the watchlist row's quality_profile_id is carried onto the lib2 artist (validated, and only when duplicate rows agree) instead of being reset to the global default. L2-007 wishlist import: spotify_data is a provider-neutral payload. The provider is resolved from the payload/source_info and non-Spotify ids go to external_ids under their own namespace, with provider-keyed album/track lookup so an existing recording is reused instead of duplicated. The namespace sanitizer now covers lib2_tracks too. L2-008 reset intent: the pre-wipe album-intent snapshot is committed to lib2_import_intent_snapshot in the same transaction as the deletes, reloaded on resume, folded into a later rebuild, and cleared only after a successful restore. L2-009 manual import: holds a ClaimKeepalive for the whole run like the automatic bootstrap does, and treats mark_done()==False as a hard failure instead of reporting done and starting artwork. L2-012 full refresh: clear_server_data no longer stamps library_rebuilt_at. The lib2 detach keeps the catalogue and play_count, so nothing needs the amnesty, and stamping it grandfathered every older download permanently while each further refresh pushed the boundary forward again. An existing stamp from a genuinely destructive older build is still honoured. L2-013 media-server mappings: backfill_legacy_mappings runs as an import finalize step, so mapping-only consumers work after the first successful upgrade instead of only after the next restart. L2-015 repair scope: build_artist_file_scope resolves the whole alias group, includes track-only credits and takes only active files, so a run started from a merged artist page covers what that page shows. L2-016 worker queue: status_counts applies the same ownership predicate as next_pending, so provider-only rows stop showing as pending work. == the lib2 halves the dev fix cannot reach == L2-005 (lib2): every enrichment worker here imports honor_stored_match from core/library2/worker_support, not from core/enrichment, so the dev fix reaches none of them. Same three-state contract: a stored id the provider could not confirm is kept rather than released to a fuzzy name search, and the failure is written to lib2_provider_attempts so the queue backs off instead of handing the entity straight back. L2-002 (lib2): the atomic publish repoints lib2_track_files, and repoint_file_path already reports a rowcount precisely so a caller can tell "the catalogue did not know this file" from "done". The lifecycle threw it away; it is returned now, which is what lets the (dev-side) publish treat a zero for an audio file as a failed publish. L2-010/L2-011 (lib2): api_export_entities was rewritten for lib2 on this branch, so the dev fix does not apply to it. Same treatment: `since` is normalised to UTC once and compared through julianday() instead of as text, the filter spans each row's parents, and the lib2 writers that change an exported field (soul_id, canonical claims, and a provider-attempt status change) now move updated_at. record_attempt touches only on a real status change, because it runs on every provider cycle. L2-014 (lib2): lib2_artists has its own soul_id_path column with the same gap. The new versioned migration proves each path locally by recomputing it and records anything it cannot prove as `unknown` rather than assuming canonical. == the legacy catalogue schema is gone == Nothing has read or written `artists`/`albums`/`tracks` at runtime for a while — `tests/library2/test_legacy_usage_ratchet.py` pins that at reads: 0, writes: 0. What was still there was the machinery that maintained them, so it goes now. A fresh install stops creating 3 tables / 215 columns / 85 indexes that nothing touches; an install that already has them keeps them until `core/library2/importer.py` has migrated its rows. Deleted outright: `core/library2/legacy_mirror.py`, whose own docstring already said it was not installed and had no caller, and everything in `core/library2/enrich.py` reachable only from it — the resync, the divergence audit, the handover bookkeeping. That module keeps the one thing the upgrade importer reads (`_ENRICHMENT_PAYLOAD` + `enrichment_columns`) and goes 748 → 131 lines. `core/library2/native_enrich.py` — what the Enrich buttons and the enrichment sweep actually call — is untouched. `database/music_database.py` loses 19 migrations that existed only to grow columns on those three tables: the per-provider `_add_*_columns` set, the art-lock and core-media backstops, the genre-format one-shot, the match-status backfill, and the three `CREATE TABLE`s with their indexes. Two are kept trimmed because their non-legacy half is live — `_add_musicbrainz_cache` (the API result cache `core.metadata` reads) and `_add_metadata_match_provenance`. Because the tables are no longer created, a fresh install reports zero source rows because they are ABSENT, and the importer's projection would raise on a table that has none of its required columns. `bootstrap.run` therefore settles an empty source into `waiting_for_source` — a state already written and checked for, but which `mark_waiting_for_source` had no caller to set. It is pinned to the watermark, so an upgrade whose rows do exist still runs normally. Two bugs the deletion uncovered, both fixed here: - `metadata_match_provenance` could not record a Library-v2 match at all. Its CHECK listed only the three legacy entity spellings while every lib2 writer and reader uses `lib2_artist`/`lib2_album`/`lib2_track`, so each write was rejected by the constraint and swallowed by the `except` around it — no manual match has ever been stored, and no chip has ever carried a `last_attempted`. The constraint is widened and an existing table is rebuilt once, carrying its rows over. (The triggers that used to fill this table automatically off the legacy `<provider>_match_status` columns had the same namespace split, so they had been writing rows nothing could read long before they were removed.) - The public REST API served `repair_status` and `repair_last_checked` as permanently NULL: they were legacy `tracks` columns that nothing had written since the repair worker went native. `_api_project_lib2` answers them from `repair_findings` instead — the newest open finding's type, and when repair last had something to say about the row. Frontend: the pre-V2 artist-grid island (`library-artist-card`, `-library.helpers`, `-library.types` and their tests) had no importer outside itself. `-library.export`, `-library.watch-all` and the two modals stay — the V2 page imports them. Tests move with their subjects rather than being dropped: the ReplayGain, similar-artists and JioSaavn fixtures seed lib2 rows, the verification-status migration test points at `lib2_track_files` (where two files for one track can be verified independently), and the stale-finding tests keep the refusal they were written for without a legacy twin to leave untouched. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
38 upstream commits, 250f7f9..5ae0a0c. Fifteen conflicts, and the shape of nearly all of them is the same one: PR Nezreka#1169 landed our dev-side findings on the LEGACY tables, and this branch already carries the lib2 twin. Those took ours. The rest needed real work. Kept ours (lib2 twin already present) - the eight provider workers: upstream's honor_stored_match grew mark_status_fn/status_column to record an UNAVAILABLE attempt; core.library2.worker_support already writes that to the attempt ledger. - soulid_worker: ten hunks, all lib2_* vs legacy table names, incl. the new _migrate_artist_soul_id_paths (PATH_MIGRATION_VERSION stays 'lib2_v1'). - repair_worker: upstream's L2-011 updated_at fix lands in _fix_unknown_artist, which this branch retired (unknown_artist_fixer is in RETIRED_JOB_IDS). - MetaSync export: _EXPORT_SPECS/_EXPORT_CHANGED_AT already do what upstream's inline specs/effective dicts do, on lib2. Took upstream's `from None` on the since-parser (B904, which took ruff red on dev). Kept both - imports/guards: upstream's Nezreka#652 quarantine source block AND our acquisition/manual-grab pipeline callbacks, in that order. - downloads/lifecycle: _FINISHED_TASK_STATUSES (ours) and _wake_waiting_batches (upstream's global concurrency cap). Ported upstream's new work onto lib2 - the owned count. Upstream moved in_library off the join and onto a flag the sync matcher writes, keeping the join as the fallback for playlists nobody has synced since. The fallback arrived reading legacy tracks/ artists; it reads lib2 here, id-first, and every hit must own an ACTIVE file — a provider-only discography row is known, not owned. - get_mirrored_playlist_status_counts, the single-playlist variant, now reads flag-first too. It feeds the sync history entry while the batched one renders the card, and one playlist reporting two different ownership figures is worse than either being a little wrong. - test_soulid_path_migration seeds lib2_artists/lib2_albums. - test_atomic_publish_wiring: the DB stubs answer a rowcount, because the repoint runs through library2.track_files.repoint_file_path and a stub that cannot say how many rows moved now reads as "matched no row", which is a FAILED publish (L2-002). The real-sqlite rollback test seeds lib2 and reads lib2_track_files back. - test_metasync_export gains upstream's minting test, on lib2. Fixed - atomic publish iterated staged files in os.walk order, which is the filesystem's business: on this box btrfs hands back 02 before 01, so a failure on the second track rolled back a different track than it did on ext4. Sorted. Upstream's own test_a_file_that_will_not_roll_back_keeps_its_db_row could not pass here until it was, and its sibling passed vacuously. - ran oxfmt over the 42 files upstream added unformatted; npm run check is a PR gate here and dev pushes do not run it. Verified: legacy-usage ratchet still 0 reads / 0 writes, 16,809 python tests pass (2 failures, both pre-existing: test_confirmed_search_route fails identically on the pre-merge tip, and the wishlist transient test is the known -n 8 isolation flake and passes serially), 7,290 vitest pass, oxfmt/oxlint clean, vite build clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…acks GET Driven by a full production wishlist report (611 rows, 3.2.3, Navidrome with media_server_connected=false). Every number below was reproduced from that report before anything was changed. Album covers (373/373 Library-v2 rows had none) `track_wishlist_payload` never built an `images` array, and the UI reads `spotify_data.album.images[0].url` — as does the import pipeline, for cover.jpg and embedded art. New `core/library2/wishlist_art.py` answers "what image URL do I hand the browser for this entity" once, in the same order the Library v2 pages use: the local artwork endpoint first (the long-term truth, already on disk, no media server involved), the provider CDN url after it as the stand-in for a cold build. The write path fills it for new rows, the read path backfills existing ones without rewriting a stored payload. Two consequences of that order, both required or the fix trades one bug for two: the import pipeline downloads from this list and cannot resolve a relative URL, so it asks for the first FETCHABLE entry instead of indexing `[0]`; and the wishlist UI had no "paint the CDN url while the local build runs" behaviour, so `WishlistCover` adds it. That is deliberately not the library page's `Artwork` component — importing it would pull the entire library page into the wishlist bundle. Artist photos (218/218 distinct URLs 404'd in 12-94 ms) They were resolved by exact name match against `lib2_artists.image_url`, which on a media-server install is a `/rest/..` path — rebuilt into an authenticated Navidrome URL that nobody can load when the server is down. Resolution now goes through the Library-v2 identity (lib2_track_id -> primary artist, else a folded name_key match) and never returns a media-server path or a provider placeholder — not even as the stand-in, since a broken stand-in is worse than none. The Last.fm default star (129 rows / 99 artists) is now rejected centrally instead of in two private copies of the same hash set. Image cache (857 rows, 602 of them pending and never served) `normalize_image_url` minted a fresh random Subsonic salt per call, so the same cover produced a new URL — and a new cache key — on every render. The salt is now derived from (password, path): stable per image, still rotating with the password, and leaking nothing the URL did not already carry. Separately, a `pending` row is a registration, not an image: it was written with `expires_at = 0` and `prune()` only deleted `expires_at > 0`, so with the size cap disabled nothing could ever reclaim it. Pending rows now have their own TTL and old zero-expiry rows drain on `last_accessed`. Provider identity (20 album groups shared across distinct entities) "Memory Reboot" and "Memory Reboot (Slowed)" shared one Spotify album id because album titles are folded with `normalize_artist_name`, which deletes exactly the words that separate a variant from its original. `titles_are_same_release()` compares the variant signature (slowed/sped/ remix/live + a trailing sequence number) while deliberately still folding edition words (remastered, deluxe) so matching coverage is unchanged. `set_library_v2_match()` additionally refuses a provider id another entity already holds; a manual match moves it instead. Recording-keyed track ids (MusicBrainz, Last.fm, AudioDB, Genius) may legitimately be shared and are exempt. Pre-existing conflicts are reported, never auto-merged. Count vs. list (614 vs 611) `GET /api/wishlist/tracks` opened by deleting rows, which is wrong on its own terms and was the only difference between it and `/api/wishlist/stats` — so the gap could never be attributed. The GET is read-only now (cleanup still runs in the maintenance automation and the processing cycle), the response reports `stored_rows`/`hidden_rows`/`duplicates_found`, unreadable rows are logged by id, and `scripts/diagnose_wishlist_visibility.py` names the dropped rows per stage against a read-only copy of the database. Upgrades are not duplicates 343 of the 373 rows are quality upgrades of files already on disk. The policy is untouched; the list view now marks them so a profile change does not read as the wishlist duplicating the library. Also: `/api/image-cache/<key>` reports why it failed in a header instead of returning an indistinguishable empty 404. Also carried in this commit: retired provenance triggers are dropped before the provenance table is rebuilt, so the rebuild no longer fails against a database that still has them. Merged upstream/dev 3.2.4 into the same commit (5ae0a0c..163d1ed, 13 commits). Two of them are this fork's own PRs coming back (Nezreka#1172 publish order + suite, Nezreka#1173 quality-profile save), so most of the overlap resolves to "we already have it". Eleven conflicts, none of them semantic: Seven are modify/delete on files this branch removed on purpose — the legacy artist-detail enhanced view (bb03187) and library-artist-card (754cbdc). Upstream's bba7c0c fixes two defects in that view: a stats bar and a section list that classified releases separately, and buckets beyond album/ep/single never rendering. Neither reaches Library v2 — `artist_detail()` classifies once, with `else: albums.append(...)` as the catch-all, so a compilation is listed AND counted, and `album_type` is lowercased on write (importer, materialize, provider adapters), so there is no strict-vs-folded comparison to disagree with. Deletions kept. `iter_staged_files` sorted its walk on both sides. Upstream's version is the better one — `_publish_order_key` reads digit runs as numbers, so a boxset's track 100 no longer sorts in among the ones. Took theirs whole; our plain `out.sort()` was the same fix, coarser. The reassign module and modal only conflicted because this branch moved them out of artist-detail and upstream reformatted them in place (oxfmt, 8e389bd). Kept ours. The listening-history-band mock took upstream's optional `_album` parameter — the required one is a TS2493 in every `mock.calls[0][3]` and oxlint --type-check gates CI on it. Nothing needed porting. The Prowlarr search throttle lands inside `ProwlarrClient.search`/`search_each_indexer`, so the lib2 manual-grab and acquisition paths inherit the pacing without a call-site change; the FLAC oversize-art fix, the no-op sync-history record and the `sync_type` filter all sit on tables Library v2 never replaced. The legacy-usage ratchet is still green (14 passed, no new reads of the retired catalogue), which is the check that would have caught an upstream feature hung on the dead tables. Suite after the merge: 16,957 passed / 3 failed, and the same three fail on the pre-merge tip in a worktree — test_confirmed_search_route's materialize guard plus two candidate_store tests that hang on the shared async loop under -n 8 and pass serially. Frontend: 7,304 tests, `npm run check` clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Library Manager v2 — a real, Lidarr-equivalent library manager, built entirely on SoulSync's own pipeline
Why I built this
SoulSync's library page has always just been a read-only mirror of whatever the media server (Plex/Jellyfin/Navidrome) reports back. No "monitor this artist," no missing-tracks visibility, no way to pick a specific release when auto-grab gets it wrong, no repair tools pointed at a single artist. It inherited every blind spot the media server has, and it couldn't do anything Lidarr does.
That always bugged me, because SoulSync already has everything a library manager needs under the hood: a real multi-source search, a download pipeline, quality enforcement, tagging, repair jobs. Lidarr has to shell out to external tools for all of that. We don't. So the actual gap wasn't "we need Lidarr's features" — it was "we need Lidarr's front end and decision layer wrapped around functionality we already have."
That's what this is. Not a reinvention of SoulSync's search/download/tag machinery — a proper Artist → Album → Track library model sitting on top of it, so you can finally manage your library the way you'd manage it in Lidarr, without needing Lidarr installed at all.
I want SoulSync's library management to be genuinely on par with Lidarr — not "good enough," actually equivalent. This PR is the first complete pass at that.
What it is, in one picture
Library v2 is not a second download/search/tagging system living next to the old one. It's a new front end and a new "what's monitored / what's wanted" model that talks to the same pipeline SoulSync already had:
flowchart TB subgraph UI["Library v2 UI (opt-in, new)"] A[Artist / Album / Track view] B[Interactive Search & Manage Tracks] C[Quality Profiles & Re-Tag Preview] end subgraph Bridge["Monitoring mirrors into the existing systems"] D[Artist monitor → Watchlist] E[Album/Track monitor → Wishlist] end subgraph Core["SoulSync's existing pipeline — unchanged, reused, not duplicated"] F[Multi-source Search] G[Download Orchestrator] H[Quality Gate + AcoustID] I[Tagging / Post-Processing] end UI --> Bridge --> Core Core -. same auto-download machinery keeps running .-> BridgeIf you never open the Library v2 page, nothing changes — it's dormant behind a feature flag. If you do open it, "Monitor this artist" doesn't invent a new download queue; it just adds the artist to the existing Watchlist, the same one the old page uses. Same for Wishlist. So the whole thing stays compatible with everything else running in the app instead of forking it.
Design rules I wouldn't break, even under pressure
These came out of a lot of back-and-forth with myself about where this could go wrong, and they held up through the whole build:
Feature checklist — where it actually lands vs. Lidarr
What's actually done (this is the bulk of the PR)
I'm not going to paste the full internal changelog here — it's long — but the shape of it:
Test status at the point of writing this: ~8,200 backend tests green, ~100 frontend tests (Vitest) green, zero lint/typecheck warnings, production build clean.
How to try it
It's off by default. To turn it on:
Set it either in
config/config.json(fresh installs, before the DB has a config row) or directly through the app's config store if you're on an existing install (the DB is the actual source of truth once it exists — editing the JSON file alone won't do anything on an established install). Restart the app afterward — the flag is only read at startup.Turning it on does not touch your existing library, files, or downloads. The only side effect is that finished downloads start getting mirrored into the new Library v2 tables in the background (so the new view stays in sync even for downloads that went through the old Wishlist/Watchlist flow) — that's bookkeeping only, it never writes to disk and never changes what gets downloaded.
What's still rough — please read this before testing
I'd rather list these myself than have someone find them and assume I didn't notice:
.jpgand servesimage/jpegregardless of what format the source image actually was. Usually harmless, occasionally shows a broken image for a non-JPEG provider result.None of these are secret landmines — they're all tracked, and none of them touch your files or your existing (non-Library-v2) workflows. But this is genuinely an early, opt-in feature, not a "flip it and forget it" one yet.
What I'm asking for
Try it on your actual library — especially if you're on a setup I can't easily replicate myself: very large libraries, path-mapped Docker volumes, multiple download sources (Usenet + torrent + Soulseek together), multiple user profiles, non-Plex media servers. Tell me what's confusing, what breaks, and what feels like it's missing compared to how you'd expect Lidarr to behave. I'd rather find the rough edges from real libraries now than after this becomes the default.
This has been a big chunk of work and I'm genuinely excited about it — but I also know "big chunk of work I'm excited about" is exactly the kind of PR that needs the most outside eyes before it merges, not the least. So: please poke at it. Break it. Tell me it's wrong. That's what this draft is for.
I screwed up — a force-push (git history cleanup) automatically closed the old PR #1025, and GitHub won't let it be reopened since then ("branch was force-pushed or recreated"). The branch/code is unchanged, only the PR container itself is gone. Here are the comments from last time so nothing gets lost:
@Nezreka (2026-07-14):
@nick2000713 (2026-07-14):
@Nezreka (2026-07-14):
@Nezreka (2026-07-14):
@nick2000713 (2026-07-14):
@nick2000713 (2026-07-14):
@Nezreka (2026-07-14):
@nick2000713 (2026-07-16):
@nick2000713 (2026-07-18):
@nick2000713 (2026-07-18):
@Nezreka (2026-07-19):
@nick2000713 (2026-07-19):