Skip to content

Feature/sweep until - #303

Closed
JamesWRC wants to merge 10 commits into
jon4hz:mainfrom
JamesWRC:feature/sweep_until
Closed

Feature/sweep until#303
JamesWRC wants to merge 10 commits into
jon4hz:mainfrom
JamesWRC:feature/sweep_until

Conversation

@JamesWRC

Copy link
Copy Markdown

Why

Myself and another person (see #158) are keen for this feature. Making it the ideal library management tool.

This filters control which content is eligible for deletion, but nothing previously controlled how much gets queued at once. On a large shared disk this can result in far more content being scheduled than the current disk pressure warrants. These new options let you express a target disk state and have jellysweep stop marking items once that target would be reached.

How it works

applySweepUntilLimit runs after the normal filter chain, before items are written to the database:

  1. Reads live disk stats via gopsutil/disk for each library's folder paths.
  2. Calculates bytes that need to be freed to reach the configured target.
  3. Subtracts bytes already pending deletion in the database (queued from previous runs, not yet deleted) so they aren't double-counted.
  4. Walks the filtered item list in order, including items until the budget is exhausted.

NOTE:

  • Shared mounts are handled correctly. If multiple libraries (e.g. Movies + TV Shows) sit on the same filesystem, they share a single budget keyed by the partition's total size. The most aggressive target across those libraries wins, and all libraries draw from the same pool, storage is not freed multiple times per library.
  • Tested on my own library (in dry run mode) and seems to be working well. My library is all on a single mount, so multiple mounts/volumes havent been tested but id imagine it would work fine. Can test the the image i have built jameswrc/jellysweep. All other filters and logic like 'Keep'ing an item marked for cleaning up, will grab the next lot of media to satisfy the quota on the next sweep.
  • All other tests have passed.
  • All code changes have been done by Claud Sonnet 4.6. All testing has been done manually and the outcome of the sweep_until_percent_used and sweep_until_gb_free values have been validated by me, and is as i expect.

Some other notes;
df reports usage as used / (used + available), not used / total — because the kernel reserves some blocks for root that are never shown as usable to normal processes. This implementation uses diskUsed + diskFree (where diskFree = Bavail via gopsutil) as the denominator, so 1sweep_until_percent_used: 67` means precisely the 67% you see in df output.

Testing

Test using sweep_until_percent_used set to 65. Should mark ~1TB

my df stats as of running this test 43560266240 27679889324 13682691796 67% /mnt/media
image

My storage was already 67% full, so setting sweep_until_percent_used to 65. should mean 2% is deleted.

Test using sweep_until_gb_free set to 14061

my df stats 43560266240 27679507848 13683073272 67% /mnt/media

13683073272 * 1024 / 1,000,000,000 = 14011
Test to delete only 50gb worth of media, 
sweep_until_gb_free: 14061
image

Commenting out both sweep_until_gb_free and sweep_until_percent_used

Would be as expected as per my config.
image

You can now set a sweep_until_gb_free or sweep_until_percent_used option in each library to only cleanup media if your available storage drops below a certain mark.

This respects all existing filter.
Copilot AI review requested due to automatic review settings March 22, 2026 12:29

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds per-library “sweep until” limits to cap how many items get queued for deletion in a cleanup run, targeting a desired disk state (percent used or GB free) to avoid over-queuing on large/shared disks.

Changes:

  • Introduces sweep_until_percent_used and sweep_until_gb_free library config options.
  • Implements applySweepUntilLimit to compute per-filesystem deletion budgets using live disk stats and already-pending DB items.
  • Updates README to document the new sweep-until behavior and configuration examples.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 6 comments.

File Description
internal/engine/engine.go Adds sweep-until limiting logic and integrates it into the mark-for-deletion pipeline.
internal/config/config.go Adds new per-library configuration fields for sweep-until targets.
README.md Documents sweep-until feature, options, and examples; updates TOC/feature list.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread README.md Outdated
Comment thread internal/engine/engine.go Outdated
Comment thread internal/engine/engine.go
Comment thread internal/engine/engine.go Outdated
Comment thread internal/engine/engine.go Outdated
Comment thread internal/config/config.go
JamesWRC and others added 2 commits March 22, 2026 23:33
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@JamesWRC
JamesWRC marked this pull request as draft March 22, 2026 12:39
JamesWRC and others added 2 commits March 23, 2026 20:42
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@JamesWRC

Copy link
Copy Markdown
Author

have tested the following cleanup_mode modes:

  • all
  • keep_episodes
  • keep_seasons

The few tests i have done with these modes seems to work as i would expect them to. With setting keep_count to 1 and 10, changed the number of items marked for clean up massively - and the GB estimated to be cleaned up looks to be about right.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread internal/engine/engine.go Outdated
Comment thread internal/engine/engine.go
Comment thread internal/engine/engine.go
Comment thread internal/config/config.go
@JamesWRC

JamesWRC commented Mar 25, 2026

Copy link
Copy Markdown
Author

Latest tests (via jameswrc/jellysweep image):

df (1K blocks) when testing: 43560266240 27753271652 13609309468 68% /mnt/media

Currently 68% full

sweep_until_percent_used: 65

Cleanup Mode Keep Count Est. GB Freed (via stats dashboard) Item Count
all 1 889 21
keep_episodes 10 891 19
keep_seasons 1 892 9

sweep_until_percent_used: 60

Cleanup Mode Keep Count Est. GB Freed (via stats dashboard) Item Count
all 10 ~3000 64
keep_episodes 10 ~3000 110
keep_seasons 1 ~3000 96

Currently free: 13935 GB

sweep_until_gb_free: 14000

Cleanup Mode Keep Count Est. GB Freed (via stats dashboard) Item Count
all 10 66 6
keep_episodes 10 67 13
keep_seasons 1 66 3

sweep_until_gb_free: 15500

Cleanup Mode Keep Count Est. GB Freed (via stats dashboard) Item Count
all 10 ~1600 35
keep_episodes 10 ~1600 30
keep_seasons 1 ~1600 20

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread internal/engine/engine.go Outdated
Comment thread internal/engine/engine.go Outdated
Comment thread internal/engine/engine.go Outdated
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@JamesWRC

JamesWRC commented Mar 25, 2026

Copy link
Copy Markdown
Author

Latest tests (via jameswrc/jellysweep image):

df (1K blocks) when testing: 43560266240 27773009416 13589571704 68% /mnt/media

Currently 68% full

sweep_until_percent_used: 65

Cleanup Mode Keep Count Est. GB Freed (via stats dashboard) Item Count
all 1 991 13
keep_episodes 10 ~1200 13
keep_seasons 1 ~1100 5

sweep_until_gb_free: 14000

Cleanup Mode Keep Count Est. GB Freed (via stats dashboard) Item Count
all 10 95 4
keep_episodes 10 681 11
keep_seasons 1 613 3

sweep_until_gb_free: 15500

Cleanup Mode Keep Count Est. GB Freed (via stats dashboard) Item Count
all 10 ~1700 17
keep_episodes 10 ~1800 17
keep_seasons 1 ~1900 11

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 6 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread internal/engine/engine.go
Comment thread internal/engine/engine.go
Comment thread internal/engine/engine.go
Comment thread internal/engine/engine.go
Comment thread internal/engine/engine.go
Comment thread internal/engine/engine.go
@JamesWRC

Copy link
Copy Markdown
Author

Closing as a better solution is in #305.

@JamesWRC JamesWRC closed this Mar 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants