Add configuration option to limit size of on-disk files

[chris124567]

Close #138

[Copilot]

Pull request overview

Adds an optional “pending on-disk bytes” limit to the Sia backend to prevent accumulating more local upload data than can be offloaded to Sia (per #138).

Changes:

• Introduces WithDiskUsageLimit and a diskUsageLimit field on the Sia backend.
• Adds Store.DiskUsage() and a SQLite implementation that computes pending on-disk bytes (objects + multipart parts), plus a test.
• Wires the limit into PutObject, UploadPart, UploadPartCopy, and CopyObject, and exposes configuration via s3d YAML config.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
sia/sia.go	Adds backend option and state for disk-usage limiting; extends Store interface.
sia/objects.go	Enforces disk-usage limit in PutObject and CopyObject.
sia/multipart.go	Enforces disk-usage limit in multipart part upload/copy flows.
sia/persist/sqlite/objects.go	Implements DiskUsage() by aggregating pending object/part sizes.
sia/persist/sqlite/objects_test.go	Adds test coverage for the new DiskUsage() query.
sia/persist/sqlite/init.sql	Adds an index related to pending objects.
sia/persist/sqlite/migrations_test.go	Updates expected schema to include the new index.
s3/s3errs/errors.go	Adds an S3 error for disk-usage limit exceeded.
cmd/s3d/config.go	Adds diskUsageLimit to YAML config struct.
cmd/s3d/main.go	Wires config value into sia.WithDiskUsageLimit.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Adds a configurable on-disk usage limit to prevent local pending uploads (including multipart parts) from growing without bound faster than they can be offloaded to Sia (closes #138).

Changes:

• Introduces a disk-usage limiting mechanism in the Sia backend that blocks new writes once the configured on-disk pending usage threshold is reached.
• Extends the SQLite store to compute pending disk usage and to return orphaned pending filenames on overwrite/copy/complete for cleanup.
• Wires the new setting into s3d config with a 10 GiB default and adds coverage for the new behavior.

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
sia/sia.go	Adds WithDiskUsageLimit, disk-usage state on the backend, and initializes disk-usage tracking from the store.
sia/objects.go	Implements reservation/release primitives and integrates limiting + overwrite cleanup into object flows.
sia/multipart.go	Applies disk-usage limiting to multipart part uploads/copies and releases usage on abort/cleanup.
sia/persist/sqlite/objects.go	Adds DiskUsage() aggregation and returns orphaned filenames from PutObject/CopyObject to enable disk cleanup.
sia/persist/sqlite/multipart.go	Returns orphaned filenames from multipart completion and extends HasMultipartUpload to indicate whether any parts exist.
sia/persist/sqlite/init.sql	Adds an index to improve pending-object queries (objects_pending_idx).
sia/persist/sqlite/migrations_test.go	Updates the baseline schema fixture used for migration consistency checks.
cmd/s3d/config.go	Adds diskUsageLimit to the YAML config schema.
cmd/s3d/main.go	Sets a default 10 GiB limit and passes the option into the Sia backend.
sia/persist/sqlite/objects_test.go	Adds store-level tests for DiskUsage() and updates call sites for signature changes.
sia/persist/sqlite/multipart_test.go	Updates multipart store tests for the new HasMultipartUpload/CompleteMultipartUpload signatures.
sia/objects_test.go	Adds end-to-end backend tests for disk usage limiting and overwrite cleanup.
sia/multipart_test.go	Updates a test call site for the new PutObject signature.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.