fix(DATAGO-122883): Add total file upload limit for projects #910

shanechen-solace · 2026-01-28T20:29:01Z

What is the purpose of this change?

Add a limit to the total size of files that can be uploaded to a project to prevent unbounded storage usage across users.

How was this change implemented?

Added DEFAULT_MAX_TOTAL_UPLOAD_SIZE_BYTES constant and validation logic in project_service.py that checks the cumulative size of user-uploaded files before accepting new uploads, rejecting requests with a 400 error when the total would exceed the limit.

Key Design Decisions

For now, the limit per-file has been set to 50mb and the total upload limit is 100mb.

The total project size limit is the primary constraint rather than enforcing strict per-file limits, providing flexibility for users to manage their file sizes as needed within the overall quota. LLM-generated artifacts are intentionally excluded from the limit (filtered by source="project") to ensure that AI-generated content doesn't consume a user's upload quota, only their manually uploaded files count.

How was this change tested?

Manual testing: Manually added files of different sizes to projects to test the upload limit behavior.
Integration tests: Added a comprehensive test suite covering limit enforcement, valid uploads, ZIP imports, edge cases, and verification that only user-uploaded files count toward the limit.
Known limitations: Some test scenarios involving file deletion and replacement cannot be fully validated with the mock artifact service used in tests, but work correctly in production with a real artifact service.

Demos

SLACK: https://solacedotcom.slack.com/archives/C09TE7FNFAR/p1770050818570359?thread_ts=1769705717.347159&cid=C09TE7FNFAR

Screen.Recording.2026-01-30.at.4.51.46.PM.mov

Is there anything the reviewers should focus on/be aware of?

The test suite is about half of this PR.

github-actions · 2026-01-28T20:32:06Z

WhiteSource Policy Violation Summary

✅︎ No Blocking Whitesource Policy Violations found in solaceai/solace-agent-mesh-ui-pr-910!

Copilot

Pull request overview

This PR adds a total file upload size limit for projects to prevent unbounded storage usage. The implementation enforces a 100MB total project limit (with 50MB per-file limit) in production, while tests use 3MB/1MB limits respectively.

Changes:

Added DEFAULT_MAX_TOTAL_UPLOAD_SIZE_BYTES constant and validation logic to check cumulative user-uploaded file sizes
LLM-generated artifacts are excluded from the limit (filtered by source="project")
Frontend and backend both validate the total upload limit before accepting new files

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
tests/integration/apis/persistence/projects/test_projects_upload_limits.py	Comprehensive test suite covering limit enforcement, ZIP imports, edge cases, and verification that only user-uploaded files count toward limits
tests/integration/apis/conftest.py	Added test configuration override for 3MB total project size limit
src/solace_agent_mesh/gateway/http_sse/services/project_service.py	Core validation logic for total upload size limits, including helper function and limit checks in project creation and artifact upload
src/solace_agent_mesh/gateway/http_sse/routers/config.py	Exposed `maxTotalUploadSizeBytes` to frontend via config endpoint
src/solace_agent_mesh/gateway/constants.py	New constants file defining all upload size limit defaults
src/solace_agent_mesh/gateway/base/app.py	Added configuration parameter for total upload size limit and migrated to use centralized constants
client/webui/frontend/src/lib/utils/file-validation.ts	Added `validateTotalUploadSize` function for frontend validation
client/webui/frontend/src/lib/contexts/ConfigContext.ts	Added `maxTotalUploadSizeBytes` to config context type
client/webui/frontend/src/lib/components/projects/KnowledgeSection.tsx	Integrated total upload size validation in file upload flow

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

tests/integration/apis/persistence/projects/test_projects_upload_limits.py

src/solace_agent_mesh/gateway/http_sse/services/project_service.py

client/webui/frontend/src/lib/utils/file-validation.ts

Copilot

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-01-29T00:31:27Z

src/solace_agent_mesh/gateway/http_sse/services/project_service.py

+        existing_artifacts = await self.get_project_artifacts(db, project_id, user_id)
+        current_project_size = sum(
+            artifact.size for artifact in existing_artifacts
+            if artifact.source == "project" and artifact.size is not None  # Only count user-uploaded files
+        )


Calling get_project_artifacts on every file upload may be inefficient for projects with many artifacts. Consider caching the artifact list or implementing a more efficient size tracking mechanism (e.g., storing total size as project metadata) to avoid repeated database queries and artifact service calls.

lgh-solace

Looks clean and has some nice additions, but perhaps goes too far on the BE? Thoughts?

client/webui/frontend/src/lib/utils/file-validation.ts

client/webui/frontend/src/lib/components/projects/KnowledgeSection.tsx

lgh-solace · 2026-01-30T22:31:38Z

src/solace_agent_mesh/gateway/http_sse/utils/helpers.py

+    return size_bytes / (1024 * 1024)
+
+
+def sanitize_log_input(value: str) -> str:


I feel this function is currently unnecessary? We are logging user_id and project_id, but those aren't user controlled, they're just IDs and UUIDs.

lgh-solace · 2026-01-30T22:32:53Z

src/solace_agent_mesh/gateway/http_sse/utils/helpers.py

+"""
+
+
+def bytes_to_mb(size_bytes: int) -> float:


This one could be useful - but not sure it's useful enough to add to the shared helper since its only one line... 🤔

Edit: Maybe we could keep it in project_service.py?

JKaram

Looking good.

Just be mindful of the comments the AI leaves. Some of them are very redundant.

client/webui/frontend/src/lib/components/projects/KnowledgeSection.tsx

JKaram · 2026-02-02T14:53:49Z

client/webui/frontend/src/lib/components/projects/KnowledgeSection.tsx

+            // 1. Validate individual file sizes
+            const fileSizeResult = validateFileSizes(files, { maxSizeBytes: maxUploadSizeBytes });
+            if (!fileSizeResult.valid) {
+                return fileSizeResult;
+            }
+
+            // 2. Validate total upload limit
+            const totalUploadSizeResult = validateTotalUploadSize(currentProjectSize, files, maxTotalUploadSizeBytes);
+            if (!totalUploadSizeResult.valid) {
+                return { valid: false, error: totalUploadSizeResult.error };
+            }
+
+            return { valid: true };


Can you remove the unnecessary comments. // 1. Validate individual file sizes ...

JKaram · 2026-02-02T14:56:07Z

client/webui/frontend/src/lib/utils/file-validation.ts

+ */
+export function validateTotalUploadSize(currentProjectSizeBytes: number, newFiles: FileList | File[], maxTotalUploadSizeBytes?: number): TotalUploadSizeValidationResult {
+    const fileArray = Array.from(newFiles);
+    const newSize = fileArray.reduce((sum, file) => sum + file.size, 0);


This would be a good place for the util I mentioned above. We might even have one already.

…nt breaking

…d-limit Signed-off-by: Shane Chen <shane.chen@solace.com>

sonarqube-solacecloud · 2026-02-09T21:23:04Z

Quality Gate failed

Failed conditions
B Security Rating on New Code (required ≥ A)

See analysis details on SonarQube

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

shanechen-solace added 3 commits January 28, 2026 12:18

fix: add a total user file upload limit and validation to projects

6df7575

test: tests for total file upload limits

fd2b574

chore: lint

d9ada70

shanechen-solace changed the title ~~fix(DATAGO-122175): Add total file upload limit~~ fix(DATAGO-122883): Add total file upload limit for projects Jan 28, 2026

shanechen-solace requested a review from Copilot January 28, 2026 20:32

Copilot AI reviewed Jan 28, 2026

View reviewed changes

shanechen-solace added 3 commits January 28, 2026 16:13

chore: improve code quality

3da88ca

chore: fix vulnerabilities

4b57dec

chore: fix vuln

c21425a

shanechen-solace requested a review from Copilot January 29, 2026 00:30

Copilot AI reviewed Jan 29, 2026

View reviewed changes

shanechen-solace marked this pull request as ready for review January 29, 2026 00:41

fix: make tests more assertive

d48b99b

shanechen-solace requested a review from lgh-solace January 30, 2026 15:19

moodiRealist requested a review from JKaram January 30, 2026 15:35

chore: update tests

f1b5162

lgh-solace reviewed Jan 30, 2026

View reviewed changes

JKaram requested changes Feb 2, 2026

View reviewed changes

shanechen-solace added 2 commits February 2, 2026 11:31

chore: reorganize

fd0905d

chore: fix typecheck failure

4440dda

shanechen-solace requested review from JKaram and lgh-solace February 2, 2026 17:25

shanechen-solace marked this pull request as draft February 3, 2026 15:54

shanechen-solace added 6 commits February 9, 2026 09:21

add batch upload limit to backend

0667eca

fix: frontend changes to new limits

e36b8ed

fix: update gateway keys, keep gateway_max_upload_size_bytes to preve…

3a654cb

…nt breaking

test: update tests

11dd562

Merge branch 'main' into shanechen/DATAGO-122883/add-total-file-uploa…

3deb21e

…d-limit Signed-off-by: Shane Chen <shane.chen@solace.com>

fix: fix problem with previous merge

53b3f97

shanechen-solace closed this Feb 10, 2026

		return size_bytes / (1024 * 1024)


		def sanitize_log_input(value: str) -> str:

fix(DATAGO-122883): Add total file upload limit for projects #910

fix(DATAGO-122883): Add total file upload limit for projects #910

Uh oh!

Conversation

shanechen-solace commented Jan 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What is the purpose of this change?

How was this change implemented?

Key Design Decisions

For now, the limit per-file has been set to 50mb and the total upload limit is 100mb.

How was this change tested?

Demos

Is there anything the reviewers should focus on/be aware of?

Uh oh!

github-actions bot commented Jan 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

WhiteSource Policy Violation Summary

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Jan 29, 2026

Choose a reason for hiding this comment

Uh oh!

lgh-solace left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

lgh-solace Jan 30, 2026

Choose a reason for hiding this comment

Uh oh!

lgh-solace Jan 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

JKaram left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

JKaram Feb 2, 2026

Choose a reason for hiding this comment

Uh oh!

JKaram Feb 2, 2026

Choose a reason for hiding this comment

Uh oh!

sonarqube-solacecloud bot commented Feb 9, 2026

Quality Gate failed

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

shanechen-solace commented Jan 28, 2026 •

edited

Loading

github-actions bot commented Jan 28, 2026 •

edited

Loading

lgh-solace Jan 30, 2026 •

edited

Loading