fix: add 50MB file size limit to prevent memory spike in editor layout#12021
Closed
warp-dev-github-integration[bot] wants to merge 1 commit into
Closed
fix: add 50MB file size limit to prevent memory spike in editor layout#12021warp-dev-github-integration[bot] wants to merge 1 commit into
warp-dev-github-integration[bot] wants to merge 1 commit into
Conversation
When a very large file is opened in the code editor, the buffer loading and layout systems can consume excessive memory. The editor's layout system (SumTree<BlockItem>) creates multiple in-memory structures per line, so a file that is large on disk can consume 5-10x more memory once fully laid out. This commit adds a 50MB file size limit check in FileModel::open and FileModel::read_content_for_file. Files exceeding this threshold are rejected with FileLoadError::FileTooLarge instead of being loaded into memory and laid out. Sentry issue: https://sentry.io/organizations/warpdotdev/issues/7259255054/ Heap profile evidence: SumTree<BlockItem>::push consumed 12.67GB (72.59% of 17.45GB total) during RenderState::layout_edit_delta. Co-Authored-By: Oz <oz-agent@warp.dev>
Contributor
|
Closing as duplicate of #12125 — both add a 50 MB FileTooLarge file size guard in FileModel::open. Keeping the newest PR. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add a 50MB file size limit to the code editor's file loading path (
FileModel::openandFileModel::read_content_for_file) to prevent excessive memory usage when opening very large files.Root Cause
Sentry alert #7259255054 reports excessive memory usage (17.45GB sampled in heap profile). Analysis of the heap profile reveals:
SumTree<BlockItem>::pushinRenderState::layout_edit_delta— 12.67GB (72.59% of total)SumTree<BufferText>::append_strinpopulate_buffer_with_read_content— 2.08GB (11.94%)SumTree<BlockItem>::push_tree/push_tree_recursive— 1.85GB (10.63%)When a very large file is opened, the entire content is loaded into the buffer, and the editor's layout system creates
BlockItemnodes for every line. Since each line generates multiple in-memory structures (text frames, selections, caret positions, etc.), a file that is large on disk can consume 5-10x more memory once fully laid out.Changes
crates/warp_util/src/file.rs: AddedFileTooLarge { size_bytes, limit_bytes }variant toFileLoadErrorcrates/warp_files/src/lib.rs: AddedMAX_EDITOR_FILE_SIZE(50MB) constant andcheck_file_sizehelper. UpdatedFileModel::openandread_content_for_fileto check file size before reading.app/src/ai/blocklist/action_model/execute.rs: Handle newFileTooLargevariant in exhaustive matchTest Plan
cargo check -p warp_files -p warp_utilpassesFileLoadError::FileTooLarge, surfacing through the existingFailedToLoadevent pathFollow-up
Conversation: https://staging.warp.dev/conversation/eed39afb-2a7e-47ef-b94b-b15dcfc91cce
Run: https://oz.staging.warp.dev/runs/019e8484-219d-70b4-9dc3-af3b4d5d9c1c
This PR was generated with Oz.