Add streaming load/save support for large editor files#158
Conversation
Agent-Logs-Url: https://github.com/gui-cs/Editor/sessions/a48dbc3d-9f40-4f88-9f28-b6d82dfe7dc6 Co-authored-by: tig <585482+tig@users.noreply.github.com>
Agent-Logs-Url: https://github.com/gui-cs/Editor/sessions/a48dbc3d-9f40-4f88-9f28-b6d82dfe7dc6 Co-authored-by: tig <585482+tig@users.noreply.github.com>
Agent-Logs-Url: https://github.com/gui-cs/Editor/sessions/a48dbc3d-9f40-4f88-9f28-b6d82dfe7dc6 Co-authored-by: tig <585482+tig@users.noreply.github.com>
Agent-Logs-Url: https://github.com/gui-cs/Editor/sessions/a48dbc3d-9f40-4f88-9f28-b6d82dfe7dc6 Co-authored-by: tig <585482+tig@users.noreply.github.com>
Agent-Logs-Url: https://github.com/gui-cs/Editor/sessions/a48dbc3d-9f40-4f88-9f28-b6d82dfe7dc6 Co-authored-by: tig <585482+tig@users.noreply.github.com>
Agent-Logs-Url: https://github.com/gui-cs/Editor/sessions/bd9d0914-6f84-4a5d-ba8a-e35a46e4a3a0 Co-authored-by: tig <585482+tig@users.noreply.github.com>
|
Just as a heads up, I was blocked by some firewall rules while working on your feedback. Expand below for details. Warning Firewall rules blocked me from connecting to one or more addresses (expand for details)I tried to connect to the following addresses, but was blocked by firewall rules:
If you need me to access, download, or install something from one of these locations, you can either:
|
…/github.com/gui-cs/Editor into copilot/resolve-open-003-large-file-support
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 089387a39f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 513f63062c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@copilot the spinnerview never updates during loading a large file. Are you sure the async path is correct and on a different thread than the ui thread? Is it notifying hte ui thread of progress in intevals that make sense? The loading shortcut's helptext should show the number of bytes (formatted nicely) while loading and the final file size when done (after the spinner is hidden) See CR feedback above and address. |
1 similar comment
|
@copilot the spinnerview never updates during loading a large file. Are you sure the async path is correct and on a different thread than the ui thread? Is it notifying hte ui thread of progress in intevals that make sense? The loading shortcut's helptext should show the number of bytes (formatted nicely) while loading and the final file size when done (after the spinner is hidden) See CR feedback above and address. |
Agent-Logs-Url: https://github.com/gui-cs/Editor/sessions/0e4ca842-fbd5-4ddf-9416-7b70a54d27f9 Co-authored-by: tig <585482+tig@users.noreply.github.com>
|
Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits. |
Agent-Logs-Url: https://github.com/gui-cs/Editor/sessions/0e4ca842-fbd5-4ddf-9416-7b70a54d27f9 Co-authored-by: tig <585482+tig@users.noreply.github.com>
…en-003-large-file-support
…e load Profiled: TextDocument.LoadAsync (rope+linetree+decode) of a 10 MiB / ~150k-line doc = 218 ms; walking every line = 5 ms. The ~10 s the user saw was entirely the Editor building + syntax-highlighting a CellVisualLine for *every* line — just to size the horizontal scrollbar — via RecomputeMaxWidth (Document swap / inline load) and UpdateMaxWidthIncremental (progressive per-chunk). Those built lines were then evicted from the LRU cache: pure throwaway work. VS Code is ~1 s because it only measures/tokenizes the viewport and estimates the rest. Fix (test-forward): - MeasureLineWidth(): documents >= 256 KiB use the line's character count (O(1), no build, no highlight) for the extent; smaller docs keep the exact tab/wide-glyph-precise computation, so existing behavior/tests are unchanged. - Draw path refines _maxVisualWidth exactly for lines it actually renders and OnDrawingContent reconciles content size once after the draw — so the horizontal scrollbar is correct for visible content and grows as wider lines scroll in (VS Code "estimate then refine"). Monotonic: no draw/layout loop. Tests written first, confirmed fail-first: - PerformanceTests/LargeDocumentLoadPerformanceTests: 10 MiB C#-highlighted Editor.LoadAsync < 3000 ms (was ~10 s). - Tests/MaxWidthEstimationTests: small doc stays exact tab-expanded; large doc uses char-length estimate (pins both branches + threshold). Suites: 467 unit + 237 integration green; dotnet format clean; ~/.tui stays clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ile-support' into copilot/resolve-open-003-large-file-support
There was a problem hiding this comment.
Pull request overview
Adds streaming file I/O support so TextDocument, Editor, and the ted sample can load/save large files without requiring a single full-document string materialization.
Changes:
- Adds
TextDocument.LoadAsync,SaveAsync, encoding metadata, and progress reporting. - Adds
Editordelegation APIs and updatestedto use stream-based open/save with status-bar progress. - Adds specs, upstream notes, and unit/integration/performance coverage for streaming behavior.
Reviewed changes
Copilot reviewed 21 out of 21 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
third_party/AvaloniaEdit/UPSTREAM.md |
Documents fork-specific streaming I/O additions. |
tests/Terminal.Gui.Editor.Tests/TextDocumentStreamingTests.cs |
Adds document/editor streaming round-trip, progress, and cancellation tests. |
tests/Terminal.Gui.Editor.PerformanceTests/StreamingLoadPerformanceTests.cs |
Adds initial-progress performance smoke test. |
tests/Terminal.Gui.Editor.IntegrationTests/TedAppTests.cs |
Updates ted tests for stream hooks and adds load/save progress coverage. |
tests/Terminal.Gui.Editor.IntegrationTests/EditorTabTests.cs |
Renames a tab behavior test. |
src/Terminal.Gui.Editor/Editor.FileIO.cs |
Adds editor-level async load/save delegation. |
src/Terminal.Gui.Editor/Editor.cs |
Avoids reading caret offset during dispose after ownership transfer. |
src/Terminal.Gui.Editor/Document/TextDocumentProgress.cs |
Adds progress reporting value type. |
src/Terminal.Gui.Editor/Document/TextDocument.cs |
Adds streaming document load/save, encoding metadata, and lazy ownership claiming. |
src/Terminal.Gui.Editor/AnsiInputProcessorState.cs |
Generalizes a Terminal.Gui version-specific comment. |
specs/public-api.md |
Documents new public streaming APIs. |
specs/file-io/spec.md |
Adds file-I/O feature specification. |
specs/decisions.md |
Records DEC-009 for streaming I/O placement. |
examples/ted/TedApp.FileOperations.cs |
Reworks ted open/save to stream files asynchronously with progress. |
examples/ted/TedApp.cs |
Adds load/save status UI and disables folding for very large documents. |
examples/ted/Program.cs |
Uses streaming open for startup file loading. |
examples/ted/InlineProgress.cs |
Adds synchronous progress helper for non-hosted scenarios. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
@copilot See the new CR feedback and address each. Ensrue there are tests that fail proving the feedback before fixing. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2d8a3698b8
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Agent-Logs-Url: https://github.com/gui-cs/Editor/sessions/d0bb09c4-3d15-4bed-8902-55c9cfc0c6b5 Co-authored-by: tig <585482+tig@users.noreply.github.com>
|
Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits. |
Agent-Logs-Url: https://github.com/gui-cs/Editor/sessions/d0bb09c4-3d15-4bed-8902-55c9cfc0c6b5 Co-authored-by: tig <585482+tig@users.noreply.github.com>
|
Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: cf73e98201
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Agent-Logs-Url: https://github.com/gui-cs/Editor/sessions/d0bb09c4-3d15-4bed-8902-55c9cfc0c6b5 Co-authored-by: tig <585482+tig@users.noreply.github.com>
|
Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits. |
Agent-Logs-Url: https://github.com/gui-cs/Editor/sessions/d0bb09c4-3d15-4bed-8902-55c9cfc0c6b5 Co-authored-by: tig <585482+tig@users.noreply.github.com>
|
Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits. |
Addressed in 4123f12. I added failing regression tests first for the load/save failure cleanup paths, legacy ReadAllText/WriteAllText hook compatibility, background-thread load assertion, and owner-thread documentation, then fixed the failures. No UI layout changes were made. Warning Firewall rules blocked me from connecting to one or more addresses (expand for details)I tried to connect to the following addresses, but was blocked by firewall rules:
If you need me to access, download, or install something from one of these locations, you can either:
|
|
@copilot CI/CD is failing |
Agent-Logs-Url: https://github.com/gui-cs/Editor/sessions/df14d2fd-3685-4dc1-86ee-a91a3ab83792 Co-authored-by: tig <585482+tig@users.noreply.github.com>
Addressed in 5483f7d. The failing CI job was Windows IntegrationTests: the canceled-save regression used |
Uh oh!
There was an error while loading. Please reload this page.