Closed
Conversation
michiosw
force-pushed
the
kontext-dev
branch
from
2b7b3d1 to
49bd073
Compare
michiosw
pushed a commit
that referenced
this pull request
Jan 10, 2026
…ai#8950) Agent wouldn't "see" attached images and would instead try to use the view_file tool: <img width="1516" height="504" alt="image" src="https://github.com/user-attachments/assets/68a705bb-f962-4fc1-9087-e932a6859b12" /> In this PR, we wrap image content items in XML tags with the name of each image (now just a numbered name like `[Image #1]`), so that the model can understand inline image references (based on name). We also put the image content items above the user message which the model seems to prefer (maybe it's more used to definitions being before references). We also tweak the view_file tool description which seemed to help a bit Results on a simple eval set of images: Before <img width="980" height="310" alt="image" src="https://github.com/user-attachments/assets/ba838651-2565-4684-a12e-81a36641bf86" /> After <img width="918" height="322" alt="image" src="https://github.com/user-attachments/assets/10a81951-7ee6-415e-a27e-e7a3fd0aee6f" /> ```json [ { "id": "single_describe", "prompt": "Describe the attached image in one sentence.", "images": ["image_a.png"] }, { "id": "single_color", "prompt": "What is the dominant color in the image? Answer with a single color word.", "images": ["image_b.png"] }, { "id": "orientation_check", "prompt": "Is the image portrait or landscape? Answer in one sentence.", "images": ["image_c.png"] }, { "id": "detail_request", "prompt": "Look closely at the image and call out any small details you notice.", "images": ["image_d.png"] }, { "id": "two_images_compare", "prompt": "I attached two images. Are they the same or different? Briefly explain.", "images": ["image_a.png", "image_b.png"] }, { "id": "two_images_captions", "prompt": "Provide a short caption for each image (Image 1, Image 2).", "images": ["image_c.png", "image_d.png"] }, { "id": "multi_image_rank", "prompt": "Rank the attached images from most colorful to least colorful.", "images": ["image_a.png", "image_b.png", "image_c.png"] }, { "id": "multi_image_choice", "prompt": "Which image looks more vibrant? Answer with 'Image 1' or 'Image 2'.", "images": ["image_b.png", "image_d.png"] } ] ```
michiosw
pushed a commit
that referenced
this pull request
Feb 7, 2026
I've seen this test fail with: ``` - Mock #1. Expected range of matching incoming requests: == 2 Number of matched incoming requests: 1 ``` This is because we pop the wrong task_complete events and then the test exits. I think this is because the MCP events are now buffered after openai#8874. So: 1. clear the buffer before we do any user message sending 2. additionally listen for task start before task complete 3. use the ID from task start to find the correct task complete event.
tumberger
pushed a commit
that referenced
this pull request
Feb 22, 2026
Fixes openai#9050 When a draft is stashed with Ctrl+C, we now persist the full draft state (text elements, local image paths, and pending paste payloads) in local history. Up/Down recall rehydrates placeholder elements and attachments so styling remains correct and large pastes still expand on submit. Persistent (cross‑session) history remains text‑only. Backtrack prefills now reuse the selected user message’s text elements and local image paths, so image placeholders/attachments rehydrate when rolling back. External editor replacements keep only attachments whose placeholders remain and then normalize image placeholders to `[Image #1]..[Image #N]` to keep the attachment mapping consistent. Docs: - docs/tui-chat-composer.md Testing: - just fix -p codex-tui - cargo test -p codex-tui Co-authored-by: Eric Traut <etraut@openai.com>
tumberger
pushed a commit
that referenced
this pull request
Feb 22, 2026
…i#10590) ## Summary This PR makes app-server-provided image URLs first-class attachments in TUI, so they survive resume/backtrack/history recall and are resubmitted correctly. <img width="715" height="491" alt="Screenshot 2026-02-12 at 8 27 08 PM" src="https://github.com/user-attachments/assets/226cbd35-8f0c-4e51-a13e-459ef5dd1927" /> Can delete the attached image upon backtracking: <img width="716" height="301" alt="Screenshot 2026-02-12 at 8 27 31 PM" src="https://github.com/user-attachments/assets/4558d230-f1bd-4eed-a093-8e1ab9c6db27" /> In both history and composer, remote images are rendered as normal `[Image #N]` placeholders, with numbering unified with local images. ## What changed - Plumb remote image URLs through TUI message state: - `UserHistoryCell` - `BacktrackSelection` - `ChatComposerHistory::HistoryEntry` - `ChatWidget::UserMessage` - Show remote images as placeholder rows inside the composer box (above textarea), and in history cells. - Support keyboard selection/deletion for remote image rows in composer (`Up`/`Down`, `Delete`/`Backspace`). - Preserve remote-image-only turns in local composer history (Up/Down recall), including restore after backtrack. - Ensure submit/queue/backtrack resubmit include remote images in model input (`UserInput::Image`), and keep request shape stable for remote-image-only turns. - Keep image numbering contiguous across remote + local images: - remote images occupy `[Image #1]..[Image #M]` - local images start at `[Image #M+1]` - deletion renumbers consistently. - In protocol conversion, increment shared image index for remote images too, so mixed remote/local image tags stay in a single sequence. - Simplify restore logic to trust in-memory attachment order (no placeholder-number parsing path). - Backtrack/replay rollback handling now queues trims through `AppEvent::ApplyThreadRollback` and syncs transcript overlay/deferred lines after trims, so overlay/transcript state stays consistent. - Trim trailing blank rendered lines from user history rendering to avoid oversized blank padding. ## Docs + tests - Updated: `docs/tui-chat-composer.md` (remote image flow, selection/deletion, numbering offsets) - Added/updated tests across `tui/src/chatwidget/tests.rs`, `tui/src/app.rs`, `tui/src/app_backtrack.rs`, `tui/src/history_cell.rs`, and `tui/src/bottom_pane/chat_composer.rs` - Added snapshot coverage for remote image composer states, including deleting the first of two remote images. ## Validation - `just fmt` - `cargo test -p codex-tui` ## Codex author `codex fork 019c2636-1571-74a1-8471-15a3b1c3f49d`
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Testing