feat: add configurable vision proxy#127
Merged
Merged
Conversation
There was a problem hiding this comment.
Pull request overview
This PR introduces a configurable “Vision Proxy” system so the extension can convert image attachments into text (via either a VS Code vision-capable LM or a user-configured HTTP endpoint) before forwarding the request to DeepSeek.
Changes:
- Added a Vision Proxy configuration webview (source selection, endpoint settings, API key handling, and connection testing).
- Implemented endpoint-based vision adapters (OpenAI Chat Completions, OpenAI Responses, Anthropic Messages) with structured diagnostics/errors and custom header support.
- Refactored vision resolution to use a
VisionDescriberabstraction and to surface “missing/failure” notices into the initial response.
Reviewed changes
Copilot reviewed 39 out of 39 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| src/runtime/provider.ts | Wires the setVisionModel command to the provider’s new configuration entrypoint. |
| src/runtime/actions.ts | Adds a URI action for opening Vision Proxy configuration and propagates action URLs into provider notices. |
| src/provider/vision/ui/style.ts | Provides CSS styling for the Vision Proxy configuration webview. |
| src/provider/vision/ui/script.ts | Implements client-side UI logic (state rendering, validation, save/test flows) for the Vision Proxy webview. |
| src/provider/vision/ui/panel.ts | Implements the webview panel host, message handling, persistence, and connection testing. |
| src/provider/vision/ui/html.ts | Generates secure webview HTML (CSP, nonce, escaped JSON bootstrapping) for the configuration panel. |
| src/provider/vision/types.ts | Introduces shared types for vision proxy sources/configs and the VisionDescriber interface. |
| src/provider/vision/sources/vscode/index.ts | Adds a VS Code LM-backed VisionDescriber with model option listing and pricing display helpers. |
| src/provider/vision/sources/endpoint/test.ts | Adds a connection test flow for endpoint-based vision proxies (captcha-style image + prompt). |
| src/provider/vision/sources/endpoint/index.ts | Implements an endpoint-backed VisionDescriber wrapper around VisionProxyClient. |
| src/provider/vision/sources/endpoint/config.ts | Adds storage + normalization/validation for endpoint configuration and API key secret handling. |
| src/provider/vision/service.ts | Introduces a vision service that selects between VS Code LM and endpoint sources and opens configuration UI. |
| src/provider/vision/resolve.ts | Refactors image resolution to use VisionDescriber, adds failure/missing notices, and improves diagnostics. |
| src/provider/vision/protocols/url.ts | Adds URL validation/resolution utilities for endpoint configurations. |
| src/provider/vision/protocols/providers/utils.ts | Adds provider utilities (base64 encoding, record guard). |
| src/provider/vision/protocols/providers/types.ts | Defines adapter interface for provider-specific request/response shapes. |
| src/provider/vision/protocols/providers/openai/responses.ts | Adds OpenAI Responses adapter (request body + response parsing). |
| src/provider/vision/protocols/providers/openai/chat.ts | Adds OpenAI Chat Completions adapter (request body + response parsing). |
| src/provider/vision/protocols/providers/index.ts | Selects the appropriate provider adapter based on config. |
| src/provider/vision/protocols/providers/anthropic/messages.ts | Adds Anthropic Messages adapter (request body + response parsing). |
| src/provider/vision/protocols/headers.ts | Adds custom header normalization and provider header construction (OpenAI/Anthropic auth). |
| src/provider/vision/protocols/errors.ts | Adds a dedicated Vision Proxy error model + diagnostics formatting and display code mapping. |
| src/provider/vision/protocols/client.ts | Adds HTTP client for vision proxy endpoints with timeout/cancellation and structured errors. |
| src/provider/vision/model.ts | Removes legacy “pick vision proxy model” implementation (superseded by new service/UI). |
| src/provider/vision/index.ts | Updates public vision exports to the new service/types surface. |
| src/provider/tools/notices.ts | Adds vision-proxy notices (missing/failure) and extends provider-notice stripping logic. |
| src/provider/tools/consts.ts | Adds notice markers for vision proxy notices. |
| src/provider/request.ts | Integrates VisionDescriber resolution into request preparation and propagates initial notices. |
| src/provider/index.ts | Switches provider vision handling to createVisionService and merges toolflow + vision notices. |
| src/provider/debug/dump.ts | Extends debug dumps to include the vision proxy source alongside model/stats. |
| src/provider/debug/diagnostics.ts | Extends diagnostics tracing to include vision proxy source. |
| src/i18n.ts | Adds i18n strings for vision proxy UI, errors, and notices (EN/ZH). |
| src/extension.ts | Updates runtime entrypoint import path (barrel export). |
| src/consts.ts | Adds URI path constant for opening vision model configuration. |
| src/client/error/network.ts | Extracts network error cause/code/category logic into a shared module. |
| src/client/error/index.ts | Refactors request error normalization to reuse the new shared network error utilities. |
| package.nls.zh-cn.json | Updates command/config descriptions to point users to the new Vision Proxy configuration UI. |
| package.nls.json | Updates command/config descriptions to point users to the new Vision Proxy configuration UI. |
| package.json | Updates configuration schema to use markdown descriptions and tags for vision-related settings. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
8a42df6 to
1ff65c0
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Verification