Skip to content

feat: add configurable vision proxy#127

Merged
Vizards merged 12 commits into
mainfrom
feat/vision-proxy-api-endpoint
Jun 2, 2026
Merged

feat: add configurable vision proxy#127
Vizards merged 12 commits into
mainfrom
feat/vision-proxy-api-endpoint

Conversation

@Vizards

@Vizards Vizards commented Jun 2, 2026

Copy link
Copy Markdown
Owner

Summary

  • add a Vision Proxy configuration panel with VS Code LM and API endpoint sources
  • add OpenAI-compatible chat/completions, OpenAI responses, and Anthropic messages endpoint support
  • add vision proxy warning notices, diagnostics, output logging, i18n, and endpoint testing UI

Verification

  • npm run format:check
  • npm run compile
  • npm run lint
  • git diff --check

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a configurable “Vision Proxy” system so the extension can convert image attachments into text (via either a VS Code vision-capable LM or a user-configured HTTP endpoint) before forwarding the request to DeepSeek.

Changes:

  • Added a Vision Proxy configuration webview (source selection, endpoint settings, API key handling, and connection testing).
  • Implemented endpoint-based vision adapters (OpenAI Chat Completions, OpenAI Responses, Anthropic Messages) with structured diagnostics/errors and custom header support.
  • Refactored vision resolution to use a VisionDescriber abstraction and to surface “missing/failure” notices into the initial response.

Reviewed changes

Copilot reviewed 39 out of 39 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
src/runtime/provider.ts Wires the setVisionModel command to the provider’s new configuration entrypoint.
src/runtime/actions.ts Adds a URI action for opening Vision Proxy configuration and propagates action URLs into provider notices.
src/provider/vision/ui/style.ts Provides CSS styling for the Vision Proxy configuration webview.
src/provider/vision/ui/script.ts Implements client-side UI logic (state rendering, validation, save/test flows) for the Vision Proxy webview.
src/provider/vision/ui/panel.ts Implements the webview panel host, message handling, persistence, and connection testing.
src/provider/vision/ui/html.ts Generates secure webview HTML (CSP, nonce, escaped JSON bootstrapping) for the configuration panel.
src/provider/vision/types.ts Introduces shared types for vision proxy sources/configs and the VisionDescriber interface.
src/provider/vision/sources/vscode/index.ts Adds a VS Code LM-backed VisionDescriber with model option listing and pricing display helpers.
src/provider/vision/sources/endpoint/test.ts Adds a connection test flow for endpoint-based vision proxies (captcha-style image + prompt).
src/provider/vision/sources/endpoint/index.ts Implements an endpoint-backed VisionDescriber wrapper around VisionProxyClient.
src/provider/vision/sources/endpoint/config.ts Adds storage + normalization/validation for endpoint configuration and API key secret handling.
src/provider/vision/service.ts Introduces a vision service that selects between VS Code LM and endpoint sources and opens configuration UI.
src/provider/vision/resolve.ts Refactors image resolution to use VisionDescriber, adds failure/missing notices, and improves diagnostics.
src/provider/vision/protocols/url.ts Adds URL validation/resolution utilities for endpoint configurations.
src/provider/vision/protocols/providers/utils.ts Adds provider utilities (base64 encoding, record guard).
src/provider/vision/protocols/providers/types.ts Defines adapter interface for provider-specific request/response shapes.
src/provider/vision/protocols/providers/openai/responses.ts Adds OpenAI Responses adapter (request body + response parsing).
src/provider/vision/protocols/providers/openai/chat.ts Adds OpenAI Chat Completions adapter (request body + response parsing).
src/provider/vision/protocols/providers/index.ts Selects the appropriate provider adapter based on config.
src/provider/vision/protocols/providers/anthropic/messages.ts Adds Anthropic Messages adapter (request body + response parsing).
src/provider/vision/protocols/headers.ts Adds custom header normalization and provider header construction (OpenAI/Anthropic auth).
src/provider/vision/protocols/errors.ts Adds a dedicated Vision Proxy error model + diagnostics formatting and display code mapping.
src/provider/vision/protocols/client.ts Adds HTTP client for vision proxy endpoints with timeout/cancellation and structured errors.
src/provider/vision/model.ts Removes legacy “pick vision proxy model” implementation (superseded by new service/UI).
src/provider/vision/index.ts Updates public vision exports to the new service/types surface.
src/provider/tools/notices.ts Adds vision-proxy notices (missing/failure) and extends provider-notice stripping logic.
src/provider/tools/consts.ts Adds notice markers for vision proxy notices.
src/provider/request.ts Integrates VisionDescriber resolution into request preparation and propagates initial notices.
src/provider/index.ts Switches provider vision handling to createVisionService and merges toolflow + vision notices.
src/provider/debug/dump.ts Extends debug dumps to include the vision proxy source alongside model/stats.
src/provider/debug/diagnostics.ts Extends diagnostics tracing to include vision proxy source.
src/i18n.ts Adds i18n strings for vision proxy UI, errors, and notices (EN/ZH).
src/extension.ts Updates runtime entrypoint import path (barrel export).
src/consts.ts Adds URI path constant for opening vision model configuration.
src/client/error/network.ts Extracts network error cause/code/category logic into a shared module.
src/client/error/index.ts Refactors request error normalization to reuse the new shared network error utilities.
package.nls.zh-cn.json Updates command/config descriptions to point users to the new Vision Proxy configuration UI.
package.nls.json Updates command/config descriptions to point users to the new Vision Proxy configuration UI.
package.json Updates configuration schema to use markdown descriptions and tags for vision-related settings.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/provider/vision/sources/endpoint/config.ts Outdated
@Vizards Vizards force-pushed the feat/vision-proxy-api-endpoint branch from 8a42df6 to 1ff65c0 Compare June 2, 2026 10:28
@Vizards Vizards requested a review from Copilot June 2, 2026 10:30

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 39 out of 39 changed files in this pull request and generated 1 comment.

Comment thread src/runtime/actions.ts Outdated

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 39 out of 39 changed files in this pull request and generated 2 comments.

Comment thread src/provider/vision/protocols/errors.ts
Comment thread src/provider/vision/protocols/client.ts

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 39 out of 39 changed files in this pull request and generated 1 comment.

Comment thread src/provider/vision/ui/html.ts

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 39 out of 39 changed files in this pull request and generated 1 comment.

Comment thread src/provider/vision/sources/vscode/index.ts

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 39 out of 39 changed files in this pull request and generated 1 comment.

Comment thread src/provider/vision/protocols/errors.ts

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 39 out of 39 changed files in this pull request and generated 3 comments.

Comment thread src/provider/vision/ui/html.ts
Comment thread src/i18n.ts
Comment thread src/i18n.ts

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 39 out of 39 changed files in this pull request and generated 2 comments.

Comment thread src/provider/vision/protocols/client.ts
Comment thread src/provider/vision/sources/endpoint/config.ts

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 39 out of 39 changed files in this pull request and generated 1 comment.

Comment thread src/provider/vision/protocols/providers/utils.ts

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 39 out of 39 changed files in this pull request and generated 2 comments.

Comment thread src/i18n.ts
Comment thread src/i18n.ts

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 39 out of 39 changed files in this pull request and generated 2 comments.

Comment thread src/provider/vision/sources/endpoint/test.ts
Comment thread src/provider/vision/protocols/errors.ts

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 39 out of 39 changed files in this pull request and generated 1 comment.

Comment thread src/provider/vision/ui/script.ts

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 39 out of 39 changed files in this pull request and generated 2 comments.

Comment thread src/provider/vision/service.ts
Comment thread src/provider/vision/service.ts

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 39 out of 39 changed files in this pull request and generated 3 comments.

Comment thread src/provider/vision/protocols/headers.ts
Comment thread src/provider/vision/protocols/headers.ts
Comment thread src/provider/vision/sources/endpoint/config.ts

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 39 out of 39 changed files in this pull request and generated no new comments.

@Vizards Vizards merged commit f5d9b8d into main Jun 2, 2026
3 checks passed
@Vizards Vizards deleted the feat/vision-proxy-api-endpoint branch June 2, 2026 13:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants