Skip to content

Comments

feat: add Vertex AI provider support#88

Open
ngattusohw wants to merge 2 commits intosteipete:mainfrom
ngattusohw:feat/vertex-ai-auth
Open

feat: add Vertex AI provider support#88
ngattusohw wants to merge 2 commits intosteipete:mainfrom
ngattusohw:feat/vertex-ai-auth

Conversation

@ngattusohw
Copy link

Summary

Adds Google Cloud Vertex AI as a new LLM provider, enabling authentication via service account credentials and Application Default Credentials (ADC) instead of requiring a GEMINI_API_KEY.

This is useful for teams and production environments that use GCP service accounts, or for anyone running on GCP infrastructure where ADC is automatically available.

New Provider

Prefix: vertex/... (e.g. summarize --model vertex/gemini-2.5-flash-preview https://example.com)

Environment Variables

Variable Description
GOOGLE_CLOUD_PROJECT GCP project ID (alias: VERTEX_AI_PROJECT_ID)
GOOGLE_CLOUD_LOCATION GCP region, e.g. us-central1 (alias: VERTEX_AI_LOCATION)
VERTEX_AI_SERVICE_ACCOUNT_KEY Inline service account JSON
GOOGLE_APPLICATION_CREDENTIALS Path to service account JSON file

Also supports GCE metadata server for auth when running on GCP.

Architecture

  • New src/llm/providers/vertex.ts with JWT-based service account auth and token caching
  • Leverages pi-ai's existing google-vertex API provider for streaming (which uses @google/genai with vertexai: true)
  • For document attachments, uses direct REST calls with bearer token auth
  • Token is cached with TTL to avoid re-signing JWTs on every call

Auth Priority

  1. VERTEX_AI_SERVICE_ACCOUNT_KEY / GOOGLE_SERVICE_ACCOUNT_KEY (inline JSON)
  2. GOOGLE_APPLICATION_CREDENTIALS (file path to SA JSON)
  3. GCE metadata server (when running on GCP)
  4. gcloud auth print-access-token (local dev fallback)

Changes

  • Added vertex to LlmProvider type union
  • Updated model-id.ts, model-spec.ts, model-auto.ts for vertex provider routing
  • Threaded vertexConfig through generate-text.ts, summary-engine.ts, summary-llm.ts, runner.ts, and daemon flow-context.ts
  • Updated CLI help text to document vertex env vars and model prefix
  • All existing tests pass (56/56 in relevant test files)

Files Changed (12)

  • src/llm/providers/vertex.tsnew Vertex AI provider
  • src/llm/providers/models.ts — added resolveVertexModel()
  • src/llm/model-id.ts — added vertex provider
  • src/llm/generate-text.ts — vertex dispatch for complete + stream
  • src/model-spec.ts — vertex model spec parsing
  • src/model-auto.ts — vertex env key detection for auto mode
  • src/run/run-env.tsvertexConfig + vertexConfigured in env state
  • src/run/summary-engine.ts — threaded vertexConfig
  • src/run/summary-llm.ts — threaded vertexConfig
  • src/run/runner.ts — destructure + pass vertexConfig
  • src/run/help.ts — CLI help text
  • src/daemon/flow-context.ts — daemon integration

AVA added 2 commits February 15, 2026 13:38
Add support for Google Cloud Vertex AI as a new provider, enabling
authentication via service account credentials and Application Default
Credentials (ADC) instead of API keys.

New provider prefix: vertex/... (e.g. vertex/gemini-2.5-flash-preview)

Environment variables:
- GOOGLE_CLOUD_PROJECT / VERTEX_AI_PROJECT_ID - GCP project ID
- GOOGLE_CLOUD_LOCATION / VERTEX_AI_LOCATION - GCP region
- VERTEX_AI_SERVICE_ACCOUNT_KEY - inline service account JSON
- GOOGLE_APPLICATION_CREDENTIALS - path to SA JSON file
- Also supports GCE metadata server for auth when on GCP

Changes:
- New vertex provider (src/llm/providers/vertex.ts) with JWT-based
  service account auth and token caching
- Leverages pi-ai's existing google-vertex API for streaming
- Added 'vertex' to LlmProvider type union
- Updated model-id, model-spec, model-auto for vertex provider
- Threaded vertexConfig through generate-text, summary-engine,
  summary-llm, runner, and daemon flow-context
- Updated CLI help text to document vertex env vars and model prefix
- All existing tests pass (56/56 in relevant test files)
Fix build errors by adding 'vertex' to provider type unions in:
- costs.ts, attachments.ts, streaming.ts, types.ts
- flows/url/markdown.ts, flows/asset/preprocess.ts
- summary-engine.ts (envHasKeyFor + error message)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant