Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 2 additions & 22 deletions .env.quickstart.example
Original file line number Diff line number Diff line change
Expand Up @@ -188,23 +188,11 @@ POSTHOG_ENABLED=false
# FIGMA_ACCESS_TOKEN=

# --- Rosetta / KnowledgeBase (project knowledge for codegen agents) ---
# Two modes — choose one:
#
# Mode 1: Bundled plugin (default, open-source quickstart)
# Generic KB, runs locally, no external credentials needed.
# Leave ROSETTA_MCP_ENABLED=false and keep the default ROSETTA_PLUGIN_PATH.
#
# Mode 2: Live MCP (project-tailored KB, Grid Dynamics managed)
# Connects to the Grid Dynamics IMS service for project-specific knowledge.
# Set ROSETTA_MCP_ENABLED=true and fill in the credentials below.
# Live MCP takes precedence when both are configured.
# The Rosetta knowledge base ships as a plugin baked into the Docker image. It is a generic KB,
# runs locally, and needs no external credentials.
#
# Docs: https://github.com/griddynamics/rosetta
#
# ROSETTA_MCP_ENABLED
# What: Toggle between bundled plugin (false) and live IMS-MCP server (true).
ROSETTA_MCP_ENABLED=false
#
# ROSETTA_PLUGIN_PATH
# What: Path to the generic KB plugin baked into the Docker image.
# How: Keep as-is. Only change if you mount a custom plugin.
Expand All @@ -215,14 +203,6 @@ ROSETTA_PLUGIN_PATH=/opt/rosetta-plugin
# the image at BUILD time (build arg, not a runtime setting). Pins the plugin version.
# How: Bump to a newer tag and rebuild the backend image to upgrade the bundled KB plugin.
ROSETTA_MARKETPLACE_REF=v2.0.55
#
# ROSETTA_API_KEY / ROSETTA_USER_EMAIL / ROSETTA_SERVER_URL
# What: Credentials for the live IMS-MCP Rosetta service.
# Only needed when ROSETTA_MCP_ENABLED=true.
# How: Obtain from your Grid Dynamics IMS admin.
# ROSETTA_API_KEY=
# ROSETTA_USER_EMAIL=
# ROSETTA_SERVER_URL=https://ims.evergreen.gcp.griddynamics.net/

# --- Agent MCP servers ---
# MCP_SERVERS_ENABLED
Expand Down
8 changes: 3 additions & 5 deletions agents/IMPLEMENTATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
- **Database**: `backend/app/database/` — firestore, emulator, in_memory (IDatabase interface)
- **Prompts**: `backend/app/prompts/agents_claude_code.py` — all agent templates
- **Schemas**: `backend/app/schemas/` — estimation_enums (statuses/checkpoints), specification (SpecReadiness)
- **Config**: `backend/app/core/config.py` — env vars including `TOKEN_ENCRYPTION_KEY`, `GITHUB_TOKEN_DEFAULT`, `GIT_USER_NAME_DEFAULT`, K8s secret key name settings; Rosetta `ims-mcp` via `uvx ims-mcp@latest` with `ROSETTA_SERVER_URL`, `ROSETTA_API_KEY`, `ROSETTA_USER_EMAIL`, `ROSETTA_IMS_VERSION` → subprocess `VERSION` (`build_rosetta_mcp_config`); `github_platform_secrets.py` loads Fernet + default PAT (K8s API or env); `workspace_pool_names.py` — pool allowlist; `artifact_subdirs.py` — `ANALYSIS_SUBDIR`, `PLANNING_SUBDIR`, `REPORT_SUBDIR` (SSOT for artifact/workspace output paths); `backend/app/prompts/mcp_workflow_registry.py` — matrix of optional agent MCPs per workflow + prune keyword field map
- **Config**: `backend/app/core/config.py` — env vars including `TOKEN_ENCRYPTION_KEY`, `GITHUB_TOKEN_DEFAULT`, `GIT_USER_NAME_DEFAULT`, K8s secret key name settings; Rosetta KB via the bundled plugin at `ROSETTA_PLUGIN_PATH` (see `app/core/rosetta_kb.py` + `WorkspaceManager.provision_rosetta_plugin`); `github_platform_secrets.py` loads Fernet + default PAT (K8s API or env); `workspace_pool_names.py` — pool allowlist; `artifact_subdirs.py` — `ANALYSIS_SUBDIR`, `PLANNING_SUBDIR`, `REPORT_SUBDIR` (SSOT for artifact/workspace output paths); `backend/app/prompts/mcp_workflow_registry.py` — matrix of optional agent MCPs per workflow + prune keyword field map
- **MCP**: `server.py` — tools: check_specification_completeness, run_planning, run_generation, check_status, retry_generation, download_estimation_outputs; workspace tools take `ctx: Context` and call `apply_mcp_project_root_from_context` so `resolve_path` / session file resolve before `gain_session.json` exists (MCP `list_roots()` → Pydantic `FileUrl` → `_project_root`).

## Key Decisions (PR83)
Expand Down Expand Up @@ -86,8 +86,6 @@
`test_estimation` (`update_completed_estimation_result`), `test_report_generation` (skipped/coverage sections),
`test_workflow_integration`, `test_model_selection`.

- **Rosetta unpack in standalone planning + Rosetta MCP on codegen (Apr 9, ops v0.4.0 #3)**: `planning_workflow` calls `unpack_rosetta_artifacts` on the primary workspace after KB init (or KB_INIT_DONE resume) and before `run_planning_agent`, matching full-estimation behavior from `prepare_parallel_workspaces`. `coding_mcp_servers_and_tools` merges `_rosetta_pair` when `ROSETTA_MCP_ENABLED` so generation/deploy phase agents get KnowledgeBase MCP like planning/KB init. `unpack_rosetta_artifacts` also maps `rosetta/skills/` and `rosetta/commands/` to `.claude/skills/` and `.claude/commands/` (same pattern as `rosetta/agents/`). Tests: `test_agents_claude_code` (coding MCP + rosetta), `test_execute_all_phases_mcps` mocks set `ROSETTA_MCP_ENABLED=False` where isolating Playwright/Figma.

- **Async spec analyze + usage counters + milestone notifications (Apr 3)**: `POST /specification/analyze` returns immediately; background task awaits `archive_analysis` before clearing `spec_analysis_in_progress` and sending email/Slack `notify_spec_check_complete`. `GET /specification/outputs/{ws}` → **410 Gone**; MCP uses short analyze timeout and `download_outputs` only. Cumulative `num_turns` / `total_tokens_used` on estimation docs via transactional `add_agent_query_totals` from successful `agent_query` (TelemetryContext handler from `build_workflow_context`, spec/planning bg tasks, estimation run/retry). `GET .../estimations/{id}/status` + MCP `check_status` expose counters and `total_tokens_used_display` (`format_token_count`). `notify_planning_complete` after `archive_planning` + `PLANNING_DONE` in `planning_workflow`. Edge-case tests: `test_specification_analyze_async_edges.py` (API key lock, `begin_analysis` failure, workflow/archive failures, archive-before-notify ordering, notify failure cleanup), `test_get_estimation_status_includes_usage_and_spec_analysis_fields`. **TODO**: crash recovery could clear stale `spec_analysis_in_progress` if the process dies mid-task.

## Recently Completed (March 2026)
Expand All @@ -104,7 +102,7 @@
flows from MCP `server.py` / sync params → estimation `parameters` → workflows. Backend `SUPPORTED_MCPS`
(`playwright`, `figma`) filters names; `FIGMA_ACCESS_TOKEN` / `FIGMA_API_KEY` + `FIGMA_MCP_*` / `PLAYWRIGHT_MCP_*`
in backend env build stdio MCP configs. Playwright wired for generation + deploy/E2E phase agents; Figma for
spec analysis, planning (with Rosetta), generation, and deploy/E2E. PostHog event `mcp_servers_configuration`
spec analysis, planning, generation, and deploy/E2E. PostHog event `mcp_servers_configuration`
records `mcp_configuration_source` (form / estimation_parameters / backend_settings / workspace_sync_params),
`mcp_configuration_raw`, resolved enablement, and `mcp_supported_ids` at that moment. API form fields + tests in
`test_mcp_config.py`, `test_mcp_configuration_telemetry.py`.
Expand Down Expand Up @@ -132,7 +130,7 @@
- **Background jobs fix**: Removed PENDING/FAILED blocking from `cleanup_workspace()`. Fixed stuck_cleaning_recovery for missing timestamps.
- **Multi-model**: Comma-separated model lists in `LLM_MEDIUM`/`LLM_HIGH`/`LLM_LOW`, round-robin assignment, OpenRouter validation.
- **Integration readiness**: SpecReadiness enum, Part F, dual plans, DEPLOY_AND_E2E_DONE checkpoint, conditional workflow.
- **KB Init**: `ROSETTA_MCP_ENABLED` flag, kb_init workflow step, rosetta artifact unpacking.
- **KB Init**: `kb_init` workflow step (bundled Rosetta plugin — see **Config** above).
- **Post-refactor audit**: State machine enforcement, rogue DB writes removed, CI guards.

## Gaps & Debt
Expand Down
12 changes: 5 additions & 7 deletions backend/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -112,13 +112,11 @@ RUN mkdir -p /root/.claude
COPY claude-settings.json /root/.claude/settings.json

# ---------------------------------------------------------------------------
# Rosetta knowledge-base plugin — bundled so KB init runs WITHOUT the ims-mcp service /
# ROSETTA_API_KEY. Plugin mode is the DEFAULT in every environment (ROSETTA_MCP_ENABLED=false,
# ROSETTA_PLUGIN_PATH=/opt/rosetta-plugin). At workspace prep time the backend copies this
# plugin's agents/skills/commands/hooks into each workspace's .claude/ (see
# WorkspaceManager.provision_rosetta_plugin); the agents discover them via
# setting_sources=["project"]. Set ROSETTA_MCP_ENABLED=true to opt back into the live MCP,
# which then ignores this path.
# Rosetta knowledge-base plugin — bundled so KB init runs without any external service or key.
# The plugin lives at ROSETTA_PLUGIN_PATH=/opt/rosetta-plugin in every environment. At workspace
# prep time the backend copies this plugin's agents/skills/commands/hooks into each workspace's
# .claude/ (see WorkspaceManager.provision_rosetta_plugin); the agents discover them via
# setting_sources=["project"].
#
# Installed from the public griddynamics/rosetta marketplace via the `claude` CLI that ships
# bundled inside claude-agent-sdk (a self-contained native binary at
Expand Down
30 changes: 9 additions & 21 deletions backend/app/core/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@
MCP_PLAYWRIGHT = "playwright"
MCP_FIGMA = "figma"
MCP_FIGMA_SERVER_KEY = "Figma" # figma-developer-mcp registers under this server name
ROSETTA_SERVER_KEY = "KnowledgeBase" # ims-mcp registers under this server name
SUPPORTED_MCPS: FrozenSet[str] = frozenset({MCP_PLAYWRIGHT, MCP_FIGMA})
MCP_SERVERS_ENABLED_ENV = "MCP_SERVERS_ENABLED"
MCP_SERVERS_ENABLED_DEFAULT = MCP_PLAYWRIGHT
Expand Down Expand Up @@ -295,28 +294,17 @@ def _empty_str_to_none_int(cls, v: object) -> object:
FIGMA_ACCESS_TOKEN: Optional[str] = None
FIGMA_API_KEY: Optional[str] = None

# KnowledgeBase/Rosetta MCP — matches `claude mcp add ... -- uvx ims-mcp@latest` env surface.
ROSETTA_MCP_COMMAND: str = "uvx"
ROSETTA_MCP_ARGS: str = "ims-mcp@latest"
ROSETTA_SERVER_URL: Optional[str] = "https://ims.evergreen.gcp.griddynamics.net/"
ROSETTA_API_KEY: Optional[str] = None
ROSETTA_USER_EMAIL: Optional[str] = None
# Passed to ims-mcp subprocess as env VERSION (e.g. r2).
ROSETTA_IMS_VERSION: str = "r2"
# Plugin mode is the DEFAULT for every environment (quickstart AND hosted): KB init
# runs against the bundled Rosetta plugin, no ims-mcp service / ROSETTA_API_KEY needed.
# Set ROSETTA_MCP_ENABLED=true only to opt back into the live ims-mcp server (it then
# wins over the plugin). See app/core/mcp_selection.py:for_kb_init.
ROSETTA_MCP_ENABLED: bool = False
# Output-dir name the KB init agent is told NOT to stage under (the plugin agent writes
# docs to their final locations directly). Also used to scope the agent's write allowlist.
ROSETTA_OUTPUT_DIR: str = "rosetta"
# Path to the Rosetta plugin bundled into the image at build time (backend/Dockerfile
# stages it here). When ROSETTA_MCP_ENABLED is False, WorkspaceManager.provision_rosetta_plugin
# copies this plugin's agents/skills/commands into each workspace's .claude/ and merges its
# hooks into .claude/settings.json so setting_sources=["project"] discovers them (no
# ~/.claude, no SDK plugins= loader). This same path is also exported as CLAUDE_PLUGIN_ROOT
# per agent (claude_code.setup_rosetta_plugin_env) so the merged hooks' ${CLAUDE_PLUGIN_ROOT}
# resolves to the read-only image plugin. Set to None / empty to disable plugin provisioning
# (KB init then no-ops unless MCP is enabled).
# stages it here). WorkspaceManager.provision_rosetta_plugin copies this plugin's
# agents/skills/commands into each workspace's .claude/ and merges its hooks into
# .claude/settings.json so setting_sources=["project"] discovers them (no ~/.claude, no
# SDK plugins= loader). This same path is also exported as CLAUDE_PLUGIN_ROOT per agent
# (claude_code.setup_rosetta_plugin_env) so the merged hooks' ${CLAUDE_PLUGIN_ROOT}
# resolves to the read-only image plugin. Set to None / empty to disable plugin
# provisioning (KB init then no-ops).
ROSETTA_PLUGIN_PATH: Optional[str] = "/opt/rosetta-plugin"

# PostHog Telemetry Configuration
Expand Down
49 changes: 5 additions & 44 deletions backend/app/core/mcp_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@
import shlex
from typing import FrozenSet, List, Literal, Optional

from app.core.config import MCP_FIGMA, MCP_FIGMA_SERVER_KEY, MCP_PLAYWRIGHT, ROSETTA_SERVER_KEY, SUPPORTED_MCPS, Settings
from app.core.tool_usage import get_figma_mcp_tools, get_playwright_mcp_tools, get_rosetta_kb_tools
from app.core.config import MCP_FIGMA, MCP_FIGMA_SERVER_KEY, MCP_PLAYWRIGHT, SUPPORTED_MCPS, Settings
from app.core.tool_usage import get_figma_mcp_tools, get_playwright_mcp_tools
from app.prompts.mcp_workflow_registry import (
CODE_GENERATION_AND_DEPLOY_OPTIONAL_MCPS,
PLANNING_OPTIONAL_AGENT_MCPS,
Expand Down Expand Up @@ -205,12 +205,6 @@ def _figma_pair(settings: Settings, enabled_mcps: FrozenSet[str]) -> tuple[dict,
return config, (get_figma_mcp_tools() if config else [])


def _rosetta_pair(settings: Settings) -> tuple[dict, List[str]]:
"""Rosetta MCP config + tools (enablement controlled by settings.ROSETTA_MCP_ENABLED)."""
config = build_rosetta_mcp_config(settings)
return config, (get_rosetta_kb_tools() if config else [])


def _combine_pairs(*pairs: tuple[dict, List[str]]) -> tuple[dict, List[str]]:
"""Merge N (config_dict, tools_list) pairs into one."""
servers: dict = {}
Expand Down Expand Up @@ -246,14 +240,13 @@ def coding_mcp_servers_and_tools(
settings: Settings,
enabled_mcps: FrozenSet[str],
) -> tuple[dict, List[str]]:
"""Playwright + Figma + optional Rosetta — used by generation and deploy/QA phase agents.
"""Playwright + Figma — used by generation and deploy/QA phase agents.

Rosetta is appended when ``ROSETTA_MCP_ENABLED`` (same as planning/KB init); not part of
``SUPPORTED_MCPS`` / prune candidates.
The Rosetta knowledge base is delivered as a provisioned plugin (vendored into each
workspace's ``.claude/``), not as an MCP server, so it is not attached here.
"""
allowed = enabled_mcps & CODE_GENERATION_AND_DEPLOY_OPTIONAL_MCPS
return _combine_pairs(
_rosetta_pair(settings),
_playwright_pair(settings, allowed),
_figma_pair(settings, allowed),
)
Expand Down Expand Up @@ -287,35 +280,3 @@ def mcp_prompt_hints(enabled_mcps: FrozenSet[str]) -> str:
if not lines:
return ""
return "\n## Available MCP Tools\n" + "\n".join(f"- {line}" for line in lines) + "\n"


def build_rosetta_mcp_config(settings: Settings) -> dict[str, dict]:
"""Build MCP server config dict for ClaudeAgentOptions.mcp_servers.

Returns an empty dict when the feature is disabled, which signals
callers to skip KB init entirely.
"""
if not settings.ROSETTA_MCP_ENABLED:
return {}

env: dict[str, str] = {}
server_url = (settings.ROSETTA_SERVER_URL or "").strip()
if server_url:
env["ROSETTA_SERVER_URL"] = server_url
rosetta_api_key = (settings.ROSETTA_API_KEY or "").strip()
if rosetta_api_key:
env["ROSETTA_API_KEY"] = rosetta_api_key
user_email = (settings.ROSETTA_USER_EMAIL or "").strip()
if user_email:
env["ROSETTA_USER_EMAIL"] = user_email
ims_version = settings.ROSETTA_IMS_VERSION.strip()
if ims_version:
env["VERSION"] = ims_version

return {
ROSETTA_SERVER_KEY: {
"command": settings.ROSETTA_MCP_COMMAND,
"args": settings.ROSETTA_MCP_ARGS.split(),
"env": env,
}
}
Loading