add management API #267
Closed
add management API #267
+610
−2
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
aarora79
added a commit
that referenced
this pull request
Dec 11, 2025
Extend registry_client.py and registry_management.py to support the new Management API endpoints from PR #267 for IAM/user management. Changes: - Add 6 Pydantic models for Management API requests/responses - Add 5 client methods: list_users, create_m2m_account, create_human_user, delete_user, list_keycloak_iam_groups - Add 5 CLI commands: user-list, user-create-m2m, user-create-human, user-delete, keycloak-groups - Update documentation with usage examples The Management API enables administrators to: - List and search Keycloak users - Create M2M service accounts with client credentials - Create human user accounts with group assignments - Delete users with confirmation prompts - List raw Keycloak IAM groups All commands require admin privileges (enforced server-side).
aarora79
added a commit
that referenced
this pull request
Dec 12, 2025
Extend registry_client.py and registry_management.py to support the new Management API endpoints from PR #267 for IAM/user management. Changes: - Add 6 Pydantic models for Management API requests/responses - Add 5 client methods: list_users, create_m2m_account, create_human_user, delete_user, list_keycloak_iam_groups - Add 5 CLI commands: user-list, user-create-m2m, user-create-human, user-delete, keycloak-groups - Update documentation with usage examples The Management API enables administrators to: - List and search Keycloak users - Create M2M service accounts with client credentials - Create human user accounts with group assignments - Delete users with confirmation prompts - List raw Keycloak IAM groups All commands require admin privileges (enforced server-side).
aarora79
added a commit
that referenced
this pull request
Dec 14, 2025
* add management API * feat: Add Management API support to registry client and CLI Extend registry_client.py and registry_management.py to support the new Management API endpoints from PR #267 for IAM/user management. Changes: - Add 6 Pydantic models for Management API requests/responses - Add 5 client methods: list_users, create_m2m_account, create_human_user, delete_user, list_keycloak_iam_groups - Add 5 CLI commands: user-list, user-create-m2m, user-create-human, user-delete, keycloak-groups - Update documentation with usage examples The Management API enables administrators to: - List and search Keycloak users - Create M2M service accounts with client credentials - Create human user accounts with group assignments - Delete users with confirmation prompts - List raw Keycloak IAM groups All commands require admin privileges (enforced server-side). * Hide internal nginx headers from OpenAPI spec Remove X-User, X-Username, X-Scopes, and X-Auth-Method headers from the OpenAPI specification by adding include_in_schema=False parameter. These headers are set internally by nginx after JWT token validation and should not be documented as client-provided parameters. Clients only need to provide the Authorization: Bearer <token> header. This change: - Cleans up API documentation - Removes confusion about which headers clients need to provide - Maintains functionality - headers still work internally - Makes it clear that JWT token alone is sufficient for authentication * Add comprehensive README for Management API tools Document how to use registry_management.py and test scripts for both local development and production AWS deployments. Includes: - Quick start examples for localhost and production - Token generation differences (generate_creds.sh vs get-m2m-token.sh) - Common management operations (users, groups, servers, agents) - Environment comparison table - Clear distinction between local and AWS token workflows Makes it easy for developers to understand: - get-m2m-token.sh is AWS-only (requires AWS credentials, uses SSM) - generate_creds.sh is localhost-only (uses docker-compose Keycloak) - Different registry URLs and Keycloak URLs for each environment * Update README to use generic enterprise domain examples Replace specific domain examples with generic enterprise domain pattern: - aroraai.people.aws.dev -> example.com - kc.us-east-1.aroraai.people.aws.dev -> keycloak.us-east-1.example.com - registry.us-east-1.aroraai.people.aws.dev -> registry.us-east-1.example.com This makes the documentation more universally applicable for enterprise deployments without referencing specific internal domains. * Add OpenAPI security scheme documentation for JWT Bearer tokens Changes: - Add formal securitySchemes component to OpenAPI spec defining Bearer authentication - Apply Bearer security requirement to all protected endpoints (excludes auth, health, public discovery) - Add enhanced tag descriptions explaining JWT requirements for each API category - Hide session cookie parameter from OpenAPI spec (internal implementation detail) - Enable persistent authorization in Swagger UI for better developer experience This ensures the OpenAPI spec clearly documents that endpoints require: `Authorization: Bearer <JWT-token>` header for authentication. The session cookie remains functional but is hidden from API documentation as it's an internal mechanism for browser-based access, not for programmatic clients. * Update API README with example URLs and localhost token generation Changes: - Revert to generic example URLs (registry.example.com, keycloak.example.com) - Add keycloak/setup/generate-agent-token.sh for localhost M2M token generation - Show both default and custom M2M bot token generation examples - Update environment summary table with both localhost token scripts - Maintain clear distinction between localhost and production workflows This ensures the README uses generic examples while providing complete token generation guidance for both localhost and AWS production environments. * Remove static API specs and add OpenAPI spec documentation - Add .kiro to .gitignore - Delete docs/api-specs/ folder containing static YAML files - Add API Specification section to api/README.md - Document how to access live OpenAPI spec from localhost and production * Add complete Management API implementation with user/group management This commit includes the full Management API implementation that was previously deployed to AWS but not committed to the repository. Core Features: - User management (create M2M/human users, list, delete) - Group management (create, delete, list groups) - Server management (register, list, toggle, remove, health check, group assignment) - Agent management (register, list, get, update, delete, toggle, discover, search, rate) - Keycloak integration for user/group operations via Admin API API Components: - registry_client.py: Python client library for Registry Management API - registry_management.py: CLI tool for all management operations - test-management-api-e2e.sh: End-to-end test suite - get-m2m-token.sh: M2M token generation for AWS deployments Backend Changes: - management_routes.py: New user/group management endpoints - server_routes.py: Enhanced group assignment endpoints - keycloak_manager.py: Keycloak Admin API integration - management.py schemas: Request/response models for user/group operations Infrastructure: - Deploy Keycloak admin password to AWS Secrets Manager - Grant ECS task role access to Keycloak admin credentials - Add keycloak_admin_password terraform variable and secret This implementation replaces the legacy cli/service_mgmt.sh and cli/user_mgmt.sh scripts with a unified, production-ready Management API. * Refactor scan_all_servers.py to use Registry Management API client Replace dependencies on deprecated service_mgmt.sh with direct usage of: - registry_client.py for fetching server list - mcp_security_scanner.py for running scans Changes: - Use RegistryClient to get server list instead of manual API calls - Call mcp_security_scanner.py directly instead of via service_mgmt.sh - Maintain all existing functionality and command-line options - Works standalone without service_mgmt.sh dependency This allows service_mgmt.sh to be deprecated while preserving the security scanning workflow for supply chain security compliance. * Use Anthropic Registry API v0.1 for server listing in scan_all_servers.py Change from generic list_servers() to anthropic_list_servers() to maintain consistency with the original implementation which used /v0.1/servers endpoint. * Fix RegistryClient initialization in scan_all_servers.py RegistryClient constructor takes registry_url and token parameters, not base_url and token_file. Load token from file first, then pass to RegistryClient constructor. Also removed duplicate token loading code. * Fix Pydantic object access pattern in scan_all_servers.py - Change from dictionary .get() to Pydantic attribute access - Access server via server_response.server (AnthropicServerDetail) - Access metadata via server.meta attribute (not server._meta or server.get()) - Simplify enabled server filtering logic - Remove unnecessary hasattr() checks for Pydantic models * Refactor TypeScript CLI to use Registry Management API - Create registry_cli_wrapper.py: Python CLI wrapper that uses Registry Management API - Replace all shell script calls with wrapper calls in cli/src/tasks/index.ts - Service management: add, delete, list/monitor operations - Group management: create, delete, list operations - User management: create M2M/human users, delete, list operations - Diagnostic tasks: Anthropic API list and get operations - Remove deprecated task references (service-test, add/remove-groups) - Update agent system prompt to remove shell script examples - All functionality now uses modern Python API (registry_client.py) This eliminates dependencies on service_mgmt.sh, user_mgmt.sh, and test_anthropic_api.py while maintaining full CLI functionality through the Management API. * Add deprecation notices to legacy CLI scripts - service_mgmt.sh: Add warning directing users to Registry Management API - user_mgmt.sh: Add warning directing users to Registry Management API - agent_mgmt.sh: Add warning directing users to Registry Management API - test_anthropic_api.py: Add deprecation notice in docstring and runtime warning All scripts now warn users on execution that they are deprecated and should use api/registry_management.py or cli/registry_cli_wrapper.py instead. * Fix TypeScript compilation errors in CLI tasks - Add missing context parameter to build() functions in user management tasks - Add missing context parameter to build() functions in diagnostic tasks - Fix process.env type mismatch with explicit type cast - Update user-create-m2m to use correct CLI wrapper arguments (user create-m2m) - Update user-create-human to use correct CLI wrapper arguments (user create-human) - Update diagnostic tasks to use Anthropic API commands (anthropic list/get) All TypeScript compilation and type checking now passes successfully. * Fix agent API endpoints in CLI executor - Change /api/agents to /agents (correct API path) - Change ?search= to ?query= parameter (correct query param) - Change Authorization to X-Authorization header (nginx proxy format) - Change gatewayToken to backendToken (correct token field) - Apply fixes to all agent commands: list, get, search, test, test-all This fixes the 404 errors when using /agents search and other agent commands. * Refactor CLI agent commands to use registry_cli_wrapper.py - Add agent commands to registry_cli_wrapper.py (list, get, search) - Update TypeScript executor to call Python wrapper instead of direct fetch - Add callRegistryWrapper helper function in executor.ts - Removes duplicated HTTP client code in favor of centralized registry_client.py * Complete agent command migration to registry_client.py - Migrate executeAgentsTest and executeAgentsTestAll to use wrapper - Remove all direct fetch() calls from agent commands - Fix token fallback: use gatewayToken if backendToken not available - All agent commands now use centralized registry_client.py * Migrate /servers command to use registry client - Replace MCP list_services tool call with anthropic_list_servers API - All server and agent listing now goes through registry_client.py - Eliminates reliance on MCP tools for registry operations - Consistent data format across all registry queries * Add dynamic version management from git tags - Create registry/version.py to get version from git describe - Update main.py to use __version__ from version module - Version format: X.Y.Z-commits-ghash (e.g., 1.0.7-21-gdee4afb) - Falls back to DEFAULT_VERSION (1.0.0) if git not available - Fixes hardcoded version='1.0.0' in FastAPI app Now /ping and OpenAPI docs will show actual git-based version * Improve version management with BUILD_VERSION support - Update version.py to check BUILD_VERSION env var first (for Docker builds) - Add BUILD_VERSION build arg to Dockerfile and Dockerfile.registry - Configure docker-compose.yml to pass BUILD_VERSION from environment - Change git error logs from warning to debug level - Support version determination priority: BUILD_VERSION > git tags > default This allows version to be set at Docker build time using: BUILD_VERSION=$(git describe --tags --always | sed 's/^v//') docker-compose build * Add automatic BUILD_VERSION determination to build scripts - Add version detection to build_and_run.sh - Add version detection to scripts/build-images.sh (for ECS deployments) - Determine version from git tags with branch info for development builds - Release builds (on tags): Use tag as-is (e.g., '1.0.7') - Development builds: Include branch name (e.g., '1.0.7-22-ge7bfb1c-test/pr-267-management-api') - Automatically pass BUILD_VERSION as build arg to all Docker builds - Log version prominently during build process Version format: - On tagged commit: v1.0.7 → 1.0.7 - Development: 1.0.7-22-ge7bfb1c-branch-name - No git: 1.0.0-dev * Add version display to UI header - Add /api/version endpoint in registry/main.py - Fetch version from API in Layout component - Display version badge in header (top right, before theme toggle) - Show on medium screens and up (hidden on mobile) - Styled with purple theme to match branding - Format: v1.0.7-22-ge7bfb1c-branch-name The version will now be visible in the UI header, making it easy to verify which version is running in any deployment. * Add registry version to agent and CLI status bar Backend: - Agent system prompt now includes registry version with clarification about different component versions - Helps agent distinguish between Registry version and MCP server versions Frontend (CLI): - Add registry version to TokenStatusFooter component - Display in status bar after cost: 'Registry: v1.0.7-25-g0099fd3-branch' - Fetch version from /api/version endpoint on initialization - Show in magenta color to stand out Agent improvements: - Fetch registry version at agent startup - Pass version to system prompt - Agent can now correctly explain version differences to users * Add version display and update documentation - Add /api/version endpoint as public (no auth required) in nginx configs - Display registry version in CLI status bar and web UI header - Remove duplicate 'v' prefix from version display - Completely rewrite service-management.md to use Registry Management API - Update README.md to highlight new Registry Management API - Replace all shell script references with Python API/client examples - Add comprehensive API usage examples with type-safe interfaces * Fix API documentation link in service-management.md * Add OpenAPI specification documentation with live and reference links * Move API Specification section to top of api/README.md * Add API Documentation link to What's New section in README * feat: add BUILD_VERSION support and OpenAPI spec reference - Add BUILD_VERSION environment variable to docker-compose.prebuilt.yml to ensure version consistency between build and prebuilt deployments - Add OpenAPI specification v1.0.8 as reference documentation - Reference spec available at api/openapi.json for offline use - Live spec always available at /openapi.json endpoint * Potential fix for code scanning alert no. 127: Clear-text logging of sensitive information Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> --------- Co-authored-by: kanghengliu <kl1099@georgetown.edu> Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Contributor
|
This PR has been merged via PR #288 (Complete Management API Implementation with Enhanced Features). PR #288 included all the management API functionality from this PR:
The management API is now fully integrated and available in the main branch. Related issue: #247 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Addresses #247
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.