Skip to content

Conversation

@aarora79
Copy link
Contributor

@aarora79 aarora79 commented Dec 9, 2025

Summary

This PR implements a complete end-to-end agent rating system for the MCP Gateway Registry, allowing users to rate agents and view aggregate ratings.

This PR contains and supersedes PR #268 which provided the initial backend implementation. This PR includes all of those changes plus the complete UI, CLI tools, and API documentation.

Closes #226

What's Included

Backend Implementation

  • Rating API Endpoints:
    • POST /api/agents/{path}/rate - Submit or update a rating (1-5 stars)
    • GET /api/agents/{path}/rating - Retrieve rating information
  • Rotating Buffer: Maintains up to 100 most recent ratings per agent to prevent unbounded growth
  • One Rating Per User: Users can update their existing rating; no duplicate ratings
  • Access Control: Users can only rate agents they have access to
  • Float Averages: Ratings are stored as integers (1-5) but averages are calculated as floats (e.g., 3.5, 4.7)

Frontend UI

  • Interactive Star Rating Widget: Click-to-rate component with dropdown interface
  • UI Flow States:
    • Default view showing average rating and count
    • Dropdown with 5-star selection
    • Hover preview with filled stars
    • Loading state during submission
    • Success state with auto-close after 2s
    • Update mode for users who already rated
  • Mobile Responsive: Touch-friendly with proper accessibility (ARIA labels)
  • Dark Mode Support: Fully themed for light and dark modes

CLI Tools

  • Rating Commands:
    • agent-rate --path <path> --rating <1-5> - Submit a rating via CLI
    • agent-rating --path <path> - View rating information via CLI
  • Token Support: Works with JSON token files for authentication
  • Error Handling: Clear error messages with retry logic

API Documentation

  • OpenAPI Specification: Complete documentation in docs/api-specs/a2a-agent-management.yaml
  • New Tag: "Agent Ratings" for rating-related endpoints
  • Schema Updates:
    • Changed num_stars type from integer to float across all agent schemas
    • Added rating_details field to AgentDetail schema
    • Added 4 new schemas: RatingRequest, RatingResponse, RatingInfoResponse, RatingDetail
  • Comprehensive Examples: Request/response examples for all endpoints and error cases

Technical Details

Files Changed

  • registry/api/agent_routes.py - Added rating endpoints
  • registry/services/agent_service.py - Rating logic with rotating buffer
  • registry/schemas/agent_models.py - Updated num_stars type (int → float)
  • api/registry_client.py - Added rating client methods
  • api/registry_management.py - Added CLI commands
  • frontend/src/components/StarRatingWidget.tsx - New interactive rating component
  • frontend/src/components/AgentCard.tsx - Integrated rating widget, fixed rating count display
  • docs/api-specs/a2a-agent-management.yaml - Complete rating API documentation

Bug Fixes

  1. Token File Parsing: CLI now correctly extracts access_token from JSON token files
  2. JSON Content-Type: Rating endpoint now correctly receives JSON instead of form-encoded data
  3. Type Mismatch: Changed num_stars from int to float to support decimal averages
  4. Rating Count Display: Fixed UI to show actual rating count from rating_details.length instead of usersCount

Data Flow

User clicks star → Dropdown opens → User selects rating → Submit → 
POST /api/agents/{path}/rate → Update/add rating → Calculate average → 
Pop oldest if >100 ratings → Save → Return average → UI updates → Success message

Security

  • JWT Bearer token authentication required
  • Access control enforced (403 errors for unauthorized access)
  • Input validation (1-5 stars only)
  • User-specific rating tracking

Testing Performed

Manual Testing

  • ✅ List agents via CLI
  • ✅ Rate agent with 5 stars via CLI
  • ✅ Get rating information via CLI
  • ✅ Update existing rating via CLI
  • ✅ Submit rating via UI
  • ✅ Update rating via UI
  • ✅ View rating details in UI
  • ✅ Token expiration handling
  • ✅ Access control (403 errors)
  • ✅ Frontend build successful
  • ✅ Rating count displays correctly

Test Commands

# Rate an agent
uv run python api/registry_management.py \
  --registry-url http://localhost \
  --token-file .oauth-tokens/admin-bot-token.json \
  agent-rate --path /test-reviewer --rating 5

# Get ratings
uv run python api/registry_management.py \
  --registry-url http://localhost \
  --token-file .oauth-tokens/admin-bot-token.json \
  agent-rating --path /test-reviewer

Breaking Changes

None - All changes are additive and backward compatible.

Performance Impact

  • Backend: Minimal impact, rotating buffer prevents unbounded growth (max 100 ratings per agent)
  • Frontend: Bundle size increase of +1.72 kB (gzipped), lazy loading component

Deployment Notes

  • Deploy backend first (new endpoints)
  • Then deploy frontend (uses new endpoints)
  • No database migrations needed
  • No configuration changes needed
  • No downtime required

Screenshots

The UI includes an interactive star rating widget on each agent card that allows users to:

  1. Click the star icon to open a rating dropdown
  2. Select 1-5 stars with hover preview
  3. Submit or update their rating
  4. See updated average rating immediately

Commits Included

This PR includes the following commits from the anrwang branch:

Related Issues/PRs


Co-authored-by: @anrwang11 (original backend implementation from PR #268)

Ubuntu and others added 5 commits December 9, 2025 19:36
- Add rating API endpoints (POST /api/agents/{path}/rate, GET /api/agents/{path}/rating)
- Implement rotating buffer for last 100 ratings
- Add CLI commands: agent-rate and agent-rating
- Update Pydantic models to support float ratings (num_stars: int -> float)
- Fix registry_client to send JSON for all agent endpoints
- Fix token file parsing to extract access_token from JSON
- Update OpenAPI spec with rating endpoints and schemas
- Change num_stars from integer to float in AgentCard and AgentInfo models
- Create StarRatingWidget component with dropdown interaction
- Implement click-to-rate functionality with 5-star selection
- Add hover preview and visual feedback
- Support rating submission and updates
- Show success/loading states with animations
- Integrate with AgentCard to replace static rating display
- Auto-close dropdown after successful submission
- Handle outside-click to close dropdown
- Full keyboard navigation support (planned)
- Mobile-responsive design with touch-friendly targets
- Changed initialCount from usersCount to rating_details.length
- Added rating_details to Agent interface TypeScript definition
- This ensures rating count matches actual number of ratings
@aarora79 aarora79 merged commit 1a22f53 into main Dec 9, 2025
4 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add API endpoints for agent rating and user tracking

2 participants