Skip to content

Add structured output quality scoring#17

Open
nguyenduc071912 wants to merge 1 commit into
Mint-Claw:mainfrom
nguyenduc071912:bounty-quality-scoring
Open

Add structured output quality scoring#17
nguyenduc071912 wants to merge 1 commit into
Mint-Claw:mainfrom
nguyenduc071912:bounty-quality-scoring

Conversation

@nguyenduc071912
Copy link
Copy Markdown

Summary

Implements the bounty request for multi-dimensional quality scoring of structured outputs.

  • Adds deterministic scoring for JSON, markdown, code, and plain text submissions
  • Scores all five requested dimensions with the requested weights:
    • Completeness: 0.30
    • Format Compliance: 0.20
    • Coverage: 0.25
    • Clarity: 0.15
    • Validity: 0.10
  • Returns the requested response shape: weighted_score, quality_rating, scores, feedback, and pass_threshold
  • Adds /api/quality-score so the scorer is available through the API
  • Adds unit tests including a 100-submission performance check under 10 seconds
  • Documents the new endpoint and rubric options in the README

Fixes #1.

Tests

python -m unittest test_quality_scoring.py -v
python -m py_compile quality_scoring.py app.py
python -c "from app import app; print(app.title); print([route.path for route in app.routes if 'quality' in route.path])"

I also verified the endpoint with fastapi.testclient.TestClient; it returned HTTP 200 and the expected scorecard schema.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BOUNTY $10] Multi-Dimensional Quality Scoring for Structured Outputs

1 participant