Skip to content

feat: add Maiat trust evaluator example for ACP jobs#193

Open
JhiNResH wants to merge 1 commit intogame-by-virtuals:mainfrom
JhiNResH:feat/maiat-evaluator-example
Open

feat: add Maiat trust evaluator example for ACP jobs#193
JhiNResH wants to merge 1 commit intogame-by-virtuals:mainfrom
JhiNResH:feat/maiat-evaluator-example

Conversation

@JhiNResH
Copy link

Summary

Adds an example showing how to use Maiat as a trust-based evaluator for ACP jobs via the on_evaluate callback.

What it does

The example provides two integration options:

Option 1: One-liner (recommended)

from maiat_evaluator import maiat_evaluator

acp_client = VirtualsACP(
    ...,
    on_evaluate=maiat_evaluator(),  # that's it
)

Option 2: Inline (no extra dependency)

Copy the inline evaluator function — zero dependencies beyond httpx.

Evaluation Logic

  1. Garbage detection — Rejects empty, too-short, or placeholder deliverables (hello, {}, etc.)
  2. Trust score check — Queries Maiat's trust oracle (18,000+ agents indexed)
  3. Auto-approve/reject — Score >= 80 auto-approves, < 30 rejects
  4. Outcome recording — Reports results back for trust score updates

Why

Currently there's no example of a trust-based evaluator in the GAME SDK examples. Most agents use a simple job.evaluate(True) which auto-approves everything — including garbage deliverables. This example shows how to add meaningful quality checks.

Files Changed

  • examples/game/maiat_evaluator_example.py (new)

Shows how to use Maiat as an on_evaluate callback to auto-verify
job deliverables. Two options:

1. One-liner with maiat-evaluator package (pip install maiat-evaluator)
2. Inline evaluator with no extra dependency

Features:
- Auto-reject garbage deliverables (empty, too short, placeholder text)
- Auto-approve trusted providers (trust score >= 80)
- Block low-trust providers (score < 30)
- Record outcomes back to Maiat for trust score updates
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant