Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 8 additions & 4 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,13 +42,17 @@ pyproject.toml Poetry-managed project metadata
# Install (editable)
pip install -e .

# Run the end-to-end demo (writes reports/ into the repo)
# Self-contained demo — bundled sample contract, OPA-only, no API keys
aicertify demo
# → writes ./aicertify_demo_report.md

# Full quickstart (uses the heavy ML evaluators)
python examples/quickstart.py

# CLI evaluation
python -m aicertify.cli \
# CLI evaluation against a user contract
aicertify evaluate \
--contract examples/sample_contract.json \
--policy aicertify/opa_policies/international/eu_ai_act/v1 \
--policy eu_ai_act \
--report-format pdf \
--output-dir reports/

Expand Down
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

## [0.7.1] — 2026-05-14

### Added

- **`aicertify demo` subcommand** — a self-contained, no-config demo entry point. Loads a bundled sample contract (`aicertify/_demo/sample_contract.json`), runs an OPA evaluation against the EU AI Act policy set, and writes `aicertify_demo_report.md` to the current directory. Requires only the `opa` binary on PATH (no API keys, no contract file). The CLI now also detects a missing `opa` binary and prints a one-line, platform-aware install command instead of stack-tracing.
- **`aicertify evaluate` subcommand** — the previous flat CLI behaviour, now under an explicit subcommand. The pre-0.7.1 invocation `aicertify --contract X --policy Y …` is still accepted (transparently routed to `evaluate`).
- **Updated README Quick Start** — collapses to three commands: `pip install aicertify`, `curl … opa`, `aicertify demo`. Honest first-install timing called out (~3–5 min for deps + the one-time OPA install).

### Changed

- **Visual refresh** — all README diagrams replaced with hand-authored, theme-aware SVGs in [`diagrams/`](diagrams/). Each diagram now ships as a paired `_light.svg` + `_dark.svg` and is embedded via `<picture>` so GitHub light- and dark-theme readers each see the variant that matches their canvas. The top-of-README logo is replaced with a hero banner SVG. The previous matplotlib generator (`diagrams/generate_diagrams.py`) and 5 baked-in PNG diagrams have been removed in favour of the hand-authored SVG system documented in [`diagrams/STYLE.md`](diagrams/STYLE.md).
Expand Down
16 changes: 8 additions & 8 deletions README.hi-IN.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,18 +46,18 @@
## Quick Start

```bash
pip install aicertify
```
pip install aicertify # पहली बार इंस्टॉल में लगभग 3–5 मिनट (langchain + transformers डाउनलोड होते हैं)

बंडल्ड डेमो चलाने के लिए (सैंपल कॉन्ट्रैक्ट + examples के लिए रिपॉज़िटरी क्लोन करें):
# OPA बाइनरी एक बार इंस्टॉल करें (~80 MB)
curl -L https://openpolicyagent.org/downloads/latest/opa_linux_amd64 -o /usr/local/bin/opa && sudo chmod +x /usr/local/bin/opa

```bash
git clone https://github.com/Principled-Evolution/aicertify.git
cd aicertify
python examples/quickstart.py
# बंडल्ड डेमो चलाएँ — कोई कॉन्ट्रैक्ट फ़ाइल नहीं, कोई API key नहीं, ~10 सेकंड
aicertify demo
```

यह क्विकस्टार्ट एक सैंपल AI एप्लिकेशन को EU AI Act पॉलिसी सेट के माध्यम से जोड़ता है और `reports/` में एक कंप्लायंस रिपोर्ट लिखता है। उसे खोलिए। यही आपके ऑडिट डिलिवरेबल का स्वरूप है — हाथ से नहीं, जनरेट होकर।
`aicertify demo` एक बंडल्ड सैंपल कॉन्ट्रैक्ट लोड करता है, उसे OPA के माध्यम से EU AI Act पॉलिसी सेट पर मूल्यांकित करता है, और मौजूदा डायरेक्टरी में `aicertify_demo_report.md` लिखता है। रिपोर्ट खोलिए — यही आपके ऑडिट डिलिवरेबल का स्वरूप है।

विस्तृत मूल्यांकन (LangFair फेयरनेस मेट्रिक्स, DeepEval कंटेंट-सेफ़्टी स्कोरिंग, PDF रिपोर्ट) के लिए [`examples/quickstart.py`](examples/quickstart.py) और [फ़ोर्क-योग्य उदाहरण बॉट्स](examples/) देखें — हर उदाहरण में `input_contract.json`, `policy_config.yaml` और `run.py` शामिल हैं।

### डेवलपमेंट के लिए

Expand Down
16 changes: 8 additions & 8 deletions README.ja-JP.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,18 +46,18 @@
## クイックスタート

```bash
pip install aicertify
```
pip install aicertify # 初回インストールは約 3〜5 分(langchain + transformers を取得)

同梱のデモを実行するには(サンプル契約と examples 一式を取得するためにリポジトリをクローンします):
# OPA バイナリを一度だけインストール(約 80 MB)
curl -L https://openpolicyagent.org/downloads/latest/opa_linux_amd64 -o /usr/local/bin/opa && sudo chmod +x /usr/local/bin/opa

```bash
git clone https://github.com/Principled-Evolution/aicertify.git
cd aicertify
python examples/quickstart.py
# 同梱のデモを実行 —— 契約ファイル不要、API キー不要、約 10 秒
aicertify demo
```

クイックスタートでは、サンプル AI アプリケーションを EU AI Act のポリシーセットに通し、コンプライアンスレポートを `reports/` に出力します。それを開いてみてください。手書きではなく、生成された監査成果物の実例です。
`aicertify demo` は同梱のサンプル契約を読み込み、OPA 経由で EU AI Act のポリシーセットに対して評価を行い、`aicertify_demo_report.md` をカレントディレクトリに書き出します。レポートを開いてみてください。それが監査成果物の実例です。

より高度な評価(LangFair の公平性メトリクス、DeepEval によるコンテンツ安全性スコアリング、PDF レポート)については、[`examples/quickstart.py`](examples/quickstart.py) と [フォーク可能なサンプルボット](examples/) を参照してください。各サンプルには `input_contract.json`、`policy_config.yaml`、`run.py` が同梱されています。

### 開発用のセットアップ

Expand Down
16 changes: 8 additions & 8 deletions README.ko-KR.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,18 +46,18 @@
## 빠른 시작

```bash
pip install aicertify
```
pip install aicertify # 첫 설치는 약 3~5분 소요 (langchain + transformers 다운로드)

샘플 계약과 예시를 함께 실행하려면 저장소를 클론하세요:
# OPA 바이너리 일회성 설치 (약 80 MB)
curl -L https://openpolicyagent.org/downloads/latest/opa_linux_amd64 -o /usr/local/bin/opa && sudo chmod +x /usr/local/bin/opa

```bash
git clone https://github.com/Principled-Evolution/aicertify.git
cd aicertify
python examples/quickstart.py
# 번들 데모 실행 — 계약 파일/ API 키 불필요, 약 10초
aicertify demo
```

빠른 시작 스크립트는 샘플 AI 애플리케이션을 EU AI Act 정책 세트에 통과시키고 컴플라이언스 리포트를 `reports/`에 작성합니다. 파일을 열어 보세요. 이것이 바로 여러분의 감사 산출물의 모습입니다 — 손으로 작성하지 않고 생성된 리포트입니다.
`aicertify demo`는 번들 샘플 계약을 로드하여 OPA를 통해 EU AI Act 정책 세트에 대해 평가하고, 현재 디렉터리에 `aicertify_demo_report.md` 파일을 작성합니다. 리포트를 열어 보세요 — 그것이 바로 감사 산출물의 모습입니다.

더 풍부한 평가(LangFair 공정성 지표, DeepEval 콘텐츠 안전성 스코어링, PDF 리포트)는 [`examples/quickstart.py`](examples/quickstart.py)와 [포크 가능한 예시 봇들](examples/)을 참고하세요. 각 예시에는 `input_contract.json`, `policy_config.yaml`, `run.py`가 포함되어 있습니다.

### 개발용 설치

Expand Down
15 changes: 8 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,18 +58,19 @@ AICertify is part of the [Open Policy Agent ecosystem](https://www.openpolicyage
## Quick Start

```bash
# 1. Install AICertify (~3–5 min on first install; pulls langchain + transformers)
pip install aicertify
```

To run the canonical demo (clone the repo for the sample contract + examples):
# 2. Install the OPA binary, one-time (~80 MB)
curl -L https://openpolicyagent.org/downloads/latest/opa_linux_amd64 -o /usr/local/bin/opa && sudo chmod +x /usr/local/bin/opa

```bash
git clone https://github.com/Principled-Evolution/aicertify.git
cd aicertify
python examples/quickstart.py
# 3. Run the bundled demo — no contract file, no API keys, ~10 seconds
aicertify demo
```

The quickstart wires a sample AI application through the EU AI Act policy set and writes a compliance report into `reports/`. Open it. That's what your audit deliverable looks like — generated, not handwritten.
`aicertify demo` loads a bundled sample contract, evaluates it against the EU AI Act policy set via OPA, and writes `aicertify_demo_report.md` to the current directory. Open the report — that's what your audit deliverable looks like.

For richer evaluations (LangFair fairness metrics, DeepEval content-safety scoring, PDF reports), see [`examples/quickstart.py`](examples/quickstart.py) and the [forkable example bots](examples/) — each ships an `input_contract.json`, a `policy_config.yaml`, and a `run.py`.

### For development

Expand Down
16 changes: 8 additions & 8 deletions README.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,18 +46,18 @@
## 快速开始

```bash
pip install aicertify
```
pip install aicertify # 首次安装约 3–5 分钟(会拉取 langchain + transformers)

运行内置演示(克隆仓库以获取示例合约和示例代码):
# 一次性安装 OPA 二进制(约 80 MB)
curl -L https://openpolicyagent.org/downloads/latest/opa_linux_amd64 -o /usr/local/bin/opa && sudo chmod +x /usr/local/bin/opa

```bash
git clone https://github.com/Principled-Evolution/aicertify.git
cd aicertify
python examples/quickstart.py
# 运行内置演示 —— 无需合约文件,无需 API key,约 10 秒
aicertify demo
```

quickstart 会将一个示例 AI 应用接入 EU AI Act 策略集,并将合规报告写入 `reports/`。打开看看 —— 这就是您的审计交付物的样貌:由系统生成,而非手工撰写。
`aicertify demo` 会加载内置的示例合约,通过 OPA 对其进行 EU AI Act 策略评估,并将 `aicertify_demo_report.md` 写入当前目录。打开报告 —— 这就是审计交付物的样貌。

如需更完整的评估(LangFair 公平性指标、DeepEval 内容安全评分、PDF 报告),请查看 [`examples/quickstart.py`](examples/quickstart.py) 以及 [可派生的示例机器人](examples/) —— 每个示例均包含 `input_contract.json`、`policy_config.yaml` 和 `run.py`。

### 用于开发

Expand Down
2 changes: 1 addition & 1 deletion aicertify/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
"""

# Version information
__version__ = "0.7.0"
__version__ = "0.7.1"

# Direct imports for developer convenience
try:
Expand Down
1 change: 1 addition & 0 deletions aicertify/_demo/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
"""Bundled demo fixtures + runner used by `aicertify demo`."""
197 changes: 197 additions & 0 deletions aicertify/_demo/runner.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,197 @@
"""Demo runner used by ``aicertify demo``.

Loads the bundled sample contract, runs an OPA evaluation against a chosen
vendored policy folder, and writes a Markdown report to the user's CWD.

Designed to work after ``pip install aicertify`` with no extra configuration
beyond the OPA binary on PATH. Heavy ML-based evaluators are skipped by
default; the OPA verdict is the substance.
"""

from __future__ import annotations

import json
import logging
import platform
import shutil
import sys
from importlib.resources import files
from pathlib import Path
from typing import Optional

logger = logging.getLogger("aicertify.demo")


DEFAULT_POLICY = "eu_ai_act"
DEFAULT_REPORT_NAME = "aicertify_demo_report.md"

# Map friendly framework names to the bundled directory under aicertify/opa_policies/
# that we use to verify the framework is present in the wheel.
_BUNDLED_POLICY_PROBE_PATH = {
"eu_ai_act": ("international", "eu_ai_act", "v1"),
"nist": ("international", "nist", "v1"),
"global": ("global", "v1"),
"global/v1": ("global", "v1"),
}


def opa_binary_path() -> Optional[str]:
"""Return the path to the opa binary on PATH, or None."""
return shutil.which("opa")


def print_opa_install_instructions() -> None:
"""Print friendly, platform-specific OPA install instructions to stderr."""
system = platform.system().lower()
if system == "linux":
url = "https://openpolicyagent.org/downloads/latest/opa_linux_amd64"
install = (
f"curl -L {url} -o /usr/local/bin/opa && sudo chmod +x /usr/local/bin/opa"
)
elif system == "darwin":
url = "https://openpolicyagent.org/downloads/latest/opa_darwin_amd64"
install = (
f"curl -L {url} -o /usr/local/bin/opa && sudo chmod +x /usr/local/bin/opa"
)
elif system == "windows":
url = "https://openpolicyagent.org/downloads/latest/opa_windows_amd64.exe"
install = f"curl -L {url} -o opa.exe (or download from {url})"
else:
url = "https://openpolicyagent.org/docs/latest/#1-download-opa"
install = f"see {url}"

msg = f"""
✗ The OPA binary was not found on PATH.

OPA (Open Policy Agent) is the engine that evaluates Rego policy files.
AICertify uses it to evaluate AI-governance rules against your AI's
captured interactions.

Install it with one command:

{install}

Then re-run: aicertify demo

More info: https://openpolicyagent.org/docs/latest/#1-download-opa
"""
print(msg, file=sys.stderr)


def bundled_contract_path() -> Path:
"""Return the path to the bundled sample contract JSON."""
return Path(str(files("aicertify._demo") / "sample_contract.json"))


def bundled_policy_path(policy: str) -> Path:
"""Return the bundled policy directory we expect to exist for ``policy``.

Used only as an existence probe so the demo can fail fast with a friendly
message if the wheel was stripped or the framework name is unknown. The
actual evaluation passes the friendly framework name (e.g. ``eu_ai_act``)
to the lib's ``find_matching_policy_folders``, which then resolves it to
the absolute directory and recurses for ``.rego`` files.
"""
probe = _BUNDLED_POLICY_PROBE_PATH.get(policy)
if probe is None:
# Unknown friendly name; fall back to treating the input as a
# path relative to opa_policies/.
probe = ("opa_policies", *policy.split("/"))
else:
probe = ("opa_policies", *probe)
p = files("aicertify")
for part in probe:
p = p / part
return Path(str(p))


async def run_demo(
output: str = DEFAULT_REPORT_NAME,
report_format: str = "markdown",
policy: str = DEFAULT_POLICY,
) -> int:
"""Run the bundled demo. Returns a shell-style exit code."""
if opa_binary_path() is None:
print_opa_install_instructions()
return 1

contract_file = bundled_contract_path()
if not contract_file.exists():
print(
f"✗ Bundled sample contract missing at {contract_file}. "
f"This is a packaging bug — please file an issue.",
file=sys.stderr,
)
return 1

policy_dir = bundled_policy_path(policy)
if not policy_dir.exists():
print(
f"✗ Bundled policy directory {policy} not found at {policy_dir}. "
f"Try one of: international/eu_ai_act/v1, global/v1, "
f"international/nist/v1",
file=sys.stderr,
)
return 1

# Load sample contract as an AiCertifyContract
from aicertify.api import load_contract

contract_data = json.loads(contract_file.read_text())
# load_contract accepts a path; serialise the bundled JSON to a tmp file
# via the API's existing path-based loader so we don't reimplement.
import tempfile

with tempfile.NamedTemporaryFile(mode="w", suffix=".json", delete=False) as tmp:
json.dump(contract_data, tmp)
tmp_path = tmp.name

try:
contract = load_contract(tmp_path)
finally:
Path(tmp_path).unlink(missing_ok=True)

output_path = Path(output).resolve()
output_dir = output_path.parent

print(
f"→ Running AICertify demo:\n"
f" contract: {contract.application_name} "
f"({len(contract.interactions)} interactions)\n"
f" policy: {policy}\n"
f" report: {report_format}\n"
)

from aicertify.api import aicertify_app_for_policy

# Pass the relative policy name (not the absolute path); the library's
# find_matching_policy_folders() rejects absolute patterns.
results = await aicertify_app_for_policy(
contract=contract,
policy_folder=policy,
output_dir=str(output_dir),
report_format=report_format,
generate_report=True,
)

# The API writes a timestamped report; surface the path it produced.
report_path = results.get("report_path")
if report_path:
print(f"\n✓ Report written to: {report_path}")
print(
f"\nOpen the report to see what an AICertify audit deliverable "
f"looks like.\n"
)
return 0

err = results.get("error")
if err:
print(f"\n✗ Demo failed: {err}", file=sys.stderr)
return 2

print(
"\n⚠ Demo completed but no report path was returned. "
"Check logs above for details.",
file=sys.stderr,
)
return 3
Loading
Loading