Skip to content

Commit b47c0cb

Browse files
e06084e06084
authored
Merge pull request #373 from MigoXLab/dev
Release 2.2.0
2 parents b10da90 + 467c5b3 commit b47c0cb

130 files changed

Lines changed: 4099 additions & 163206 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.claude/skills/dingo-verify/SKILL.md‎

Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,95 @@
1+
---
2+
name: dingo-verify
3+
description: >
4+
Use when the user wants to fact-check an article or verify factual claims
5+
in a document. Triggers on: "fact-check", "verify article", "check facts",
6+
"文章事实核查", "验证文章". Runs Dingo's ArticleFactChecker via SDK
7+
to extract and verify all factual claims with web search evidence.
8+
argument-hint: [article-path]
9+
allowed-tools: Bash Read Glob
10+
---
11+
12+
# Dingo Article Fact-Checker
13+
14+
Verify factual claims in articles using Dingo's ArticleFactChecker agent.
15+
16+
## Prerequisites
17+
18+
Before running, verify:
19+
1. Dingo is installed: `python -c "from dingo.config import InputArgs; print('OK')"`
20+
2. `OPENAI_API_KEY` is set (required)
21+
3. `TAVILY_API_KEY` is set (optional, enables web search verification)
22+
23+
If prerequisites fail, help the user fix them:
24+
- Missing dingo: `pip install -e .` (from project root) or `pip install dingo-python`
25+
- Missing LangChain: `pip install -r requirements/agent.txt`
26+
- Missing API key: `export OPENAI_API_KEY='your-key'`
27+
28+
## Usage
29+
30+
Run the fact-check script with the article path:
31+
32+
```bash
33+
python ${CLAUDE_SKILL_DIR}/scripts/fact_check.py $ARGUMENTS
34+
```
35+
36+
### Optional arguments
37+
38+
- `--model MODEL`: Override LLM model (default: env `OPENAI_MODEL` or `gpt-5.4-mini`)
39+
- `--max-claims N`: Max claims to extract (default: 50, reduce for faster runs)
40+
- `--max-concurrent N`: Parallel verification slots (default: 5)
41+
42+
### Supported file formats
43+
44+
- `.md`, `.txt`: Markdown/plaintext articles (auto-wrapped for Dingo)
45+
- `.jsonl`: JSONL with `{"content": "..."}` per line
46+
- `.json`: JSON array format
47+
48+
## Presenting Results
49+
50+
The script outputs JSON to stdout. Parse it and present to the user:
51+
52+
### Success output
53+
54+
Present a formatted report with these sections:
55+
56+
1. **Summary**: total claims, accuracy score, false/unverifiable counts
57+
2. **False Claims Table**: if any false claims found, show claim vs truth vs evidence
58+
3. **All Claims Overview**: list all claims with their verdicts (TRUE/FALSE/UNVERIFIABLE)
59+
4. **Output Path**: where the full Dingo report is saved
60+
61+
Example presentation:
62+
63+
```
64+
## Article Fact-Check Report
65+
66+
**Accuracy**: 73.3% (11/15 claims verified true)
67+
- Verified True: 11
68+
- Verified False: 2
69+
- Unverifiable: 2
70+
71+
### False Claims Found
72+
73+
| # | Article Claimed | Actual Truth | Evidence |
74+
|---|----------------|-------------|----------|
75+
| 1 | "released in Nov 2024" | Released Dec 5, 2024 | Official announcement |
76+
77+
### Full Report
78+
Saved to: outputs/20260318_143022_abc123/
79+
```
80+
81+
### Error output
82+
83+
If the script exits with code 1, it prints error JSON to stderr.
84+
Read the `error` and `hint` fields and help the user resolve the issue.
85+
86+
## Performance Notes
87+
88+
- Single article: typically 2-5 minutes depending on claim count and model speed
89+
- Progress is printed to stderr during execution
90+
- For faster runs: use `--max-claims 10 --model gpt-5.4-mini`
91+
92+
## Advanced Configuration
93+
94+
For model selection, claim types, and tuning options, see:
95+
[references/advanced-config.md](references/advanced-config.md)

‎.claude/skills/dingo-verify/references/advanced-config.md‎

Lines changed: 100 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,100 @@
1+
# Advanced Configuration
2+
3+
## Model Selection
4+
5+
### OpenAI Models (recommended)
6+
7+
| Model | Speed | Quality | Cost (per 1M tokens) | Notes |
8+
|-------|-------|---------|---------------------|-------|
9+
| `gpt-5.4-mini` | Fast | Very Good | $0.75 / $4.50 | **Default**, best balance for fact-checking |
10+
| `gpt-5.4` | Medium | Excellent | $2.50 / $15.00 | Flagship, 33% fewer factual errors vs 5.2 |
11+
| `gpt-5.4-nano` | Very Fast | Good | $0.20 / $1.25 | Budget option for simple claims |
12+
| `o3` | Slow | Best reasoning | $2.00 / $8.00 | Best for complex multi-step verification |
13+
| `o4-mini` | Medium | Good reasoning | $1.10 / $4.40 | Budget reasoning model |
14+
15+
### Alternative Providers (via OPENAI_BASE_URL)
16+
17+
| Model | Provider | Cost (per 1M tokens) | Notes |
18+
|-------|----------|---------------------|-------|
19+
| `deepseek-chat` | DeepSeek | $0.14 / $0.28 | Very cheap, good quality, tool use supported |
20+
| `gemini-2.5-flash` | Google (via proxy) | $0.30 / $2.50 | Fast, 1M context, requires OpenAI-compatible proxy |
21+
| Custom model | Self-hosted | Varies | Set via `OPENAI_BASE_URL` |
22+
23+
Override model: `--model gpt-5.4` or `export OPENAI_MODEL=gpt-5.4`
24+
25+
### Model Selection Guide
26+
27+
- **Quick demo**: `gpt-5.4-nano` or `deepseek-chat` (fastest, cheapest)
28+
- **Standard use**: `gpt-5.4-mini` (default, best balance)
29+
- **High accuracy**: `gpt-5.4` (flagship, fewer factual errors)
30+
- **Complex reasoning**: `o3` (best for institutional/attribution claims)
31+
32+
## Claim Types
33+
34+
ArticleFactChecker recognizes 8 claim types:
35+
36+
| Type | Description | Example |
37+
|------|-------------|---------|
38+
| `factual` | General factual statements | "Python was created in 1991" |
39+
| `statistical` | Numbers, percentages, metrics | "GPT-4 achieves 86.4% on MMLU" |
40+
| `attribution` | Who said/did what | "Elon Musk announced..." |
41+
| `institutional` | Organization affiliations | "Released by Tsinghua University" |
42+
| `temporal` | Dates and timelines | "Launched on December 5, 2024" |
43+
| `comparative` | Comparisons between entities | "Faster than GPT-3.5" |
44+
| `monetary` | Financial figures | "Raised $100M in Series B" |
45+
| `technical` | Technical specs and capabilities | "Supports 128K context window" |
46+
47+
## Tuning Parameters
48+
49+
### `--max-claims N` (default: 50)
50+
51+
Controls how many claims are extracted from the article.
52+
53+
- **10-20**: Quick scan, good for short articles or demos
54+
- **30-50**: Standard, covers most article claims
55+
- **50+**: Thorough, may increase execution time significantly
56+
57+
### `--max-concurrent N` (default: 5)
58+
59+
Controls parallel claim verification.
60+
61+
- **1-3**: Conservative, avoids API rate limits
62+
- **5**: Default balance of speed and reliability
63+
- **10**: Fast but may hit rate limits on some APIs
64+
65+
## Environment Variables
66+
67+
| Variable | Required | Default | Description |
68+
|----------|----------|---------|-------------|
69+
| `OPENAI_API_KEY` | Yes | - | API key for LLM calls |
70+
| `OPENAI_BASE_URL` | No | `https://api.openai.com/v1` | Custom API endpoint |
71+
| `OPENAI_MODEL` | No | `gpt-5.4-mini` | Default model |
72+
| `TAVILY_API_KEY` | No | - | Enables web search verification |
73+
74+
## Output Artifacts
75+
76+
Dingo saves detailed output to `outputs/<timestamp>/`:
77+
78+
| File | Content |
79+
|------|---------|
80+
| `summary.json` | Overall evaluation statistics |
81+
| `content/QUALITY_BAD_*.jsonl` | Per-item results grouped by error type |
82+
83+
ArticleFactChecker also saves intermediate artifacts:
84+
85+
| File | Content |
86+
|------|---------|
87+
| `article_content.md` | Original article text |
88+
| `claims_extracted.jsonl` | Extracted claims (one per line) |
89+
| `claims_verification.jsonl` | Per-claim verification details |
90+
| `verification_report.json` | Full structured verification report |
91+
92+
## Troubleshooting
93+
94+
| Problem | Solution |
95+
|---------|----------|
96+
| "Dingo SDK not installed" | `pip install -e .` from project root |
97+
| "LangChain not installed" | `pip install -r requirements/agent.txt` |
98+
| Timeout errors | Use `--model gpt-5.4-mini` and `--max-claims 20` |
99+
| Rate limit errors | Reduce `--max-concurrent` to 2-3 |
100+
| Empty results | Check that article has verifiable factual claims |

0 commit comments

Comments
 (0)