Skip to content

Commit af7dfa2

Browse files
authored
docs: update readme (MigoXLab#368)
1 parent 1ee36fb commit af7dfa2

3 files changed

Lines changed: 247 additions & 232 deletions

File tree

README.md

Lines changed: 53 additions & 79 deletions
Original file line numberDiff line numberDiff line change
@@ -197,36 +197,7 @@ dingo eval --input .github/env/local_plaintext.json
197197
dingo eval --input .github/env/local_json.json
198198
```
199199

200-
## GUI Visualization
201-
202-
After evaluation (with `result_save.bad=True`), a frontend page will be automatically generated. To manually start the frontend:
203-
204-
```shell
205-
python -m dingo.run.vsl --input output_directory
206-
```
207-
208-
Where `output_directory` contains the evaluation results with a `summary.json` file.
209-
210-
![GUI output](docs/assets/dingo_gui.jpg)
211-
212-
## Online Demo
213-
Try Dingo on our online demo: [(Hugging Face)🤗](https://huggingface.co/spaces/DataEval/dingo)
214-
215-
## Local Demo
216-
Try Dingo in local:
217-
218-
```shell
219-
cd app_gradio
220-
python app.py
221-
```
222-
223-
![Gradio demo](docs/assets/gradio_demo.png)
224-
225-
226-
## Google Colab Demo
227-
Experience Dingo interactively with Google Colab notebook: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/DataEval/dingo/blob/dev/examples/colab/dingo_colab_demo.ipynb)
228-
229-
200+
---
230201

231202
# MCP Server
232203

@@ -255,6 +226,58 @@ https://github.com/user-attachments/assets/aca26f4c-3f2e-445e-9ef9-9331c4d7a37b
255226

256227
This video demonstrates step-by-step how to use Dingo MCP server with Cursor.
257228

229+
---
230+
231+
# 📚 Data Quality Metrics
232+
233+
Dingo provides **70+ evaluation metrics** across multiple dimensions, combining rule-based speed with LLM-based depth.
234+
235+
## Metric Categories
236+
237+
| Category | Examples | Use Case |
238+
|----------|----------|----------|
239+
| **Pretrain Text Quality** | Completeness, Effectiveness, Similarity, Security | LLM pre-training data filtering |
240+
| **SFT Data Quality** | Honest, Helpful, Harmless (3H) | Instruction fine-tuning data |
241+
| **RAG Evaluation** | Faithfulness, Context Precision, Answer Relevancy | RAG system assessment |
242+
| **Hallucination Detection** | HHEM-2.1-Open, Factuality Check | Production AI reliability |
243+
| **Classification** | Topic categorization, Content labeling | Data organization |
244+
| **Multimodal** | Image-text relevance, VLM quality, OCR visual evaluation | Vision-language data |
245+
| **Security** | PII detection, Perspective API toxicity | Privacy and safety |
246+
247+
📊 **[View Complete Metrics Documentation →](docs/metrics.md)**
248+
📖 **[RAG Evaluation Guide →](docs/rag_evaluation_metrics.md)** | **[中文版](docs/rag_evaluation_metrics_zh.md)**
249+
🔍 **[Hallucination Detection Guide →](docs/hallucination_detection_guide.md)** | **[中文版](docs/hallucination_guide.md)**
250+
**[Factuality Assessment Guide →](docs/factuality_assessment_guide.md)** | **[中文版](docs/factcheck_guide.md)**
251+
👁️ **[VLM Render Judge Guide →](docs/en/vlm_render_judge_guide.md)** | **[中文版](docs/vlm_render_judge_guide.md)**
252+
253+
Most metrics are backed by academic research to ensure scientific rigor.
254+
255+
## Quick Metric Usage
256+
257+
```python
258+
llm_config = {
259+
"model": "gpt-4o",
260+
"key": "YOUR_API_KEY",
261+
"api_url": "https://api.openai.com/v1/chat/completions"
262+
}
263+
264+
input_data = {
265+
"evaluator": [
266+
{
267+
"fields": {"content": "content"},
268+
"evals": [
269+
{"name": "RuleAbnormalChar"}, # Rule-based (fast)
270+
{"name": "LLMTextQualityV5", "config": llm_config} # LLM-based (deep)
271+
]
272+
}
273+
]
274+
}
275+
```
276+
277+
**Customization**: All prompts are defined in `dingo/model/llm/` directory (organized by category: `text_quality/`, `rag/`, `hhh/`, etc.). Extend or modify them for domain-specific requirements.
278+
279+
280+
---
258281

259282
# 🎓 Key Concepts for Practitioners
260283

@@ -324,55 +347,6 @@ class MyCustomRule(BaseRule):
324347

325348
---
326349

327-
# 📚 Data Quality Metrics
328-
329-
Dingo provides **70+ evaluation metrics** across multiple dimensions, combining rule-based speed with LLM-based depth.
330-
331-
## Metric Categories
332-
333-
| Category | Examples | Use Case |
334-
|----------|----------|----------|
335-
| **Pretrain Text Quality** | Completeness, Effectiveness, Similarity, Security | LLM pre-training data filtering |
336-
| **SFT Data Quality** | Honest, Helpful, Harmless (3H) | Instruction fine-tuning data |
337-
| **RAG Evaluation** | Faithfulness, Context Precision, Answer Relevancy | RAG system assessment |
338-
| **Hallucination Detection** | HHEM-2.1-Open, Factuality Check | Production AI reliability |
339-
| **Classification** | Topic categorization, Content labeling | Data organization |
340-
| **Multimodal** | Image-text relevance, VLM quality, OCR visual evaluation | Vision-language data |
341-
| **Security** | PII detection, Perspective API toxicity | Privacy and safety |
342-
343-
📊 **[View Complete Metrics Documentation →](docs/metrics.md)**
344-
📖 **[RAG Evaluation Guide →](docs/rag_evaluation_metrics.md)** | **[中文版](docs/rag_evaluation_metrics_zh.md)**
345-
🔍 **[Hallucination Detection Guide →](docs/hallucination_detection_guide.md)** | **[中文版](docs/hallucination_guide.md)**
346-
**[Factuality Assessment Guide →](docs/factuality_assessment_guide.md)** | **[中文版](docs/factcheck_guide.md)**
347-
👁️ **[VLM Render Judge Guide →](docs/en/vlm_render_judge_guide.md)** | **[中文版](docs/vlm_render_judge_guide.md)**
348-
349-
Most metrics are backed by academic research to ensure scientific rigor.
350-
351-
## Quick Metric Usage
352-
353-
```python
354-
llm_config = {
355-
"model": "gpt-4o",
356-
"key": "YOUR_API_KEY",
357-
"api_url": "https://api.openai.com/v1/chat/completions"
358-
}
359-
360-
input_data = {
361-
"evaluator": [
362-
{
363-
"fields": {"content": "content"},
364-
"evals": [
365-
{"name": "RuleAbnormalChar"}, # Rule-based (fast)
366-
{"name": "LLMTextQualityV5", "config": llm_config} # LLM-based (deep)
367-
]
368-
}
369-
]
370-
}
371-
```
372-
373-
**Customization**: All prompts are defined in `dingo/model/llm/` directory (organized by category: `text_quality/`, `rag/`, `hhh/`, etc.). Extend or modify them for domain-specific requirements.
374-
375-
376350
# 🌟 Feature Highlights
377351

378352
## 📊 Multi-Source Data Integration

0 commit comments

Comments
 (0)