Skip to content

Commit 5faf984

Browse files
authored
Collector docs improved (#27)
1 parent e364f25 commit 5faf984

3 files changed

Lines changed: 30 additions & 1 deletion

File tree

docs/cookbooks/batch-evaluation.mdx

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -441,6 +441,8 @@ def find_failures(results: List[EvalTask]) -> List[Dict]:
441441

442442
### Generating Reports
443443

444+
> **Tip:** For pytest-based batch runs, you can use the built-in `--fe-output` flag to export results as JSON or HTML without writing custom reporting code. See [Output Collectors](/docs/advanced/output-collectors) for details.
445+
444446
```python
445447
# reporting.py
446448
import json

docs/core-concepts/pytest.mdx

Lines changed: 27 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,8 @@ sidebar:
99
anchor: running-tests
1010
- title: Understanding Output
1111
anchor: understanding-output
12+
- title: Exporting Results
13+
anchor: exporting-results
1214
- title: Fixtures
1315
anchor: fixtures
1416
- title: CI/CD Integration
@@ -74,6 +76,23 @@ Evaluation Results:
7476
be perceived as unhelpful or rude.
7577
```
7678

79+
### Exporting Results
80+
81+
You can export evaluation results to JSON or HTML, or print a summary table to the console:
82+
83+
```bash
84+
# Export as JSON
85+
pytest tests/ --fe-output json:results.json
86+
87+
# Export as HTML report
88+
pytest tests/ --fe-output html:report.html
89+
90+
# Print a summary table after the run
91+
pytest tests/ --fe-summary
92+
```
93+
94+
For full details on output formats, custom reporters, and programmatic access, see [Output Collectors](/docs/advanced/output-collectors).
95+
7796
## Configuration
7897

7998
### Using pyproject.toml (Recommended)
@@ -202,7 +221,14 @@ jobs:
202221
- name: Run evaluations
203222
env:
204223
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
205-
run: pytest tests/ -v --tb=short
224+
run: pytest tests/ -v --tb=short --fe-output json:results.json --fe-summary
225+
226+
- name: Upload results
227+
if: always()
228+
uses: actions/upload-artifact@v4
229+
with:
230+
name: evaluation-results
231+
path: results.json
206232
```
207233
208234
### Separating Fast and Slow Tests

docs/getting-started/quickstart.mdx

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -229,4 +229,5 @@ You've got the fundamentals down. Here's where to go deeper:
229229
- **Evaluate RAG pipelines**[RAG Metrics](/docs/rag-metrics/faithfulness)
230230
- **Build custom metrics**[Custom Metrics](/docs/advanced/custom-metrics)
231231
- **Set up CI/CD**[CI/CD Setup](/docs/cookbooks/ci-cd-setup)
232+
- **Export results**[Output Collectors](/docs/advanced/output-collectors) for JSON, HTML reports, and custom formats
232233
- **Monitor production**[fasteval-observe](/docs/plugins/observe/overview)

0 commit comments

Comments
 (0)