-
Notifications
You must be signed in to change notification settings - Fork 0
Open
Description
need to provide options for strucutred responses
maybe in templates to request json or as an additional bench-level meta-task
ollama CLI has --format json option that seems to work, need to figure out way to use that through python/ how to let users pass that
https://github.com/ollama/ollama-python/blob/main/examples/structured-outputs.py
Still to do:
- make it work for other runners
- decide how to store format type in each task format or if per benchmark?
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels