Skip to content

Task response formats #43

@brownsarahm

Description

@brownsarahm

need to provide options for strucutred responses

maybe in templates to request json or as an additional bench-level meta-task

ollama CLI has --format json option that seems to work, need to figure out way to use that through python/ how to let users pass that

https://github.com/ollama/ollama-python/blob/main/examples/structured-outputs.py

Still to do:

  • make it work for other runners
  • decide how to store format type in each task format or if per benchmark?

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions