Skip to content

Create evaluation data set for focused scenarios #29

@ddlliu

Description

@ddlliu

Current Status:
Xlsx-lookup (single table)​ | 121 queries
Xlsx-aggregation (single table)​ | 34 queries
Xlsx-lookup (multiple tables)​ | 10 queries
Xlsx-aggregation (multiple tables)​ | 10 queries
PDF-lookup (single table)​ | 77 queries
DOCX-lookup (single table)​ | 43 queries
PPTX-lookup (single table)​ | 0 queries
file generation​ | 8 queries

Strategy
files + manually crafted few shot examples --> synthetic question and answers --> run code interpreter --> questions and code interpreter generated answers --> manual filtering --> manual labeling --> curated q/a pairs

Pointer of more details

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions