Summary
Add ImgEdit benchmark for image editing evaluation with 8 edit type subsets.
Dataset
- Source:
sysuyy/ImgEdit (HuggingFace) + GitHub JSON from PKU-YuanGroup/ImgEdit
- Edit types: replace, add, remove, adjust, extract, style, background, compose
- Collate:
prompt_with_auxiliaries_collate
Implementation
- Add
setup_imgedit_dataset in src/pruna/data/datasets/prompt.py
- Support
subset param for filtering edit types
- Register in
base_datasets
- Add
BenchmarkInfo entry with metrics: ["accuracy"], subsets list
- Auxiliaries should include
image (input), judge_prompt, subset
- Add test
Acceptance
PrunaDataModule.from_string("ImgEdit") works (all subsets)
PrunaDataModule.from_string("ImgEdit", subset="replace") works
- Auxiliaries include
image, judge_prompt, subset fields
- Test passes
Summary
Add ImgEdit benchmark for image editing evaluation with 8 edit type subsets.
Dataset
sysuyy/ImgEdit(HuggingFace) + GitHub JSON fromPKU-YuanGroup/ImgEditprompt_with_auxiliaries_collateImplementation
setup_imgedit_datasetinsrc/pruna/data/datasets/prompt.pysubsetparam for filtering edit typesbase_datasetsBenchmarkInfoentry with metrics:["accuracy"], subsets listimage(input),judge_prompt,subsetAcceptance
PrunaDataModule.from_string("ImgEdit")works (all subsets)PrunaDataModule.from_string("ImgEdit", subset="replace")worksimage,judge_prompt,subsetfields