Skip to content

feat(flydsl2flydsl): add flydsl2flydsl task type with FlyDSL kernel test examples#39

Open
johayang-amd wants to merge 3 commits intomainfrom
feature/flydsl-support
Open

feat(flydsl2flydsl): add flydsl2flydsl task type with FlyDSL kernel test examples#39
johayang-amd wants to merge 3 commits intomainfrom
feature/flydsl-support

Conversation

@johayang-amd
Copy link
Copy Markdown
Collaborator

Add flydsl2flydsl task type with FlyDSL kernel test examples

Summary

Adds a new flydsl2flydsl task type for benchmarking FlyDSL (FlyDSL Python DSL) kernel optimization on AMD GPUs.

Framework changes:

  • src/prompt_builder.py — route flydsl2flydsl to the new task type persona
  • src/prompts/task_type.py — add FlyDSL optimization specialist persona
  • src/prompts/cheatsheet/default_cheatsheet.yaml — register FlyDSL cheatsheet
  • src/prompts/cheatsheet/flydsl_cheatsheet.md — FlyDSL optimization guide for agent prompt context

4 initial kernel tasks (sourced from FlyDSL):

  • rmsnorm_kernel — RMSNorm with float32 accumulation
  • layernorm_kernel — LayerNorm with mean/variance reduction
  • softmax_kernel — Numerically stable softmax using exp2
  • fused_rope_cache_kernel — Fused Q/K RoPE rotation + KV cache write

Each task includes kernel.py, test_kernel_harness.py (supports --correctness / --full-benchmark), and config.yaml. Harnesses write build/performance_report.json for evaluator compatibility.

Scope

This PR is strictly additive. All existing task types are unaffected — no existing configs, evaluator, scorer, or agent templates are modified.

Testing

  • Validate tasks with task_validator
  • End-to-end benchmark run

@johayang-amd johayang-amd requested a review from peyron-amd May 7, 2026 14:18
@johayang-amd johayang-amd changed the title Add flydsl2flydsl task type with FlyDSL kernel test examples feat(flydsl2flydsl): add flydsl2flydsl task type with FlyDSL kernel test examples May 7, 2026
@johayang-amd johayang-amd force-pushed the feature/flydsl-support branch from d1916ea to 19210bb Compare May 7, 2026 18:15
@irvineoy
Copy link
Copy Markdown
Collaborator

irvineoy commented May 8, 2026

Two medium follow-ups worth addressing before merge:

  1. The compile_command for the new FlyDSL tasks currently only does an AST parse of kernel.py. That catches syntax errors, but it does not validate that FlyDSL can be imported, that the builder/JIT path is valid, or that the generated module can be constructed. Could we strengthen this to at least import the kernel and instantiate/build a small representative module, if runtime cost is acceptable?

  2. The PR introduces a new FlyDSL task family, but I do not see an install/dependency story for FlyDSL in requirements.txt, docs, or task setup. If FlyDSL is expected to be preinstalled in the benchmark image, please document that assumption; otherwise please add the required dependency/setup instructions so these tasks are reproducible in a clean environment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants