subset metrics

[fzhml]

Thank you for your excellent work. The paper presents pass@1/pass@3/pass^3/average metrics for the full 200-sentence test set. Are there any breakdowns by subsets available?

[icexiaoche]

Hi fzhml,
Thank you for your interest in Claw-Anything!

Regarding the metrics for the different subsets, we have provided the pass@1 scores for both cli-only and cli+gui in the appendix of our paper. We plan to include the detailed data for pass@3/pass^3/average in a future update of the paper.

Please stay tuned for further updates on Claw-Anything!

Best regards,
Yusong Lin

[fzhml]

thanks for your reply！
I have one more question: could you share which judge model was adopted for evaluating each different subset in your experiments?

[icexiaoche]

Thank you for your follow-up question.

In our experiments, Claude Sonnet 4.6 was adopted as the standard judge model across all tasks.

Please let us know if you have any further questions!

[fzhml]

I noticed the paper uses Claude Sonnet 4.5. Can performance results be consistent across different judge models?