Request for Evaluation Results for Academic Research

[kinokopro]

Hi,

I am currently writing a research paper and would like to conduct a statistical analysis of existing work in the related work section. Specifically, I want to analyze the error types in different methods' results - how many are logical errors versus syntax/execution errors.

I have already obtained the results of DAIL-SQL on the Spider dataset. I noticed that your work has achieved excellent scores on the BIRD dataset leaderboard.

Would it be possible for you to share the detailed results that correspond to your leaderboard score? This would allow me to directly evaluate and analyze the error distributions.

My Commitment:
I will clearly cite and acknowledge the data source in my paper
I will comply with any usage guidelines or requirements you may have
The data will be used solely for academic research purposes
Thank you very much for considering this request. I greatly appreciate your contribution to the community and look forward to your response.

If you are willing to share the data, please feel free to contact me at: yangh50060@hdu.edu.cn

Best regards！

[Peng-fei-Wang]

Hi @kinokopro ,

Thanks for your interest in our work.

We are happy to share our Dev set predictions to support your research. Regarding the Test set, as it is private and evaluated officially by the BIRD organizers, we only receive the final scores and do not have access to the prediction files.

I will send the Dev set results to your email (yangh50060@hdu.edu.cn) shortly.

[wanteatfruit]

Hi @Peng-fei-Wang

Thank you for this amazing work! I'm a PhD student at NUS doing relevant research. I noticed that your work got the first place of Spider 2.0-Snow benchmark (https://spider2-sql.github.io/). Would it be possible share the predictions on the Spider 2.0-Snow benchmark? My email is junwenan@u.nus.edu

Thank you!

[Peng-fei-Wang]

Hi @wanteatfruit ,

To clarify, we have not evaluated our work on the Spider 2.0-Snow benchmark. Since this would require setting up the Snowflake environment and involves high API overhead, it is currently not in our immediate roadmap.

We apologize for any confusion and thank you for following our research!

[wanteatfruit]

Hi @Peng-fei-Wang,

Thank you for the clarification! I realized I might have mistakenly overlooked that leaderboard; I sincerely apologize for this. Still, is there any chance I could also have the BIRD predictions?

[Peng-fei-Wang]

Hi @Peng-fei-Wang,

Thank you for the clarification! I realized I might have mistakenly overlooked that leaderboard; I sincerely apologize for this. Still, is there any chance I could also have the BIRD predictions?

Hi @wanteatfruit,
Sure! I've just sent the Dev set results to your email address.

[wanteatfruit]

Hi @Peng-fei-Wang ,
Received, thank you!