[Urgent] Single Agent Forecasting Benchmark Re-open for IROS deadline

[ctyfang]

Thank you guys for your work with the dataset and in maintaining the benchmark. Could you please re-open the Single Agent Forecasting competition? I need it for an active resubmission to IROS. Thank you very much.

[ctyfang]

I see that some others are also interested in the same #329

[HighqualityZ]

@ctyfang I'm facing the same issue with the closed benchmark. Did the organizers reach out to you regarding the re-opening? I'm particularly curious if you managed to complete your research without the test set metrics if they didn't reply in time for your deadline.

[Lukas88664]

@benjaminrwilson hello

[kkurise]

@wqi @benjaminrwilson @neeharperi Has EvalAI replied to any of your messages?

[James-Hays]

At this point, it looks like we sadly need to declare the AV1 motion forecasting benchmark deprecated. I would recommend people evaluate single agent forecasting on the AV2 motion forecasting dataset.

We have no copy of the AV1 test set with Eval AI's deletion of our challenge. Older backups were lost with the closure of Argo.

AV2 is backed up and all test sets have been made public so this won't happen with AV2.

[ctyfang]

What are the implications for the single agent forecasting challenge if the test set is public?

[James-Hays]

Are you asking about prize competitions for AV2 forecasting? We aren't hosting such a competition this year and probably won't in the future.

If you're asking more generally about how much you can trust a benchmark if the test set is public, then yes, that is a concern. But plenty of influential test sets have public test sets or instead report numbers on the validation set (e.g. COCO minival).

[yhwang1520-gif]

However, submissions for the Argoverse 2 challenge currently appear to be unavailable. Could you please check and help resolve this issue?

[neeharperi]

We are no longer actively maintaining the Argoverse 2 motion forecasting challenge on Eval AI. We have released the test-set labels for this challenge on argoverse.org and encourage you to evaluate your methods locally.

[yhwang1520-gif]

thanks Neehar Peri ***@***.***> 于2026年3月12日周四 01:07写道：

…

*neeharperi* left a comment (argoverse/av2-api#335) <#335 (comment)> We are no longer actively maintaining the Argoverse 2 motion forecasting challenge on Eval AI. We have released the test-set labels for this challenge on argoverse.org <https://www.argoverse.org/av2.html#download-link> and encourage you to evaluate your methods locally. — Reply to this email directly, view it on GitHub <#335 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/B3S2CW4QYF4BX6DIS5P47Z34QGMNLAVCNFSM6AAAAACV2XXPA6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHM2DANBQG4ZTKNBZHE> . You are receiving this because you commented.Message ID: ***@***.***>