SemanticKITTI single-scan segmentation submissions repeatedly fail with "Execution Time Limit exceeded"

[vsrters]

Dear CodaBench / SemanticKITTI team,

I hope you are doing well.
I recently submitted results to the SemanticKITTI single-scan semantic segmentation benchmark, but the submissions repeatedly failed with the following error:

Execution Time Limit exceeded. Limit was 1500 seconds.

To further check whether this was caused by my current prediction files, I also re-submitted a previous submission package that had been successfully evaluated before. However, it now fails with the same timeout error. Therefore, it seems that the issue may not be related to the submission format or prediction files.

Previous successful submission:
ID: 695866, score: 65.5

Recent failed submissions:
ID: 713500
ID: 713396

The package structure is the standard SemanticKITTI format:

sequences/11/predictions/.label
...
sequences/21/predictions/.label

Could you please help check whether the evaluation worker or scoring queue for this benchmark is running normally?

Thank you very much for your help.

[zhangsongrong]

Excuse me, I would like to ask if there is a submission channel for multiple scans.
This option was available on the original CodaLab platform, but I cannot find the corresponding entry on Codabench.

[Didayolo]

Dear @vsrters,

There are 5 different SemanticKITTI challenges on Codabench. Please provide the URL of the challenge you are participating in so we can investigate the problem.

@zhangsongrong

Excuse me, I would like to ask if there is a submission channel for multiple scans.

I do not get what you mean here, please try to describe better your request. Also, if it is not related to the SemanticKITTI challenge, please open a separate issue on the repository.

[zhangsongrong]

Dear @Didayolo，

Hello, I would like to inquire about Multiple Scans in the SemanticKITTI Semantic Segmentation challenge.
The challenge is split into two tracks: Multiple Scans and Single Scans. However, only the Single Scans competition track is available on Codabench. I would like to ask whether the Multiple Scans track still exists.
The link on Codabench is https://www.codabench.org/competitions/12448/#/results-tab, and the original CodaLab link is https://codalab.lisn.upsaclay.fr/competitions/6280#results.

[vsrters]

Dear @Didayolo
Thank you very much for your reply.
The challenge URL is:
https://www.codabench.org/competitions/12448/
It is the SemanticKITTI: Semantic Segmentation benchmark for the single-scan semantic segmentation task.
My recent submissions to this challenge repeatedly failed with:
Execution Time Limit exceeded. Limit was 1500 seconds.