Some Issues on ATM Model Training.

[cuite1999]

ATM is really a very delicate work, which is very rewarding.
I encountered some problems in the process of reproducing the results of the ATM paper. I fully trained the track model and the action model according to the method and process you provided. However, whether I used the entire dataset training data, or used 20% of the training data to train the action model, or used the tracking transformer model you provided, plus 20% of the data training action model, the results during testing could not reach the results in the paper. Even the success rate of many tasks is 0. There must be something wrong with my reproduction process. Can you share your training or testing log, or the final model you obtained by training under different data set partitions?
Sorry to bother you.

[AlvinWen428]

Hello, thanks for your nice words. The policy checkpoints trained with 20% data have been provided in README. I wonder whether you have tried to evaluate them and how is the performance. I'm so sorry that I didn't save the training logs.

[rshkong]

ATM is truly a remarkable piece of work, and I appreciate all the effort you’ve put into it. I’ve encountered similar difficulties reproducing the results. I’m using an RTX 4090 with 24 GB of GPU memory, and due to CUDA memory limitations, I had to reduce my batch size to 32. Unfortunately, on the LIBERO goal suite 9 out of 10 tasks still have a success rate of 0. I also tried gradient accumulation to compensate, but it didn’t improve performance. Given these hardware constraints, I’m not sure how to get better training results. I would really appreciate any advice or suggestions you could share.

[fsdqbs]

ATM 确实是一项了不起的工作，我感谢您为它付出的所有努力。我在重现结果时也遇到了类似的困难。我使用的是具有 24 GB GPU 内存的 RTX 4090，由于 CUDA 内存限制，我不得不将批处理大小减少到 32。遗憾的是，在 LIBERO 目标套件中，10 个任务中有 9 个的成功率仍然为 0。我还尝试了梯度累积来补偿，但它并没有提高性能。鉴于这些硬件限制，我不确定如何获得更好的训练结果。我非常感谢您能分享的任何意见或建议。

Hello, do you have this error in the evaluation stage: KeyError: 'MountedPanda', if so, is it convenient to tell me the solution? Thank you very much!

[rshkong]

ATM 确实是一项了不起的工作，我感谢您为它付出的所有努力。我在重现结果时也遇到了类似的困难。我使用的是具有 24 GB GPU 内存的 RTX 4090，由于 CUDA 内存限制，我不得不将批处理大小减少到 32。遗憾的是，在 LIBERO 目标套件中，10 个任务中有 9 个的成功率仍然为 0。我还尝试了梯度累积来补偿，但它并没有提高性能。鉴于这些硬件限制，我不确定如何获得更好的训练结果。我非常感谢您能分享的任何意见或建议。

Hello, do you have this error in the evaluation stage: KeyError: 'MountedPanda', if so, is it convenient to tell me the solution? Thank you very much!

No, I haven't encountered this error. I’ve figured out why my previous training didn’t work — it was because I had modified the preprocessing script to handle CUDA memory limitations. However, I made a mistake in the code, which caused the dataset to include only the first 50 frames for each task. After fixing the preprocessing and handling the dataset properly, I was able to reproduce the training results.

[fsdqbs]

ATM 确实是一项了不起的工作，我感谢您为它付出的所有努力。我在重现结果时也遇到了类似的困难。我使用的是具有 24 GB GPU 内存的 RTX 4090，由于 CUDA 内存限制，我不得不将批处理大小减少到 32。遗憾的是，在 LIBERO 目标套件中，10 个任务中有 9 个的成功率仍然为 0。我还尝试了梯度累积来补偿，但它并没有提高性能。鉴于这些硬件限制，我不确定如何获得更好的训练结果。我非常感谢您能分享的任何意见或建议。

Hello, do you have this error in the evaluation stage: KeyError: 'MountedPanda', if so, is it convenient to tell me the solution? Thank you very much!

No, I haven't encountered this error. I’ve figured out why my previous training didn’t work — it was because I had modified the preprocessing script to handle CUDA memory limitations. However, I made a mistake in the code, which caused the dataset to include only the first 50 frames for each task. After fixing the preprocessing and handling the dataset properly, I was able to reproduce the training results.

Thank you very much for your answer! I have resolved the error that occurred on my end, and I hope everything goes smoothly for you!

[fsdqbs]

ATM 确实是一项了不起的工作，我非常感谢您为此付出的所有努力。我在复现结果时也遇到了类似的困难。我使用的是配备 24 GB GPU 内存的 RTX 4090，由于 CUDA 内存限制，我不得不将批次大小减小到 32。遗憾的是，在 LIBERO 目标套件中，10 个任务中有 9 个的成功率仍然为 0。我也尝试过梯度累积来弥补，但性能并没有提高。考虑到这些硬件限制，我不确定如何获得更好的训练结果。如果您能分享任何建议，我将不胜感激。

Sorry to bother you again. I would like to ask if there was a mujoco simulation environment during your replication process. I found that in my replication process, only evaluation videos and success rates appeared at the end, and there was no real-time simulation scene of mujoco. Do you have it? If so, how do you need to modify it? I have set has_renderer to True, but I found that the real-time simulation scene still did not appear.