Can I use the LMDrive dataset to train previous works?

[jje526]

Thank you for your outstanding work. I am currently trying to simulate your projects — LM Drive, ReasonNet, and InterFuser. Based on the papers, it seems that all three projects use similar datasets (8 towns + 21 weather conditions, 3M/2M/3M frames, respectively).

Is it possible to use the publicly available LM Drive dataset to train the other models (ReasonNet and InterFuser) as well?

[deepcs233]

It's ok.

[jje526]

Hi, thanks so much for your reply!

I have successfully completed ReasonNet training by reusing and integrating your excellent works
I have a few questions regarding my reimplementation of Reasonnet based on LMDrive github & Dataset

1. I assumed LMDrive’s vision encoder and ReasonNet’s perception module are basically the same, and built my code that way. Are they actually identical? (From the paper details, they look almost the same.)

2. For the occupancy ground truth — is it okay to use the BEV map object probability (first traffic channel) at the corresponding future timestep? Or should I actually generate a new dataset for this? I was able to obtain the following results using this GT together with det_loss, but I’m wondering if this is the same as what you originally intended.

3. About the occupancy map estimation:
   You mention in the paper: “2) the occupancy map feature, which is then applied with convolution operation to generate the occupancy map Ot ∈ RTf×H×W.”
   Following this, I reduced the output from the occupancy decoder (dim=256) through two 3×3 convolution layers: 256 → 64 → 4 (Tf). Does this match what you intended?

4. And for the occupancy queries — should they be implemented as learnable parameters?

Thanks again for all your amazing contributions to the E2E algorithm.
Even a partial answer would mean a lot! 🙌