train/validation split in the Domino model for the DrivAerML data set

[kk98kk]

Hi everyone,

I’m currently exploring the Domino model and trying to reproduce some of the results.

I found this section about the dataset split:
https://github.com/NVIDIA/physicsnemo-cfd/blob/main/workflows/bench_example/drivaer_ml_files/README.md#drivaerml-dataset-files

From what I can see, it lists which runs are used for training and testing, but I couldn’t find any information on how the training data itself was further divided into training and validation subsets.

Does anybody know how that split was done (e.g., which runs were used for training, validation and testing explicitly)
Having that detail would be very helpful for reproducibility.

Thanks a lot in advance!

[coreyjadams]

Hi @kk98kk - I didn't realize discussions were open here, sorry!

Take a look at this paper: https://arxiv.org/pdf/2507.10747

In particular, in section 2.2.1, second paragraph:

As of this writing, the DrivAerML dataset [18] does not provide guidance on train-validation split. To
enable consistent comparison, we propose the split in the benchmarking code. The dataset is split as 90% for
training and 10% for validation. For selecting the validation set, the entire dataset is sorted by drag force. Then,
10% of the validation set is chosen to be the top of the sorted dataset, and another 10% is chosen to be the
bottom of the sorted dataset. The rest of the 80% of the validation set is chosen randomly from the remaining
dataset. This allows us to include some out-of-distribution data in the validation set, which is important for the
generalizability of the models. Figure 1 shows the training-validation split pictorially

[kk98kk]

Hi,

Thanks for the paper — I went through it earlier already, but I’m still unclear about how the dataset is split among the train, validation, and test folders for the Domino architecture.

As I understand it, the workflow for the Domino architecture requires three folders:

• Train folder: used to train the model
• Validation folder: used to validate the trained models
• Test folder: used to evaluate the model on unseen geometries

For the test folder, it’s mentioned that it contains 10% of the dataset, where:

• 20% of that subset are out-of-distribution (10% of the top highest and 10% of the top lowest drag-sorted models), and
• the remaining 80% are random samples.

The test runs seem to be explicitly defined here:
https://github.com/NVIDIA/physicsnemo-cfd/blob/main/workflows/bench_example/drivaer_ml_files/train.csv

However, I couldn’t find any information on how the remaining runs are split between training and validation.
Am I missing something here? Or are the validation and test sets actually the same — which wouldn’t make much sense to me?