Linear Mode Connectivity under Data Shifts for Deep Ensembles of Image Classifiers

This repository contains code and experimental results for "Linear Mode Connectivity under Data Shifts for Deep Ensembles of Image Classifiers". Link to arXiv: https://arxiv.org/abs/2511.04514.

Repository Overview

1. Experiments on MNIST and CIFAR-10 with MLP, ResNet, and VGG models

   • codes contains:
     • scripts for model training, linear interpolation, and ensemble prediction
     • a .pptx presentation with an overview of the relevant code
     • notebooks, which contains Jupyter notebooks for plotting experimental results and figures for the paper (see also figures)
   • experimental_results contains automatically generated files such as learning curves and interpolation results; trained models can be provided upon request
   • ppts contains presentations summarizing the experimental results, including tables and selected plots of learning curves, interpolations, and polar plots

2. Experiments on the BigEarthNet dataset with a VGG-like model

   The bigearth directory contains:

   • BigEarthNetList contains metadata for the BigEarthNet dataset (https://bigearth.net/)
   • Rebasin contains scripts for model training and linear interpolation
   • outputs contains trained models and generated files such as learning curves
   • results contains interpolation results

3. Poster

   • PDF of a poster presented at the Helmholtz Imaging Conference 2025 (https://helmholtz-imaging.de/news/helmholtz-imaging-conference-2025/)

[!Note] Note on weight matching

The repository contains a weight_matching option, but it was not used in the experiments presented in the paper. It is included for reference only.

Requirements

The setup_scripts directory contains code for setting up the environment and downloading the large-scale BigEarthNet dataset. The project dependencies are listed in setup_scripts/requirements.txt.

Installation

Clone the repository:

git clone https://github.com/DLR-KI/LMC.git
cd LMC

Create and activate a conda environment:

conda create -n lmc
conda activate lmc

Install the dependencies:

bash setup_project.sh

In case (on Linux systems) one of the scripts don't run, try running the following command for that script:

dos2unix script/path/my_script.sh
chmod +x script/path/my_script.sh

Instructions for reproducing the experimental results

Before starting please navigate into the codes directory:

cd codes

Reproduce the experimental results on MNIST and CIFAR-10

• Model training (example arguments). Run the following commands in the terminal:

  • MLP

    python3 mlp_subsets_training.py --seed 1 --data 'cifar' --percentage 20 --bn_layer 1 --epochs 500 --save_model 500

  • ResNet

    python3 resnet_subsets_training.py --seed 1 --data 'mnist' --depth 10 --epochs 50 --save_model 50

  • VGG

    python3 vgg_subsets_training.py --seed 20 --vgg_model 'VGG11' --noise 'random'

  • Alternatively, run the provided bash scripts to train multiple models sequentially with different hyperparameters:

    bash train_mlp.sh
    bash train_resnet.sh
    bash train_vgg11.sh
    bash train_vgg19.sh

• Linear interpolation (example arguments). Run the following command in the terminal:

  • MLP

    python3 interpolation_mlp.py  --data 'cifar' --split_percentage 50  --training_batch_size 1024 --training_lr 0.001 --epoch 500 --initialization 'diff'

  • ResNet

    python3 interpolation_resnet.py --data 'mnist' --split_percentage 50 --depth 22 --training_batch_size 32 --training_lr 0.0001 --epoch 100  --noise 'same'

  • VGG

    python3 interpolation_vgg.py --training_data 1 --split_percentage 50  --subset_index 1 --initialization 'diff' --training_batch 1024 --training_lr 0.0001

• Ensemble prediction with selected models

  • Ensemble constructed via LMC (example arguments):

    python3 ensemble_prediction_LMC.py --model 'resnet'

  • Ensemble constructed from different random seeds (example arguments):

    python3 ensemble_prediction_DIFF_SEEDS.py --model 'mlp' --percentage 20

Reproduce the experimental results on BigEarthNet

• Model training (example arguments). Run the following commands in the terminal:

  python train.py -s 100 -b "B02" -e 100 -n 256

• Alternatively, run the provided bash scripts to train multiple models sequentially with different hyperparameters, using either single-band or multi-band inputs:

  bash train_single.sh
  bash train_multiple.sh

• Linear interpolation (example arguments). Select two epoch_X_model_dict files from either bigearth/outputs/1_bands or bigearth/outputs/3_bands. Run the following command in the terminal:

  python test2.py -p1 '<path to model 1>' -p2 '<path to model 2>'

Authors and acknowledgment

Authors in alphabetical order:

• Carolyna Hepburn
• Tobias Zielke

Acknowledgments:

• Jakob Gawlikowski
• Mauricio Ortiz Torres

Citation

For citation details, please refer to the metadata provided in the CITATION.cff file associated with this work.

Licenses

Please see the file LICENSE.md for further information about how the content is licensed.