TRADES Adversarial Training Code

[TalRub104]

Hi,

Can you please upload your code for the TRADES adversarial training part (specifically for restoring Figure 5 in the CIFAR-10 case)?

[HashmatShadab]

Hi @TalRub104, thank you for reaching out! We followed the training recipe provided in (https://github.com/dedeswim/vits-robustness-torch) for adversarial fine-tuning of vision models. You can utilize their codebase to run adversarial training. Let us know if you have any further questions!

[TalRub104]

Thank you for your reference.
I would appreciate it if you could review the steps I made and the current issue I am failing with:

0. Cloning the https://github.com/dedeswim/vits-robustness-torch repository and setting up the environment

1. Working with the following command line:
   python -m torch.distributed.launch --nproc_per_node=8 train.py CIFA10 --dataset torch/CIFAR10 --experiment vssm_tiny_v2_TRADE_finetuning_none_reg --output output/finetuning_TRADE/vssm_tiny_v2/ --model vssm_tiny_v2 --config configs/xcit-adv-finetuning.yaml --finetune pretrained_weights/vssm_tiny_0230_ckpt_epoch_262.pth --mean 0.4914 0.4822 0.4465 --std 0.2471 0.2435 0.2616 --normalize-model

2. Adding 'reset_classifier' function to VSSM class:
   def reset_classifier(self, num_classes, global_pool=''): self.num_classes = num_classes self.classifier.head = nn.Linear(self.n_channels, num_classes) if num_classes > 0 else nn.Identity()

3. Registering the 'vssm_tiny_v2' model with timm using the following in vmamba.py:
   `
   @register_model
   def vssm_tiny_v2(pretrained=False, **kwargs):

   model = VSSM(
   patch_size=4,
   in_chans=3,
   num_classes=1000,
   depths=[2, 2, 5, 2],
   dims=96,
   # ===================
   ssm_d_state=1,
   ssm_ratio=2.0,
   ssm_rank_ratio=2.0,
   ssm_dt_rank="auto", ####
   ssm_act_layer="silu",
   ssm_conv=3,
   ssm_conv_bias=False,
   ssm_drop_rate=0.0,
   ssm_init="v0", #####
   forward_type="v3noz",
   # ===================
   mlp_ratio=4.0,
   mlp_act_layer="gelu",
   mlp_drop_rate=0.0, ####
   # ===================
   drop_path_rate=0.2,
   patch_norm=True, #####
   norm_layer="ln", #####
   downsample_version="v3",
   patchembed_version="v2", ###gmlp=false
   use_checkpoint=False,
   )
   return model`

4. Adding to the VSSM class 'default_cfg' field (as it is needed in line 63 https://github.com/dedeswim/vits-robustness-torch/blob/master/train.py.
   Here i am not sure if i should use the 'xcit-adv-training-cifar10.yaml' config as follows:
   config_path = "/vits-robustness-torch/configs/xcit-adv-training-cifar10.yaml"
   with open(config_path, "r") as file:
   yaml_cfg = yaml.safe_load(file)
   self.default_cfg = yaml_cfg

or should i define the following default_cfg :

self.default_cfg = {
'input_size': (3, 32, 32), # CIFAR-10 input size (channels x height x width)
'mean': [0.4914, 0.4822, 0.4465], # CIFAR-10 mean
'std': [0.2470, 0.2435, 0.2616], # CIFAR-10 std
'num_classes': 10, # CIFAR-10 has 10 classes
'interpolation': 'bilinear', # Interpolation method for resizing
'crop_pct': 0.875, # Crop percentage (for augmentations, if needed)
'input_range': (0.0, 1.0), # Normalized range for input
}

I am not exactly sure how my self.default_cfg should be defined.

When i choose the first option, the interpolation value is set as an empty string (''). This causes the following error:

File "/home/gamir/edocohen/talr/DeepLearningCourseProject/vits-robustness-torch/train.py", line 64, in main
data_config, loader_eval, loader_train = setup_data(args,
File "/home/gamir/edocohen/talr/DeepLearningCourseProject/vits-robustness-torch/src/setup_task.py", line 254, in setup_data
data_config, loader_eval, loader_train = setup_data(args,
File "/home/gamir/edocohen/talr/DeepLearningCourseProject/vits-robustness-torch/src/setup_task.py", line 254, in setup_data
loader_eval = create_loader_v2(
File "/home/gamir/edocohen/talr/anaconda3/envs/VMamba_env20/lib/python3.10/site-packages/timm/data/loader.py", line 71, in create_loader_v2
dataset.transform = create_transform_v2(
File "/home/gamir/edocohen/talr/anaconda3/envs/VMamba_env20/lib/python3.10/site-packages/timm/data/transforms_factory.py", line 229, in create_transform_v2
loader_eval = create_loader_v2(
File "/home/gamir/edocohen/talr/anaconda3/envs/VMamba_env20/lib/python3.10/site-packages/timm/data/loader.py", line 71, in create_loader_v2
transform = transforms_imagenet_eval(
File "/home/gamir/edocohen/talr/anaconda3/envs/VMamba_env20/lib/python3.10/site-packages/timm/data/transforms_factory.py", line 159, in transforms_imagenet_eval
dataset.transform = create_transform_v2(
File "/home/gamir/edocohen/talr/anaconda3/envs/VMamba_env20/lib/python3.10/site-packages/timm/data/transforms_factory.py", line 229, in create_transform_v2
transforms.Resize(scale_size, transforms.InterpolationMode(interpolation)),
File "/home/gamir/edocohen/talr/anaconda3/envs/VMamba_env20/lib/python3.10/enum.py", line 385, in call
transform = transforms_imagenet_eval(
File "/home/gamir/edocohen/talr/anaconda3/envs/VMamba_env20/lib/python3.10/site-packages/timm/data/transforms_factory.py", line 159, in transforms_imagenet_eval
return cls.new(cls, value)
transforms.Resize(scale_size, transforms.InterpolationMode(interpolation)), File "/home/gamir/edocohen/talr/anaconda3/envs/VMamba_env20/lib/python3.10/enum.py", line 710, in new

File "/home/gamir/edocohen/talr/anaconda3/envs/VMamba_env20/lib/python3.10/enum.py", line 385, in call
raise ve_exc
ValueError: '' is not a valid InterpolationMode
return cls.new(cls, value)
File "/home/gamir/edocohen/talr/anaconda3/envs/VMamba_env20/lib/python3.10/enum.py", line 710, in new
raise ve_exc
ValueError: '' is not a valid InterpolationMode