Running SimCLR

[josejhlee]

Firstly, thank you for your incredible input.
Im trying to reproduce SimCLR with 8xA5000(24GB) w/ ffcv. I'd like to ask:

1. If the config for simclr_ffcv sufficient for the reproduction.
2. What changes do you recommend me to implement.

Thank you!

[erow]

Hi, thank you for your attention.

You can run the code to reproduce SimCLR:

torchrun --nproc_per_node=8 main_pretrain.py  --data_path=${train_path} --data_set=ffcv     --epochs 1000 --warmup_epochs 40 --blr 1.5e-4 --weight_decay 0.05  --opt_betas 0.9 0.95 --batch_size 128    --cfgs configs/simclr_ffcv.gin

However, to reproduce the result in the paper needs careful parameter tuning. Many factors may reduce the performance: 1. ffcv will compress images. 2. the default optimizer is adam. 3. batch size is not large enough.

Good luck!