Integrate SARATR-X

[lucianchauvin]

Paper Data/Code

[lucianchauvin]

• torch_model can be thought of basically as a feature extractor
• lightning model facilitates the training and stores embeddings/weights

currently trying to implement their from .SARATRX.classification.model.hivit import HiViT torch.nn model. Since they have the torch.nn model has already been done theoretically; we only need to make the lightning model

[lucianchauvin]

1. Try running some of the SARATR-X examples, and run them on MSTAR
2. Look into how to create a PyTorch lightning config (see what type of config file Anomalib is looking for)
3. Simply just integrate the nn.module within the AnomalibModule class
4. Look into how were training just the head.

SARATR-X/pre_train/main_finetune.py

Zero-shot
Finetuning either just the head, or the whole network

for just the head reference classification/trainers/MIM_linear.py

[lucianchauvin]

[lucianchauvin]

[lucianchauvin]

[lucianchauvin]

324 keys, missing about 20

[lucianchauvin]

Seen here

Removing validation gives:

and then OOM

HiViTMAE has a in_chans but hard codes this:

I fixed this in their code just to test it out, but then I ran into some other issues. It looks like the image must be greyscale before it hits anything to do with the model.