Feature visualiser

[alismil]

Code for the feature visualisations detailed in this document.

Copied the following scripts from this github corresponding to this paper:

• topomoe/inspection/feature_visualiser.py
• topomoe/inspection/image_augmentation.py
• topomoe/visualise_features.py

visualise_features.py is the main script used to train feature visualisations. Given you have a model checkpoint and you have extracted features from it using topomoe/extract_features.py you can run it for example as:

python3 topomoe/visualise_features.py --layer 4 --feature 93 --lr 1 --steps 500 --model "topomoe_tiny_3s_patch16_128" --cuda False --model_args_path "topomoe/topomoe_01/10_topomoe_3s_wl-0.1/args.yaml" --checkpoint_path "topomoe/topomoe_01/10_topomoe_3s_wl-0.1/checkpoints/ckpt-best.pt"

Also added some functions in topomoe/inspection/most_activating_images.py which are detailed in the 'Visualizations for topomoe_tiny_3s_patch16_128 document' and usage examples given in notebooks/feature_analysis.ipynb.

TODO

• Add comments to all functions in most_activating_images.py
• Run similar analysis on a ViT of the same size
• Visualise most activating features per-patch
• Run 3D visualisations using per-patch PCA