Paper: The Fossil Frontier: An answer to the 3-billion fossil question (Link)
As a joint project between UiT and Equinor as part of the SFI Visual Intelligence consortium, our paper "The 3-billion fossil question: How to automate classification of microfossils" introduced the use of self-supervised learning for creating a DNN which excels at image feature extraction for images of microfossils. Further work in our upcoming publication "The Fossil Frontier: Answering the 3-billion fossil question" builds on this, using a data-curation step which increases the performance on the downstream tasks of CBIR and image classification.
Fig. 1 Images in benchmark dataset plotted by t-SNE embedding of their embeddings generated by our feature extractor.
-
Benchmarking data used in our publications:
- 697 labelled images from 20 genera of dinoflagellate cycts and other palynomorphs from the well NO 6407/6-5
- These images are distributed under the Norwegian Licence for Open Government Data (NLOD) 2.0, as adaptations of palynology slide scans created by the Norwegian Offshore Directorate
-
Code for benchmarking the performance of our networks against dino-vits8 and dino-vits16 from Meta is also provided. You can choose to download only the weights of the pretrained backbone used for downstream tasks, or the full checkpoint which contains backbone and projection head weights for both student and teacher networks from the DINO training. We also provide the detailed arguments and training logs, in addition to a comparison to pretrained ViTs.
| arch | params | nn f1 | p@n_k | download | |||
|---|---|---|---|---|---|---|---|
| SCAMPI ViT-S/16 | 21M | 0.91 | 0.60 | backbone only | full ckpt | args | logs |
| SCAMPI ViT-B/16 | 85M | 0.90 | 0.60 | backbone only | full ckpt | args | logs |
| DINO ViT-S/16 | 21M | 0.78 | 0.44 | backbone only | full ckpt | args | logs |
| DINO ViT-S/8 | 21M | 0.84 | 0.46 | backbone only | full ckpt | args | logs |
| DINO ViT-B/16 | 85M | 0.81 | 0.47 | backbone only | full ckpt | args | logs |
| DINO ViT-B/8 | 85M | 0.82 | 0.45 | backbone only | full ckpt | args | logs |
The ViTs presented in this work are backbone models for various downstream tasks. See example_usage.py for a minimum working example of how to create image embeddings using the ViT-S model.
To install necessary dependencies using conda:
conda env create --name scampi-benchmark --file requirements.txt
To reproduce the evaluation from the paper:
python run_evaluation.py
We are excited to see future development in this field. If you have an public model with and open license on HuggingFace which could challenge our metrics on this benchmark we would welcome a PR to include it in the evaluation.
We are grateful to Equinor for permission to release the labelled data, and to Martin Pearce for performing the labelling.