Suboptimal Reproduced Results on iNat2018

Hello! I finally collected the necessary data files for the iNat2018 task and got the whole thing running. Unfortunately the result for SH-SIREN was 0.5688 for Top 1 and 0.6887 for Top 3, which is even lower than the image-only baseline. Is it necessary to change the code to reproduce the results in the paper, or am I missing something again? Thank you!