Tilt ordering for exposure in chlamydataset2relion5.py

[maurerv]

Hi,

Thanks a lot for sharing this dataset.

I ran chlamydataset2relion5.py on the example test case in the repo (01082023_BrnoKrios_Arctis_WebUI_Position_8) and noticed unexpected values in _rlnMicrographPreExposure. From 01082023_BrnoKrios_Arctis_WebUI_Position_8.star.txt (adding .txt for GitHub file upload):

_rlnTomoNominalStageTiltAngle _rlnMicrographPreExposure
-55.02 3.500401
-52.01 7.000802

This does not match the mapping acquired from combining (column bind) collected_tilts.star and dose.star from metadata/tomolist, as well as the corresponding mdoc file, which all contain an exposure of 3.500401 for tilt -15.03. Does the output above correspond to the intended behavior?

Digging into chlamy2relion5.py, the mismatch described above seems to arise from read_acquisition_order_dose_star:122, which removes excluded tilts but does not sort the remainder to match the order imposed by AreTomo/*dose-filt.tlt, which is later assumed in collect_tomogram_data:268.

Happy to provide more details if needed :)

[rdrighetto]

Hi @maurerv thanks for reporting this! I just submitted PR #22 that should fix this.
If you wanna test the new code right away please go to my fork:
https://github.com/rdrighetto/ChlamyAnnotations/tree/dosefix

If you confirm the issue is fixed then I guess @paloha can merge 😉
Please let us know if you need further help.

[maurerv]

Hi @rdrighetto,

Thank you for the fix! I will also take you up on your offer as I was indeed wondering about two more things in the script :)

• The produced star files will always use a _rlnMicrographOriginalPixelSize of 1.96 and _rlnTomoTomogramBinning of 4.0 (i.e., a pixel size of 7.84 Apx), although the header of some reconstructions contains 7.28 Apx (for instance 08042022_BrnoKrios_Arctis_grid5_gistest_Position_4 with a PixelSpacing of 1.82 in the mdoc). Are these reconstructions actually at 7.28 Apx or is the header faulty?

• On EMPIAR I found CTF specification in ctffind4/ctfphaseflip_ctffind4.txt, tiltctf/ctfphaseflip_ctffind4.txt, metadata/tomolist/determined_defocii.star, which can be slightly different. Which of these are relevant when working on the ctf3d_bin4 reconstructions?

[rdrighetto]

Hi @maurerv,

• The produced star files will always use a _rlnMicrographOriginalPixelSize of 1.96 and _rlnTomoTomogramBinning of 4.0 (i.e., a pixel size of 7.84 Apx), although the header of some reconstructions contains 7.28 Apx (for instance 08042022_BrnoKrios_Arctis_grid5_gistest_Position_4 with a PixelSpacing of 1.82 in the mdoc). Are these reconstructions actually at 7.28 Apx or is the header faulty?

For simplicity, we processed the entire dataset assuming 1.96 A/px, which was the pixel size of the initial sessions -- so the chlamydataset2relion5.py script assumes this value to be consistent with what we used for defocus estimations, as it's intended for those using the preprocessed data. However, since sessions spanned a long time period and were done on at least 3 different microscopes, there might be variations in the true pixel size for each session. We now know that a pixel size of 1.91 A/px (calibrated against high-resolution maps generated from the data) is probably more accurate on average (as indicated in the TomoGuide) -- still, the true value for each session may be different. I don't know if the mdocs are reliable for that.

• On EMPIAR I found CTF specification in ctffind4/ctfphaseflip_ctffind4.txt, tiltctf/ctfphaseflip_ctffind4.txt, metadata/tomolist/determined_defocii.star, which can be slightly different. Which of these are relevant when working on the ctf3d_bin4 reconstructions?

Thanks for pointing this out. It is a bit confusing, I know. Initially we used CTFFIND4 for defocus estimations, but later we redid it using TILTCTF which is much more accurate for tomography data. This is what was used to generate the ctf3d_bin4 volumes, so you should take defocus from tiltctf/ctfphaseflip_tiltctf.txt. The file metadata/tomolist/determined_defocii.star can also be used as it contains the same values, only with more decimals.

Hope this clarifies it! Thanks for using the Chlamy dataset!

[maurerv]

Perfect, thanks a lot!