[Bug]: `.jpk-qi-data` files not loading

[bicarlsen]

Describe the bug

Using AFMReader.jpk.load_jpk on a .jpk-qi-data is throwing the error TiffFileError: not a TIFF file b'PK\x03\x04'. In issue #154 @derollins states that this should be working now.

I tried with multiple images which open fine in Gwyddion.

Copy of the output

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
c:\Users\user\Downloads\.venv-afm-analysis\Lib\site-packages\tifffile\tifffile.py in ?(self, file, mode, name, offset, size, omexml, superres, _multifile, _useframes, _parent, **is_flags)
   4359         except Exception:
   4360             fh.close()
-> 4361             raise

KeyError: b'PK'

The above exception was the direct cause of the following exception:

TiffFileError                             Traceback (most recent call last)
<ipython-input-12-f981e5507437> in ?()
      1 # %%
----> 2 jpk.load_jpk("qi_data.jpk-qi-data", "height")

c:\Users\user\Downloads\.venv-afm-analysis\Lib\site-packages\AFMReader\jpk.py in ?(file_path, channel, config_path, flip_image)
    215     filename = file_path.stem
    216     jpk_tags = _load_jpk_tags(config_path)
    217     try:
    218         tif = tifffile.TiffFile(file_path)
--> 219     except FileNotFoundError:
    220         logger.error(f"[{filename}] File not found : {file_path}")
    221         raise
    222     # Obtain channel list for all channels in file

c:\Users\user\Downloads\.venv-afm-analysis\Lib\site-packages\tifffile\tifffile.py in ?(self, file, mode, name, offset, size, omexml, superres, _multifile, _useframes, _parent, **is_flags)
   4357                 self.pages.useframes = True
   4358 
   4359         except Exception:
   4360             fh.close()
-> 4361             raise

TiffFileError: not a TIFF file b'PK\x03\x04'

To Reproduce

from AFMReader import jpk 

jpk.load_jpk("qi_data.jpk-qi-data", "height")

I also tried the height_trace and other channels as shown in #155 with the same result.

AFMReader Version

0.0.6

Python Version

3.13

Operating System

Windows

Python Packages

• AFMReader: 0.0.6
• Python: 3.13
  (These aren't listed in the available options.)

[derollins]

Hi @bicarlsen

The file discussed in #154 was a .jpk-qi file, I believe the .jpk-qi-data is a different format and from the print-out may not contain a tiff stack like the other .jpk-* tiff formats. Although that is speculation, as I don't know anything about that particular format.

I'd check in Gwyddion what the actual channel name is and try that, and then try to append _trace or _retrace to that if that doesn't work.

Alternatively, it could be a Python version or dependency issue, AFMReader is only tested to work on Python 3.10-Python 3.12 so creating a new environment with one of these versions of python my fix the issue.

If you can share the file, I could have a go trying to open in and work out what is going wrong.

UPDATE: https://sourceforge.net/p/gwyddion/discussion/fileformats/thread/f66faa4d0e/
This seems to suggest that this file format is not a tiff stack, so won't be able to be read with .load_jpk()

[bicarlsen]

The test file I have is 500 MB, so couldn't share it directly here. Here is a Google Drive link for it though.

[bicarlsen]

This seems to be the relevant Gwyddion source code for loading a .jpk-qi-data file.

[derollins]

Thanks for that. I've had a look and as suspected the .jpk-qi-data is a zipped file with thousands of .data files.

In the Gwyddion source code this is a 'jpkforce' type file, while AFMReader only currently loads what are called 'jpkscan' files within Gwyddion (TIFF with particular metadata tags).

I'd suggest maybe opening up a new feature request issue for these types of file. Alternatively, you might consider trying afmformats which may be able to open these files.

[bicarlsen]

Thanks for the info @derollins. Yeah, afmformats can read it (Although it's very slow. Two hours to open a 500 MB file. Gwyddion also took about a minute to open it, but JPK software does it almost instantly, so not sure hwo they can do it so fast.)

Shall I still open a feature request for AFMReader to be able to read these files, or do you want to keep it outsourced to afmformats?

[derollins]

I'm not sure what the development priorities for the team managing AFMReader are currently and whether expanding the file formats used is part of their active work although @ns-rse might know.

It might be worth putting up a feature request and someone might have some time to start working of a reader within AFMReader ( or potentially contribute it yourself) . However the fact that these files contain a force curve data for each pixel may mean it is slightly outside the current scope of the project.

An easier approach might be trying to improve the performance of afmformats and getting in touch with the developers.

[bicarlsen]

Also see AFM-analysis/afmformats#41