Merge pull request #9 from azazellochg/devel

3.0b3

Author: azazellochg

.github/workflows/publish_and_tag.yml

@@ -12,6 +12,9 @@ env:
 jobs:
   deploy:
     runs-on: ubuntu-latest
+    permissions:
+    # IMPORTANT: this permission is mandatory for Trusted Publishing
+      id-token: write
     steps:
     - uses: actions/checkout@v4
     - uses: actions/setup-python@v5
@@ -24,13 +27,10 @@ jobs:
     - name: Check extra requirements
       if: ${{ hashFiles('requirements.txt') != '' }}
       run: pip install -r requirements.txt
-    - name: publish
-      env:
-        TWINE_USERNAME: __token__
-        TWINE_PASSWORD: ${{ secrets.PYPI_API_TOKEN }}
-      run: |
-        python -m build --no-isolation
-        twine upload dist/*
+    - name: Build dist
+      run: python -m build --no-isolation
+    - name: Publish package distributions to PyPI
+      uses: pypa/gh-action-pypi-publish@release/v1
     - name: Get version and tag
       run: |
         export PACKAGE_VERSION=$(python -c "import $FOLDER_WITH_VERSION; print('VERSION', 'v'+$FOLDER_WITH_VERSION.__version__)" | grep VERSION | sed "s/VERSION //g")

README.rst

@@ -34,6 +34,8 @@ New changes and this whole product is distributed under either version 3 of the
 Documentation
 -------------
 
+The source code is available at https://github.com/azazellochg/pytemscript
+
 The documentation can be found at https://pytemscript.readthedocs.io
 
 Quick example

docs/acquisition.rst

@@ -0,0 +1,107 @@
+Image acquisition
+=================
+
+The acquisition can become quite cumbersome due to many different cameras from various manufacturers installed on a microscope.
+Here we describe supported / tested cameras and explain how they can be controlled by ``pytemscript``
+
+List of tested cameras:
+
+ * Orius CCD (SC200W (830), SC200B (830))
+ * Ceta 16M
+ * Ceta D
+ * Falcon 3EC
+ * Falcon 4(i)
+ * K2
+ * K3
+
+All methods described below return a 16-bit unsigned integer (equivalent to MRC mode 6) :meth:`~pytemscript.modules.Image` object.
+If movies are being acquired asynchronously, their format can be different.
+
+Standard scripting
+------------------
+
+Gatan CCD cameras are usually embedded by TFS and can be controlled via standard scripting. This requires both Digital Micrograph
+and TIA to be opened as well as the current camera selected in the Microscope User Interface (CCD/TV camera panel).
+
+.. code-block:: python
+
+    microscope = Microscope()
+    acq = microscope.acquisition
+    img = acq.acquire_tem_image("BM-Orius", AcqImageSize.FULL, exp_time=1.0, binning=2)
+
+.. warning:: If you need to change the camera, after doing so in the Microscope interface, you have to reconnect the microscope client since the COM interface needs to be reinitialised.
+
+For Gatan K2/K3 cameras (if they are embedded by TFS), standard scripting can only return unaligned average image,
+there are no options to acquire movies or change the mode (linear/counting).
+You can only modify binning or exposure time.
+
+TecnaiCCD plugin
+----------------
+
+FEI has created their own plugin for Gatan CCD cameras. The plugin needs to be installed on Gatan PC inside Digital Micrograph.
+Digital Micrograph and TIA need to be opened as well as the current camera selected in the Microscope User Interface (CCD/TV camera panel).
+The advantage of this method over standard scripting is ~20 % speed improvement for both acquisition and image return, because the plugin
+interacts directly with Digital Micrograph and does not return the image to TIA.
+
+.. code-block:: python
+
+    microscope = Microscope(useTecnaiCCD=True)
+    acq = microscope.acquisition
+    img = acq.acquire_tem_image("BM-Orius", AcqImageSize.FULL, exp_time=1.0, binning=2, use_tecnaiccd=True)
+
+SerialEMCCD plugin
+------------------
+
+David Mastronarde has created a SerialEM `plugin <https://github.com/mastcu/SerialEMCCD>`_ to control both Gatan CCDs and advanced cameras like K2 or K3.
+The plugin has to be installed on Gatan PC inside Digital Micrograph, which is normally done during SerialEM installation.
+The connection to the plugin is established via a socket interface created by ``pytemscript`` (same way as Leginon does it).
+Digital Micrograph needs to be opened. SerialEM does not have to be running.
+
+The plugin provides multiple options for movie acquisition, frame alignment etc.
+
+.. warning:: In development, not available yet
+
+Advanced scripting
+------------------
+
+This scripting interface was developed by TFS for their newer cameras like Ceta and Falcon.
+The currently supported cameras are Ceta 1, Ceta 2, Falcon 3 and Falcon 4(i).
+The interface includes new features like movie acquisition, counting mode, EER format etc.
+Movies are offloaded asynchronously to the storage server, while the returned image is an average (aligned or not).
+
+There's no need to open TIA or select the camera in the microscope interface.
+
+See details for :meth:`~pytemscript.modules.Acquisition.acquire_tem_image`
+
+.. code-block:: python
+
+    microscope = Microscope()
+    acq = microscope.acquisition
+    img = acq.acquire_tem_image("BM-Falcon", AcqImageSize.FULL, exp_time=5.0, binning=1, electron_counting=True, align_image=True, group_frames=2)
+
+.. note:: Advanced scripting features like "Camera Electron Counting" and "Camera Dose Fractions" require separate licenses from TFS.
+
+Speed up the acquisition
+------------------------
+
+By default, ``pytemscript`` will use `AsSafeArray` method to convert the COM image object to a numpy array via standard or advanced scripting.
+Depending on the image size this method can be very slow (several seconds). There's a trick to save the image object to a temporary file
+(`AsFile` COM method) and then read it, which seems to work much faster (up to 3x). However, this requires an extra `imageio` dependency for reading the temporary file.
+
+.. warning:: On some systems, saving to a file can fail with a COM error due to incomplete implementation, so you will have to stick to the default `AsSafeArray` method.
+
+If you want to try this method, add a couple of kwargs to your acquisition command:
+
+.. code-block:: python
+
+    microscope = Microscope()
+    acq = microscope.acquisition
+    img = acq.acquire_tem_image("BM-Falcon", AcqImageSize.FULL, exp_time=5.0, use_safearray=False, use_asfile=True)
+
+
+STEM acquisition
+----------------
+
+STEM detectors have to be embedded by FEI and selected in the Microscope User Interface (STEM user panel). They are controlled by standard scripting.
+
+.. note:: Be aware that the acquisition starts immediately without waiting for STEM detectors insertion to finish. It's probably better to manually insert them first in the microscope interface.

docs/changelog.rst

@@ -23,11 +23,6 @@ Version 3.0
     - Tundra (Win10, Python 3.11)
     - Titan Krios G1 (Win7, Python 3.6), G2, G3i (Win10, Python 3.8), G4 (Win10, Python 3.8)
 
-* Future plans:
-
-    - UTAPI client
-    - Acquisition series
-
 Version 2.0.0
 ^^^^^^^^^^^^^
 

docs/components/events.rst

@@ -5,7 +5,7 @@ You can receive events from hand panel buttons when using the local client on th
 can be assigned with a custom Python function that will be executed upon pressing. We provide the :meth:`~pytemscript.modules.ButtonHandler` class
 that takes care of assigning events.
 
-.. warning:: Don't forget to clear the custom button assignment at the end using `clear()` method. This will restore the previous assignment.
+.. note:: Don't forget to clear the custom button assignment at the end using `clear()` method. This will restore the previous assignment.
 
 See example below:
 

docs/components/index.rst

@@ -71,7 +71,8 @@ Vectors
 -------
 
 Some attributes handle two dimensional vectors that have X and Y values (e.g. image shift or gun tilt). These
-attributes accept and return a :meth:`~pytemscript.modules.Vector` of two floats. Vectors can be multiplied, subtracted etc.:
+attributes accept and return a :meth:`~pytemscript.modules.Vector` of two floats. Vectors can be multiplied, subtracted etc. as shown below.
+You can also use a list or a tuple to set vector attributes.
 
 .. code-block:: python
 
@@ -80,6 +81,8 @@ attributes accept and return a :meth:`~pytemscript.modules.Vector` of two floats
     shift += (0.4, 0.2)
     shift *= 2
     microscope.optics.illumination.beam_shift = shift
+    projection.image_shift = (0.05, 0.1)
+    projection.image_shift = [0.05, 0.1]
 
 .. autoclass:: pytemscript.modules.Vector
     :members: set_limits, check_limits, get, set

docs/conf.py

@@ -21,7 +21,7 @@
 author = 'Tore Niermann, Grigory Sharov'
 
 # The full version, including alpha/beta/rc tags
-release = '3.0b1'
+release = '3.0b3'
 
 # -- General configuration ---------------------------------------------------
 # If your documentation needs a minimal Sphinx version, state it here.

docs/index.rst

@@ -18,6 +18,8 @@ New changes and this whole product is distributed under either version 3 of the
 Documentation
 -------------
 
+The source code is available at https://github.com/azazellochg/pytemscript
+
 The documentation can be found at https://pytemscript.readthedocs.io
 
 .. toctree::
@@ -27,6 +29,7 @@ The documentation can be found at https://pytemscript.readthedocs.io
    installation
    components/index
    getting_started
+   acquisition
    remote
    changelog
 

docs/installation.rst

@@ -1,13 +1,22 @@
 Installation
 ============
 
-Requirements:
+Prerequisites for the FEI or Thermo Fisher Scientific microscope:
+
+    * TEM Scripting
+    * TEM Advanced scripting (optional)
+    * LowDose (optional)
+    * TecnaiCCD plugin for Digital Micrograph (optional)
+    * SerialEMCCD plugin for Digital Micrograph (optional)
+
+Requirements for this package:
 
     * python 3.4 or newer
     * comtypes
     * mrcfile (to save MRC files)
     * numpy
     * pillow (to save non-MRC files)
+    * imageio (optional, to speed up image acquisition)
 
 Online installation on Windows
 ##############################

docs/remote.rst

@@ -12,7 +12,7 @@ Socket-based client
 In this mode the pytemscript socket server must run on the microscope PC (Windows).
 By default, it will listen for clients on port 39000.
 
-.. warning::
+.. danger::
 
     The server provides no means of security or authorization control itself.
     Thus it is highly recommended to let the server only listen to internal networks or at least route it through a reverse proxy, which implements sufficient security.
@@ -42,7 +42,7 @@ Then you can connect to the server as shown below:
 
 Diagnostic messages are saved to ``socket_client.log`` and ``socket_server.log`` as well as printed to the console. Log files are rotated weekly at midnight.
 
-To shutdown pytemscript-server, press Ctrl+C in the console.
+To shutdown pytemscript-server, press Ctrl+C in the server console.
 
 UTAPI client
 ------------
@@ -55,7 +55,8 @@ you can search for ``utapi_server.exe`` in the Task Manager. The server is liste
 **46699**. Under the hood UTAPI utilizes gRPC (Google Remote Procedure Calls) framework that uses protocol
 buffers for communication.
 
-Pytemscript converts its API commands to UTAPI calls. The client requires extra dependencies to be installed:
+Pytemscript converts its API commands to UTAPI calls. The client only supports Python 3.8+ and requires
+a few extra dependencies to be installed:
 
 .. code-block:: python