feat!: MATLAB‑aligned masks & auto routines; require Python 3.11+

## Summary

This merge brings the Python NanoLocz library into algorithmic alignment with the MATLAB reference, with a strong focus on mask semantics, numerical parity, and automated routines. It clarifies contracts across modules, reduces hidden numerical drift, and makes validation and future extension easier.

### Key changes
1) MATLAB‑aligned mask semantics (across all leveling/thresholding)

All thresholding methods now return **boolean *exclusion* masks** `(True = excluded / masked, False = valid)`.
Leveling functions convert public exclusion masks to internal validity via a shared helper and use **NaN‑outside semantics** for fitting (excluded pixels are ignored in fits but preserved in outputs).
Mask contract and behaviour are documented in `level.py`, `level_weighted.py`, `level_auto.py`, and `thresholder.py`.

2) Numerical parity in core levelling (`level.py`)

Polynomial fits use 1‑based indices and population standard deviation (ddof=0) to mirror MATLAB polyfit(..., mu).
Reworked methods: `plane`, `line`, `med_line`, `med_line_y`, `smed_line`, `mean_plane`.

Fit only over valid pixels; explicit low‑sample fallbacks; preserve excluded pixels.
Document MATLAB‑specific gates (e.g., column stage in line applies only when `polyy > 0`).


`get_background(...)` guarantees background = input − levelled under the same semantics.

3) Automated routines completed & aligned (`level_auto.py`)

`multi-plane-edges` and `multi-plane-otsu` fully implemented (replacing placeholders) and missing plane→histogram passes restored.
Adds anisotropy‑gated med_line preconditioning (post plane(1,1) triggers) to match MATLAB’s adaptive behaviour.
Implements MATLAB‑style Gaussian histogram fitting (gauss1) for adaptive threshold bounds (`gauss_fi`t, `gauss_peaks`, `gauss_holes`), with a documented frame‑wise vs stack‑wise difference.

4) API clean-up & consistency

New public entry point: `apply_thresholder(...)` replaces the ambiguous `thresholder()` within the `thresholder` module. 
Internal callers updated.
Consistent function contracts across `level`, `level_weighted`, `thresholder`, and `level_auto`.
`thresholder.py` now returns boolean masks (no NaN masks) and clarifies/expands methods:

`selection`, `histogram`, `otsu`,
`auto edges`, `hist edges`, `otsu edges`,
`hist skel`, `otsu skel`, `line_step`, `adaptive`.



5) Region‑weighted leveling improvements (`level_weighted.py`)

Regions derived from validity masks using 8‑connectivity with `min area = floor(1% of H×W)` (MATLAB rule).
Weighted aggregation uses a 2% weight threshold (weights ≤2% zeroed; not renormalized).
Evaluation uses 1‑based coordinates; detailed docstrings describe behavior and edge cases.

6) Documentation & examples

Adds `CITATION.cff` for software citation.
README updates: requirements (Python 3.11+), quickstart with apply_thresholder and apply_level_weighted, routine summaries, and citations/links.
Tutorial notebook refreshed to demonstrate the new API and pipeline narrative.

7) Build, tooling & CI

**Minimum Python is now 3.11**; CI matrix tests 3.11/3.12/3.13; classifiers and tool targets updated (Black/Ruff/Mypy → py311).
Dependency pin: scikit-image >= 0.26, < 0.27.
Pre‑commit: --show-diff-on-failure, refined excludes (docs, notebooks, tests/resources, etc.).

8) Tests & resources

Adds tests/conftest.py fixture for loading NPZ test data and AFM resource file(s) under tests/resources/.


**Breaking changes**
- **Python version**
  Minimum supported Python is 3.11 (3.10 dropped).

- Thresholding & mask polarity
  Use `apply_thresholder(...)`. It returns a boolean exclusion mask `(True = excluded)`.
  If you previously expected NaN masks or “True = valid” logic, invert with ~mask where a validity mask is needed.

- Leveling inputs
  Public leveling methods now expect exclusion masks. If you were passing validity masks, invert them first.

- Behavioral alignment tweaks
  `level_line`: column stage runs only when polyy > 0.
  `med_line`: interprets polyx as a gain on the row‑median when polyx > 0 (else 1.0).
  Region‑weighted methods implement MATLAB’s min‑area and weight thresholds.

Author: derollins

.github/workflows/pre-commit.yaml

@@ -24,4 +24,4 @@ jobs:
           pip install pre-commit
 
       - name: Run pre-commit hooks
-        run: pre-commit run --all-files
+        run: pre-commit run --all-files --show-diff-on-failure

.github/workflows/tests.yaml

@@ -14,7 +14,7 @@ jobs:
       fail-fast: false
       matrix:
         os: [ubuntu-latest, macos-latest, windows-latest]
-        python-version: ["3.10", "3.11", "3.12"]
+        python-version: ["3.11", "3.12", "3.13"]
 
     steps:
       - name: Checkout repository

.pre-commit-config.yaml

@@ -4,14 +4,14 @@ repos:
     rev: v4.5.0
     hooks:
       - id: trailing-whitespace
-        exclude: ^(docs/|notebooks/|LICENSE)$
+        exclude: (^docs/|^tests/resources/|^notebooks/|^LICENSE$)
       - id: end-of-file-fixer
-        exclude: ^(docs/|notebooks/|LICENSE)$
+        exclude: (^docs/|^tests/resources/|^notebooks/|^LICENSE$)
       - id: check-yaml
-        exclude: ^(docs/|notebooks/|LICENSE)$
+        exclude: (^docs/|^notebooks/|^LICENSE$)
       - id: check-added-large-files
         args: ["--maxkb=500"]
-        exclude: ^(docs/|notebooks/|LICENSE)$
+        exclude: (^docs/|^tests/resources/|^notebooks/|^LICENSE$)
       - id: check-merge-conflict
       - id: check-toml
 
@@ -22,23 +22,23 @@ repos:
       - id: isort
         args: ["--profile", "black"]
         language_version: python3
-        exclude: ^(docs/|notebooks/|LICENSE)$
+        exclude: (^docs/|^notebooks/|^LICENSE$)
 
   # --- Ruff Lint & Format ---
   - repo: https://github.com/astral-sh/ruff-pre-commit
     rev: v0.3.7
     hooks:
       - id: ruff
         args: ["--fix"]
-        exclude: ^(docs/|notebooks/|LICENSE)$
+        exclude: (^docs/|^notebooks/|^LICENSE$)
 
   # --- Black Code Formatter ---
   - repo: https://github.com/psf/black
     rev: 23.7.0
     hooks:
       - id: black
         language_version: python3
-        exclude: ^(docs/|notebooks/|LICENSE)$
+        exclude: (^docs/|^notebooks/|^LICENSE$)
 
   # --- Mypy (type checking) ---
   - repo: https://github.com/pre-commit/mirrors-mypy
@@ -59,7 +59,7 @@ repos:
     hooks:
       - id: markdownlint-cli2
         args: ['--fix','--config','.markdownlint.yaml']
-        exclude: ^(docs/|notebooks/|LICENSE)$
+        exclude: (^docs/|^notebooks/|^LICENSE$)
 
   # --- Remove Notebook Metadata ---
   - repo: https://github.com/kynan/nbstripout
@@ -74,4 +74,4 @@ repos:
     hooks:
       - id: codespell
         types: [text]
-        exclude: ^(docs/_build/|build/|dist/|notebooks/|LICENSE)$
+        exclude: (^docs/_build/|^tests/resources/|^build/|^dist/|^notebooks/|^LICENSE$)

CITATION.cff

@@ -0,0 +1,31 @@
+# This CITATION.cff file was generated with cffinit.
+# Visit https://bit.ly/cffinit to generate yours today!
+
+cff-version: 1.2.0
+title: Python-Nanolocz-Library
+message: >-
+  If you use this software, please cite it using the
+  metadata from this file.
+type: software
+authors:
+  - given-names: Daniel E
+    family-names: Rollins
+    email: d.e.rollins@leed.ac.uk
+    affiliation: University of Leeds
+  - given-names: George R
+    family-names: Heath
+    affiliation: University of Leeds
+    email: G.R.Heath@leeds.ac.uk
+    orcid: 'https://orcid.org/0000-0001-6431-2191'
+repository-code: 'https://github.com/derollins/Python-Nanolocz-Library'
+abstract: >-
+  A Python implementation of the NanoLocz library
+  (https://github.com/George-R-Heath/NanoLocz-Matlab-Library).
+keywords:
+  - AFM
+  - Atomic Force MIcroscopy
+  - High Speed AFM
+  - Levelling
+  - image processing
+  - image analysis
+license: GPL-3.0

README.md

@@ -39,7 +39,7 @@ pip install .
 
 ### Requirements
 
-This library requires **Python 3.10 or newer** and uses modern scientific Python packages to replace MATLAB functionality
+This library requires **Python 3.11 or newer** and uses modern scientific Python packages to replace MATLAB functionality
 from the original NanoLocz platform:
 
 - **NumPy** – Core numerical operations and array handling (replaces MATLAB’s matrix operations).
@@ -61,7 +61,7 @@ easily extensible for AFM workflows.
 import numpy as np
 from pnanolocz_lib.level import apply_level
 from pnanolocz_lib.level_auto import apply_level_auto
-from pnanolocz_lib.thresholder import thresholder
+from pnanolocz_lib.thresholder import apply_thresholder
 from pnanolocz_lib.level_weighted import apply_weighted_level
 
 # 1) Polynomial plane leveling
@@ -78,7 +78,7 @@ stack = np.load("stack.npy")      # (N,H,W)
 out = apply_level_auto(stack, routine="multi-plane-otsu")
 
 # 4) Otsu mask
-otsu_mask = thresholder(img, method="otsu", limits=None)
+mask = apply_thresholder(img, method="otsu", limits=None)
 ```
 
 ---
@@ -124,7 +124,11 @@ otsu_mask = thresholder(img, method="otsu", limits=None)
 - **`pnanolocz_lib.thresholder`**
   Intensity / edge detection: histogram, Otsu, auto edges, skeleton, step detection.
 
-    Available thresholds:
+  Typical usage involves calling the `apply_thresholder()` function with an image (2D)
+  or image stack (3D) and specifying the desired method and polynomial orders.
+  (see Quickstart above for an example)
+
+    Available thresholder functions:
 
 | Method       | Description |
 |--------------|-------------|
@@ -168,8 +172,13 @@ otsu_mask = thresholder(img, method="otsu", limits=None)
 ## 📝 Citation
 
 If you use this library, please cite:
-Heath, G.R. et al. *NanoLocz: Image analysis platform for AFM, high‑speed AFM and localization AFM.*
-Small Methods 2024, 2301766. <https://doi.org/10.1002/smtd.202301766>
+> Heath, G.R. et al. *NanoLocz: Image analysis platform for AFM, high‑speed AFM and localization AFM.*
+> Small Methods 2024, 2301766. <https://doi.org/10.1002/smtd.202301766>
+
+and
+
+> Rollins, D. E., & Heath, G. R. (2025). *Python-NanoLocz-Library: A Python implementation of the NanoLocz AFM leveling and
+> analysis tools*. University of Leeds. <https://github.com/derollins/Python-Nanolocz-Library>
 
 ---
 

notebooks/Test pnanolocz-lib.ipynb

@@ -16,6 +16,8 @@
    "outputs": [],
    "source": [
     "data_path = r\"C:/Users/ggjh246/OneDrive - University of Leeds/Code/playNano_testdata/save-2025.05.20-12.57.06.187.h5-jpk\"\n",
+    "# Un- comment the following line if you do not have playnano installed in your enviroment. \n",
+    "#%pip install numpy pandas matplotlib playnano\n",
     "from playnano.io.loader import load_afm_stack\n",
     "afm_stack = load_afm_stack(data_path, channel = \"height_trace\")"
    ]
@@ -40,24 +42,18 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from pnanolocz_lib import level_auto, level, level_weighted\n",
+    "from pnanolocz_lib import level_auto, level, level_weighted, thresholder\n",
     "\n",
     "frames = afm_stack.n_frames\n",
-    "frame_ind = range(0, frames)\n",
-    "\n",
-    "plane_levelled = level.apply_level(afm_stack.data, 2, 2, \"plane\")\n",
-    "plt.imshow(plane_levelled[5], cmap=\"afmhot\")\n"
+    "frame_ind = range(0, frames)"
    ]
   },
   {
-   "cell_type": "code",
-   "execution_count": null,
+   "cell_type": "markdown",
    "id": "4",
    "metadata": {},
-   "outputs": [],
    "source": [
-    "levelled = level.apply_level(plane_levelled, 1, 0, \"med_line\",)\n",
-    "plt.imshow(levelled[5], cmap=\"afmhot\")\n"
+    "### Apply a quadratic plane fit"
    ]
   },
   {
@@ -67,17 +63,16 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "auto_leveled = level_auto.apply_level_auto(afm_stack.data, \"multi-plane-otsu\")"
+    "plane_levelled = level.apply_level(afm_stack.data, 2, 2, \"plane\")\n",
+    "plt.imshow(plane_levelled[5], cmap=\"afmhot\")"
    ]
   },
   {
-   "cell_type": "code",
-   "execution_count": null,
+   "cell_type": "markdown",
    "id": "6",
    "metadata": {},
-   "outputs": [],
    "source": [
-    "plt.imshow(auto_leveled[5], cmap=\"afmhot\")"
+    "### Mask all the pixels above 0"
    ]
   },
   {
@@ -87,18 +82,16 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from pnanolocz_lib.thresholder import thresholder\n",
-    "mask_hist = thresholder(plane_levelled, 'histogram', limits = (0.2, 100),invert = False)"
+    "mask_hist = thresholder.apply_thresholder(plane_levelled, 'histogram', limits = (float('-inf'), 0.2),invert = False)\n",
+    "plt.imshow(mask_hist[5], interpolation= 'none')"
    ]
   },
   {
-   "cell_type": "code",
-   "execution_count": null,
+   "cell_type": "markdown",
    "id": "8",
    "metadata": {},
-   "outputs": [],
    "source": [
-    "plt.imshow(mask_hist[5], interpolation= 'none')"
+    "### Apply a line fit from the background (not masked) to the image"
    ]
   },
   {
@@ -108,20 +101,16 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from pnanolocz_lib.level_weighted import apply_level_weighted\n",
-    "lev_weight = apply_level_weighted(plane_levelled, 1, 1, \"smed_line\", mask=mask_hist)\n",
-    "plt.imshow(lev_weight[5], cmap=\"afmhot\")"
+    "masked_line_levelled = level.apply_level(plane_levelled, 1, 0, \"line\", mask_hist)\n",
+    "plt.imshow(masked_line_levelled[5], cmap=\"afmhot\")"
    ]
   },
   {
-   "cell_type": "code",
-   "execution_count": null,
+   "cell_type": "markdown",
    "id": "10",
    "metadata": {},
-   "outputs": [],
    "source": [
-    "masked_line_levelled = level.apply_level(plane_levelled, 1, 0, \"line\", mask_hist)\n",
-    "plt.imshow(masked_line_levelled[5], cmap=\"afmhot\")"
+    "### Rather than manually applying steps, use the \"multi-plane-otsu\" auto level routine"
    ]
   },
   {
@@ -130,17 +119,17 @@
    "id": "11",
    "metadata": {},
    "outputs": [],
-   "source": []
+   "source": [
+    "auto_leveled = level_auto.apply_level_auto(afm_stack.data, \"multi-plane-otsu\")\n",
+    "plt.imshow(auto_leveled[5], cmap=\"afmhot\")"
+   ]
   },
   {
-   "cell_type": "code",
-   "execution_count": null,
+   "cell_type": "markdown",
    "id": "12",
    "metadata": {},
-   "outputs": [],
    "source": [
-    "hist2 = thresholder(masked_line_levelled, \"histogram\", limits=(0.4,100), invert=False)\n",
-    "plt.imshow(hist2[5],  interpolation= 'none')"
+    "### Use `get_background` to see the values subtracted from a leveled image."
    ]
   },
   {
@@ -150,8 +139,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "maksed_linemed_levelled = level.apply_level(masked_line_levelled, 1, 0, \"med_line\", mask_hist)\n",
-    "plt.imshow(maksed_linemed_levelled[1], cmap=\"afmhot\")"
+    "from pnanolocz_lib.level import get_background"
    ]
   },
   {
@@ -161,7 +149,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "mask_hist2 =thresholder(maksed_linemed_levelled, 'histogram', limits= (0.5,100), invert=False)"
+    "bg = get_background(afm_stack.data, 1, 1, 'plane')"
    ]
   },
   {
@@ -171,8 +159,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "mask_hist2 = thresholder(maksed_linemed_levelled, 'histogram', limits = (0.5,100),invert = False)\n",
-    "plt.imshow(mask_hist2[0])"
+    "plt.imshow(bg[5])"
    ]
   },
   {
@@ -181,55 +168,6 @@
    "id": "16",
    "metadata": {},
    "outputs": [],
-   "source": [
-    "manual_levelled = level.apply_level(masked_plane_levelled, 1, 0, \"line\", mask_hist2)\n",
-    "plt.imshow(manual_levelled[0], cmap=\"afmhot\")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "17",
-   "metadata": {},
-   "outputs": [],
-   "source": []
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "18",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from pnanolocz_lib.level import get_background"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "19",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "bg = get_background(afm_stack.data, 1, 0, 'med_line')"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "20",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "plt.imshow(bg[0])"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "21",
-   "metadata": {},
-   "outputs": [],
    "source": []
   }
  ],
@@ -249,7 +187,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.11.13"
+   "version": "3.12.12"
   }
  },
  "nbformat": 4,