v2.4.0

What's Changed

• Introduces Pydantic Dataclasses for the different levels of data analysed and passed around between processing, TopoStats, GrainCrops, DisorderedTrace, Node, OrderedTrace, MatchedBranch, UnmatchedBranch, giving strong typing of attributes to facilitate cleaner development going forward.
• Revises and updates the .topostats HDF5 format to reflect the internal structure introduced by the above dataclasses. These are compressed format HDF5 and can easily be read by AFMReader which returns a dictionary representation. TopoStats contains functions to take these dictionaries and recreate the TopoStats objects with the same nested structure of attributes they were created from.
• All tests have been updated to reflect the new Dataclasses
• Switches to using Syrupy for regression tests rather than pytest-regtest
• Adds advanced documentation on the classes structure and notes on extending TopoStats.
• Revised some of the internal nomenclature to make it clearer what the required parameters are at different stages (mainly distinguishing between the different forms of skeletons required at different stages).
• Removes ImageGrainCrop and GrainCropsDirection classes. Instead the threshold and "direction" of a grain is stored as an attribute of the GrainCrop class.
• Filenames used in logging and for the creation of per-image output now include the file extension. This is so that older Bruker files which have the same stem (the part before the file extension) do not have data over-written. As a consequence the .topostats files that are written at the end of processing now reside under <output_dir>/processed/topostats/*.topostats as having a file and directory of the same name in the same location isn't possible.

New Contributions

This release see's new contributions from...

• @tobyallwood
• @ahobbs7

Thank you 🙏

Full Changelog: v2.3.2...v2.4.0

v2.3.2

What's Changed

• The topostats create-config options have changed. To get a simplified configuration rather than specifying the flag --simple which has been removed users should provide an argument to the --config flag of either default (used if not specified) simple, mplstyle or var_to_label. If the optional flag -f/--filename is omitted then the output filename is determined by the --config option.
• CSV output has changed, files have new names.
• An entry point for grains processing is now available topostats grains.
• How to add functionality has been added.
• Support for .stp and .top files and old style Bruker extensions (i.e. .001, .002 etc.).
• Improves and expands documentation.
• Migrates tutorial notebooks from Jupyter to Marimo.
• Output now includes TopoStat version with detailed information on commits if running from Git installed versions.
• In whole image plots grains can now be optionally numbered. Plotting is now more selective but configurable too.
• There are now entry points to run filtering (topostats filter), grain finding (topostats grain) and grainstats (topostats grainstats), but preceding stages are required for these to run (i.e. you can not run topostats grains without having first run topostats filters, nor can you run topostats grainstats without first having run topostats grainstats). Each entry point has its own help, see topostats --help and topostats <subcommand> --help

Full Commit list

• Tidy up package and build with only necessary files by @ns-rse in #1065
• feature: expand objects returned by LoadScan.load_topostats() by @ns-rse in #1068
• Combine grain crops rather than overwriting them by @llwiggins in #1070
• feature: Adds filter sub-command by @ns-rse in #1071
• fix: Updates io to work with AFMReader only returning data by @ns-rse in #1091
• Added rough grainstats summary documentation and metric images by @Max-Gamill in #1087
• fix(processing): create grains/dnatracing directories if needed by @ns-rse in #1095
• feature(grainstats): Adds entry point for grainstats by @ns-rse in #1094
• Refactor pipeline to use grain crop classes by @SylviaWhittle in #1022
• feature(grains|io): Method to convert GrainCrop to dictionary by @ns-rse in #1104
• Add unet option: remove_disconnected_grains by @SylviaWhittle in #1108
• feature(grains|io): GrainCropsDirection to dictionary by @ns-rse in #1105
• feature(grains|io): ImageGrainCrop to dictionary by @ns-rse in #1106
• feature(grains): Add stats and height_profiles to GrainCrop by @ns-rse in #1118
• Adds a class name option to the config file and grainstats by @Max-Gamill in #1122
• style(test_processing): Adds pytest.param( id="") by @ns-rse in #1131
• Selective plotting of TopoStats images by @Max-Gamill in #1127
• Extends height-based pruning beyond endpoints by @Max-Gamill in #1132
• Add code for loading .stp and .top files by @EvaMartinC in #1136
• Add multi class thresholding (& refactor early find_grains()) by @SylviaWhittle in #1123
• Improve curvature reliability by @SylviaWhittle in #1141
• feature: Introduce TopoStats class and saving to HDF5 .topostats by @ns-rse in #1145
• Add unet confidence as a configurable parameter by @SylviaWhittle in #1156
• Minimum Python >= 3.10 by @ns-rse in #1157
• feat: Support loading of .jpk-qi-image files by @ns-rse in #1178
• Support old Bruker file extensions by @ns-rse in #1182
• include topostats version in output config file by @tobyallwood in #1197
• Llwiggins/new notebooks by @llwiggins in #1208
• Changed output CSV names and added basic / full option in config by @tobyallwood in #1218
• feature: write YAML configuration files from other packages by @ns-rse in #1223
• Add option to number grain masks in plots by @tobyallwood in #1230
• fix: no commit in released versions to report by @ns-rse in #1247

New Contributors

Thanks to our new contributors 👏

• @EvaMartinC made their first contribution in #1136
• @tobyallwood made their first contribution in #1197

Full Changelog: v2.3.0...v2.3.2

v2.3.2rc2

What's Changed

• fix: no commit in released versions to report by @ns-rse in #1247

Full Changelog: v2.3.2rc1...v2.3.2rc2

v2.3.2rc1

What's Changed

• The topostats create-config options have changed. To get a simplified configuration rather than specifying the flag --simple which has been removed users should provide an argument to the --config flag of either default (used if not specified) simple, mplstyle or var_to_label. If the optional flag -f/--filename is omitted then the output filename is determined by the --config option.
• CSV output has changed, files have new names.
• An entry point for grains processing is now available topostats grains.
• How to add functionality has been added.
• Support for .stp and .top files and old style Bruker extensions (i.e. .001, .002 etc.).
• Improves and expands documentation.
• Migrates tutorial notebooks from Jupyter to Marimo.
• Output now includes TopoStat version with detailed information on commits if running from Git installed versions.
• In whole image plots grains can now be optionally numbered. Plotting is now more selective but configurable too.
• There are now entry points to run filtering (topostats filter), grain finding (topostats grain) and grainstats (topostats grainstats), but preceding stages are required for these to run (i.e. you can not run topostats grains without having first run topostats filters, nor can you run topostats grainstats without first having run topostats grainstats).

Full Commit list

• Tidy up package and build with only necessary files by @ns-rse in #1065
• feature: expand objects returned by LoadScan.load_topostats() by @ns-rse in #1068
• Combine grain crops rather than overwriting them by @llwiggins in #1070
• feature: Adds filter sub-command by @ns-rse in #1071
• fix: Updates io to work with AFMReader only returning data by @ns-rse in #1091
• Added rough grainstats summary documentation and metric images by @Max-Gamill in #1087
• fix(processing): create grains/dnatracing directories if needed by @ns-rse in #1095
• feature(grainstats): Adds entry point for grainstats by @ns-rse in #1094
• Refactor pipeline to use grain crop classes by @SylviaWhittle in #1022
• feature(grains|io): Method to convert GrainCrop to dictionary by @ns-rse in #1104
• Add unet option: remove_disconnected_grains by @SylviaWhittle in #1108
• feature(grains|io): GrainCropsDirection to dictionary by @ns-rse in #1105
• feature(grains|io): ImageGrainCrop to dictionary by @ns-rse in #1106
• feature(grains): Add stats and height_profiles to GrainCrop by @ns-rse in #1118
• Adds a class name option to the config file and grainstats by @Max-Gamill in #1122
• style(test_processing): Adds pytest.param( id="") by @ns-rse in #1131
• Selective plotting of TopoStats images by @Max-Gamill in #1127
• Extends height-based pruning beyond endpoints by @Max-Gamill in #1132
• Add code for loading .stp and .top files by @EvaMartinC in #1136
• Add multi class thresholding (& refactor early find_grains()) by @SylviaWhittle in #1123
• Improve curvature reliability by @SylviaWhittle in #1141
• feature: Introduce TopoStats class and saving to HDF5 .topostats by @ns-rse in #1145
• Add unet confidence as a configurable parameter by @SylviaWhittle in #1156
• Minimum Python >= 3.10 by @ns-rse in #1157
• feat: Support loading of .jpk-qi-image files by @ns-rse in #1178
• Support old Bruker file extensions by @ns-rse in #1182
• include topostats version in output config file by @tobyallwood in #1197
• Llwiggins/new notebooks by @llwiggins in #1208
• Changed output CSV names and added basic / full option in config by @tobyallwood in #1218
• feature: write YAML configuration files from other packages by @ns-rse in #1223
• Add option to number grain masks in plots by @tobyallwood in #1230

New Contributors

Thanks to our new contributors 👏

• @EvaMartinC made their first contribution in #1136
• @tobyallwood made their first contribution in #1197

Full Changelog: v2.3.0...v2.3.2rc1

v2.3.0

What's Changed

• Fixes for height-biasing of skeletons by @llwiggins in #815
• tests: Adapts get_skeleton() tests to the getSkeleton() class by @ns-rse in #822
• chore: Linting dnatracing; setup skeletons for remaining tests; split pruning to submodule by @ns-rse in #825
• Update dnatracing docstrings by @MaxGamill-Sheffield in #826
• chore(tracing,skeleton): Line length linting; correcting tpyos by @ns-rse in #827
• tests(utils/pruning): Tests for utils and some of the pruning classes by @ns-rse in #835
• tests: Moves dummy skeleton & heights fixtures to tests/conftest.py by @ns-rse in #844
• chore(workflows): updating to setup-python@v5 by @ns-rse in #857
• feature(measure): Adds the height_profiles sub-module by @ns-rse in #859
• [pre-commit.ci] pre-commit-autoupdate by @pre-commit-ci in #867
• tests(pruning): Tests TopoStatsPrune and convPrune by @ns-rse in #848
• Bound aspect ratio between 0 and 1 by @llwiggins in #864
• fix: Removes saving of plots to pickles by @ns-rse in #872
• Incorporating height profile into GrainStats by @ns-rse in #871
• Re-add changes for connect_nodes_nearest by @SylviaWhittle in #879
• Fixing broken tests (Part I) by @ns-rse in #874
• [pre-commit.ci] pre-commit-autoupdate by @pre-commit-ci in #883
• Sylvia whittle/800 analyse nodes by @SylviaWhittle in #888
• Maxgamill sheffield/800 btr splining by @MaxGamill-Sheffield in #889
• Add U-Net grain segmentation option by @SylviaWhittle in #890
• Seperate out Splining from dnaTracing by @MaxGamill-Sheffield in #886
• Fix height profile tests for macOS by @SylviaWhittle in #894
• [pre-commit.ci] pre-commit-autoupdate by @pre-commit-ci in #895
• Add multi-class deep learning segmentation support by @SylviaWhittle in #893
• Adding RI code to handle and break odd crossings by @MaxGamill-Sheffield in #891
• tests(plotting): revert molecule_number > grain_number in tests by @ns-rse in #882
• test: Adds pytest-icdiff to dev dependencies by @ns-rse in #905
• chore: fix linting errors by @ns-rse in #900
• tests(processing): Improves logic and tests for check_run_steps() by @ns-rse in #903
• fix: reinstate grains try except by @ns-rse in #906
• test(utils): Fixing broken tests by @ns-rse in #907
• tests(plottingfuncs): fixes failing tests by @ns-rse in #909
• Add trace smoothing tests by @SylviaWhittle in #892
• tests: Remove restriction on pytest<8.0.0 by @ns-rse in #917
• tests: Remove restriction on pytest<8.0.0 (cherry pick) by @ns-rse in #918
• fix(plotting): remove dilation if images are large by @ns-rse in #914
• Add basic smoke tests for topology branch by @SylviaWhittle in #897
• ci: Run tests workflow on all pull_request by @ns-rse in #925
• ci: Add local pytest --testmon to pre-commit by @ns-rse in #926
• ci: restrict supported python version to <3.12 by @ns-rse in #931
• Adds topological features into better tracing by @MaxGamill-Sheffield in #898
• ci: Explicit types of pull requests and paths to run CI tests by @ns-rse in #933
• style(tracing/nodestats): Fix all numpydoc-validation errors by @ns-rse in #937
• Fix processing tests by @SylviaWhittle in #939
• 800 : Migrate / delete old dnatracing tests by @SylviaWhittle in #941
• Fix Grainstats failing test on 800-better-tracing by @MaxGamill-Sheffield in #940
• Fixes failing pruning tests on 800-better-tracing by @MaxGamill-Sheffield in #938
• ci: Adds tmate to tests.yaml workflow by @ns-rse in #946
• fix(disordered_tracing): force branch_type to int64 by @ns-rse in #949
• style(theme): RT01 Numpydoc Validation by @ns-rse in #952
• style(tracingfuncs): Numpydoc validation, PEP8 & tidying by @ns-rse in #948
• docs: Document refactored tracing by @ns-rse in #936
• 950 better tracing angle out of range by @MaxGamill-Sheffield in #956
• ci: gh-pages building correctly by @ns-rse in #961
• ci: gh-pages building correctly by @ns-rse in #963
• docs: lower-case link by @ns-rse in #964
• ci(sphinx): Python 3.12 > 3.11 by @ns-rse in #965
• Fixes the dictionary keys docs formation by @MaxGamill-Sheffield in #968
• Adding the Better Tracing Suite into Main by @MaxGamill-Sheffield in #932
• Remove Python 3.8 from pyproject.toml classifiers by @ns-rse in #971
• Removing errant notebook by @ns-rse in #970
• Fixes pd.concat error halting processing by @MaxGamill-Sheffield in #972
• tests(utils): parameterise test_create_empty_dataframe() by @ns-rse in #977
• chore: Adds fair-software.eu badge to README.md by @ns-rse in #979
• Fixes None tracing outputs attempted to be concatenated by @MaxGamill-Sheffield in #973
• tests(disordered_tracing): Bulks out missing unittests by @ns-rse in #978
• feature: Print 'art' above completion message by @ns-rse in #980
• General Cleanup and Following the Better Tracing Merger by @MaxGamill-Sheffield in #976
• fix: Remove duplicate modification of TensorFlow log level by @ns-rse in #982
• refactor(entry_points): Removing legacy entry points by @ns-rse in #985
• Add support for partial config files by @SylviaWhittle in #981
• [pre-commit.ci] pre-commit-autoupdate by @pre-commit-ci in #991
• wip: Adding all configuration options as arguments by @ns-rse in #988
• Add documentation for flattening and grains by @SylviaWhittle in #993
• update readme.md with new swiss army knife processing order by @MaxGamill-Sheffield in #997
• refactor(run_modules): Move common setup to individual functions by @ns-rse in #996
• ci: Adds a Pull Request template by @ns-rse in #1000
• Hariborings vetting to main by @SylviaWhittle in #992
• tests(processing): Update regression test targets to include dna width by @ns-rse in #1002
• package: Unpin topoly version by @ns-rse in #995
• doc: Reduce Python version to 3.11 in .readthedocs.yaml by @ns-rse in #1004
• style: Changes prettier mirror used by pre-commit by @ns-rse in #1011
• dependabot for GitHub Actions by @ns-rse in #1012
• config: Removes target-version from ruff configuration by @ns-rse in #1013
• ci: Adds pygrep-hooks and fixes errors by @ns-rse in #1014
• Build(deps): Bump actions/add-to-project from 0.5.0 to 1.0.2 by @dependabot in #1015
• Build(deps): Bump codecov/codecov-action from 4 to 5 by @dependabot in #1016
• ci: group dependabot github-actions updates by @ns-rse in #1017
• Add basic curvature by @SylviaWhittle in #974
• chore: Updates publicat...

v2.3.1

What's Changed

• Added average grain width measurement to disordered tracing by @tcatley in #999
• Combine grain crops rather than overwriting them by @llwiggins in #1070

New Contributors

• @dependabot made their first contribution in #1015
• @tcatley made their first contribution in #999

Full Changelog: v2.2.1...v2.3.1

v2.2.1

Highlights

• Basic .tif file support (currently no metadata is included).
• Saves results in HDF5 file.
• Supports Python 3.12
• Improved .asd file format via AFMReader (nee topofileformats)
• New submodule for feret calculations.
• Support for very small (pm) and large (mm) scans.

What's Changed

• pytest<8.0.0 by @ns-rse in #788
• Add the option to save as a tif file by @derollins in #778
• Save grain trace data to HDF5 format by @SylviaWhittle in #790
• Added 'pm' and 'mm' for very small and large scans by @ns-rse in #801
• Added 'pm' and 'mm' for very small and large scans by @MaxGamill-Sheffield in #796
• Remove saving of gaussian filtered arrays to .npy files by @ns-rse in #804
• Adding 3.12 to workflow test matrix by @ns-rse in #691
• Sets the type of all arguments that are files/paths to Path by @ns-rse in #809
• Fixing DPI settings by @ns-rse in #808
• Tidies up Images.plot_and_save() by @ns-rse in #810
• Adds numpydoc-validation hook to pre-commit by @ns-rse in #816
• New submodule for feret calculations by @ns-rse in #755
• refactor(grainstats): GrainStats.get_max_min_feret() > measure.feret by @ns-rse in #823
• chore: Update codecov version used in test workflow by @ns-rse in #831
• bug: Addresses importlib.resources.open_text() deprecation by @ns-rse in #830
• Resolve .gwy file not loading and add channel support for .gwy files by @SylviaWhittle in #837
• chore: Update dependency to AFMReader by @ns-rse in #852

Full Changelog: v2.2.0...v2.2.1

TopoStats v2.2.post0

The tag v2.2.0 appears to have already been used on PyPI (although no trace of it remains) and its impossible to make releases with versions that have already been used. This is therefore post0 as the release is identical and complies with such minor changes as described in the Python packaging post-releases page.

Please refer to v2.2.0 release notes for details of changes.

v2.2.0

User Interface

• Switched to topostats <action> and removed original run_topostats, for more details please refer to the usage documentation.
• Introduces a complete matplotlibrc params file for customising plots en-masse
  • Added topostats create-matplibrc command (with options) for generating user customisable figures.
• .asd file support @SylviaWhittle

Bugs

• Plotting of images that touch the border are now fixed @MaxGamill-Sheffield
• Improved linear splinning @SylviaWhittle
• Flattening runs when scar removal is disabled (which is now the default).

Miscellaneous

• Various minor updates to make the package compliant with Scientific Python Repository Review (still a few more steps to go though).

Development

• Adds snoop to aid with debugging.
• Additional pre-commit hooks are now included that prevent accidental commits to main.
• Issue templates have been improved to use Issue Forms.

Full Changelog: v2.1.2...v2.2.0

v2.1.2

Various small changes fixing aspects of plotting with regards to DPI and axis scales. Dependencies are updated and improved and igor2 instead of the unmaintained igor is now used for loading .ibw. Linting is improved and considerably faster in the pre-commit pipeline courtesy with ruff rather than flake8 and introducing pydocstyle linting.

Importantly scar removal is now disabled by default, its a time consuming step and should only be enable if you know you have scars in your scans that you want removing.

What's Changed

• Switched to igor2 instead of igor for loading .ibw files which allows unpinning of numpy dependency #685
• Added ruff and pydocstyle to the pre-commit configuration and removed flake8 (ruff covers the same issues) by @ns-rse in #670
• Moves DPI configuration from default_config to plotting_dictionary
• Fix broken reference to 'image', now 'image_original' by @SylviaWhittle in #654
• Updates violin plots to work with Seaborn-0.13 and adds legend only to plots with multiple images by @ns-rse in #680
• Disable scars by default and fix KDE plotting bug by @SylviaWhittle in #678
• Add a zeroing step to filters by @SylviaWhittle in #688
• Fix plotting in pixels for topostats files by @SylviaWhittle in #658
• Adds details for image_stats.csv by @ns-rse in #664
• Changed dict crop_size -1 names to match processing by @MaxGamill-Sheffield in #665
• Lower default DPI of individual molecule plots by @SylviaWhittle in #667
• Remove redundant test by @ns-rse in #689

Full Changelog: v2.1.1...v2.1.2