Introduce CellTable to store cell coords #32
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR introduces CellTable and CellTable3D, more compact representations of the long list of tiny cell-footprint arrays SPRM accesses via the ROI_coords list-of-lists. Iterators are provided to walk efficiently through the new tables.
Images also get a new "cache" functionality whereby a value can be associated with a name and then retrieved later via
cache_set()andcache_get()methods. This is used because the existing code keeps a large data structure around so that a small summary can be extracted later. Using these methods, the summaries are extracted early and the larger dataset is freed.The gist of the representation is that the input masks are sorted by cell ID, leading to a representation in which all coordinates associated with a given cell are grouped together. These are also sorted by Z, Y, and X coordinates so that the coordinates for a given cell appear in the expected order. This sorting is done in-place. The end result is a faster, much more compact representation. A great deal of memory is saved by eliminating the array description information associated with each small cell array. A second array is allocated listing the number of consecutive entries in each cell to facilitate traversal.
I have some worries about the 3D case, because the cell iterators return voxels in ZYX order as per "normal" arrays while some existing code reformats things into YXZ order to simplify the general case logic. Since I don't have a good 3D dataset for testing I have been unable to validate this case.
This version includes yet more diagnostic output, which will have to be cleaned up in a later PR.
The PR would benefit from a minor rebase, I think.