Skip to content

Add DataTable source Transform and lazy coordinate conversion#199

Draft
slimbuck wants to merge 9 commits intoplaycanvas:mainfrom
slimbuck:trans-dev
Draft

Add DataTable source Transform and lazy coordinate conversion#199
slimbuck wants to merge 9 commits intoplaycanvas:mainfrom
slimbuck:trans-dev

Conversation

@slimbuck
Copy link
Copy Markdown
Member

@slimbuck slimbuck commented Apr 7, 2026

Summary

Introduces a source coordinate transform on DataTable (translation, rotation, uniform scale) so splat data can stay in raw column form while the library knows how that data maps into engine space. Translate / rotate / scale processing steps now compose into DataTable.transform instead of rewriting columns immediately. Writers convert to each output format’s expected space (e.g. PLY) via convertToSpace, using on-demand column transforms (positions, quaternions, log-scales when scaled, SH when rotated) where possible.

Why

  • Avoids repeated full-table rewrites when chaining geometric operations.
  • Makes format conventions explicit (Transform.PLY, etc.) on read and reconciles them on write.
  • Keeps filters semantically correct: filter-by-value can bake the pending transform for geometry/SH columns before comparison; filter-box / filter-sphere map the query region into raw space when a non-identity transform is pending.

Key changes

  • Transform in src/lib/utils/math.ts: TRS helpers (fromEulers, mul, invert, transformPoint, …), Transform.IDENTITY, Transform.PLY (Z-by-180° convention for PLY-class formats).
  • DataTable: required transform; constructor optional second argument; clone preserves it.
  • convertToSpace, transformColumns, computeWriteTransform, transformAABB: internal/lazy conversion pipeline; convertToSpace is exported from the package (replaces the old transform export).
  • processDataTable: geometric actions update result.transform; spatial filters account for pending transform as above.
  • combine: if inputs disagree on transform, logs a warning and converts all tables to engine space (Transform.IDENTITY) before merging.
  • Readers attach the appropriate initial transform (e.g. Transform.PLY for splat-class inputs; LCC env table uses its existing euler convention).
  • Writers call convertToSpace(..., <format transform>) so emitted files stay in the correct convention.

Tests

  • Expanded transforms.test.mjs and new source-transform.test.mjs covering Transform, column transformation, combine with mixed transforms, processDataTable, and related behavior.

Breaking change

  • Removed the transform export from the public API.
  • Added Transform and convertToSpace as the supported surface for coordinate-space work. Callers that used transform directly should migrate to composing DataTable.transform and/or convertToSpace as appropriate.

@slimbuck slimbuck requested a review from Copilot April 7, 2026 22:33
@slimbuck slimbuck self-assigned this Apr 7, 2026
@slimbuck slimbuck added the enhancement New feature or request label Apr 7, 2026
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a DataTable.transform (translation/rotation/uniform scale) and a lazy coordinate-space conversion pipeline so geometric operations compose into metadata instead of eagerly rewriting column data, with writers converting into each format’s expected convention on demand.

Changes:

  • Added Transform utilities (incl. Transform.IDENTITY / Transform.PLY) and exported Transform + convertToSpace as the new public coordinate-space API.
  • Reworked transform handling: processDataTable composes TRS into DataTable.transform, spatial filters account for pending transforms, and combine normalizes mixed-transform inputs.
  • Updated readers/writers to set initial transforms on read and convert to required output space on write; expanded/added tests for transform behavior and round-trips.

Reviewed changes

Copilot reviewed 24 out of 24 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
test/transforms.test.mjs Updates transform tests to validate transform composition + on-demand column conversion.
test/source-transform.test.mjs New comprehensive tests for Transform, transformColumns, convertToSpace, combine behavior, and spatial filters.
test/formats.test.mjs Adjusts format tests to tag PLY fixtures with Transform.PLY.
test/decimate.test.mjs Updates decimate integration expectations for composed transforms.
src/lib/writers/write-voxel.ts Writes voxel data by converting required columns to engine space via transformColumns.
src/lib/writers/write-sog.ts Converts inputs to engine space for SOG output; bumps meta version.
src/lib/writers/write-ply.ts Converts element DataTables to PLY convention on write.
src/lib/writers/write-lod.ts Converts main/env tables to engine space before LOD output.
src/lib/writers/write-glb.ts Converts to engine space before GLB packing.
src/lib/writers/write-compressed-ply.ts Converts to PLY convention before compressed PLY write.
src/lib/write.ts Minor object shorthand tweak when passing dataTable.
src/lib/utils/math.ts Adds the Transform class (TRS + composition/inversion helpers).
src/lib/readers/read-spz.ts Tags SPZ reads with Transform.PLY.
src/lib/readers/read-splat.ts Tags SPLAT reads with Transform.PLY.
src/lib/readers/read-sog.ts Uses meta version to decide whether to apply legacy Transform.PLY.
src/lib/readers/read-ply.ts Tags PLY reads with Transform.PLY (incl. compressed PLY path).
src/lib/readers/read-lcc.ts Attaches LCC’s euler-based transform to main/env tables.
src/lib/readers/read-ksplat.ts Tags KSPLAT reads with Transform.PLY.
src/lib/process.ts Composes TRS into DataTable.transform; updates filtering to respect/bake transforms where needed.
src/lib/index.ts Public API: removes transform export; exports Transform + convertToSpace.
src/lib/data-table/transform.ts Replaces in-place transform with transformColumns, computeWriteTransform, convertToSpace, and transformAABB.
src/lib/data-table/decimate.ts Preserves DataTable.transform through decimation output.
src/lib/data-table/data-table.ts Adds transform to DataTable, constructor arg, and clone preservation.
src/lib/data-table/combine.ts Converts to engine space when combining tables with differing transforms.
Comments suppressed due to low confidence (1)

src/lib/process.ts:338

  • In filterNaN, columnNames is taken from the original dataTable rather than the current result. If a previous action removed columns (e.g. filterBands) or otherwise changed the table shape, this can cause the predicate to read missing keys (often undefined/stale values) and incorrectly drop rows. Use result.columnNames (or recompute per-iteration from result) so the predicate only checks columns that actually exist on the table being filtered.
            case 'filterNaN': {
                const infOk = new Set(['opacity']);
                const negInfOk = new Set(['scale_0', 'scale_1', 'scale_2']);
                const columnNames = dataTable.columnNames;


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 24 out of 24 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 24 out of 24 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

src/lib/process.ts:340

  • filterNaN iterates dataTable.columnNames (the original input) rather than the current result.columnNames. If earlier actions changed the column set (e.g. filterBands), the predicate will see row[key] === undefined for removed columns, isFinite(undefined) will be false, and all rows will be dropped. Use result.columnNames (or iterate result.columns) when building the key list.
            case 'filterNaN': {
                const infOk = new Set(['opacity']);
                const negInfOk = new Set(['scale_0', 'scale_1', 'scale_2']);
                const columnNames = dataTable.columnNames;

                const predicate = (row: any, rowIndex: number) => {
                    for (const key of columnNames) {

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants