Add DataTable source Transform and lazy coordinate conversion#199
Add DataTable source Transform and lazy coordinate conversion#199slimbuck wants to merge 9 commits intoplaycanvas:mainfrom
Transform and lazy coordinate conversion#199Conversation
There was a problem hiding this comment.
Pull request overview
This PR introduces a DataTable.transform (translation/rotation/uniform scale) and a lazy coordinate-space conversion pipeline so geometric operations compose into metadata instead of eagerly rewriting column data, with writers converting into each format’s expected convention on demand.
Changes:
- Added
Transformutilities (incl.Transform.IDENTITY/Transform.PLY) and exportedTransform+convertToSpaceas the new public coordinate-space API. - Reworked transform handling:
processDataTablecomposes TRS intoDataTable.transform, spatial filters account for pending transforms, andcombinenormalizes mixed-transform inputs. - Updated readers/writers to set initial transforms on read and convert to required output space on write; expanded/added tests for transform behavior and round-trips.
Reviewed changes
Copilot reviewed 24 out of 24 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| test/transforms.test.mjs | Updates transform tests to validate transform composition + on-demand column conversion. |
| test/source-transform.test.mjs | New comprehensive tests for Transform, transformColumns, convertToSpace, combine behavior, and spatial filters. |
| test/formats.test.mjs | Adjusts format tests to tag PLY fixtures with Transform.PLY. |
| test/decimate.test.mjs | Updates decimate integration expectations for composed transforms. |
| src/lib/writers/write-voxel.ts | Writes voxel data by converting required columns to engine space via transformColumns. |
| src/lib/writers/write-sog.ts | Converts inputs to engine space for SOG output; bumps meta version. |
| src/lib/writers/write-ply.ts | Converts element DataTables to PLY convention on write. |
| src/lib/writers/write-lod.ts | Converts main/env tables to engine space before LOD output. |
| src/lib/writers/write-glb.ts | Converts to engine space before GLB packing. |
| src/lib/writers/write-compressed-ply.ts | Converts to PLY convention before compressed PLY write. |
| src/lib/write.ts | Minor object shorthand tweak when passing dataTable. |
| src/lib/utils/math.ts | Adds the Transform class (TRS + composition/inversion helpers). |
| src/lib/readers/read-spz.ts | Tags SPZ reads with Transform.PLY. |
| src/lib/readers/read-splat.ts | Tags SPLAT reads with Transform.PLY. |
| src/lib/readers/read-sog.ts | Uses meta version to decide whether to apply legacy Transform.PLY. |
| src/lib/readers/read-ply.ts | Tags PLY reads with Transform.PLY (incl. compressed PLY path). |
| src/lib/readers/read-lcc.ts | Attaches LCC’s euler-based transform to main/env tables. |
| src/lib/readers/read-ksplat.ts | Tags KSPLAT reads with Transform.PLY. |
| src/lib/process.ts | Composes TRS into DataTable.transform; updates filtering to respect/bake transforms where needed. |
| src/lib/index.ts | Public API: removes transform export; exports Transform + convertToSpace. |
| src/lib/data-table/transform.ts | Replaces in-place transform with transformColumns, computeWriteTransform, convertToSpace, and transformAABB. |
| src/lib/data-table/decimate.ts | Preserves DataTable.transform through decimation output. |
| src/lib/data-table/data-table.ts | Adds transform to DataTable, constructor arg, and clone preservation. |
| src/lib/data-table/combine.ts | Converts to engine space when combining tables with differing transforms. |
Comments suppressed due to low confidence (1)
src/lib/process.ts:338
- In
filterNaN,columnNamesis taken from the originaldataTablerather than the currentresult. If a previous action removed columns (e.g.filterBands) or otherwise changed the table shape, this can cause the predicate to read missing keys (oftenundefined/stale values) and incorrectly drop rows. Useresult.columnNames(or recompute per-iteration fromresult) so the predicate only checks columns that actually exist on the table being filtered.
case 'filterNaN': {
const infOk = new Set(['opacity']);
const negInfOk = new Set(['scale_0', 'scale_1', 'scale_2']);
const columnNames = dataTable.columnNames;
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 24 out of 24 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 24 out of 24 changed files in this pull request and generated 1 comment.
Comments suppressed due to low confidence (1)
src/lib/process.ts:340
filterNaNiteratesdataTable.columnNames(the original input) rather than the currentresult.columnNames. If earlier actions changed the column set (e.g.filterBands), the predicate will seerow[key] === undefinedfor removed columns,isFinite(undefined)will be false, and all rows will be dropped. Useresult.columnNames(or iterateresult.columns) when building the key list.
case 'filterNaN': {
const infOk = new Set(['opacity']);
const negInfOk = new Set(['scale_0', 'scale_1', 'scale_2']);
const columnNames = dataTable.columnNames;
const predicate = (row: any, rowIndex: number) => {
for (const key of columnNames) {
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Summary
Introduces a source coordinate transform on
DataTable(translation, rotation, uniform scale) so splat data can stay in raw column form while the library knows how that data maps into engine space. Translate / rotate / scale processing steps now compose intoDataTable.transforminstead of rewriting columns immediately. Writers convert to each output format’s expected space (e.g. PLY) viaconvertToSpace, using on-demand column transforms (positions, quaternions, log-scales when scaled, SH when rotated) where possible.Why
Transform.PLY, etc.) on read and reconciles them on write.filter-by-valuecan bake the pending transform for geometry/SH columns before comparison; filter-box / filter-sphere map the query region into raw space when a non-identity transform is pending.Key changes
Transforminsrc/lib/utils/math.ts: TRS helpers (fromEulers,mul,invert,transformPoint, …),Transform.IDENTITY,Transform.PLY(Z-by-180° convention for PLY-class formats).DataTable: requiredtransform; constructor optional second argument; clone preserves it.convertToSpace,transformColumns,computeWriteTransform,transformAABB: internal/lazy conversion pipeline;convertToSpaceis exported from the package (replaces the oldtransformexport).processDataTable: geometric actions updateresult.transform; spatial filters account for pending transform as above.combine: if inputs disagree on transform, logs a warning and converts all tables to engine space (Transform.IDENTITY) before merging.Transform.PLYfor splat-class inputs; LCC env table uses its existing euler convention).convertToSpace(..., <format transform>)so emitted files stay in the correct convention.Tests
transforms.test.mjsand newsource-transform.test.mjscoveringTransform, column transformation, combine with mixed transforms,processDataTable, and related behavior.Breaking change
transformexport from the public API.TransformandconvertToSpaceas the supported surface for coordinate-space work. Callers that usedtransformdirectly should migrate to composingDataTable.transformand/orconvertToSpaceas appropriate.