Skip to content

Commit 175074b

Browse files
author
DavidQ
committed
BUILD_PR_LEVEL_10_6_SAMPLE_SCHEMA_AND_STANDALONE_TOOL_DATA_FLOW_VALIDATION
- Added standalone sample schema and tool data-flow validation - Fixed targeted Asset Browser, Asset Pipeline, and Palette sample loading/display failures - Prevented fallback/demo data from masking invalid payloads PR Details: - Validates standalone tool mode receives/displays sample data - No hardcoded asset paths - No runtime validators - No start_of_day changes
1 parent d82c46b commit 175074b

16 files changed

Lines changed: 1452 additions & 108 deletions

‎docs/dev/reports/expected_delta_template.md‎

Lines changed: 24 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,23 +1,35 @@
11
# Expected Codex Return / Delta Template
22

33
## Expected Changed Files
4-
- tool files only where hidden fallback/hardcoded paths are removed
5-
- tests for no hidden coupling
6-
- `docs/dev/reports/level_10_5_no_hidden_tool_coupling_report.md`
7-
- `docs/dev/reports/level_10_5_hardcoded_asset_path_audit.md`
4+
- affected `samples/phase-*/####/**`
5+
- affected standalone tool data-loading/binding files
6+
- `tests/runtime/SampleStandaloneToolDataFlow.test.mjs` or equivalent
7+
- `docs/dev/reports/level_10_6_sample_schema_validation_report.md`
8+
- `docs/dev/reports/level_10_6_standalone_tool_data_flow_report.md`
89
- `docs/dev/roadmaps/MASTER_ROADMAP_ENGINE.md` if status update needed
910

11+
## Required Targeted Regression Results
12+
- `sample_0204_asset_browser_pipeline_input_loaded=true`
13+
- `sample_1413_asset_browser_or_pipeline_input_loaded=true`
14+
- `sample_1505_asset_browser_pipeline_input_loaded=true`
15+
- `sample_0510_asset_pipeline_input_loaded=true`
16+
- `sample_1417_asset_pipeline_input_loaded=true`
17+
- `sample_0213_palette_displayed=true`
18+
- `sample_0308_palette_displayed=true`
19+
- `sample_0313_palette_displayed=true`
20+
1021
## Expected Validation Summary
11-
- `tools_scanned=<count>`
12-
- `silent_fallbacks_removed=<count>`
13-
- `hardcoded_asset_paths_removed=<count>`
14-
- `remaining_silent_fallbacks=0`
15-
- `remaining_hardcoded_json_asset_paths=0`
16-
- `tools_empty_state_without_input=true`
17-
- `tools_render_manifest_input=true`
22+
- `sample_payload_files_scanned=<count>`
23+
- `schema_resolution_failures=0`
24+
- `tool_payload_contract_failures=0`
25+
- `standalone_data_flow_failures=0`
26+
- `palette_display_failures=0`
27+
- `pipeline_input_failures=0`
28+
- `silent_fallbacks_used=0`
29+
- `hardcoded_asset_paths_added=0`
1830
- `start_of_day_changes=0`
1931

2032
## Expected Delta ZIP
2133
Codex must create:
2234

23-
`tmp/BUILD_PR_LEVEL_10_5_NO_HIDDEN_TOOL_COUPLING_VALIDATION_delta.zip`
35+
`tmp/BUILD_PR_LEVEL_10_6_SAMPLE_SCHEMA_AND_STANDALONE_TOOL_DATA_FLOW_VALIDATION_delta.zip`

‎docs/dev/reports/level_10_6_sample_schema_validation_report.md‎

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
# Level 10.6 Sample Schema Validation Report
2+
3+
- Generated: 2026-04-26
4+
- BUILD: `BUILD_PR_LEVEL_10_6_SAMPLE_SCHEMA_AND_STANDALONE_TOOL_DATA_FLOW_VALIDATION`
5+
6+
## Scope
7+
8+
- Audited sample tool payloads from `samples/metadata/samples.index.metadata.json` `roundtripToolPresets` universe.
9+
- Audited all sample palette files matching `samples/**/sample.*.palette.json`.
10+
11+
## Files Scanned
12+
13+
- `samples/**/sample.*.*.json` (roundtrip-mapped payloads): `62`
14+
- `samples/**/sample.*.palette.json`: `20`
15+
16+
## Schema Reference Validation
17+
18+
- `$schema` missing failures: `0`
19+
- `$schema` unresolved failures: `0`
20+
21+
## Payload Contract Validation
22+
23+
- Tool payload contract failures: `0`
24+
- Palette contract failures: `0`
25+
- Roundtrip preset-path failures: `0`
26+
27+
## Targeted Fixes Applied
28+
29+
- Updated metadata preset routes from legacy dashed naming to existing dotted payload files where present.
30+
- File: `samples/metadata/samples.index.metadata.json`
31+
- Rewrites applied: `53`
32+
- Fixed standalone sample preset extraction for config-wrapped payload contracts:
33+
- `tools/Asset Browser/main.js`
34+
- `tools/Asset Pipeline Tool/main.js`
35+
- `tools/Palette Browser/main.js`
36+
- Added explicit palette payloads for targeted palette-browser samples:
37+
- `samples/phase-02/0213/sample.0213.palette-browser.json`
38+
- `samples/phase-03/0308/sample.0308.palette-browser.json`
39+
- `samples/phase-03/0313/sample.0313.palette-browser.json`
40+
41+
## Fixed Samples (Targeted)
42+
43+
- Asset Browser / Import Hub: `0204`, `1413`, `1505`
44+
- Asset Pipeline Tool: `0510`, `1413`, `1417`
45+
- Palette Browser: `0213`, `0308`, `0313`
46+
47+
## Deferred / Blockers
48+
49+
- None in schema/reference contract validation scope.
50+
- Standalone non-target data-flow failures are documented in:
51+
- `docs/dev/reports/level_10_6_standalone_tool_data_flow_report.md`

‎docs/dev/reports/level_10_6_standalone_tool_data_flow_report.md‎

Lines changed: 91 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,91 @@
1+
# Level 10.6 Standalone Tool Data Flow Report
2+
3+
- Generated: 2026-04-26
4+
- BUILD: `BUILD_PR_LEVEL_10_6_SAMPLE_SCHEMA_AND_STANDALONE_TOOL_DATA_FLOW_VALIDATION`
5+
- Test command: `npm run test:sample-standalone:data-flow`
6+
- Raw runtime summary: `tmp/sample-standalone-tool-data-flow-results.json`
7+
8+
## Automated Coverage Added
9+
10+
- Added runtime test:
11+
- `tests/runtime/SampleStandaloneToolDataFlow.test.mjs`
12+
- Added script:
13+
- `package.json` -> `test:sample-standalone:data-flow`
14+
15+
Validation performed by test:
16+
17+
1. Discover sample roundtrip preset rows from metadata.
18+
2. Validate each mapped preset path exists.
19+
3. Validate schema/contract coverage for roundtrip tool payloads and all sample palette files.
20+
4. Run generic standalone open-route checks across all roundtrip rows.
21+
5. Run strict UI/state assertions for targeted failures.
22+
23+
## Targeted Regression Results
24+
25+
All targeted flows now pass with explicit payload-to-UI/state evidence.
26+
27+
### Asset Browser / Import Hub
28+
29+
- `0204` -> status: `Loaded preset from sample 0204.`
30+
- `1413` -> status: `Loaded preset from sample 1413.`
31+
- `1505` -> status: `Loaded preset from sample 1505.`
32+
- Pipeline/import input bind is verified via `importNameInput` and category/destination state.
33+
34+
### Asset Pipeline Tool
35+
36+
- `0510` -> status: `Loaded preset from sample 0510.`
37+
- `1413` -> status: `Loaded preset from sample 1413.`
38+
- `1417` -> status: `Loaded preset from sample 1417.`
39+
- Pipeline input bind is verified via parsed `assetPipelineInput` payload and non-empty domain records.
40+
41+
### Palette Browser
42+
43+
- `0213` -> palette title: `Sample 0213 Palette`, swatches: `13`
44+
- `0308` -> palette title: `Sample 0308 Palette`, swatches: `5`
45+
- `0313` -> palette title: `Sample 0313 Palette`, swatches: `5`
46+
- Palette state is verified as loaded (not empty/none state).
47+
48+
## Generic Roundtrip Data-Flow Scan
49+
50+
- Total roundtrip rows checked: `62`
51+
- Generic failure signals detected: `25`
52+
- These are non-targeted tools/slices and were not expanded in this PR to preserve one-PR purpose and minimal scope.
53+
54+
Generic failures detected:
55+
56+
- `0201:3d-camera-path-editor:preset load failure signal`
57+
- `0202:3d-camera-path-editor:preset load failure signal`
58+
- `0204:3d-asset-viewer:preset load failure signal`
59+
- `0210:physics-sandbox:preset load failure signal`
60+
- `0220:3d-camera-path-editor:preset load failure signal`
61+
- `0221:tile-model-converter:preset load failure signal`
62+
- `0221:3d-json-payload-normalizer:preset load failure signal`
63+
- `0303:physics-sandbox:preset load failure signal`
64+
- `0305:3d-json-payload-normalizer:preset load failure signal`
65+
- `0305:tile-model-converter:preset load failure signal`
66+
- `0306:parallax-editor:preset load failure signal`
67+
- `0512:performance-profiler:preset load failure signal`
68+
- `0708:replay-visualizer:preset load failure signal`
69+
- `1204:parallax-editor:preset load failure signal`
70+
- `1205:parallax-editor:preset load failure signal`
71+
- `1208:3d-asset-viewer:preset load failure signal`
72+
- `1208:3d-json-payload-normalizer:preset load failure signal`
73+
- `1208:parallax-editor:preset load failure signal`
74+
- `1209:tile-model-converter:preset load failure signal`
75+
- `1315:replay-visualizer:preset load failure signal`
76+
- `1319:performance-profiler:preset load failure signal`
77+
- `1406:replay-visualizer:preset load failure signal`
78+
- `1407:performance-profiler:preset load failure signal`
79+
- `1413:3d-asset-viewer:preset load failure signal`
80+
- `1606:physics-sandbox:preset load failure signal`
81+
82+
## No Hidden Coupling / No Fallback Notes
83+
84+
- No default/demo data was added to mask invalid preset payloads.
85+
- Targeted tools now explicitly accept sample payload contract shape where sample payload is wrapped in `config`.
86+
- Invalid preset parsing still surfaces visible failure status (`Preset load failed: ...`) rather than silent fallback.
87+
88+
## Roadmap Status Update
89+
90+
- No Level 10.6 status marker exists in `docs/dev/roadmaps/MASTER_ROADMAP_ENGINE.md`.
91+
- No roadmap prose/status changes were applied in this PR.

‎docs/operations/dev/codex_commands.md‎

Lines changed: 43 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -2,42 +2,53 @@ MODEL: GPT-5.3-codex
22
REASONING: high
33

44
TASK:
5-
Apply BUILD_PR_LEVEL_10_5_NO_HIDDEN_TOOL_COUPLING_VALIDATION.
5+
Apply BUILD_PR_LEVEL_10_6_SAMPLE_SCHEMA_AND_STANDALONE_TOOL_DATA_FLOW_VALIDATION.
66

77
NON-NEGOTIABLE RULES:
8-
- Do not allow silent fallback data.
9-
- Do not allow hardcoded asset paths.
10-
- Tools must use explicit manifest/input data.
11-
- No input means safe empty state, not demo/sample data.
8+
- Do not use silent fallback data.
9+
- Do not add hardcoded asset paths.
10+
- Do not hide invalid sample payloads by loading defaults.
11+
- Valid sample data must be explicitly received and displayed by the target tool.
1212

1313
STEPS:
14-
1. Read docs/pr/PLAN_PR_LEVEL_10_5_NO_HIDDEN_TOOL_COUPLING_VALIDATION.md.
15-
2. Read docs/pr/BUILD_PR_LEVEL_10_5_NO_HIDDEN_TOOL_COUPLING_VALIDATION.md.
16-
3. Scan all active first-class tools for:
17-
- silent fallback sample/demo data
18-
- hardcoded JSON asset paths
19-
- implicit sample loading
20-
- fetches to old sample/tool demo JSON
21-
4. Remove/fix any discovered fallback behavior.
22-
5. Ensure tools with no input show safe empty state.
23-
6. Ensure tools with manifest slice input render the provided data.
24-
7. Add/update regression tests for:
25-
- no hidden data without input
26-
- no hardcoded JSON asset fetches
27-
- manifest input rendering
28-
8. Write reports:
29-
- docs/dev/reports/level_10_5_no_hidden_tool_coupling_report.md
30-
- docs/dev/reports/level_10_5_hardcoded_asset_path_audit.md
31-
9. Update docs/dev/roadmaps/MASTER_ROADMAP_ENGINE.md status only if needed:
32-
- [ ] -> [.]
33-
- [.] -> [x]
34-
- no prose rewrite/delete
35-
10. Do not add validators.
36-
11. Do not modify start_of_day.
37-
12. Create Codex delta ZIP:
38-
tmp/BUILD_PR_LEVEL_10_5_NO_HIDDEN_TOOL_COUPLING_VALIDATION_delta.zip
14+
1. Read docs/pr/PLAN_PR_LEVEL_10_6_SAMPLE_SCHEMA_AND_STANDALONE_TOOL_DATA_FLOW_VALIDATION.md.
15+
2. Read docs/pr/BUILD_PR_LEVEL_10_6_SAMPLE_SCHEMA_AND_STANDALONE_TOOL_DATA_FLOW_VALIDATION.md.
16+
3. Audit all sample payload files:
17+
- samples/**/sample.*.*.json
18+
- samples/**/sample.*.palette.json
19+
4. Validate schema references and tool payload contracts.
20+
5. Add/update an automated standalone sample tool data-flow test:
21+
- discover samples
22+
- open/route to target tool
23+
- verify data is received
24+
- verify UI/state displays expected data
25+
6. Specifically fix/test:
26+
- sample 0204 -> Asset Browser / Import Hub Pipeline input
27+
- sample 1413 -> Asset Browser / Import Hub and/or Asset Pipeline Tool Pipeline input
28+
- sample 1505 -> Asset Browser / Import Hub Pipeline input
29+
- sample 0510 -> Asset Pipeline Tool Pipeline input
30+
- sample 1417 -> Asset Pipeline Tool Pipeline input
31+
- sample 0213 -> palette displays
32+
- sample 0308 -> palette displays
33+
- sample 0313 -> palette displays
34+
7. Update tool standalone binding only where needed:
35+
- Asset Browser / Import Hub Pipeline input
36+
- Asset Pipeline Tool Pipeline input
37+
- Palette Browser palette display
38+
8. Ensure no fallback/demo data is loaded.
39+
9. Write reports:
40+
- docs/dev/reports/level_10_6_sample_schema_validation_report.md
41+
- docs/dev/reports/level_10_6_standalone_tool_data_flow_report.md
42+
10. Update docs/dev/roadmaps/MASTER_ROADMAP_ENGINE.md status only if needed:
43+
- [ ] -> [.]
44+
- [.] -> [x]
45+
- no prose rewrite/delete
46+
11. Do not add runtime validators.
47+
12. Do not modify start_of_day.
48+
13. Create Codex delta ZIP:
49+
tmp/BUILD_PR_LEVEL_10_6_SAMPLE_SCHEMA_AND_STANDALONE_TOOL_DATA_FLOW_VALIDATION_delta.zip
3950

4051
ACCEPTANCE:
41-
- no silent fallback data remains
42-
- no hardcoded JSON asset paths remain
52+
- all targeted sample failures fixed or reported with exact blocker
53+
- automated test catches standalone tool data-flow failures
4354
- delta ZIP exists

‎docs/operations/dev/commit_comment.txt‎

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,11 @@
1-
BUILD_PR_LEVEL_10_5_NO_HIDDEN_TOOL_COUPLING_VALIDATION
1+
BUILD_PR_LEVEL_10_6_SAMPLE_SCHEMA_AND_STANDALONE_TOOL_DATA_FLOW_VALIDATION
22

3-
- Removed hidden fallback/sample/demo data paths from tools
4-
- Removed hardcoded JSON asset paths
5-
- Enforced manifest-input-only tool behavior
6-
- Added regression coverage for safe empty state and manifest input rendering
3+
- Added standalone sample schema and tool data-flow validation
4+
- Fixed targeted Asset Browser, Asset Pipeline, and Palette sample loading/display failures
5+
- Prevented fallback/demo data from masking invalid payloads
76

87
PR Details:
9-
- Final Phase 10 tool purity gate
10-
- No validators
8+
- Validates standalone tool mode receives/displays sample data
9+
- No hardcoded asset paths
10+
- No runtime validators
1111
- No start_of_day changes

0 commit comments

Comments
 (0)