Skip to content

feat(pj_base): pj.data_processors.v1 — data-only SDK service for plugin-created transforms#125

Open
facontidavide wants to merge 1 commit into
mainfrom
feat/data-processors-service
Open

feat(pj_base): pj.data_processors.v1 — data-only SDK service for plugin-created transforms#125
facontidavide wants to merge 1 commit into
mainfrom
feat/data-processors-service

Conversation

@facontidavide

Copy link
Copy Markdown
Contributor

What

Adds the pj.data_processors.v1 host service — a data-only C ABI that lets a toolbox plugin create catalog-resident Transform nodes in the host. Only strings cross the boundary (script source + input/output names + params JSON + an id); the host owns all execution.

This is M4.1 of the PJ4 Data Processors plan (the SDK half). The PJ4 host bridge (M4.2) that turns these calls into live DerivedEngine nodes lands separately in the application repo.

Why this shape

  • Plugin creates, host executes. Because the host owns the script text + VM, a transform survives plugin unload and session reload — the originating plugin is needed only for re-editing.
  • No C++ across the DSO boundary. Nothing executable crosses; this deliberately avoids the C++-vtable / library_owner UAF that sank the earlier filter-registry attempt.
  • The native door is WASM, not a DSO kernel. The payload is binary-safe (PJ_string_view_t {data,size}), so a future host-owned WASM/Python backend ships as bytes through this same surface — purely additive, survives unload, sandboxed — rather than a native kernel vtable that would dangle on unload.
  • Transforms only. outputs is required non-empty (named catalog topics); host-internal per-curve Filters are not created through this ABI but share the same DataProcessor substrate.

Changes

  • PJ_data_processors_host_vtable_t (create / remove / list / config), protocol_version 1, struct_size-gated, ABI-appendable; all slots [main-thread], PJ_NOEXCEPT.
  • sdk::DataProcessorsHostView C++ sugar + DataProcessorsHostService trait.
  • View now rejects an empty output list before reaching the vtable.
  • Doc-comments: binary-safety, per-plugin isolation, transactional create, the WASM forward-compat door.

Versioning

Additive new service → Conan MINOR 0.8.00.9.0; abi/baseline.abi untouched (additions-only).

Tests

Fake-host gtest suite, 8 cases (6 baseline + CreateRejectsEmptyOutputs, BinarySafePayloadRoundTrips). Full SDK build + 47/47 tests green; pre-commit (clang-format v22.1.0) clean.

🤖 Generated with Claude Code

…ugin-created transforms

Adds the `pj.data_processors.v1` host service: a DATA-ONLY C ABI that lets a
toolbox plugin create catalog-resident "transform" nodes in the host. Only
strings cross the boundary (script source + input/output names + params JSON +
an id); the host owns all execution, so a transform survives plugin unload and
session reload — the originating plugin is needed only for re-editing.

- PJ_data_processors_host_vtable_t (create/remove/list/config), protocol_version 1,
  struct_size-gated and ABI-appendable; all slots [main-thread], PJ_NOEXCEPT.
- sdk::DataProcessorsHostView C++ sugar + DataProcessorsHostService trait.
- Payload (script/params) is binary-safe (PJ_string_view_t {data,size}), so the
  native "door" is WASM bytes through this same surface — a future host-owned
  WASM/Python backend is purely additive and survives unload, deliberately NOT a
  C++ kernel vtable that would dangle on DSO unload.
- The view now rejects an empty output list (transforms create named topics, so
  >=1 output is required) before it ever reaches the vtable.
- Fake-host gtest suite (8 cases) incl. a binary-safe payload round-trip and the
  empty-outputs guard.

Additive new service -> Conan MINOR (0.8.0 -> 0.9.0); abi/baseline.abi untouched.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant