Skip to content

feat: new manifest format#261

Draft
atilafassina wants to merge 17 commits intomainfrom
manifest-ax
Draft

feat: new manifest format#261
atilafassina wants to merge 17 commits intomainfrom
manifest-ax

Conversation

@atilafassina
Copy link
Copy Markdown
Contributor

@atilafassina atilafassina commented Apr 8, 2026

Summary

Evolves the AppKit plugin manifest contract from v1.0 to v2.0 and swaps the canonical authoring surface from hand-written JSON Schema to Zod via the Standard Schema interface. AJV, the json-schema-to-typescript codegen pipeline, and the standalone semantic-validator delete; refinement and .transform() on Zod schemas replace them.

Important

~91% of insertions are auto-regenerated artifacts — two JSON Schema files (docs/static/schemas/{plugin-manifest,template-plugins}.schema.json) account for +10,400 lines.

.gitattributes marks all generated artifacts linguist-generated=true. GitHub collapses them in the file list — substantive review surface is ~32 files / ~2,300 insertions.

What changes vs main

Contract

  • discoveryDescriptor is a discriminated union: { type: "kind", resourceKind, ... } | { type: "cli", cliCommand, ... }. Six known kinds: warehouse, genie_space, volume, postgres_project, postgres_branch, postgres_database.
  • cli variant rejects shell metacharacters (;, |, &, backtick, $, newlines) on cliCommand and shortcut. .describe() notes direct authors to kind for first-party resources and flag the cli shape as minimal-and-may-tighten.
  • RESOURCE_KIND_COMMANDS map next to the schema; commands verified against Databricks CLI v0.299.0.
  • origin on template fields is a .transform() output (drift-impossible).
  • rules.never[] / rules.must[] items capped at maxLength: 120.
  • New volume MUST rule: prompt for catalog and schema before listing volumes.
  • 4 core plugin manifests (analytics, files, genie, lakebase) migrated to the kind variant; lakebase gains a project field.

Authoring surface

  • New: packages/shared/src/schemas/manifest.ts (Zod schemas + RESOURCE_KIND_COMMANDS + TEMPLATE_SCAFFOLDING).
  • New: tools/generate-json-schema.ts (Zod → draft-07 JSON Schema, emits to docs/static/schemas/).
  • New runtime dep on packages/shared: @standard-schema/spec.

Deletions

  • Deps removed: ajv, ajv-formats, json-schema-to-typescript.
  • packages/shared/src/schemas/{plugin-manifest.schema.json,template-plugins.schema.json,plugin-manifest.generated.ts}.
  • tools/generate-schema-types.ts, docs/scripts/copy-schemas.ts.
  • validate-manifest.ts (498 → 177 lines): AJV plumbing + runSemanticValidation + helpers.
  • enrichFieldsWithOrigin, validateDiscoveryOrigin.

Migrated to Zod-direct

  • schema-resources.ts and tools/generate-registry-types.ts (no more runtime JSON-schema reads).
  • manifest-types.ts is now a re-export shim (z.infer types + StandardSchemaV1).

Deferred to follow-up PRs

Change Note
TypeScript-authored manifests (defineManifest({ ... })) stacked PR link coming soon
Docs for the v2.0 format stacked PR link coming soon
Branded field-name keys for compile-time dependsOn checks not a must-have, larger implementation change
cli variant full hardening shell-metacharacter denylist + describe notes shipped here. Remaining: argv-array form (executor-side spawn-not-shell), output-shape contract (replace jq path with declared field names + unwrap). Tied to the executor PR.

Note

Plugin authors continue to write manifest.json. More about that in tbd: follow-up PR.

@atilafassina atilafassina marked this pull request as ready for review April 9, 2026 08:16
Copilot AI review requested due to automatic review settings April 9, 2026 08:16
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR upgrades the AppKit plugin manifest ecosystem to a v2.0 template manifest format to support smarter databricks apps init scaffolding, including field discovery metadata, post-scaffold user instructions, computed field origins, and semantic (cross-field) validation during plugin validate.

Changes:

  • Bump template plugin manifest version to 2.0 and add a top-level scaffolding descriptor.
  • Extend plugin/template schemas with discovery (CLI command-based value discovery) and postScaffold steps; generate/propagate computed origin into template manifests during sync.
  • Add semantic validation (dependsOn cycles/dangling refs, <PROFILE> placeholder, discovery/origin coherence, postScaffold structure) and associated tests/docs updates.

Reviewed changes

Copilot reviewed 20 out of 20 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
template/appkit.plugins.json Updates template manifest to v2.0 and annotates built-in plugins with discovery/origin/postScaffold plus scaffolding descriptor.
packages/shared/src/schemas/template-plugins.schema.json Expands template manifest schema to support v2.0 and scaffolding descriptor; inlines field/requirement defs for origin.
packages/shared/src/schemas/plugin-manifest.schema.json Adds discovery to resource fields and postScaffold to plugin manifests.
packages/shared/src/schemas/plugin-manifest.generated.ts Updates generated TS types to include discovery/postScaffold types.
packages/shared/src/plugin.ts Re-exports new generated types (DiscoveryDescriptor, PostScaffoldStep).
packages/shared/src/cli/commands/plugin/validate/validate.ts Runs semantic validation and formats semantic errors/warnings in CLI output.
packages/shared/src/cli/commands/plugin/validate/validate-manifest.ts Implements semantic validation rules and issue formatting.
packages/shared/src/cli/commands/plugin/validate/validate-manifest.test.ts Adds test coverage for new semantic validation behavior.
packages/shared/src/cli/commands/plugin/sync/sync.ts Computes/injects origin, bumps template manifest to v2.0, and adds scaffolding descriptor on write.
packages/shared/src/cli/commands/plugin/sync/sync.test.ts Adds unit tests for origin computation.
packages/shared/src/cli/commands/plugin/manifest-types.ts Adds scaffolding descriptor types and exports new manifest-related types.
packages/appkit/src/plugins/lakebase/manifest.json Adds discovery descriptors and postScaffold steps to the lakebase built-in plugin manifest.
packages/appkit/src/plugins/genie/manifest.json Adds schema reference, discovery descriptor, and postScaffold steps to genie plugin manifest.
packages/appkit/src/plugins/files/manifest.json Adds discovery descriptor and postScaffold steps to files plugin manifest.
packages/appkit/src/plugins/analytics/manifest.json Adds discovery descriptor and postScaffold steps to analytics plugin manifest.
docs/static/schemas/template-plugins.schema.json Publishes updated template plugins schema for docs site.
docs/static/schemas/plugin-manifest.schema.json Publishes updated plugin manifest schema for docs site.
docs/static/appkit-ui/styles.gen.css Updates generated UI styles (tailwind output) used by docs UI.
docs/docs/api/appkit/Interface.ResourceFieldEntry.md Documents the new discovery field on ResourceFieldEntry.
docs/docs/api/appkit/Interface.PluginManifest.md Documents the new postScaffold field on PluginManifest.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread packages/shared/src/cli/commands/plugin/validate/validate-manifest.ts Outdated
Comment thread packages/shared/src/schemas/plugin-manifest.generated.ts Outdated
Comment thread template/appkit.plugins.json Outdated
Comment thread packages/appkit/src/plugins/lakebase/manifest.json Outdated
@atilafassina atilafassina changed the base branch from main to pkosiec/template-artifact April 10, 2026 10:21
Base automatically changed from pkosiec/template-artifact to main April 14, 2026 07:52
@keugenek
Copy link
Copy Markdown
Contributor

@atilafassina this one looks ok and evolution, however little heavy code-wise (let me post comments from isaac about these) and mixing concepts little bit. What do you think of this:

Alternatives worth weighing

Zod as single source of truth (my default for a TS-first ecosystem):

  • Types, runtime validation, and semantic checks in one file.
  • zod-to-json-schema emits the $schema JSON for VSCode intellisense.
  • Cycle/dangling-ref checks become .refine() calls in the same module as the
    schema.
  • Deletes plugin-manifest.generated.ts and the Ajv layer.

TS-authored manifests compiled to JSON:

  • manifest.ts → defineManifest({...}) with branded types for field names
    (compile-time dependsOn checking, no DFS needed).
  • sync lowers to .json for non-TS consumers.
  • Strongest author ergonomics; agent still reads JSON.

Split concerns into two files:

  • manifest.json — strict contract (resources, fields, env), stable schema.
  • scaffold.yaml or .md — agent hints (discovery, prompts, postScaffold), free
    to evolve.
  • Today they're fused, which is why the schema keeps growing.

Copy link
Copy Markdown
Contributor

@keugenek keugenek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall direction looks right — v2.0 versioning via if/then is correct and co-located manifest.json per plugin is the right locality. A few shape-level concerns worth addressing before or soon after this lands. Happy to chat on any of them.

}
}
},
"discoveryDescriptor": {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cliCommand is a string template with a <PROFILE> placeholder plus a jq path in selectField. The only validation is "contains <PROFILE>" — nothing checks the CLI output shape, so paginated or wrapped responses ({warehouses:[...]}) will fail at runtime. Consider {resourceKind, select, display} and letting the CLI own the command.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll change the
- cliCommand gets a .refine() denylist of shell operators
- cli variant's .describe() gains a "use kind where possible; cli is an escape hatch and may tighten further" note. Surfaces in the published JSON. This mains we can tighten it up / deprecate going forward.

I'm a bit reluctant about making it more structured/strict right now because we are still a bit short in plugin quantity, so new use-cases and usage may surface here.

Wdyt?

Comment thread packages/shared/src/cli/commands/plugin/sync/sync.ts Outdated
},
"additionalProperties": false
},
"scaffoldingDescriptor": {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rules.never[] / rules.must[] are LLM prompts inside the schema. No validator can check what actually matters (agent behavior), and the schema file becomes part prompt. Worth splitting into a sibling scaffold.prompts.md.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My reluctance to an *.md is that this will create contention between that and the plugin skill. I don't want freeform text here. These keys are short directives for things the plugin needs the LLM to always or never do in order to function efficiently.

So, while we can't validate the string fully, I'm capping it to a max-length. That and a clear validation error will nudge the plugin author in the right direction.

Comment thread docs/static/schemas/template-plugins.schema.json Outdated
Comment thread .claude/scheduled_tasks.lock Outdated
atilafassina added a commit that referenced this pull request Apr 21, 2026
Accidentally committed in a1c30e3; it's ephemeral Claude Code loop
state, not source. Flagged in PR #261 review.

Signed-off-by: Atila Fassina <atila@fassina.eu>
@MarioCadenas MarioCadenas requested a review from a team as a code owner May 6, 2026 10:59
atilafassina added a commit that referenced this pull request May 7, 2026
Accidentally committed in a1c30e3; it's ephemeral Claude Code loop
state, not source. Flagged in PR #261 review.

Signed-off-by: Atila Fassina <atila@fassina.eu>
@atilafassina atilafassina marked this pull request as draft May 7, 2026 20:12
@atilafassina atilafassina removed the request for review from arsenyinfo May 7, 2026 20:12
…tScaffold, and scaffolding

Xavier loop: iteration 1 — Phase 1 (Schema Definitions & Type Generation)

Co-authored-by: Isaac
Signed-off-by: Atila Fassina <atila@fassina.eu>
…te manifest emission

Xavier loop: iteration 2 — Phase 2 (Origin Computation & Sync Enrichment)

Co-authored-by: Isaac
Signed-off-by: Atila Fassina <atila@fassina.eu>
…, and postScaffold

Xavier loop: iteration 3 — Phase 3 (Semantic Validation)

Co-authored-by: Isaac
Signed-off-by: Atila Fassina <atila@fassina.eu>
…ostScaffold steps

Xavier loop: iteration 4 — Phase 4 (Core Plugin Manifest Annotations)

Co-authored-by: Isaac
Signed-off-by: Atila Fassina <atila@fassina.eu>
…t for origin support

JSON Schema Draft-07 additionalProperties:false blocks allOf composition.
Inlined both defs in template schema so origin validates correctly.

Xavier loop: iteration 5 — Phase 5 (Integration & Backpressure)

Co-authored-by: Isaac
Signed-off-by: Atila Fassina <atila@fassina.eu>
Accidentally committed in a1c30e3; it's ephemeral Claude Code loop
state, not source. Flagged in PR #261 review.

Signed-off-by: Atila Fassina <atila@fassina.eu>
Add Zod schemas mirroring the existing plugin and template manifest JSON
Schemas, the @standard-schema/spec dep that consumer code will use in
phase 2, and a Zod→JSON Schema generator wired into build:package and
generate:types. AJV continues to run in validate-manifest.ts; this is
purely additive groundwork.

Parity test uses fixture equivalence (Strategy B) — Zod 4's toJSONSchema
emits per-type permission constraints as oneOf-of-discriminated-variants
while the hand-written schema uses allOf+if/then over $defs/$ref, so byte
parity is structurally infeasible. The test asserts AJV-with-legacy and
Zod-with-new return matching accept/reject verdicts on the four core
plugin manifests plus 5 synthetic plugin and 3 synthetic template fixtures
(12 cases total).

Build-pipeline byproducts of running pnpm build && pnpm docs:build cleanly
are also captured: docs/static/schemas/plugin-manifest.schema.json loses a
description field that copy-schemas.ts overwrote from the package-internal
source (where the description was never present), and template/appkit.plugins.json
gains origin enrichment on jobs.id and serving.name fields the parent
PRD's enrichFieldsWithOrigin pass missed.

Co-authored-by: Isaac
Signed-off-by: Atila Fassina <atila@fassina.eu>
Replace AJV with Zod-via-Standard-Schema in validate-manifest.ts. The CLI
validator now calls `~standard.validate` against the Zod schemas authored
in phase 1; consumer code never imports zod directly.

Cycle detection, dangling-reference checks, and the <PROFILE> placeholder
constraint move from the standalone runSemanticValidation pass into Zod
refinements co-located with the shape:

- resourceRequirementSchema gains a superRefine running DFS over
  discovery.dependsOn — dangling refs emit at fields.<name>.discovery.dependsOn,
  cycles emit at the resource root with the existing 'a → b → c → a' chain.
- discoveryDescriptorSchema.refine() enforces the <PROFILE> placeholder
  on cliCommand.
- postScaffoldStepSchema.instruction tightens to z.string().min(1).

origin-drift detection (validateDiscoveryOrigin) is dropped — origin
becomes a transform in phase 3, eliminating the desync surface entirely.

validate-manifest.ts shrinks from 498 to 177 lines: loadSchema,
getPluginValidator, getTemplateValidator, the AJV compile cache, the
JSON-pointer humanizer, the AJV error formatter, runSemanticValidation,
validateDependsOn, validateDiscoveryProfile, validateDiscoveryOrigin,
validatePostScaffold, and formatSemanticIssues all delete. Tests rewrite
to drive validateManifest end-to-end and assert on the resulting
SemanticIssue shape.

validateManifest returns the original input object as `manifest` rather
than result.output — Zod parsing is used purely as a verifier here so
property order is preserved for round-trip writers like add-resource.
Phase 3 will introduce the first real transform (origin), at which point
output-vs-input distinction becomes intentional.

ajv and ajv-formats remain in packages/shared/package.json for the
phase-1 parity test (deletes in phase 5).

Co-authored-by: Isaac
Signed-off-by: Atila Fassina <atila@fassina.eu>
templateFieldEntrySchema gains a .transform() that computes origin from
localOnly/value/resolve and emits it on every parse. origin is accepted as
optional input but always overwritten — drift-by-construction is now
structurally impossible. The new sync.test.ts case verifies this: a field
with value: "5432" and stale input origin: "user" parses to "static".

enrichFieldsWithOrigin and its mutation pass delete from sync.ts. The
template manifest is now built by parsing each field through
templateFieldEntrySchema before serialization. Per-field parse (rather
than whole-manifest parse) is chosen because Zod 4's strict-object parse
reorders keys aggressively, churning resource and plugin entries; per-field
parse leaves the surrounding structure in input order. Sync output is
byte-identical to phase 2 (md5 verified).

computeOrigin and Origin type delete from manifest-types.ts and sync.ts.
The replacement, computeOriginFromField, is private to manifest.ts and
invoked only by the transform — nothing else in the codebase needs origin
computation now that validateDiscoveryOrigin (deleted in phase 2) is gone.

generate-json-schema.ts passes io: "input" to z.toJSONSchema so the
transform doesn't break schema emission. The published JSON Schema
describes what plugin authors write (no origin slot), not the transformed
output — exactly the right semantic for IDE intellisense and external
validators.

template/appkit.plugins.json regenerated by build pipeline (pre-existing
drift, not introduced here).

Co-authored-by: Isaac
Signed-off-by: Atila Fassina <atila@fassina.eu>
Replace the free-form discoveryDescriptorSchema with a discriminated
union on `type`:

- kind variant: { type: "kind", resourceKind, select?, display?, dependsOn?, shortcut? }
  resourceKind enum is the closed set of databricks resources AppKit owns
  command templates for: warehouse, genie_space, postgres_branch,
  postgres_database, volume.
- cli variant: { type: "cli", cliCommand, selectField, displayField?, dependsOn?, shortcut? }
  preserves the existing free-form shape as an escape hatch for bespoke
  resources not yet in the kind enum.

The <PROFILE> placeholder .refine() moves from the top-level descriptor
to the cli variant only — kind has no cliCommand to validate. The cycle
and dangling-reference DFS on resourceRequirementSchema reads
field.discovery?.dependsOn generically and continues to work across both
variants.

A typed RESOURCE_KIND_COMMANDS map ships next to the schema. Each entry
declares the CLI command template (with <PROFILE> placeholder + optional
{<fieldName>} placeholders for dependsOn substitution) and an optional
unwrap path for wrapped responses. Volume's catalog/schema parent
context is documented in a code comment as a phase 6 MUST-rule concern,
not a schema construct.

The four core plugin manifests migrate to the kind variant:
- analytics: { type: "kind", resourceKind: "warehouse" }
- genie: { type: "kind", resourceKind: "genie_space" }
- lakebase: branch → postgres_branch, database → postgres_database (dependsOn: "branch")
- files: { type: "kind", resourceKind: "volume", select: "full_name" }

Lakebase carries select: "name" (non-default for postgres_branch and
postgres_database); files carries select: "full_name". Defaults are kind-
specific identifiers and live in the command map / runner (out of scope).

The Zod-derived ResourceRequirement is a discriminated union (per-type
permission tightness baked in). Two consumer interface declarations in
packages/shared/src/plugin.ts and packages/appkit/src/registry/types.ts
previously did `interface ResourceRequirement extends GeneratedResourceRequirement` —
TS interfaces cannot extend union types. Both flatten to structural
interfaces with `permission: string`. This matches the previous
consumer-facing shape (legacy generated permission was already loose
string); per-variant tightness is enforced at schema parse time, where
it belongs. add-resource.ts's literal entry construction casts to
ResourceRequirement for the same reason.

manifest-types.ts re-export source switches from the legacy
plugin-manifest.generated to the canonical Zod schemas/manifest. The
generated.ts file is now orphaned (no source imports it) and deletes
in phase 5.

The phase 1 parity test (json-schema-parity.test.ts) deletes — it
asserted AJV-with-legacy-schema and Zod-with-new-schema return matching
verdicts on the four core plugin manifests, but those manifests now use
type: "kind" which the legacy schema doesn't understand. The test was
the phase-1 transition gate; once the contract intentionally diverges
it loses meaning.

template/appkit.plugins.json and docs/docs/api/appkit/* re-emitted by
the build pipeline.

Co-authored-by: Isaac
Signed-off-by: Atila Fassina <atila@fassina.eu>
Now that Zod is canonical and the validation runtime calls
~standard.validate, the legacy artifacts have nothing left to do. This
phase deletes them and the build steps that produced them.

Deleted:
- packages/shared/src/schemas/plugin-manifest.generated.ts (orphaned
  since phase 4 switched manifest-types.ts re-exports to Zod).
- packages/shared/src/schemas/plugin-manifest.schema.json,
  template-plugins.schema.json (legacy hand-written JSON Schema; Zod
  is now the source, JSON Schema is generated into docs/static/schemas
  by tools/generate-json-schema.ts).
- tools/generate-schema-types.ts (JSON Schema → TS codegen, replaced by
  tools/generate-json-schema.ts going the other way).
- docs/scripts/copy-schemas.ts (copied the legacy schemas to docs/static,
  now no-op).

Dependencies removed from packages/shared:
- ajv, ajv-formats — runtime validator gone in phase 2.
- json-schema-to-typescript — codegen tool gone above.

Build pipeline updated:
- root package.json generate:types and packages/shared build:package
  scripts drop generate-schema-types.ts.
- packages/shared/tsdown.config.ts drops the copy: block (the .json
  files no longer exist).
- docs/package.json drops the copy-schemas script and removes it from
  the gen chain.
- knip.json drops json-schema-to-typescript from ignoreDependencies.
- .github/workflows/ci.yml `Check generated types are up to date` step
  drops the deleted plugin-manifest.generated.ts and adds
  docs/static/schemas/*.schema.json (now owned by generate-json-schema.ts).

Source migrations to Zod:
- schema-resources.ts: was reading plugin-manifest.schema.json at
  runtime to derive resource type options and per-type permissions.
  Now imports the per-type permission schemas and resourceTypeSchema
  from the Zod module and reads .options. No filesystem reads, no
  caching, no defensive null branches — values are module-level
  constants now. Public API preserved.
- tools/generate-registry-types.ts: hidden consumer that also read
  the legacy JSON schema. Same migration.
- packages/shared/src/cli/commands/plugin/manifest-types.ts: shrunk to
  a thin re-export shim of z.infer types and StandardSchemaV1.

Type-level fix:
- TemplatePlugin / TemplateResourceRequirement / TemplateFieldEntry /
  TemplatePluginsManifest type aliases switched to z.input instead of
  z.infer/z.output. The field-level origin transform makes origin
  REQUIRED on z.output, but consumer code (sync.ts) constructs
  template plugins without origin before writeManifest runs the
  transform at write time. z.input gives the pre-transform shape,
  matching the runtime invariant.

Stale JSDoc references to GeneratedPluginManifest and
plugin-manifest.generated.ts updated.

The published JSON Schema URL is unchanged. Plugin authors' VSCode
intellisense continues to work; the docs/static/schemas/*.json files
are now byte-stable across runs (generated solely by the Zod-fed
generate-json-schema.ts) and contain the new discriminated-union
discovery shape.

Co-authored-by: Isaac
Signed-off-by: Atila Fassina <atila@fassina.eu>
Final phase of the manifest zod refactor. Three small, targeted changes
to the scaffolding descriptor:

1. Rule items are constrained to ≤ 120 chars via z.string().max(120)
   on both rules.never[] and rules.must[]. The schema can't validate
   prose content but it can stop a directive from growing into a
   paragraph — enforces the "short directive" intent at the only
   boundary the schema can express.

2. TEMPLATE_SCAFFOLDING moves from sync.ts into the schema module.
   The constant lives next to the schemas it conforms to, with a
   `satisfies z.infer&lt;typeof scaffoldingDescriptorSchema&gt;` clause for
   compile-time validation against the input shape. sync.ts imports it.

3. New MUST rule directive describing volume parent-context handling:
   "When discovering volume resources, prompt the user for catalog
   and schema before listing volumes." The kind variant for `volume`
   doesn't model catalog/schema parents in the schema (per PRD design
   decision #7 — hierarchical context as MUST rule, not schema
   structure); this directive carries the requirement to LLM
   scaffolding agents instead.

Tests added under "scaffolding rule item maxLength (Phase 6)":
- never[]/must[] items exceeding 120 chars produce errors with the
  right path and message.
- 120 chars exactly is accepted (≤ semantics).
- A mixed-length array flags only the offending entry.
- TEMPLATE_SCAFFOLDING parses cleanly against scaffoldingDescriptorSchema.
- The synced template manifest carries the new volume MUST rule string.

template/appkit.plugins.json regenerated by sync:template — the new
rule string is now in scaffolding.rules.must.

Co-authored-by: Isaac
Signed-off-by: Atila Fassina <atila@fassina.eu>
CI's `Check generated types are up to date` step was failing because
two zod versions live in the workspace:
- 4.1.13 hoisted at root (transitive via clean-app's
  eslint-plugin-react-hooks → zod-validation-error peer)
- 4.3.6 in packages/shared (explicit dep)

`tools/generate-json-schema.ts` imports zod from its own location,
which resolves to root's 4.1.13. `packages/shared/src/schemas/manifest.ts`
imports zod, resolving to shared's 4.3.6. The two runtimes operate on
each other's schema objects, and the older zod's `toJSONSchema` doesn't
extract all the constraints (pattern, minLength, propertyNames) that
the newer zod baked into the schemas. CI's pnpm install resolves them
consistently and emits the richer output, which then drifts from what's
committed.

Adding zod@4.3.6 as a root devDependency makes pnpm hoist the matching
version to the top-level node_modules. The generator now resolves the
same zod runtime as the schema module, and the JSON Schema output is
byte-stable across local and CI.

Regenerated docs/static/schemas/*.schema.json carry the now-emitted
constraints (~135 minLength/pattern entries on plugin-manifest, similar
on template-plugins). The constraints were always in the Zod schemas
since phase 1 — they just weren't surviving the cross-version emit.

Co-authored-by: Isaac
Signed-off-by: Atila Fassina <atila@fassina.eu>
… strict config

Three real bugs flagged by the multi-model review of PR #261, fixed in
this iteration. (CRITICAL cliCommand RCE hardening + HIGH z.lazy() perf
deferred to follow-up PRs.)

1. RESOURCE_KIND_COMMANDS strings now match the real Databricks CLI
   (verified against v0.299.0 --help output):
   - `genie list` → `genie list-spaces`
   - `volumes list <catalog>.<schema>` → `volumes list {catalog} {schema}`
     (two separate positionals, prompted via the volume MUST rule)
   - `postgres list-branches` → `postgres list-branches {project}` with
     a project parent (covered by Fix 2 below)

2. Lakebase branch discovery is now actually runnable:
   - resourceKindSchema gains `postgres_project`. RESOURCE_KIND_COMMANDS
     gains the corresponding `databricks postgres list-projects` entry.
   - lakebase/manifest.json gains a new `project` field with
     `discovery: { type: "kind", resourceKind: "postgres_project", select: "name" }`.
   - The existing `branch` field's discovery adds `dependsOn: "project"`,
     so the parent project name flows into the branch listing command.

3. configSchemaPropertySchema and configSchemaSchema gain `.strict()`,
   so plugin config-schema typos no longer pass validation silently.
   `additionalProperties` (a standard JSON Schema keyword used by three
   core plugins — serving, vector-search, genie — inside nested property
   entries) is added explicitly as
   `z.union([z.boolean(), configSchemaPropertySchema]).optional()` so
   those manifests keep validating; this is a deliberate canonical
   addition, not a loosening of strict mode.

Auto-regenerated by the build pipeline:
- docs/static/schemas/{plugin-manifest,template-plugins}.schema.json
- template/appkit.plugins.json

Backpressure: typecheck=0, test=0 (108 files / 2136 tests), build=0,
docs:build=0, knip=0, check:fix=0.

Co-authored-by: Isaac
Signed-off-by: Atila Fassina <atila@fassina.eu>
GitHub collapses these in the diff view by default and excludes them
from language stats. The files are emitted by the build pipeline and
should not need reviewer attention.

- docs/static/schemas/*.schema.json — emitted by tools/generate-json-schema.ts
- template/appkit.plugins.json — emitted by pnpm sync:template
- packages/appkit/src/registry/types.generated.ts — generate-registry-types.ts
- packages/appkit/src/plugins/*-exports.generated.ts — generate-plugin-entries.ts
- docs/docs/api/** — typedoc API reference
- pnpm-lock.yaml — pnpm

Reduces perceived PR size on this branch by ~10k lines (two regenerated
JSON Schema files alone account for ~91% of insertions on PR #261).

Co-authored-by: Isaac
Signed-off-by: Atila Fassina <atila@fassina.eu>
The `cli` variant of discoveryDescriptor accepts a free-form Databricks
CLI command supplied by the plugin author. With no further constraint
beyond the existing `<PROFILE>` placeholder check, the shape is open to
two unrelated foot-guns reviewers flagged:

- output-shape brittleness: a plugin writes `selectField: ".id"` but the
  CLI returns a wrapped object (e.g. `{warehouses: [...]}`); jq fails
  silently at scaffold time
- shell-injection-if-executed: when an executor lands and passes the
  string to a shell, statement separators / pipes / command substitution
  / redirects all become attack surface

The `kind` variant addresses both for first-party plugins (AppKit owns
the command and unwrap rules). The `cli` variant is the escape hatch for
third-party plugins that need bespoke commands. Tighten it cheaply now,
before anyone ships against the loose shape:

- new SHELL_METACHAR_RE blocks `;`, `|`, `&`, backtick, `$`, newlines on
  both `cliCommand` and `shortcut`. Angle brackets are still permitted
  so `<PROFILE>` (and future `<…>` placeholders) work.
- describes on cliCommand and the variant overall direct authors to use
  `kind` for first-party resources and call out that the cli shape is
  intentionally minimal and may tighten further.

Not a security boundary on its own — executors must still spawn(argv)
not shell-exec the string. argv-array form, denylist of shell operators
in argv, and an output-shape contract are all separate decisions tied
to the executor PR.

Two new test cases cover the two refinements (cliCommand + shortcut).

Co-authored-by: Isaac
Signed-off-by: Atila Fassina <atila@fassina.eu>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants