Lambé 0.10.0: WASM-clean library, frontmatter-aware Markdown, idiom redirects#8
Merged
Conversation
Largest concrete change: the published library now has zero dart:io
imports, so the lambé playground in arda-web (which compiles with
`dart compile wasm`) gets a clean platform compatibility report from
pana and runs without dart:io bridges.
Breaking — library API surface
==============================
Removed from `package:lambe`:
- `loadSchemaFromFile`
- `loadSchemaForData`
Both helpers used `dart:io` (file IO). They moved to `bin/schema_io.dart`
where the CLI consumes them. `mergeSchemaWithData` (pure) stays in
`lib/src/schema/loader.dart` and remains exported.
REPL and readline (CLI-only) moved out of lib/ into bin/:
- `lib/src/repl.dart` → `bin/repl.dart`
- `lib/src/readline.dart` → `bin/readline.dart`
- `lib/src/highlight_grammar.dart` → `bin/highlight_grammar.dart`
bin/lam.dart and bin/repl.dart updated to use `package:lambe/...` for
the lib/ imports they still need, plus relative imports for sibling
bin/ files.
Markdown frontmatter
====================
`parseInput(text, Format.markdown)` now uses
`parseMarkdownWithFrontmatter` from rumil_parsers 0.8.1. A leading
`---` YAML frontmatter block becomes a sibling `frontmatter` field on
the document instead of being absorbed into the body as prose.
Files without frontmatter parse byte-identically to before. The
frontmatter conversion goes through rumil_parsers' yamlToNative so
anchors and aliases resolve correctly without lambé reimplementing
the walk.
Foreign-idiom redirects (jq compatibility)
==========================================
Adds redirect hints for the rest of the common jq habits a model
might draft. Pipe-op redirects (fire on `... | name(...)`):
`getpath`, `setpath`, `env`, `gsub`, `sub`, `test`, `match`, `scan`,
`splits`, `tojson`, `fromjson`. Inline-idiom redirects: `@uri`,
`@html`, `@sh`, `@json`, `$ENV` and other variable-binding forms.
The regex family (`test`, `match`, `sub`) previously hit the
closest-match suggestion ("did you mean text/map/sum?"). Now they
get the regex-family explanation directly.
Heterogeneous-list shape descriptions
=====================================
`SList` gains an optional `sampledKinds: List<Shape>?` field, populated
when widening to `SAny` because of observed heterogeneity. Empty
lists keep `sampledKinds: null` (no observations to report).
`renderJsonSchema` reads it and emits, for example,
`"sampled: number, string, boolean, null, array (heterogeneous)"`
instead of the generic `"sampled, may be heterogeneous"`.
CLI / error message polish
==========================
- Output-shape error messages clarified: `Append one of these stages
to the end of your query (keep your existing flags such as -t hcl):`
instead of the ambiguous `Try appending one of:`. The format name
interpolates dynamically so HCL / TOML / CSV all read correctly.
- `--schema <data-file>` migration hint when the argument has a data
extension (`.json`, `.yaml`, etc.) rather than `*.schema.json`,
pointing at the new `--print-shape` flag instead of dumping usage.
- `.jsonlines` extension joins `.ndjson` and `.jsonl` in
auto-detecting --ndjson mode.
Tests
=====
- 4 new tests in `test/markdown_text_test.dart` for the frontmatter
contract (text op no longer scoops frontmatter; frontmatter
addressable; absent case unchanged; child shape preserved).
- 2 tests in `test/shape_test.dart` updated to assert the new
`sampledKinds` contract on heterogeneous lists.
- `test/schema_loader_test.dart` imports the moved file-loaders via
relative path `../bin/schema_io.dart`.
1657 tests pass, format clean, analyze clean.
Performance
===========
Bench (lambé library bench, median of 7, after warm-up) versus
lambé 0.9.0 with rumil 0.7.0 / rumil_parsers 0.8.0:
Workload AOT 0.9.0 0.10.0 Δ WASM 0.9.0 0.10.0 Δ
--print-shape on 50k items 742.2 ms 693.8 ms -6.5% 319.9 ms 290.1 ms -9.3%
filter+length on 50k items 748.5 ms 704.8 ms -5.8% 324.9 ms 285.9 ms -12.0%
group_by(.role) on 1k records 30.7 ms 27.1 ms -11.7% 12.4 ms 11.8 ms -4.8%
Comes from rumil's hot/cold dispatch split (4-6% on synthetic
format benches), compounded through lambé's deeper call chain.
CHANGELOG entry for 0.10.0 covering the breaking library API removal, the frontmatter parser adoption, foreign-idiom redirect coverage, heterogeneous-list description upgrade, error message polish, performance numbers (5-12% AOT, 5-12% WASM on representative workloads), and the rumil_parsers ^0.8.0 → ^0.8.1 dependency bump. jq-compatibility aliases now documented across the canonical surfaces: - `tonumber` (jq alias for `to_number`) — was implemented in 0.9 but undocumented; now appears in jq-to-lambe.md, syntax.md, lam.1.md, AGENTS.md. - `add` (jq alias for `sum`) — same story; same surfaces. `//` documentation tightened in jq-to-lambe.md: it's the null-fallback operator (returns the right-hand side when the left evaluates to null), not an error-handler. The doc previously said "not yet supported" — that was stale; 0.9 implemented it. Skill file (.claude/skills/lambe/SKILL.md) gains a sandbox note: when an agent harness reports `lam: command not found`, the fall-back is the absolute path `~/.pub-cache/bin/lam`. This is a shell-PATH issue, not a lambé issue. Documented so future readers of the skill don't bounce off it. doc/lam.1 regenerated from doc/lam.1.md via tool/manpage.dart so the committed mandoc round-trips cleanly with the source. pubspec bumped to 0.10.0; rumil_parsers constraint bumped to ^0.8.1.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Polish release built on rumil 0.7.1 / rumil_parsers 0.8.1.
dart:io-free.loadSchemaFromFile/loadSchemaForDatamove tobin/schema_io.dart;mergeSchemaWithDatastays in the public API. This is the breaking change of the release — needed for the lambé playground in arda-web to compile to WASM without bridges.parseInput(text, Format.markdown)now usesparseMarkdownWithFrontmatter. Files without frontmatter parse byte-identically to before; files with frontmatter gain afrontmatterkey addressable via.frontmatter.title,.frontmatter.tags[0], etc.getpath/setpath/tojson/fromjson/regex family/@uri/@html/@sh/@json/$ENV.test/match/subno longer produce misleading "did you mean text/map/sum?" guesses.Append one of these stages to the end of your query (keep your existing flags such as -t hcl):with the format name interpolated.--schema <data-file>migration hint. When the argument has a data extension, the error suggestslam --print-shape <file>..jsonlinesextension auto-implies--ndjson(joining.ndjsonand.jsonl).--print-shapeon a mixed array now emitssampled: number, string, boolean, null, array (heterogeneous).tonumberandaddjq-compatibility aliases now documented. Surface was already implemented in 0.9; 0.10 documents it acrossdoc/jq-to-lambe.md,doc/syntax.md,doc/lam.1.md, andAGENTS.md.Performance
End-to-end CLI / library benchmark (median of 7 runs, after warm-up), comparing 0.9.0 (rumil 0.7.0 / rumil_parsers 0.8.0) against this release:
--print-shapeon 50k items.items | filter(.value > 50000) | lengthgroup_by(.role)on 1k recordsThe win comes from rumil's hot/cold dispatch split (4–6% on synthetic format benches; compounded through lambé's deeper call chain). WASM is also the relevant runtime for the lambé playground in arda-web; users get faster live-explain feedback in the browser.
Test plan
dart pub publish --dry-runreports 0 warningslib/has nodart:ioimports)