Add RustPython generated WASM WS module#63
Draft
jayvdb wants to merge 88 commits into
Draft
Conversation
Two changes in one commit so the next CI run isolates them together: - build-ws-wasm-agent: drop `RUSTUP_TOOLCHAIN=nightly` + `-Z build-std= std,panic_abort` and let the build use the workspace default stable toolchain. The custom-target rustflags (mvp + mutable-globals + sign-ext + nontrapping-fptoint) stay; without build-std the precompiled std doesn't get the mvp target-feature treatment, but the agent doesn't need that for correctness. This also unblocks the Windows cmd-PATH issue we hit on 6092b62 (only fired when mise swapped to the nightly rust exec_env on a bash-parent task spawn). - PR matrix re-includes ubuntu-latest alongside windows-latest so we can tell apart "real config regression" (both fail) from "Windows- specific mise PATH/exec_env interaction" (only Windows fails). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Stable rust + the trimmed wasm target (mvp resets every feature; we restore mutable-globals, sign-ext, nontrapping-fptoint) means the precompiled std doesn't carry reference-types either. wasm-bindgen 0.2.122 then can't find `__wbindgen_externref_table_dealloc` when post-processing the .wasm: error: failed to find the `__wbindgen_externref_table_dealloc` function Error: Running the wasm-bindgen CLI Captured on 177898f, ubuntu-latest job 82263633026. Add +reference-types so the externref tables wasm-bindgen depends on are emitted. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Even without `RUSTUP_TOOLCHAIN=nightly`, a task-level `[tasks.build-ws-wasm-agent.env]` block on Windows triggered mise's per-task env-composition path that drops the `%LOCALAPPDATA%\mise\shims` directory from PATH, so the cmd subprocess can't find wasm-pack: $ cmd /s /c "wasm-pack build . --target web --no-opt" 'wasm-pack' is not recognized as an internal or external command Other build-ws-*-module tasks (no env override) found wasm-pack in the same step. Move `CARGO_TARGET_WASM32_UNKNOWN_UNKNOWN_RUSTFLAGS` to global [env]. It's wasm32-target-scoped (won't leak to host builds), so every build-ws-*-module that targets wasm32-unknown-unknown picks up the same trimmed mvp+features value -- harmless for them, fixes the PATH dropout on wasm-agent. Captured on 3a1e0c8, windows job 82264573380. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Pattern discovered on 0490dbc (Windows job 82266241297): every build-ws-*-module task in config.rust.toml finds wasm-pack via the shim dir under bash-parent steps; only build-ws-wasm-agent (defined in base config.toml) fails with `'wasm-pack' is not recognized`. mise's child-env composition on Windows apparently drops the `%LOCALAPPDATA%\mise\shims` PATH entry for base-config tasks spawned under a bash-parent step, but keeps it for tasks declared in a language-env config. This was already flagged as a migration in config.rust.toml's header comment ("moving those out is a later pass") -- we're doing the later pass now to unblock the bisect. Also drops the stale RUSTFLAGS rationale comment from the task -- the wasm_rustflags var is hoisted to global [env] and uses the target-scoped CARGO_TARGET_WASM32_UNKNOWN_UNKNOWN_RUSTFLAGS, so the "leaks into wasm-bindgen-cli host build" concern no longer applies. The build-modules glob `build-ws-*` in base config still picks up the task because it's merged into mise's task table when MISE_ENV includes `rust` -- which every workflow + Dockerfile uses. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Same pattern as build-ws-wasm-agent on d5bb7e0: every config.rust.toml task finds cargo via the shim dir under a bash-parent step; only the base-config build-et-cli task failed with `'cargo' is not recognized` in the cmd subprocess. Moving the task into the rust env config keeps the PATH entry that mise's per-env composition apparently preserves. Captured on d5bb7e0, windows job 82267796152. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
et-cli is a workspace-wide prereq: dart, js, python, and rust env build-ws-*-module tasks all `depends = ["build-et-cli"]`, so the task has to live in the always-loaded base config (not config.rust.toml, which only loads when MISE_ENV includes rust). To work around the Windows base-config + bash-parent PATH dropout that made bare `cargo` fail in the cmd subprocess (windows job 82267796152 on d5bb7e0), set `shell = "bash -euo pipefail -c"` on the task. Git Bash sources its own PATH from mise's shim dir, so `cargo` resolves cleanly there. Pure-cargo invocation -- no nested wasm-pack -> cargo metadata chain that would trip Git Bash's child- process cargo lookup. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Job-level `defaults.run.shell` is now conditional on matrix.os:
- Linux: `bash --noprofile --norc -euo pipefail {0}` (workflow default)
- Windows: `C:\Users\runneradmin\AppData\Local\mise\installs\http-
busybox\1.37.0\ash.exe --noprofile --norc -euo pipefail {0}`
The Windows Docker images already use this busybox-w32 ash as the
shell mise tasks invoke via `MISE_BASH_PATH`. Reusing it as the
workflow step shell avoids Git Bash's MSYS2 fork emulation + PATH
translation, which is what made mise's child-cmd env composition
drop the `%LOCALAPPDATA%\mise\shims` PATH entry for base-config
tasks (the `'wasm-pack' is not recognized` / `'cargo' is not
recognized` chain).
`Set HOME = USERPROFILE on Windows` explicitly overrides back to
`shell: bash` because it runs BEFORE `Install mise + tools` -- the
step that installs http:busybox in `_setup_all`. All later
run-bodies use ash, which is enough for `mise run ...` invocations.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Captured on 4d703cf windows job 82271660399: the `Diagnostic: prefetch fetch-mnist-rclone (bash + xtrace)` step exited with code 2 before any of its body ran, because busybox-w32 ash doesn't accept GNU bash's `--noprofile --norc` long flags. ash doesn't read profile files in non-login mode anyway, so just drop the flags. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
`disable_backends = ["vfox"]` kills the `DEBUG [vfox] Getting metadata for yarn` noise that swamps MISE_DEBUG=1 logs. Source: mise's `load_tools` proactively walks the registry and instantiates every tool whose registry entry declares `idiomatic_files`. yarn's registry sets `idiomatic_files = ["package.json"]` and its first backend is `vfox:mise-plugins/vfox-yarn`, so each `mise run` re-creates the vfox Plugin and runs `Plugin::load()` per call -- log line per call. Only tool in our `[tools]` that defaults to vfox is `chromedriver`, which has `asdf:mise-plugins/mise-chromedriver` as its second backend -- mise falls through to that automatically. `idiomatic_version_file_enable_tools = []` is mise's default, but making it explicit guards against a future bump flipping the default on. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Only consumers are `test-ws-wasm-agent-chrome` / `ws-e2e-chrome`, neither in the standard CI `test` flow. With chromedriver gone, the last [tools] entry that defaulted to vfox is out, so the `disable_backends = ["vfox"]` in [settings] has no [tools] fallout. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Windows + ubuntu both green at 914c143. Two changes: - test.yaml MISE_ENV: rust,python,zig,dotnet,js (next bisect step). js pulls in npm:onnxruntime-web + npm:pnpm (macos/x64) + the har1/face-detection module builds. - docker-windows.yaml: drop `if: false` and narrow the PR matrix to the single windows-2025 / servercore entry while we ramp back up. Full four-entry matrix kept commented for restoration. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The augeas / dart-typegen / gnupg-w32 jobs need to run on pull_request so we can iterate before they hit main -- workflow_dispatch isn't selectable on branches not yet on main. The `pull_request` trigger already filters by paths to just upstream-cache.yaml + install-mise/**, so a typical PR doesn't accidentally trigger the heavy build matrix. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…as knobs
- test.yaml: MISE_ENV expands to include java
- config.java.toml: pin java to 26.0.1 (matches the existing maven pin pattern)
- config.windows.toml: [env] JAVA_HOME -- mise's java plugin exports it on
Linux/macOS via shell activation, but `mise run <task>` on Windows doesn't
propagate the export, so mvn.cmd aborts with "JAVA_HOME environment variable
is not defined correctly" (servercore job 82284035647)
- config.windows.toml: [tools."http:dart-typegen"] consumes the dart-typegen-v1
release asset (built by upstream-cache.yaml's dart-typegen job)
- config.dart.toml: os-scope cargo:dart-typegen to non-Windows (Windows uses
the http:dart-typegen prebuilt above)
- upstream-cache.yaml/augeas: lift pkgx's augeas.net recipe knobs --
autoreconf --force --install, CFLAGS=-Wno-implicit-function-declaration,
--disable-debug; add `make` to the MSYS2 install list; replace banned
trailing-backslash DLL list with shell-var concatenation
- upstream-cache.yaml/gnupg-w32: revert to system curl + 7z (preinstalled on
ubuntu-latest); the mise:maint route via xh + 7zz tripped mise 2026.6.5's
aqua install-dir resolution
- Dockerfile.{windows,nanoserver}: drop cargo:dart-typegen from
MISE_DISABLE_TOOLS now that it's os-scoped
- config/upstream-cache.toml: real sha256 for dart-typegen 0.1.13
- CLAUDE.md: add "use pkgx pantry recipe as canonical source" to the
upstream-cache entry guide
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Removing the cargo:dart-typegen MISE_DISABLE_TOOLS override left an `env:` key with only comments and no real entries, which fails the workflow parse with /jobs/rust/env "value must be object or string". Drop the whole block. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The gnupg-w32 job's first successful run uploaded gnupg-w32-2.5.20_20260513-x86_64-pc-windows.tar.gz to the gnupg-w32-v1 release. Record its sha256 in config/upstream-cache.toml and add a matching [tools."http:gnupg-w32"] entry in config.windows.toml so mise installs it on Windows hosts. Also: allowlist cargo:dart-typegen in the mise os-scope policy. The tool is os-scoped to linux/macos because the Windows lane uses http:dart-typegen, but checksums.rego (which can't cross-reference the two entries) was flagging the os-scope as a coverage violation. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
augeas's headers use bare `uint` (info.h:64, augeas.c:1210, etc.) but mingw-w64's <sys/types.h> doesn't define that typedef -- u_int is the portable name. Preprocessor rewriting via -Duint=unsigned avoids patching augeas itself. Failure observed in job 82288094830 on cfa309c. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
augtool.c uses pwd.h's getpwuid_r() + getuid() to find the user's home dir for ~/.augeas readline history. mingw-w64 ships neither. Wrap the lookup in _WIN32 guards and fall back to the existing $HOME / $USERPROFILE env-var path. Patch lives at config/augeas/win32-augtool.patch and is applied by the augeas job before autoreconf. Also: split the configure curl line out of the >120-char trap; lift checksums.rego's sprintf template to module scope (regal's non-loop-expression rule). Failure observed in job 82288834608 on 93a198a. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- config/upstream-cache.toml -> config/upstream-cache/data.toml - config/augeas/ -> config/upstream-cache/patches/ Updates references in .editorconfig, .mise/config*.toml, the upstream-cache.yaml workflow, the checksums.rego policy, and CLAUDE.md. Also adds config/upstream-cache/patches/** to the workflow's pull_request.paths so an edited patch retriggers the build. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Both shows as "skipped" via `if: false` on their sole job, so PR runner capacity goes to the upstream-cache.yaml augeas job we're currently iterating on. Restore by removing the `if: false` line in each workflow. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Extends config/upstream-cache/patches/win32-augtool.patch:
- internal.h gets a `typedef unsigned int uint;` under `#ifdef _WIN32`,
which propagates to every .c through the existing #include "internal.h"
chain (info.h, augeas.c, augrun.c, fa.c, get.c, lens.c, pathx.c, ...
all use bare `uint`). Replaces the previous `-Duint=unsigned` CFLAG,
which collided with autoconf's auto-generated `typedef ... uint;` in
config.h ("error: two or more data types in declaration specifiers"
at config.h:2202 on job 82290162010).
- augtool.c: `#include <pwd.h>`, `struct sigaction`, `sigemptyset()`,
`sigaction()`, and the sigint_handler() function all get wrapped in
`#ifndef _WIN32` -- mingw-w64 ships none of them. Ctrl-C on Windows
now no-ops in augtool (acceptable trade for the readline-history
fallback path; the alternative was a Win32 SetConsoleCtrlHandler
shim that would have doubled the patch size for a niche tool).
Also drops `-Duint=unsigned` from the workflow CFLAGS (the typedef
above does the same job without colliding with config.h).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
`patch -p1` reads the git-style `index <pre>..<post>` line and a
leading-zero pre-hash means "this file does not exist yet, create
it". When src/internal.h DID exist, patch refused the hunk:
The next patch would create the file src/internal.h, which
already exists! Assume -R? [n]
Apply anyway? [n]
Skipping patch.
(job 82293824097 on 68e7fbc.) Use the real release-1.14.1 blob hash
(`482648a6`) on the pre side so patch treats it as a modify, not a
create.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
syntax.c uses glob()/globfree() to enumerate `<dir>/*.aug` lens files
in interpreter_init -- mingw-w64 has no <glob.h>:
syntax.c:29:10: fatal error: glob.h: No such file or directory
(job 82294486522 on d0a172c.)
Extends config/upstream-cache/patches/win32-augtool.patch with two
hunks:
- New file src/win32-glob-shim.h with a minimal Win32 implementation
(FindFirstFileA / FindNextFileA), covering just the API surface
augeas uses: GLOB_NOSORT / GLOB_APPEND / GLOB_NOMATCH return codes,
gl_pathc + gl_pathv fields, single-directory single-wildcard
patterns.
- syntax.c: replace `#include <glob.h>` with a platform switch that
picks the shim under `_WIN32` and keeps the real <glob.h> elsewhere.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
MSYS2 bash interprets the Windows-form \$RUNNER_TEMP (`D:\a\_temp`) as
escape sequences -- `\a` -> BEL, `\_` -> `_` -- so paths flowed into
libtool as `D:a_temp/install/lib`:
libtool: error: argument to -rpath is not absolute:
D:a_temp/install/lib
(job 82331162358 on 338ad82.) Convert via `cygpath -m` so the
Windows-form path uses forward slashes (`D:/a/_temp`), which is
absolute AND has no backslashes for bash to misread.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The build, link and install all succeeded on 79108ae -- only the final packaging step bombs because tar (Git Bash / MSYS2) parses `D:\a\...` as `host:path`: tar (child): Cannot connect to D: resolve failed tar: D:\a\_temp/augeas-1.14.1-x86_64-pc-windows-mingw.tar.gz: Cannot write: Broken pipe (job 82334098051.) Switch to the same `(cd dir && tar ...)` shape the dart-typegen job uses, but referencing the cygpath -m converted $runner_temp_m so the output path is forward-slash absolute and tar parses it as a filename instead of a host. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
cygpath -m gave us forward slashes, but GNU tar still parses any colon
in a path as `host:path` (rsh remote spec):
tar: D\:/a/_temp/augeas-1.14.1-x86_64-pc-windows-mingw.tar.gz:
Cannot write: Broken pipe
(job 82336918318 on 2b81850.) --force-local disables that parsing so
absolute Windows-form paths flow through as plain filenames.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
augeas-v1 release now ships `augeas-1.14.1-x86_64-pc-windows-mingw.tar.gz` (sha 6d97229c…); record the sha in config/upstream-cache/data.toml and register `[tools."http:augeas"]` in .mise/config.windows.toml so Windows hosts mise-install it. Use augtool with the Toml lens to do the in-place .mise/config.toml + config/upstream-cache/data.toml edits the publish-* maint tasks need. Replaces cargo:toml-cli, which had no Windows quickinstall asset (was on the MISE_DISABLE_TOOLS Windows carve-out + a config.windows.toml install_env override, both of which are no longer needed). Linux/macOS workstations are expected to have augtool on PATH (e.g. `apt install augeas-tools`, `brew install augeas`); the rp-native publish task still self-skips its config.toml edits when augtool is absent and prints the values to paste manually. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.