Skip to content

Commit 145af94

Browse files
Update docs
1 parent 2d3c59a commit 145af94

File tree

8 files changed

+396
-338
lines changed

8 files changed

+396
-338
lines changed

.github/workflows/deploy.yml

Lines changed: 37 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,17 @@
11
# Deploy multilingual docs to GitHub Pages.
22
#
33
# This workflow replaces GitHub's default Jekyll deployment so we can:
4-
# 1. Build the multilingual browser WASM binary via Python + Cranelift.
5-
# 2. Generate the WAT text format alongside it (for educational display).
6-
# 3. Copy both into assets/wasm/ and build the Jekyll site.
4+
# 1. Build the browser WASM bundle from demo.ml via WATCodeGenerator.
5+
# `multilingual build-wasm-bundle demo.ml` emits:
6+
# - module.wat (WebAssembly Text — educational display in docs)
7+
# - module.wasm (binary — executed in the browser REPL)
8+
# - host_shim.js (JS host import stubs for print_str, print_f64, …)
9+
# - abi_manifest.json (ABI metadata)
10+
# 2. Copy all bundle artifacts into assets/wasm/.
11+
# 3. Build the Jekyll site.
712
# 4. Deploy _site/ to GitHub Pages.
813
#
9-
# The WASM build is purely Python-driven: multilingualprogramming[wasm]
10-
# uses the Cranelift compiler (via wasmtime) to produce .wasm binaries.
14+
# The build is purely Python-driven (multilingualprogramming[wasm]).
1115
# No Rust toolchain or wasm-pack is required.
1216
#
1317
# Trigger: every push to main, or manually via workflow_dispatch.
@@ -37,6 +41,9 @@ jobs:
3741
runs-on: ubuntu-latest
3842

3943
steps:
44+
- name: Checkout docs repo (for demo.ml)
45+
uses: actions/checkout@v4
46+
4047
- name: Checkout multilingual source
4148
uses: actions/checkout@v4
4249
with:
@@ -48,47 +55,45 @@ jobs:
4855
with:
4956
python-version: '3.12'
5057
cache: pip
51-
cache-dependency-path: multilingual-src/setup.cfg
58+
cache-dependency-path: multilingual-src/pyproject.toml
5259

5360
- name: Install multilingual with WASM extras
5461
working-directory: multilingual-src
5562
run: pip install -e ".[dev,wasm]"
5663

57-
- name: Build browser WASM bundle
58-
working-directory: multilingual-src
64+
- name: Build browser WASM bundle from demo.ml
5965
run: |
60-
# build_wasm.sh uses WasmGenerator + Cranelift to produce
61-
# .wasm binaries in wasm/pkg/.
62-
# Pass --target browser so the script emits a browser-compatible
63-
# binary (no wasmtime WASI imports).
64-
mkdir -p wasm/pkg
65-
if [ -f build_wasm.sh ]; then
66-
bash build_wasm.sh --target browser --out-dir wasm/pkg
67-
else
68-
python -m multilingualprogramming build-wasm \
69-
--target browser \
70-
--out-dir wasm/pkg
71-
fi
72-
73-
# Install the WebAssembly Binary Toolkit so we can generate the WAT
74-
# (text format) alongside the binary for display in the docs.
75-
- name: Install wabt (wasm2wat)
66+
# WATCodeGenerator compiles demo.ml → WAT text → WASM binary.
67+
# Artifacts land in wasm-out/:
68+
# module.wat, module.wasm, host_shim.js, abi_manifest.json
69+
mkdir -p wasm-out
70+
multilingual build-wasm-bundle demo.ml --out-dir wasm-out
71+
72+
# wabt provides wasm2wat for validating / inspecting the binary.
73+
# The primary .wat is already emitted by build-wasm-bundle above,
74+
# but wasm2wat gives a round-trip check and canonical text form.
75+
- name: Install wabt (wasm-validate, wasm2wat)
7676
run: sudo apt-get install -y wabt
7777

78-
- name: Generate WAT text format
79-
working-directory: multilingual-src/wasm/pkg
78+
- name: Validate and normalise WAT
79+
run: |
80+
wasm-validate wasm-out/module.wasm
81+
# Overwrite with canonical round-trip WAT (catches any encoding drift).
82+
wasm2wat wasm-out/module.wasm -o wasm-out/module.wat
83+
84+
- name: Rename artifacts to stable names
8085
run: |
81-
# Convert every .wasm binary to its human-readable .wat counterpart.
82-
for f in *.wasm; do
83-
wasm2wat "$f" -o "${f%.wasm}.wat" || true
84-
done
86+
# Jekyll references these as assets/wasm/multilingual.*
87+
mv wasm-out/module.wasm wasm-out/multilingual.wasm
88+
mv wasm-out/module.wat wasm-out/multilingual.wat
89+
mv wasm-out/host_shim.js wasm-out/host_shim.js # keep name
90+
mv wasm-out/abi_manifest.json wasm-out/abi_manifest.json
8591
8692
- name: Upload WASM artefacts
8793
uses: actions/upload-artifact@v4
8894
with:
8995
name: wasm-pkg
90-
# Includes both .wasm binaries and .wat text files.
91-
path: multilingual-src/wasm/pkg/
96+
path: wasm-out/
9297
retention-days: 1
9398

9499
# ── Job 2: Build Jekyll ───────────────────────────────────────────────────

_config.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@ title: "Multilingual Programming Language"
22
description: "Write code in 17 human languages. One formal core. Infinite expression."
33
url: "https://multilingualprogramming.github.io"
44
baseurl: "/docs"
5-
version: "0.4.0"
5+
version: "0.5.1"
66

77
# Author
88
author:

assets/js/main.js

Lines changed: 54 additions & 56 deletions
Original file line numberDiff line numberDiff line change
@@ -4,15 +4,25 @@
44

55
/* ================================================================
66
WASM MODULE LOADER
7-
Lazily instantiates assets/wasm/multilingual.wasm — the full
8-
multilingual interpreter compiled to WASM via Python + Cranelift
9-
(multilingualprogramming[wasm]). No wasm-bindgen glue file.
7+
Lazily instantiates assets/wasm/multilingual.wasm — the demo.ml
8+
program compiled to WASM by `multilingual build-wasm-bundle` via
9+
WATCodeGenerator (multilingualprogramming[wasm]).
1010
11-
Expected binary exports:
12-
run_code(ptr: i32, len: i32) -> i32 source in, JSON result out
13-
__wasm_alloc(size: i32) -> i32 linear-memory allocator
14-
__wasm_free(ptr: i32, size: i32) linear-memory free
15-
memory WebAssembly.Memory
11+
The WAT backend uses host-import callbacks for all output rather
12+
than returning values. The browser must supply these imports:
13+
14+
env.print_str(ptr: i32, len: i32) — print a UTF-8 string slice
15+
env.print_f64(val: f64) — print a number
16+
env.print_bool(val: i32) — print True (1) or False (0)
17+
env.print_sep() — print an argument separator (space)
18+
env.print_newline() — print a newline
19+
20+
Output is captured into a string buffer that is reset before each
21+
call to __main().
22+
23+
The Run buttons on individual code examples call __main() and show
24+
the captured output. For a fully interactive experience with
25+
arbitrary code, visit the live playground linked on the WASM page.
1626
1727
The loader resolves the base URL from the <meta name="base-url"> tag
1828
so paths work identically on local dev and under /docs on GitHub Pages.
@@ -23,55 +33,45 @@
2333
const MLWasm = (() => {
2434
let _instance = null;
2535
let _memory = null;
36+
let _outputBuf = '';
2637
let _modulePromise = null;
2738

28-
const enc = new TextEncoder();
2939
const dec = new TextDecoder();
3040

31-
/* Instantiate the binary directly — no wasm-bindgen glue file needed.
32-
* The Cranelift-compiled binary exports:
33-
* run_code(ptr: i32, len: i32) -> i32 (returns ptr to JSON result)
34-
* __wasm_alloc(size: i32) -> i32
35-
* __wasm_free(ptr: i32, size: i32)
36-
* memory (WebAssembly.Memory) */
41+
/* Host imports required by every WAT module produced by WATCodeGenerator. */
42+
const importObject = {
43+
env: {
44+
print_str(ptr, len) {
45+
_outputBuf += dec.decode(new Uint8Array(_memory.buffer, ptr, len));
46+
},
47+
print_f64(val) {
48+
/* Match Python's default float repr: drop trailing .0 for integers. */
49+
_outputBuf += Number.isInteger(val) ? String(val) : String(val);
50+
},
51+
print_bool(val) { _outputBuf += val ? 'True' : 'False'; },
52+
print_sep() { _outputBuf += ' '; },
53+
print_newline() { _outputBuf += '\n'; },
54+
},
55+
};
56+
3757
function load() {
3858
if (_modulePromise) return _modulePromise;
3959

4060
const wasmUrl = baseUrl + '/assets/wasm/multilingual.wasm';
4161

42-
_modulePromise = WebAssembly.instantiateStreaming(fetch(wasmUrl), {
43-
env: { abort: () => {} },
44-
}).then(({ instance }) => {
45-
_instance = instance;
46-
_memory = instance.exports.memory;
47-
return true;
48-
});
62+
_modulePromise = WebAssembly.instantiateStreaming(fetch(wasmUrl), importObject)
63+
.then(({ instance }) => {
64+
_instance = instance;
65+
_memory = instance.exports.memory;
66+
return true;
67+
});
4968

5069
return _modulePromise;
5170
}
5271

53-
/* Write a JS string into WASM linear memory; return { ptr, len }. */
54-
function writeStr(str) {
55-
const bytes = enc.encode(str);
56-
const ptr = _instance.exports.__wasm_alloc(bytes.length + 1);
57-
const view = new Uint8Array(_memory.buffer);
58-
view.set(bytes, ptr);
59-
view[ptr + bytes.length] = 0;
60-
return { ptr, len: bytes.length };
61-
}
62-
63-
/* Read a null-terminated UTF-8 string from WASM linear memory. */
64-
function readStr(ptr) {
65-
const view = new Uint8Array(_memory.buffer);
66-
let end = ptr;
67-
while (view[end] !== 0) end++;
68-
return dec.decode(view.subarray(ptr, end));
69-
}
70-
7172
/* ---- WAT text loader ------------------------------------------------- */
72-
/* Fetches the pre-generated .wat file (produced by wasm2wat in CI) once
73-
* and caches the text. Used purely for display — execution always uses
74-
* the binary. Returns Promise<string>. */
73+
/* Fetches the pre-built .wat file once for educational display.
74+
* Execution always uses the binary — WAT is read-only. */
7575
let _watPromise = null;
7676

7777
function loadWat() {
@@ -86,26 +86,24 @@
8686

8787
/* Public API */
8888
return {
89-
/* Returns a Promise<{ stdout: string, stderr: string }>. */
90-
execute(src) {
89+
/* Run the compiled demo and return { stdout, stderr }.
90+
* The WASM binary is a pre-compiled program (demo.ml); it always
91+
* runs the same code regardless of the `src` argument. The `src`
92+
* parameter is kept for API symmetry with the REPL textarea. */
93+
execute(_src) {
9194
return load().then(() => {
92-
const { ptr, len } = writeStr(src);
93-
let resultPtr;
95+
_outputBuf = '';
9496
try {
95-
resultPtr = _instance.exports.run_code(ptr, len);
96-
} finally {
97-
_instance.exports.__wasm_free(ptr, len + 1);
97+
_instance.exports.__main();
98+
} catch (e) {
99+
return { stdout: _outputBuf, stderr: String(e) };
98100
}
99-
const raw = readStr(resultPtr);
100-
let out;
101-
try { out = JSON.parse(raw); }
102-
catch { out = { stdout: raw, stderr: '' }; }
103-
return { stdout: out.stdout || '', stderr: out.stderr || '' };
101+
return { stdout: _outputBuf, stderr: '' };
104102
});
105103
},
106104
/* Expose the load promise so the toggle button can show readiness. */
107105
get ready() { return load(); },
108-
/* Returns a Promise<string> with the full WAT text. */
106+
/* Returns a Promise<string> with the WAT text format. */
109107
get wat() { return loadWat(); },
110108
};
111109
})();

codegen/api.md

Lines changed: 57 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,7 @@ from multilingualprogramming.codegen import (
3131
WasmGenerator,
3232
RuntimeBuiltins,
3333
)
34+
from multilingualprogramming.codegen.wat_generator import WATCodeGenerator
3435

3536
from multilingualprogramming.core.lowering import lower_to_core_ir
3637
from multilingualprogramming.runtime.backend_selector import (
@@ -396,9 +397,63 @@ exec(python_code, namespace)
396397

397398
---
398399

400+
## WATCodeGenerator
401+
402+
Generates WebAssembly Text Format (WAT) directly from a Core IR AST. This is the primary WASM backend — no Rust toolchain or Cranelift is required.
403+
404+
**File**: `multilingualprogramming/codegen/wat_generator.py`
405+
406+
```python
407+
class WATCodeGenerator:
408+
def __init__(self)
409+
```
410+
411+
### Methods
412+
413+
#### `generate(ast: ASTNode) -> str`
414+
415+
Compile the full program AST to a WAT module string:
416+
417+
```python
418+
from multilingualprogramming.codegen.wat_generator import WATCodeGenerator
419+
420+
gen = WATCodeGenerator()
421+
wat_text = gen.generate(ast)
422+
423+
# Write to file for inspection or wat2wasm assembly
424+
with open("program.wat", "w") as f:
425+
f.write(wat_text)
426+
```
427+
428+
The returned WAT module:
429+
- Imports host callbacks: `env.print_str`, `env.print_f64`, `env.print_bool`, `env.print_sep`, `env.print_newline`
430+
- Exports `__main` as the program entry point
431+
- Exports `memory` for host access to linear memory
432+
433+
### CLI: `build-wasm-bundle`
434+
435+
The high-level command that runs the full pipeline (parse → WATCodeGenerator → wat2wasm → artifact packaging):
436+
437+
```bash
438+
multilingual build-wasm-bundle program.ml --out-dir wasm-out
439+
```
440+
441+
Output artifacts:
442+
443+
| File | Description |
444+
|------|-------------|
445+
| `module.wat` | WAT source text |
446+
| `module.wasm` | Binary WASM for browser |
447+
| `host_shim.js` | JS host import stubs |
448+
| `abi_manifest.json` | ABI metadata |
449+
450+
See [WASM Architecture](/wasm/architecture/) for the full pipeline and host import protocol.
451+
452+
---
453+
399454
## WasmGenerator
400455

401-
Generates WebAssembly bytecode from a Core IR AST (requires the WASM optional dependency).
456+
Legacy generator that produces Rust intermediate code, which is then compiled to WASM externally. Requires a Rust toolchain. For most use cases, prefer `WATCodeGenerator` via `multilingual build-wasm-bundle`.
402457

403458
```python
404459
class WasmGenerator:
@@ -409,30 +464,25 @@ class WasmGenerator:
409464

410465
#### `generate_wasm(ast, function_name: str) -> bytes`
411466

412-
Compile a function from the AST to WASM bytecode:
413-
414467
```python
415468
from multilingualprogramming.codegen import WasmGenerator
416469

417470
wasm_gen = WasmGenerator()
418471
wasm_bytes = wasm_gen.generate_wasm(ast, function_name="fibonacci")
419472

420-
# Write to file
421473
with open("fibonacci.wasm", "wb") as f:
422474
f.write(wasm_bytes)
423475
```
424476

425477
#### `generate_rust(ast, function_name: str) -> str`
426478

427-
Get the Rust intermediate representation (before WASM compilation):
479+
Get the Rust intermediate representation:
428480

429481
```python
430482
rust_code = wasm_gen.generate_rust(ast, function_name="fibonacci")
431483
print(rust_code)
432484
```
433485

434-
See [WASM Architecture](/wasm/architecture/) for details on the compilation chain.
435-
436486
---
437487

438488
## BackendSelector

0 commit comments

Comments
 (0)