Skip to content

Commit 5162fc6

Browse files
johnnohjjohnnohj
committed
Continued changes
1 parent 8657ad5 commit 5162fc6

14 files changed

Lines changed: 800 additions & 38 deletions

‎ports/wasm/ARCHITECTURE.md‎

Lines changed: 115 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,115 @@
1+
# CircuitPython WASM Port Architecture
2+
3+
## CircuitPython Configuration Layers (From Bottom Up)
4+
5+
### Layer 1: Core Python (`py/`)
6+
- `py/circuitpy_mkenv.mk` - CircuitPython environment setup
7+
- `py/circuitpy_defns.mk` - Automatic module/source discovery based on CIRCUITPY_* flags
8+
- `py/py.mk` - Core Python source files
9+
- `py/mkrules.mk` - Build rules
10+
11+
### Layer 2: Port Configuration (`ports/webassembly/`)
12+
- `Makefile` - Port-specific build orchestration
13+
- `mpconfigport.h` - Port-wide Python configuration (what Python features are enabled)
14+
- Port source files (`mphalport.c`, `main.c`, etc.)
15+
16+
### Layer 3: Board Configuration (`ports/webassembly/boards/BOARD/`)
17+
- `mpconfigboard.h` - Board hardware configuration (pins, peripherals)
18+
- `mpconfigboard.mk` - Which CircuitPython modules to enable (CIRCUITPY_ANALOGIO=1, etc.)
19+
- `pins.c` - Board pin definitions
20+
- `board.c` - Board-specific initialization
21+
22+
### Layer 4: WASM-Specific Integration
23+
**This is where we need clarity!**
24+
25+
Current files without clear layer assignment:
26+
- `shared_memory.c/h` - Virtual hardware state (belongs at port level)
27+
- `virtual_hardware.c/h` - Hardware simulation (belongs at port level)
28+
- `background.c/h` - Background task coordination (belongs at port level)
29+
- JavaScript files (api.js, virtual_clock.js, etc.) - WASM loader/runtime
30+
31+
## Full CircuitPython Port Requirements
32+
33+
### A. Supervisor Layer (`supervisor/`)
34+
**We have:**
35+
- ✅ `supervisor/port.c` - Port initialization, timing, idle
36+
- ✅ `supervisor/shared/tick.c` - Timing system
37+
- ✅ `supervisor/shared/background_callback.c` - Background tasks
38+
- ✅ `supervisor/shared/translate/translate.c` - String compression/decompression
39+
40+
**We're missing:**
41+
- ❌ Heap management (`port_heap_init`, `port_malloc`, `port_free`)
42+
- ❌ Boot sequence integration (boot.py, code.py auto-run)
43+
- ❌ USB/Serial integration
44+
- ❌ Safe mode handling
45+
- ❌ CircuitPython REPL prompt
46+
47+
### B. Hardware Abstraction (`common-hal/`)
48+
**We have:**
49+
- ✅ analogio (AnalogIn, AnalogOut)
50+
- ✅ busio (I2C, SPI, UART)
51+
- ✅ digitalio (DigitalInOut)
52+
- ✅ microcontroller (Pin, Processor, cpu)
53+
- ✅ time (monotonic, sleep)
54+
55+
**Integration status:**
56+
- ⚠️ Hardware is defined but NOT connected to virtual_hardware simulation
57+
- ⚠️ No GPIO state actually changes in JavaScript when Python calls GPIO functions
58+
59+
### C. Module Ecosystem
60+
**Currently enabled (7 modules):**
61+
- analogio, board, busio, digitalio, microcontroller, time, zlib
62+
63+
**Typical full build (40+ modules):**
64+
- USB (usb_cdc, usb_hid, usb_midi)
65+
- Storage (storage, os, sys)
66+
- Display (displayio, vectorio, terminalio)
67+
- Networking (wifi, socketpool, ssl)
68+
- Audio (audiocore, audioio)
69+
- etc.
70+
71+
## Key Integration Gaps
72+
73+
### 1. Virtual Hardware Not Wired Up
74+
```c
75+
// common-hal/digitalio/DigitalInOut.c does:
76+
void common_hal_digitalio_digitalinout_set_value(digitalio_digitalinout_obj_t *self, bool value) {
77+
// This writes to some local state...
78+
// But virtual_gpio_set_output_value() is NEVER called!
79+
// So JavaScript never sees the GPIO change!
80+
}
81+
```
82+
83+
### 2. Timing System Incomplete
84+
- `port_get_raw_ticks()` reads from `virtual_clock_hw.ticks_32khz`
85+
- But supervisor timing may not be properly using this
86+
- Need to verify timing integration end-to-end
87+
88+
### 3. Memory Management Missing
89+
- Using Emscripten's malloc instead of CircuitPython's managed heap
90+
- No `gc_collect()` integration with port heap
91+
- No heap statistics
92+
93+
### 4. Boot Sequence Missing
94+
- No supervisor boot (boot.py, code.py)
95+
- No USB/serial console
96+
- No CircuitPython REPL prompt (just bare MicroPython REPL)
97+
98+
## Questions to Answer
99+
100+
1. **Board vs Variant**: Do we even need board abstraction for WASM? Or should WASM be a "port" with different "configurations" (similar to Unix port variants)?
101+
102+
2. **Virtual Hardware Ownership**: Should `virtual_hardware.c` be:
103+
- At port level (webassembly/virtual_hardware.c) ✓ current
104+
- Part of common-hal (webassembly/common-hal/virtual/hardware.c)
105+
- Shared infrastructure (supervisor/virtual_hardware.c)
106+
107+
3. **JavaScript Integration**: Where should the JS boundary be?
108+
- Current: Scattered (mphalport exports, library.js exports, etc.)
109+
- Better: Defined interface layer?
110+
111+
4. **Configuration Philosophy**:
112+
- MicroPython approach: Variants for different configs (standard, minimal, etc.)
113+
- CircuitPython approach: Boards for different hardware
114+
- WASM needs: Different simulation configurations?
115+

‎ports/wasm/DESIGN.md‎

Lines changed: 170 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,170 @@
1+
# CircuitPython WASM Port Design Philosophy
2+
3+
## The Triple Nature of WASM
4+
5+
WASM is simultaneously three things that are separate in hardware:
6+
7+
| Layer | Hardware | WASM |
8+
|-------|----------|------|
9+
| **Port** | ARM Cortex-M, RISC-V, etc. | Emscripten/WebAssembly compilation |
10+
| **Silicon** | The actual chip (RP2040, SAMD51, etc.) | Virtual hardware simulation (`virtual_hardware.c`) |
11+
| **Board** | PCB with specific pins/peripherals wired | Configuration of which peripherals exist |
12+
13+
In hardware, these are physically separate:
14+
- **Port**: The CPU architecture (different compiler, different instructions)
15+
- **Silicon**: The actual chip design (GPIO registers at specific addresses)
16+
- **Board**: How the chip is wired on a PCB (which pins go where)
17+
18+
**In WASM, these collapse into one:**
19+
- We compile to WASM (port)
20+
- We simulate the chip (silicon)
21+
- We configure what exists (board)
22+
23+
All in the same codebase.
24+
25+
## Proposed Layer Architecture
26+
27+
### Layer 1: Port (`ports/webassembly/`)
28+
**Responsibility**: WASM/Emscripten-specific infrastructure that would be the same regardless of what we're simulating.
29+
30+
**Files:**
31+
- `Makefile` - Build orchestration (Emscripten flags, JS library integration)
32+
- `mpconfigport.h` - Python VM configuration for WASM
33+
- `mphalport.c/h` - HAL for WASM/Emscripten (timing, stdout, etc.)
34+
- `main.c` - Entry point, REPL setup
35+
- **C/JS Boundary:**
36+
- `library.js` - Emscripten library functions (exported to WASM)
37+
- `api.js` - JavaScript API (what developers use)
38+
39+
### Layer 2: Silicon (`ports/webassembly/silicon/`)
40+
**Responsibility**: The virtual hardware simulation - the "chip" we're simulating.
41+
42+
**Files:**
43+
- `virtual_hardware.c/h` - Hardware simulation (GPIO, I2C, SPI, UART state)
44+
- `shared_memory.h` - C/JS shared memory layout (the "hardware registers")
45+
- `virtual_clock.c/h` - Virtual timing system
46+
- **Common-HAL implementations** (`common-hal/*/`)
47+
- These call into virtual_hardware functions
48+
- This is where Python `pin.value = True` becomes `virtual_gpio_set_value()`
49+
50+
**JavaScript Runtime Integration:**
51+
- `virtual_clock.js` - JS side of timing
52+
- `virtual_gpio.js` - JS side of GPIO (future)
53+
- `virtual_i2c.js` - JS side of I2C simulation (future)
54+
55+
### Layer 3: Variant (`ports/webassembly/variants/VARIANT/`)
56+
**Responsibility**: Different simulation configurations, NOT different physical boards.
57+
58+
**Variants we might have:**
59+
- `standard/` - Full CircuitPython with all common modules
60+
- `minimal/` - Bare minimum for testing (like MicroPython unix/minimal)
61+
- `display/` - Standard + displayio for graphics testing
62+
- `networking/` - Standard + wifi/socketpool for network testing
63+
- `sensors/` - Standard + common sensor libraries pre-loaded
64+
65+
**Files in each variant:**
66+
- `mpconfigvariant.mk` - Which CircuitPython modules to enable
67+
- `mpconfigvariant.h` - Variant-specific Python VM settings (if needed)
68+
69+
**NO board-specific pins.c** - Pins are virtual, defined once at silicon level.
70+
71+
## The C/JS Boundary
72+
73+
This is critical and currently unclear. Here's the proposed clean separation:
74+
75+
### C Side Exports (WASM → JavaScript)
76+
**In `library.js` (Emscripten `--js-library`):**
77+
```c
78+
// These are the ONLY functions C calls into JavaScript:
79+
EM_JS(void, js_gpio_changed, (int pin, int value), {
80+
// Notify JS that GPIO changed
81+
});
82+
83+
EM_JS(void, js_i2c_transaction, (int address, uint8_t* data, int len), {
84+
// Notify JS of I2C transaction
85+
});
86+
```
87+
88+
### JavaScript Side Exports (JavaScript → WASM)
89+
**In `api.js` (loaded via `--pre-js`):**
90+
```javascript
91+
// These are the ONLY functions JavaScript calls into C:
92+
export async function loadCircuitPython(options) {
93+
const Module = await _createCircuitPythonModule();
94+
95+
return {
96+
// High-level API
97+
runPython: (code) => Module.ccall('mp_js_do_str', ...),
98+
runFile: (path) => Module.ccall('mp_js_do_file', ...),
99+
100+
// Hardware access (read-only from JS)
101+
getGPIOValue: (pin) => Module._virtual_gpio_get_value(pin),
102+
getI2CState: () => Module._virtual_i2c_get_state(),
103+
104+
// Virtual clock control (JS drives the clock)
105+
advanceTime: (ms) => Module._virtual_clock_advance(ms),
106+
107+
// Filesystem
108+
FS: Module.FS,
109+
};
110+
}
111+
```
112+
113+
## Ownership and Responsibility
114+
115+
| Component | Owner | Responsibility |
116+
|-----------|-------|----------------|
117+
| **CircuitPython VM** | CircuitPython core | Python execution |
118+
| **Supervisor** | CircuitPython core | Boot sequence, timing, safe mode |
119+
| **Common-HAL** | Port (webassembly) | Hardware abstraction API implementation |
120+
| **Virtual Hardware** | Port (silicon layer) | Simulated chip state |
121+
| **Shared Memory** | Port (silicon layer) | C/JS data bridge |
122+
| **JavaScript Runtime** | Port (api.js) | Browser/Node.js integration |
123+
| **Virtual Devices** | TypeScript/JS | Simulated I2C devices, displays, etc. |
124+
125+
## Build Flow
126+
127+
```
128+
User runs: make VARIANT=standard
129+
130+
1. circuitpy_mkenv.mk loads variant configuration
131+
2. mpconfigvariant.mk sets CIRCUITPY_* flags
132+
3. circuitpy_defns.mk discovers modules to build based on flags
133+
4. Port Makefile compiles:
134+
- CircuitPython core (py/)
135+
- Supervisor (supervisor/)
136+
- Common-HAL implementations (common-hal/)
137+
- Virtual hardware (silicon/)
138+
- Port infrastructure (mphalport.c, etc.)
139+
5. Emscripten links everything to circuitpython.wasm
140+
6. api.js wraps WASM in JavaScript API
141+
7. Output: build-VARIANT/circuitpython.mjs + circuitpython.wasm
142+
```
143+
144+
## Why Variant, Not Board?
145+
146+
**Board abstraction implies:**
147+
- Different physical hardware layouts
148+
- Different pin mappings (board.D0 on Pico vs Feather)
149+
- Different peripheral availability (this board has WiFi, that doesn't)
150+
151+
**Variant abstraction implies:**
152+
- Different feature sets of the same virtual platform
153+
- Different module combinations for different use cases
154+
- Same "hardware" (all 64 GPIO, all peripherals), just different software builds
155+
156+
For WASM, we're NOT representing different physical boards. We're representing different **software configurations** of the same virtual platform. That's a variant, not a board.
157+
158+
## Next Steps
159+
160+
1. Restructure to variant-based build
161+
2. Move virtual_hardware to `silicon/` subdirectory for clarity
162+
3. Define clean C/JS boundary with documented exports
163+
4. Create multiple variants (minimal, standard, display)
164+
5. Document the "silicon specification" - what hardware we're simulating
165+
166+
This gives us:
167+
- **Clear ownership** - Who owns each layer
168+
- **Clean boundaries** - Well-defined interfaces between layers
169+
- **Honest naming** - We're not pretending to be physical boards
170+
- **Flexibility** - Easy to add new simulation features or variants

‎ports/wasm/Makefile‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -67,7 +67,7 @@ EXPORTED_FUNCTIONS_EXTRA += ,\
6767
_board_serial_input_available,\
6868
_board_serial_set_output_callback,\
6969
_board_serial_repl_process_string,\
70-
_get_virtual_hardware_ptr,\
70+
_get_virtual_clock_hw_ptr,\
7171
_virtual_gpio_set_input_value,\
7272
_virtual_gpio_get_output_value,\
7373
_virtual_gpio_get_direction,\

0 commit comments

Comments
 (0)