Skip to content

Commit 28e8ec6

Browse files
nev-offloadnev-offload
committed
Add host:// import scheme design and spec
1 parent 8cd4bfa commit 28e8ec6

2 files changed

Lines changed: 619 additions & 0 deletions

File tree

‎HOST_FUNCTIONS_DESIGN.md‎

Lines changed: 221 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,221 @@
1+
# Host Functions Design
2+
3+
## Overview
4+
5+
Host functions are capabilities provided by the runtime, not implemented in JavaScript. They represent interfaces that the host must fulfill. By making this explicit with a `host://` import scheme, we:
6+
7+
1. **Clarify the runtime contract** — What the host must provide
8+
2. **Enable alternative hosts** — Different runtimes can implement the same interfaces
9+
3. **Separate concerns** — Pure library code vs host-dependent code
10+
4. **Make dependencies visible** — Easy to see what host capabilities a module needs
11+
12+
## Current Host Operations
13+
14+
These are the Rust ops currently exposed to JavaScript:
15+
16+
### File System (`host://fs`)
17+
- `readFile(path: string): Promise<string>`
18+
- `readFileBinary(path: string): Promise<Uint8Array>`
19+
- `writeFile(path: string, content: string): Promise<void>`
20+
- `writeFileBinary(path: string, content: Uint8Array): Promise<void>`
21+
- `isFile(path: string): Promise<boolean>`
22+
- `exists(path: string): Promise<boolean>`
23+
- `lstat(path: string): Promise<FileStat>`
24+
- `mkdir(path: string): Promise<void>`
25+
- `readdir(path: string): Promise<string[]>`
26+
- `tmpdir(): string`
27+
28+
### HTTP (`host://http`)
29+
- `fetch(url: string, options?: RequestInit): Promise<Response>`
30+
31+
### HTTP Server (`host://http/server`)
32+
- `serve(options: ServeOptions): Server`
33+
34+
### Process (`host://process`)
35+
- `spawn(cmd: string[], options?: SpawnOptions): Process`
36+
37+
### Timers (`host://time`)
38+
- `setTimeout(callback: () => void, ms: number): number`
39+
- `clearTimeout(id: number): void`
40+
- `setInterval(callback: () => void, ms: number): number`
41+
- `clearInterval(id: number): void`
42+
43+
### Watch (`host://watch`)
44+
- `watchFile(path: string): AsyncIterable<WatchEvent>`
45+
- `watchDirectory(path: string): AsyncIterable<WatchEvent>`
46+
47+
### Crypto (`host://crypto`)
48+
- `randomBytes(length: number): Uint8Array`
49+
50+
### Console (`host://console`)
51+
- `log(...args: any[]): void`
52+
- `debug(...args: any[]): void`
53+
54+
## URL Scheme Design
55+
56+
```
57+
host://<namespace>/<optional-path>
58+
```
59+
60+
### Proposed Namespaces
61+
62+
| Namespace | Description | Exports |
63+
|-----------|-------------|---------|
64+
| `host://fs` | File system operations | readFile, writeFile, isFile, exists, lstat, mkdir, readdir, tmpdir |
65+
| `host://http` | HTTP client | fetch |
66+
| `host://http/server` | HTTP server | serve |
67+
| `host://process` | Subprocess spawning | spawn |
68+
| `host://time` | Timers | setTimeout, clearTimeout, setInterval, clearInterval |
69+
| `host://watch` | File watching | watchFile, watchDirectory |
70+
| `host://crypto` | Cryptographic utilities | randomBytes |
71+
| `host://console` | Logging | log, debug |
72+
73+
### Import Examples
74+
75+
```typescript
76+
// Before (importing from "funee")
77+
import { readFile, writeFile } from "funee";
78+
import { fetch } from "funee";
79+
import { serve } from "funee";
80+
import { spawn } from "funee";
81+
import { log, debug } from "funee";
82+
83+
// After (importing from host://)
84+
import { readFile, writeFile, isFile } from "host://fs";
85+
import { fetch } from "host://http";
86+
import { serve } from "host://http/server";
87+
import { spawn } from "host://process";
88+
import { log, debug } from "host://console";
89+
import { setTimeout, clearTimeout } from "host://time";
90+
import { watchFile, watchDirectory } from "host://watch";
91+
import { randomBytes } from "host://crypto";
92+
```
93+
94+
## What Stays in "funee"?
95+
96+
The `"funee"` import becomes pure JavaScript/TypeScript library code:
97+
98+
```typescript
99+
import {
100+
// Macros (compile-time, not host functions)
101+
closure, canonicalName, definition,
102+
Closure, CanonicalName, Definition, createMacro,
103+
104+
// Assertions (pure JS)
105+
assertThat, is, not, both, contains, matches,
106+
greaterThan, lessThan,
107+
108+
// Testing framework (pure JS, uses host://time internally)
109+
scenario, runScenarios, runScenariosWatch,
110+
111+
// Streams/axax (pure JS async iterables)
112+
fromArray, toArray, map, filter, reduce, pipe,
113+
114+
// Utilities (pure JS)
115+
join, // path joining is pure string manipulation
116+
} from "funee";
117+
```
118+
119+
## Implementation Plan
120+
121+
### Phase 1: Bundler Support for `host://` URLs
122+
123+
1. **Modify `load_module.rs`** to recognize `host://` scheme
124+
2. **Create host module stubs** that re-export from runtime bootstrap
125+
3. **Update `http_loader.rs`** to handle `host://` specially (not HTTP fetch)
126+
127+
### Phase 2: Create Host Module Definitions
128+
129+
Create TypeScript declaration files for each host namespace:
130+
131+
```
132+
funee-lib/
133+
host/
134+
fs.ts # host://fs
135+
http.ts # host://http
136+
server.ts # host://http/server
137+
process.ts # host://process
138+
time.ts # host://time
139+
watch.ts # host://watch
140+
crypto.ts # host://crypto
141+
console.ts # host://console
142+
```
143+
144+
### Phase 3: Update funee-lib
145+
146+
1. **Move host function wrappers** to `funee-lib/host/*.ts`
147+
2. **Update internal imports** in funee-lib to use `host://`
148+
3. **Re-export from funee-lib/index.ts** for backward compatibility (deprecation period)
149+
150+
### Phase 4: Update Tests & Examples
151+
152+
1. Update all test fixtures to use `host://` imports
153+
2. Update examples
154+
3. Update README
155+
156+
### Phase 5: Deprecation Path
157+
158+
For backward compatibility:
159+
```typescript
160+
// funee-lib/index.ts
161+
// @deprecated Use host://fs instead
162+
export { readFile, writeFile } from "host://fs";
163+
// @deprecated Use host://http instead
164+
export { fetch } from "host://http";
165+
```
166+
167+
## Alternative Considered: `runtime://` or `builtin://`
168+
169+
| Scheme | Pros | Cons |
170+
|--------|------|------|
171+
| `host://` | Clear "provided by host" semantics | Could confuse with hostnames |
172+
| `runtime://` | Clear it's runtime-provided | Longer |
173+
| `builtin://` | Node.js precedent | Less clear about replaceability |
174+
| `native://` | Clear it's native code | Could confuse with native modules |
175+
176+
**Decision:** `host://` — Short, clear semantics about the host/guest boundary.
177+
178+
## Type Safety
179+
180+
Each `host://` module has full TypeScript types:
181+
182+
```typescript
183+
// host://fs types
184+
export declare function readFile(path: string): Promise<string>;
185+
export declare function writeFile(path: string, content: string): Promise<void>;
186+
export declare function isFile(path: string): Promise<boolean>;
187+
// ...
188+
189+
// host://http types
190+
export declare function fetch(
191+
url: string | URL,
192+
init?: RequestInit
193+
): Promise<Response>;
194+
195+
// host://process types
196+
export interface SpawnOptions {
197+
cmd: string[];
198+
cwd?: string;
199+
env?: Record<string, string>;
200+
stdin?: "piped" | "inherit" | "null";
201+
stdout?: "piped" | "inherit" | "null";
202+
stderr?: "piped" | "inherit" | "null";
203+
}
204+
export declare function spawn(options: SpawnOptions): Process;
205+
export declare function spawn(cmd: string, args?: string[]): Promise<CommandOutput>;
206+
```
207+
208+
## Benefits
209+
210+
1. **Explicit host dependency** — Easy to see what a module needs from the host
211+
2. **Portable code** — Pure `"funee"` imports work anywhere
212+
3. **Alternative runtimes** — Browser, edge workers, etc. can provide different `host://` implementations
213+
4. **Tree-shaking** — Unused host modules aren't loaded
214+
5. **Documentation** — The URL itself documents what capability is needed
215+
6. **Testing** — Easy to mock `host://` imports in tests
216+
217+
## Open Questions
218+
219+
1. **Should timers be globals or imports?** — Currently `setTimeout` is a global. Keep as global for compatibility?
220+
2. **Version in URL?** — `host://fs@1` for future breaking changes?
221+
3. **Capability detection?** — `import { available } from "host://fs"` to check if host provides it?

0 commit comments

Comments
 (0)