Skip to content

Commit 3d0add0

Browse files
benoitcbenoitc
committed
Document OWN_GIL subinterpreter parallelism
1 parent 84a8bb8 commit 3d0add0

2 files changed

Lines changed: 248 additions & 3 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,15 @@
44

55
### Added
66

7+
- **OWN_GIL Subinterpreter Thread Pool** - True parallelism with Python 3.12+ subinterpreters
8+
- Each subinterpreter runs in its own thread with its own GIL (`Py_GIL_OWN`)
9+
- Thread pool manages N subinterpreters for parallel Python execution
10+
- `py:context(N)` returns the Nth context PID for explicit context selection
11+
- `py_context_router` provides scheduler-affinity routing for automatic distribution
12+
- Cast operations are 25-30% faster compared to worker mode
13+
- Full isolation between subinterpreters (separate namespaces, modules, state)
14+
- New C files: `py_subinterp_pool.c`, `py_subinterp_pool.h`
15+
716
- **`erlang.reactor` module** - FD-based protocol handling for building custom servers
817
- `reactor.Protocol` - Base class for implementing protocols
918
- `reactor.serve(sock, protocol_factory)` - Serve connections using a protocol
@@ -112,6 +121,11 @@
112121
- **Subinterpreter cleanup and thread worker re-registration** - Fixed cleanup
113122
issues when subinterpreters are destroyed and recreated
114123

124+
- **ProcessError exception class identity in subinterpreters** - Fixed exception
125+
class mismatch when raising `erlang.ProcessError` in subinterpreter contexts.
126+
The exception class is now looked up from the current interpreter's `erlang`
127+
module at runtime instead of using a global variable.
128+
115129
- **Thread worker handlers not re-registering after app restart** - Workers now
116130
properly re-register when application restarts
117131

‎docs/scalability.md‎

Lines changed: 234 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -30,14 +30,174 @@ When running on a free-threaded Python build (compiled with `--disable-gil`), er
3030

3131
### Sub-interpreter Mode (Python 3.12+)
3232

33-
Uses Python's sub-interpreter feature with per-interpreter GIL. Each sub-interpreter has its own GIL, allowing true parallel execution across interpreters.
33+
Uses Python's sub-interpreter feature with per-interpreter GIL (`Py_GIL_OWN`). Each sub-interpreter runs in its own dedicated thread with its own GIL, enabling true parallel execution across interpreters.
34+
35+
**Architecture:**
36+
- Thread pool manages N subinterpreters (default: number of schedulers)
37+
- Each subinterpreter has its own thread, GIL, and Python state
38+
- Requests are routed to subinterpreters via `py_context_router`
39+
- 25-30% faster cast operations compared to worker mode
3440

3541
**Note:** Each sub-interpreter has isolated state. Use the [Shared State](#shared-state) API to share data between workers.
3642

43+
**Explicit Context Selection:**
44+
```erlang
45+
%% Get a specific context by index (1-based)
46+
Ctx = py:context(1),
47+
{ok, Result} = py:call(Ctx, math, sqrt, [16]).
48+
49+
%% Or use automatic scheduler-affinity routing
50+
{ok, Result} = py:call(math, sqrt, [16]).
51+
```
52+
3753
### Multi-Executor Mode (Python < 3.12)
3854

3955
Runs N executor threads that share the GIL. Requests are distributed round-robin across executors. Good for I/O-bound workloads where Python releases the GIL during I/O operations.
4056

57+
## Choosing the Right Mode
58+
59+
### Mode Comparison
60+
61+
| Aspect | Free-Threaded | Subinterpreter | Multi-Executor |
62+
|--------|---------------|----------------|----------------|
63+
| **Parallelism** | True N-way | True N-way | GIL contention |
64+
| **State Isolation** | Shared | Isolated | Shared |
65+
| **Memory Overhead** | Low | Higher (per-interp) | Low |
66+
| **Module Compatibility** | Limited | Most modules | All modules |
67+
| **Python Version** | 3.13+ (nogil) | 3.12+ | Any |
68+
69+
### When to Use Each Mode
70+
71+
**Use Free-Threaded (Python 3.13t) when:**
72+
- You need maximum parallelism with shared state
73+
- Your libraries are GIL-free compatible
74+
- You're running CPU-bound workloads
75+
- Memory efficiency is important
76+
77+
**Use Subinterpreters (Python 3.12+) when:**
78+
- You need parallelism with state isolation
79+
- You want crash isolation between contexts
80+
- You're running untrusted or unstable code
81+
- You need predictable per-request state
82+
83+
**Use Multi-Executor (Python < 3.12) when:**
84+
- Running on older Python versions
85+
- Your workload is I/O-bound (GIL released during I/O)
86+
- You need compatibility with all Python modules
87+
- Shared state between workers is required
88+
89+
### Pros and Cons
90+
91+
**Subinterpreter Mode Pros:**
92+
- True parallelism without GIL contention
93+
- Complete isolation (crashes don't affect other contexts)
94+
- Each context has clean namespace (no state bleed)
95+
- 25-30% faster cast operations vs worker mode
96+
97+
**Subinterpreter Mode Cons:**
98+
- Higher memory usage (each interpreter loads modules separately)
99+
- Some C extensions don't support subinterpreters
100+
- No shared state between contexts (use Shared State API)
101+
- asyncio event loop integration requires main interpreter
102+
103+
**Free-Threaded Mode Pros:**
104+
- True parallelism with shared state
105+
- Lower memory overhead than subinterpreters
106+
- Simplest mental model (like regular threading)
107+
108+
**Free-Threaded Mode Cons:**
109+
- Requires Python 3.13+ built with `--disable-gil`
110+
- Many C extensions not yet compatible
111+
- Shared state requires careful synchronization
112+
- Still experimental
113+
114+
## Subinterpreter Architecture
115+
116+
### Design Overview
117+
118+
```
119+
┌─────────────────────────────────────────────────────────────────┐
120+
│ Erlang VM (BEAM) │
121+
├─────────────────────────────────────────────────────────────────┤
122+
│ py_context_router │
123+
│ ┌─────────────────────────────────────────────────────────┐ │
124+
│ │ Scheduler 1 ──► Context 1 (pid) │ │
125+
│ │ Scheduler 2 ──► Context 2 (pid) │ │
126+
│ │ Scheduler N ──► Context N (pid) │ │
127+
│ └─────────────────────────────────────────────────────────┘ │
128+
│ │ │ │ │
129+
│ ▼ ▼ ▼ │
130+
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
131+
│ │ Context │ │ Context │ │ Context │ │
132+
│ │ Process │ │ Process │ │ Process │ │
133+
│ │ (gen_srv)│ │ (gen_srv)│ │ (gen_srv)│ │
134+
│ └────┬─────┘ └────┬─────┘ └────┬─────┘ │
135+
└───────┼──────────────┼──────────────┼───────────────────────────┘
136+
│ │ │
137+
▼ ▼ ▼
138+
┌─────────────────────────────────────────────────────────────────┐
139+
│ Subinterpreter Thread Pool │
140+
├─────────────────────────────────────────────────────────────────┤
141+
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
142+
│ │ Thread 1 │ │ Thread 2 │ │ Thread N │ │
143+
│ │ ┌──────────┐ │ │ ┌──────────┐ │ │ ┌──────────┐ │ │
144+
│ │ │ Interp │ │ │ │ Interp │ │ │ │ Interp │ │ │
145+
│ │ │ (GIL 1) │ │ │ │ (GIL 2) │ │ │ │ (GIL N) │ │ │
146+
│ │ └──────────┘ │ │ └──────────┘ │ │ └──────────┘ │ │
147+
│ └──────────────┘ └──────────────┘ └──────────────┘ │
148+
│ │
149+
│ Each thread owns its interpreter's GIL (Py_GIL_OWN) │
150+
│ No GIL contention between threads │
151+
└─────────────────────────────────────────────────────────────────┘
152+
```
153+
154+
### Key Components
155+
156+
**py_context_router**: Routes requests to context processes based on scheduler affinity or explicit binding.
157+
158+
**py_context_process**: Gen_server that owns a Python context reference and handles call/eval/exec operations.
159+
160+
**Subinterpreter Thread Pool (C)**: Manages N threads, each with its own Python subinterpreter created with `Py_NewInterpreterFromConfig()` and `Py_GIL_OWN`.
161+
162+
### Request Flow
163+
164+
1. Erlang process calls `py:call(Module, Func, Args)`
165+
2. `py_context_router` selects context based on scheduler ID
166+
3. Request sent to `py_context_process` gen_server
167+
4. Gen_server calls NIF which executes on subinterpreter's thread
168+
5. Result returned through gen_server to caller
169+
170+
### Pool Size
171+
172+
The subinterpreter pool size is configured at two levels:
173+
174+
| Level | Default | Max |
175+
|-------|---------|-----|
176+
| **Erlang (py_context_router)** | `erlang:system_info(schedulers)` | configurable |
177+
| **C pool (py_subinterp_pool)** | 32 | 64 |
178+
179+
On a typical 8-core machine, 8 context processes are started, each with one subinterpreter slot.
180+
181+
**Configuration via sys.config:**
182+
```erlang
183+
{erlang_python, [
184+
{num_contexts, 16} %% Override scheduler count
185+
]}
186+
```
187+
188+
**Configuration at runtime:**
189+
```erlang
190+
%% Start with custom pool size
191+
py_context_router:start(#{contexts => 16}).
192+
```
193+
194+
### Thread Safety
195+
196+
- Each subinterpreter has its own GIL (no cross-interpreter contention)
197+
- NIF calls are serialized per-context via gen_server
198+
- Erlang message passing provides synchronization
199+
- C code uses atomics for cross-thread state (`thread_running` flag)
200+
41201
## Rate Limiting
42202

43203
All Python calls pass through an ETS-based counting semaphore that prevents overload:
@@ -106,12 +266,83 @@ This allows your application to implement backpressure or shed load gracefully.
106266

107267
## Parallel Execution with Sub-interpreters
108268

109-
For CPU-bound workloads on Python 3.12+, use explicit parallel execution:
269+
For CPU-bound workloads on Python 3.12+, erlang_python provides true parallelism via OWN_GIL subinterpreters.
270+
271+
### Check Support
110272

111273
```erlang
112-
%% Check if supported
274+
%% Check if subinterpreters are supported (Python 3.12+)
113275
true = py:subinterp_supported().
114276

277+
%% Check current execution mode
278+
subinterp = py:execution_mode().
279+
```
280+
281+
### Using the Context Router
282+
283+
The context router automatically distributes calls across subinterpreters:
284+
285+
```erlang
286+
%% Start contexts (usually done by application startup)
287+
{ok, _} = py:start_contexts().
288+
289+
%% Calls are automatically routed to subinterpreters
290+
{ok, 4.0} = py:call(math, sqrt, [16]).
291+
{ok, 6} = py:eval(<<"2 + 4">>).
292+
ok = py:exec(<<"x = 42">>).
293+
```
294+
295+
### Explicit Context Selection
296+
297+
For fine-grained control, use explicit context selection:
298+
299+
```erlang
300+
%% Get a specific context by index (1-based)
301+
Ctx = py:context(1),
302+
303+
%% All operations on this context share state
304+
ok = py:exec(Ctx, <<"my_var = 'hello'">>),
305+
{ok, <<"hello">>} = py:eval(Ctx, <<"my_var">>),
306+
{ok, 4.0} = py:call(Ctx, math, sqrt, [16]).
307+
308+
%% Different context has isolated state
309+
Ctx2 = py:context(2),
310+
{error, _} = py:eval(Ctx2, <<"my_var">>). %% Not defined in Ctx2
311+
```
312+
313+
### Context Router API
314+
315+
```erlang
316+
%% Start router with default number of contexts (scheduler count)
317+
{ok, Contexts} = py_context_router:start().
318+
319+
%% Start with custom number of contexts
320+
{ok, Contexts} = py_context_router:start(#{contexts => 8}).
321+
322+
%% Get context for current scheduler (automatic affinity)
323+
Ctx = py_context_router:get_context().
324+
325+
%% Get specific context by index
326+
Ctx = py_context_router:get_context(1).
327+
328+
%% Bind current process to a specific context
329+
ok = py_context_router:bind_context(Ctx).
330+
331+
%% Unbind (return to scheduler-based routing)
332+
ok = py_context_router:unbind_context().
333+
334+
%% Get number of active contexts
335+
N = py_context_router:num_contexts().
336+
337+
%% Stop all contexts
338+
ok = py_context_router:stop().
339+
```
340+
341+
### Parallel Execution
342+
343+
Execute multiple calls in parallel across subinterpreters:
344+
345+
```erlang
115346
%% Execute multiple calls in parallel
116347
{ok, Results} = py:parallel([
117348
{math, sqrt, [16]},

0 commit comments

Comments
 (0)