Skip to content

Commit 74ecf34

Browse files
committed
docs(readme): add 'How packaging works' section (per-platform layout)
Document, per platform, where the stdlib and site-packages live, how native extension modules are shipped (Android: jniLibs mmap'd from the APK via a custom importer; iOS: xcframeworks + AppleFrameworkLoader; macOS: universal .so; Linux/Windows: on-disk; Web: Pyodide wheels), architecture handling, and how the user's app.zip is produced and extracted at runtime. Replace the stale Android note (enableUncompressedNativeLibs / extractNativeLibs) with the new no-config guidance (minSdk 23+).
1 parent 96d92c1 commit 74ecf34

1 file changed

Lines changed: 48 additions & 9 deletions

File tree

src/serious_python/README.md

Lines changed: 48 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -207,6 +207,47 @@ Additional Python binary packages for iOS and Android can be built with adding a
207207

208208
Request additional packages for iOS and Android on [Flet Discussions - Packages](https://github.com/flet-dev/flet/discussions/categories/packages).
209209

210+
## How packaging works
211+
212+
`dart run serious_python:main package` assembles two things, which the platform plugin then bundles into your Flutter app:
213+
214+
1. **The CPython runtime + standard library** — a per-target build downloaded from [flet-dev/python-build](https://github.com/flet-dev/python-build) (and, for native extensions, [mobile-forge](https://github.com/flet-dev/mobile-forge)) and bundled by the plugin at build time.
215+
2. **Your app + its dependencies** — your Python sources are zipped into an **asset** (`app/app.zip` by default), and `pip`-installed packages are placed where each platform expects them.
216+
217+
At runtime the plugin sets `PYTHONHOME` / `PYTHONPATH` (or, on Android, installs a custom importer) so the interpreter finds the stdlib, your dependencies, and your app.
218+
219+
The on-disk layout differs per platform, mostly because each OS has different rules for shipping **native (compiled) extension modules** — the `.so`/`.pyd`/`.dylib` files inside packages like `numpy`:
220+
221+
| Platform | Standard library | Site-packages (deps) | Native extension modules | Architectures |
222+
| --- | --- | --- | --- | --- |
223+
| **Android** | `stdlib.zip` asset, read via `zipimport` | `sitepackages.zip` asset, read via `zipimport` | relocated to `jniLibs/<abi>/`, **memory-mapped from the APK** (no extraction), resolved by a custom importer | natives per-ABI in `jniLibs`; pure zips are ABI-common (shipped once) |
224+
| **iOS** | dir inside the framework resource bundle | dir inside the framework resource bundle | each `.so` wrapped in a signed `.framework` inside an `.xcframework`, loaded via CPython's `AppleFrameworkLoader` (`.fwork` markers) | device `arm64` + simulator `arm64`/`x86_64` xcframework slices |
225+
| **macOS** | dir inside the framework resource bundle | dir (universal) | universal (`lipo`'d `arm64`+`x86_64`) `.so`, loaded directly | `arm64`+`x86_64` merged into fat binaries |
226+
| **Linux** | `<exe-dir>/python<X.Y>/` | `<exe-dir>/site-packages/` | on-disk `.so` (in `lib-dynload` / package dirs) | one of `x86_64` / `aarch64` per build |
227+
| **Windows** | `<exe-dir>/Lib/` | `<exe-dir>/site-packages/` | on-disk `.pyd`/`.dll` in `<exe-dir>/DLLs/` | `x86_64` |
228+
| **Web** | bundled inside Pyodide | `__pypackages__/` inside `app.zip` | Pyodide WebAssembly wheels | `wasm32` |
229+
230+
### Your app program (all platforms)
231+
232+
`package` copies your Python sources into a temp dir (honoring `--exclude` globs, optionally compiling to `.pyc` with `--compile-app`), zips it to `app/app.zip` (override with `-a`/`--asset`), and writes an `app.zip.hash` next to it. At runtime the asset is extracted once to the app's support directory (`<app-support>/flet/app`), guarded by a hash + an optional `invalidateKey` (typically your app version) so it only re-extracts when the bundle changes — debug builds always re-extract. Your app dir is placed first on `sys.path`; a sibling `__pypackages__/` is also added (so you can vendor pure-Python deps next to your code).
233+
234+
`pip install` output goes to `build/site-packages` by default (override with the `SERIOUS_PYTHON_SITE_PACKAGES` env var). For mobile, packages are installed **per architecture** (a `sitecustomize.py` shim spoofs the wheel platform tag so the correct mobile wheels resolve), then merged or split per platform as shown above.
235+
236+
### Android specifics
237+
238+
- **Pure Python** (stdlib + dependencies) ships in two **stored** (uncompressed) ABI-common zips — `stdlib.zip` and `sitepackages.zip` — copied once to the app's files dir and imported in place via `zipimport`. Final `sys.path` (highest first): your app dir, the extract dir, `sitepackages.zip`, `stdlib.zip`.
239+
- **Native modules** (stdlib `lib-dynload` and site-package extensions) are relocated to `jniLibs/<abi>/lib<mangled>.so` and loaded **directly from the APK** (memory-mapped, never extracted to disk); a `sys.meta_path` finder resolves them from `.soref` markers left in the zips. This is why Android needs **no** `useLegacyPackaging` / `keepDebugSymbols` config and the stdlib is **not** duplicated per ABI.
240+
- **Path-hungry packages** (those that read bundled data via `__file__` / `pkg_resources` rather than `importlib.resources`) can be shipped extracted to disk instead of inside the zip — list them (comma-separated relative paths) in `SERIOUS_PYTHON_ANDROID_EXTRACT_PACKAGES`; they go into `extract.zip` and are unpacked to disk at first launch.
241+
- Works for both **single APK** (`flutter build apk`) and **Play Store App Bundles** (per-ABI config splits); under legacy packaging / `minSdk < 23` the same finder falls back to loading from the extracted `nativeLibraryDir`.
242+
243+
### iOS / macOS specifics
244+
245+
The CPython runtime, stdlib, and (on iOS) native extensions are bundled into `serious_python_darwin.framework` as resources. On **iOS**, the App Store forbids loose `.dylib`s, so every native extension `.so` is repackaged into a signed `.framework` inside an `.xcframework`, with a `.fwork` text marker left at the module's import path; CPython's `AppleFrameworkLoader` reads the marker and loads the framework binary. On **macOS**, native extensions stay as plain `.so`, merged into universal (`arm64`+`x86_64`) binaries at package time. `PYTHONHOME` is the framework's resource path; `sys.path` includes `<resources>/site-packages`, `<resources>/stdlib`, and `<resources>/stdlib/lib-dynload`.
246+
247+
### Linux / Windows specifics
248+
249+
The CPython runtime (`libpython3.so` + `libpython<X.Y>.so` on Linux; `python3.dll` + `python<XY>.dll` on Windows), `libdart_bridge`, the stdlib, and native modules are copied next to your app's executable at build time. `PYTHONHOME` is the executable's directory. On Windows, extension modules (`.pyd`) and their dependent DLLs live in `<exe-dir>/DLLs/`, which is added to `sys.path`.
250+
210251
## Platform notes
211252

212253
### Build matrix
@@ -238,18 +279,16 @@ MACOSX_DEPLOYMENT_TARGET = 10.15;
238279

239280
### Android
240281

241-
To make `serious_python` work in your own Android app:
282+
No special native-library packaging config is required (see [How packaging works](#how-packaging-works)). serious_python loads native modules directly from the APK and ships pure Python in stored asset zips, so you don't need `useLegacyPackaging`, `keepDebugSymbols`, `extractNativeLibs`, or `android.bundle.enableUncompressedNativeLibs`. Just use a `minSdk` of 23+ so native libs stay uncompressed/page-aligned in the APK:
242283

243-
If you build an App Bundle Edit `android/gradle.properties` and add the flag:
244-
245-
```
246-
android.bundle.enableUncompressedNativeLibs=false
284+
```kotlin
285+
android {
286+
defaultConfig {
287+
minSdk = 23
288+
}
289+
}
247290
```
248291

249-
If you build an APK Make sure `android/app/src/AndroidManifest.xml` has `android:extractNativeLibs="true"` in the `<application>` tag.
250-
251-
For more information, see the [public issue](https://issuetracker.google.com/issues/147096055).
252-
253292
## Troubleshooting
254293

255294
### Detailed logging

0 commit comments

Comments
 (0)