Android: memory-map native modules from the APK (drop fake-.so / useLegacyPackaging)

src/serious_python/CHANGELOG.md

@@ -1,7 +1,8 @@
 ## 3.0.0
 
-* **New in-process transport (dart_bridge FFI).** `SeriousPython.run` can now run the embedded interpreter **in-process** through the `dart_bridge` FFI bridge instead of talking to it over a socket. The Python lifecycle (initialize / run / teardown) is absorbed into the `dart_bridge` native library on every platform — `dart_bridge.xcframework` (iOS/macOS), `libdart_bridge.so` (Android/Linux), and `dart_bridge.dll` / `dart_bridge.pyd` (Windows) — and a new `PythonBridge` API exposes a MsgPack control channel plus dedicated binary data channels between Dart and Python. See the `bridge_example` app. The bundled `dart_bridge` is **1.2.3**.
-* **Breaking change:** requires Flutter **3.44.2** / Dart 3.12+. The Android plugin moves to AGP **8.11.1**, `compileSdk` **36**, Java **17**, and the Kotlin-DSL Gradle build; the removed `android.bundle.enableUncompressedNativeLibs` flag is replaced with `useLegacyPackaging` + `keepDebugSymbols` for the bundled `libpython*.so`.
+* **New in-process transport (dart_bridge FFI).** `SeriousPython.run` can now run the embedded interpreter **in-process** through the `dart_bridge` FFI bridge instead of talking to it over a socket. The Python lifecycle (initialize / run / teardown) is absorbed into the `dart_bridge` native library on every platform — `dart_bridge.xcframework` (iOS/macOS), `libdart_bridge.so` (Android/Linux), and `dart_bridge.dll` / `dart_bridge.pyd` (Windows) — and a new `PythonBridge` API exposes a MsgPack control channel plus dedicated binary data channels between Dart and Python. See the `bridge_example` app. The bundled `dart_bridge` is **1.4.0**.
+* **Android native packaging — memory-mapped from the APK.** Python extension modules are relocated into `jniLibs` and loaded **directly from the APK** (mmap, no extraction) by a custom importer that resolves them from `.soref` markers; pure Python ships in stored, ABI-common asset zips read via `zipimport` (no per-ABI duplication). Apps no longer need `useLegacyPackaging` / `keepDebugSymbols` — the brittle per-app packaging config is gone. Set **`SERIOUS_PYTHON_ANDROID_EXTRACT_PACKAGES`** (comma-separated relative paths) to ship path-hungry packages extracted to disk. The dart-bridge Android binary uses the full CPython API (`PyConfig`) to install the importer before `site` runs.
+* **Breaking change:** requires Flutter **3.44.2** / Dart 3.12+. The Android plugin moves to AGP **8.11.1**, `compileSdk` **36**, Java **17**, and the Kotlin-DSL Gradle build (`build.gradle.kts`).
 * Python runtime versions are now a committed snapshot of `flet-dev/python-build`'s date-keyed `manifest.json`, generated by `dart run serious_python:gen_version_tables`. **`SERIOUS_PYTHON_VERSION`** (short, e.g. `3.14`) is the single input — the full CPython version, python-build release date, Pyodide version + platform tag, and `dart_bridge` version all derive from it. `SERIOUS_PYTHON_FULL_VERSION`, `SERIOUS_PYTHON_BUILD_DATE`, and `DART_BRIDGE_VERSION` remain as rarely-needed escape hatches. The native build configs (Android `build.gradle`, Darwin podspec, Linux/Windows `CMakeLists.txt`) read the generated `python_versions.properties`, and a CI job fails if the snapshots drift from the manifest. This replaces the per-config hardcoded defaults and the `flet build`-exported `SERIOUS_PYTHON_FULL_VERSION` / `SERIOUS_PYTHON_BUILD_DATE` introduced in 2.0.0.
 * Bundle **3.12.13 / 3.13.14 / 3.14.6** (python-build `20260614`); Pyodide **0.27.7 / 0.29.4 / 314.0.0** (314.0.0 GA, up from the 314.0.0a2 in 2.0.0).
 * Add **`dart run serious_python:main version [--json]`** — prints the serious_python version, the pinned python-build release, and the supported Python / Pyodide / dart_bridge matrix.

src/serious_python/README.md

@@ -207,6 +207,47 @@ Additional Python binary packages for iOS and Android can be built with adding a
 
 Request additional packages for iOS and Android on [Flet Discussions - Packages](https://github.com/flet-dev/flet/discussions/categories/packages).
 
+## How packaging works
+
+`dart run serious_python:main package` assembles two things, which the platform plugin then bundles into your Flutter app:
+
+1. **The CPython runtime + standard library** — a per-target build downloaded from [flet-dev/python-build](https://github.com/flet-dev/python-build) (and, for native extensions, [mobile-forge](https://github.com/flet-dev/mobile-forge)) and bundled by the plugin at build time.
+2. **Your app + its dependencies** — your Python sources are zipped into an **asset** (`app/app.zip` by default), and `pip`-installed packages are placed where each platform expects them.
+
+At runtime the plugin sets `PYTHONHOME` / `PYTHONPATH` (or, on Android, installs a custom importer) so the interpreter finds the stdlib, your dependencies, and your app.
+
+The on-disk layout differs per platform, mostly because each OS has different rules for shipping **native (compiled) extension modules** — the `.so`/`.pyd`/`.dylib` files inside packages like `numpy`:
+
+| Platform | Standard library | Site-packages (deps) | Native extension modules | Architectures |
+| --- | --- | --- | --- | --- |
+| **Android** | `stdlib.zip` asset, read via `zipimport` | `sitepackages.zip` asset, read via `zipimport` | relocated to `jniLibs/<abi>/`, **memory-mapped from the APK** (no extraction), resolved by a custom importer | natives per-ABI in `jniLibs`; pure zips are ABI-common (shipped once) |
+| **iOS** | dir inside the framework resource bundle | dir inside the framework resource bundle | each `.so` wrapped in a signed `.framework` inside an `.xcframework`, loaded via CPython's `AppleFrameworkLoader` (`.fwork` markers) | device `arm64` + simulator `arm64`/`x86_64` xcframework slices |
+| **macOS** | dir inside the framework resource bundle | dir (universal) | universal (`lipo`'d `arm64`+`x86_64`) `.so`, loaded directly | `arm64`+`x86_64` merged into fat binaries |
+| **Linux** | `<exe-dir>/python<X.Y>/` | `<exe-dir>/site-packages/` | on-disk `.so` (in `lib-dynload` / package dirs) | one of `x86_64` / `aarch64` per build |
+| **Windows** | `<exe-dir>/Lib/` | `<exe-dir>/site-packages/` | on-disk `.pyd`/`.dll` in `<exe-dir>/DLLs/` | `x86_64` |
+| **Web** | bundled inside Pyodide | `__pypackages__/` inside `app.zip` | Pyodide WebAssembly wheels | `wasm32` |
+
+### Your app program (all platforms)
+
+`package` copies your Python sources into a temp dir (honoring `--exclude` globs, optionally compiling to `.pyc` with `--compile-app`), zips it to `app/app.zip` (override with `-a`/`--asset`), and writes an `app.zip.hash` next to it. At runtime the asset is extracted once to the app's support directory (`<app-support>/flet/app`), guarded by a hash + an optional `invalidateKey` (typically your app version) so it only re-extracts when the bundle changes — debug builds always re-extract. Your app dir is placed first on `sys.path`; a sibling `__pypackages__/` is also added (so you can vendor pure-Python deps next to your code).
+
+`pip install` output goes to `build/site-packages` by default (override with the `SERIOUS_PYTHON_SITE_PACKAGES` env var). For mobile, packages are installed **per architecture** (a `sitecustomize.py` shim spoofs the wheel platform tag so the correct mobile wheels resolve), then merged or split per platform as shown above.
+
+### Android specifics
+
+- **Pure Python** (stdlib + dependencies) ships in two **stored** (uncompressed) ABI-common zips — `stdlib.zip` and `sitepackages.zip` — copied once to the app's files dir and imported in place via `zipimport`. Final `sys.path` (highest first): your app dir, the extract dir, `sitepackages.zip`, `stdlib.zip`.
+- **Native modules** (stdlib `lib-dynload` and site-package extensions) are relocated to `jniLibs/<abi>/lib<mangled>.so` and loaded **directly from the APK** (memory-mapped, never extracted to disk); a `sys.meta_path` finder resolves them from `.soref` markers left in the zips. This is why Android needs **no** `useLegacyPackaging` / `keepDebugSymbols` config and the stdlib is **not** duplicated per ABI.
+- **Path-hungry packages** (those that read bundled data via `__file__` / `pkg_resources` rather than `importlib.resources`) can be shipped extracted to disk instead of inside the zip — list them (comma-separated relative paths) in `SERIOUS_PYTHON_ANDROID_EXTRACT_PACKAGES`; they go into `extract.zip` and are unpacked to disk at first launch.
+- Works for both **single APK** (`flutter build apk`) and **Play Store App Bundles** (per-ABI config splits); under legacy packaging / `minSdk < 23` the same finder falls back to loading from the extracted `nativeLibraryDir`.
+
+### iOS / macOS specifics
+
+The CPython runtime, stdlib, and (on iOS) native extensions are bundled into `serious_python_darwin.framework` as resources. On **iOS**, the App Store forbids loose `.dylib`s, so every native extension `.so` is repackaged into a signed `.framework` inside an `.xcframework`, with a `.fwork` text marker left at the module's import path; CPython's `AppleFrameworkLoader` reads the marker and loads the framework binary. On **macOS**, native extensions stay as plain `.so`, merged into universal (`arm64`+`x86_64`) binaries at package time. `PYTHONHOME` is the framework's resource path; `sys.path` includes `<resources>/site-packages`, `<resources>/stdlib`, and `<resources>/stdlib/lib-dynload`.
+
+### Linux / Windows specifics
+
+The CPython runtime (`libpython3.so` + `libpython<X.Y>.so` on Linux; `python3.dll` + `python<XY>.dll` on Windows), `libdart_bridge`, the stdlib, and native modules are copied next to your app's executable at build time. `PYTHONHOME` is the executable's directory. On Windows, extension modules (`.pyd`) and their dependent DLLs live in `<exe-dir>/DLLs/`, which is added to `sys.path`.
+
 ## Platform notes
 
 ### Build matrix
@@ -238,18 +279,16 @@ MACOSX_DEPLOYMENT_TARGET = 10.15;
 
 ### Android
 
-To make `serious_python` work in your own Android app:
+No special native-library packaging config is required (see [How packaging works](#how-packaging-works)). serious_python loads native modules directly from the APK and ships pure Python in stored asset zips, so you don't need `useLegacyPackaging`, `keepDebugSymbols`, `extractNativeLibs`, or `android.bundle.enableUncompressedNativeLibs`. Just use a `minSdk` of 23+ so native libs stay uncompressed/page-aligned in the APK:
 
-If you build an App Bundle Edit `android/gradle.properties` and add the flag:
-
-```
-android.bundle.enableUncompressedNativeLibs=false
+```kotlin
+android {
+    defaultConfig {
+        minSdk = 23
+    }
+}
 ```
 
-If you build an APK Make sure `android/app/src/AndroidManifest.xml` has `android:extractNativeLibs="true"` in the `<application>` tag.
-
-For more information, see the [public issue](https://issuetracker.google.com/issues/147096055).
-
 ## Troubleshooting
 
 ### Detailed logging

src/serious_python/example/flask_example/README.md

@@ -37,21 +37,16 @@ export SERIOUS_PYTHON_SITE_PACKAGES=$(pwd)/build/site-packages
 dart run serious_python:main package app/src -p Linux -r -r -r app/src/requirements.txt
 ```
 
-Important: to make `serious_python` work in your own Android app, the bundled
-`libpython*.so` must be shipped uncompressed and extracted so the embedded
-interpreter can `dlopen` them at runtime. In `android/app/build.gradle.kts`:
+For Android, no special native-library packaging config is required.
+`serious_python` relocates Python extension modules into `jniLibs` and loads them
+directly from the APK (memory-mapped, no extraction), and ships pure Python in
+stored asset zips. Just use a `minSdk` of 23+ so native libs stay uncompressed and
+page-aligned in the APK:
 
 ```kotlin
 android {
-    packaging {
-        jniLibs {
-            useLegacyPackaging = true
-        }
+    defaultConfig {
+        minSdk = 23
     }
 }
-```
-
-`useLegacyPackaging = true` is the modern replacement (AGP 8.1+) for both the
-old `android.bundle.enableUncompressedNativeLibs=false` gradle property and the
-`android:extractNativeLibs="true"` manifest attribute, and it covers both APK and
-App Bundle builds. See the [public issue](https://issuetracker.google.com/issues/147096055).
+```

src/serious_python/example/flask_example/android/app/src/main/AndroidManifest.xml

@@ -1,4 +1,7 @@
 <manifest xmlns:android="http://schemas.android.com/apk/res/android">
+    <!-- The Flask app binds a local socket server, so it needs INTERNET in all build
+         types (the debug/profile manifests already add it for tooling). -->
+    <uses-permission android:name="android.permission.INTERNET"/>
     <application
         android:label="flask_example"
         android:name="${applicationName}"

src/serious_python/example/run_example/README.md

@@ -43,29 +43,23 @@ For Pyodide:
 dart run serious_python:main package app/src -p Emscripten -r -r -r app/src/requirements.txt
 ```
 
-For Android:
+For Android, no special native-library packaging config is required. serious_python
+relocates Python extension modules into `jniLibs` and loads them directly from the APK
+(memory-mapped, no extraction), and ships pure Python in stored asset zips read via
+`zipimport`. Just use a `minSdk` of 23+ so native libs stay uncompressed/page-aligned in
+the APK:
 
 In `android/app/build.gradle.kts`:
 
 ```kotlin
 android {
-    // serious_python bundles libpython*.so. Use legacy (extracted, uncompressed)
-    // packaging so the embedded interpreter can dlopen them at runtime, and keep
-    // their symbols so they are not stripped.
-    packaging {
-        jniLibs {
-            useLegacyPackaging = true
-            keepDebugSymbols += setOf(
-                "*/arm64-v8a/libpython*.so",
-                "*/armeabi-v7a/libpython*.so",
-                "*/x86/libpython*.so",
-                "*/x86_64/libpython*.so",
-            )
-        }
-    }
-
     defaultConfig {
         minSdk = 23
     }
 }
-```
+```
+
+To ship a path-hungry package **extracted to disk** instead of inside the zip — for
+packages that read bundled data via `__file__` / `pkg_resources` rather than
+`importlib.resources` — set `SERIOUS_PYTHON_ANDROID_EXTRACT_PACKAGES` to a comma-separated
+list of relative package paths before building.

src/serious_python/example/run_example/android/app/build.gradle.kts

@@ -16,20 +16,9 @@ android {
         targetCompatibility = JavaVersion.VERSION_17
     }
 
-    // serious_python bundles libpython*.so. Use legacy (extracted, uncompressed)
-    // packaging so the embedded interpreter can dlopen them at runtime, and keep
-    // their symbols so they are not stripped.
-    packaging {
-        jniLibs {
-            useLegacyPackaging = true
-            keepDebugSymbols += setOf(
-                "*/arm64-v8a/libpython*.so",
-                "*/armeabi-v7a/libpython*.so",
-                "*/x86/libpython*.so",
-                "*/x86_64/libpython*.so",
-            )
-        }
-    }
+    // No special native-library packaging is needed: serious_python loads extension
+    // modules directly from the APK (mmap) via its finder. Modern packaging
+    // (useLegacyPackaging=false) is AGP's default at minSdk 23+.
 
     defaultConfig {
         // TODO: Specify your own unique Application ID (https://developer.android.com/studio/build/application-id.html).

src/serious_python/lib/src/python_versions.dart

@@ -11,7 +11,7 @@ const dartBridgeVersionEnvironmentVariable = "DART_BRIDGE_VERSION";
 
 /// python-build release the bundled runtimes come from (YYYYMMDD).
 const pythonReleaseDate = "20260614";
-const dartBridgeVersion = "1.3.2";
+const dartBridgeVersion = "1.4.0";
 const defaultPythonVersion = "3.14";
 
 class PythonRelease {

src/serious_python_android/CHANGELOG.md

@@ -1,7 +1,8 @@
 ## 3.0.0
 
-* **In-process Python (dart_bridge FFI).** The Python lifecycle now runs through `libdart_bridge.so` (from `flet-dev/dart-bridge` **1.2.3**) instead of a socket transport; `extractNativeLibs=true` keeps `libpython*.so` extractable for `dlopen`.
-* **Breaking change:** requires Flutter **3.44.2**. Moves to AGP **8.11.1**, Gradle 8.11.1, `compileSdk` **36** and Java **17**; `useLegacyPackaging` + `keepDebugSymbols` for the bundled `libpython*.so` replace the removed `android.bundle.enableUncompressedNativeLibs`.
+* **In-process Python (dart_bridge FFI).** The Python lifecycle now runs through `libdart_bridge.so` (from `flet-dev/dart-bridge` **1.4.0**) instead of a socket transport.
+* **Native modules are memory-mapped from the APK — no more `useLegacyPackaging`.** Python extension modules (stdlib `lib-dynload` and site-packages) are relocated into `jniLibs/<abi>/lib<mangled>.so` and loaded directly from the APK by a custom `sys.meta_path` finder that resolves them from `.soref` markers — no extraction to disk, still ABI-split by the Play Store. Pure Python ships in **stored, ABI-common asset zips** read via `zipimport`, so the stdlib is no longer duplicated per ABI. This replaces the previous scheme of zipping stdlib/site-packages into fake `lib*.so` files (which required `useLegacyPackaging`). The dart-bridge Android binary now uses the full CPython API (`PyConfig`) to install the finder before `site` runs. Set **`SERIOUS_PYTHON_ANDROID_EXTRACT_PACKAGES`** (comma-separated relative paths) to ship path-hungry packages extracted to disk instead.
+* **Breaking change:** requires Flutter **3.44.2**. Moves to AGP **8.11.1**, Gradle 8.11.1, `compileSdk` **36**, Java **17**, and the **Kotlin-DSL** Gradle build (`build.gradle.kts`).
 * `build.gradle` resolves the Python version from the generated `python_versions.properties` (a snapshot of python-build's `manifest.json`): `SERIOUS_PYTHON_VERSION` selects the version; the full version, build date and `dart_bridge` version derive from the table, with `SERIOUS_PYTHON_FULL_VERSION` / `SERIOUS_PYTHON_BUILD_DATE` / `DART_BRIDGE_VERSION` left as escape hatches. Downloads continue to use python-build's date-keyed release scheme.
 * Remove the scaffold `getPlatformVersion` method.
 

src/serious_python_android/android/.gitignore

@@ -8,3 +8,4 @@
 /captures
 .cxx
 /src/main/jniLibs
+/src/main/assets