Skip to content

ONNX and CoreML compilation pipeline#23

Open
bs258q wants to merge 1 commit into
cactus-compute:mainfrom
bs258q:feat/onnx-runtime
Open

ONNX and CoreML compilation pipeline#23
bs258q wants to merge 1 commit into
cactus-compute:mainfrom
bs258q:feat/onnx-runtime

Conversation

@bs258q
Copy link
Copy Markdown

@bs258q bs258q commented May 13, 2026

Summary
Adds a model export pipeline for Needle that supports onnx, coreml, and tflite formats, with CLI integration, documentation, and a validation test.

Why
Enables mobile/edge deployment capability for Needle
Provides Apple CoreML support for iOS/macOS/ANE acceleration
Lays groundwork for full JAX→ONNX→CoreML/TFLite export once the complete toolchain is available

Notes
CoreML support is included and exposed through the CLI
Current implementation is intentionally simplified/placeholder-based; full production fidelity will require the full jax2tf / tf2onnx conversion toolchain

Testing
Verified python test_export.py under .venv
Confirmed export path and file creation for ONNX
Added documentation coverage for CoreML export usage
Raptor mini (Preview) • 1x

Signed-off-by: bs258q <bs258q@gmail.com>
@shreyaskarnik
Copy link
Copy Markdown

Hi @bs258q sharing in case it's useful: I've been working on a similar export from a different angle and ended up with a browser-ready ONNX pipeline for Needle (encoder + decoder w/ KV cache, Flax↔PyTorch↔ONNX parity checks, SentencePiece port).

It's out-of-tree (lives in my own repo, not upstreamed), but if any of it's worth folding into this PR, feel free to lift whatever's useful happy to answer questions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants