… to post link binary payloads (#200) ## Description This PR introduces the encoderfile embedded binary format and packaging pipeline. Instead of embedding model artifacts via linker sections or include_bytes!, the runtime binary now has a self-describing payload appended at EOF, located via a fixed-size footer and protobuf manifest. Model artifacts (weights, tokenizer, config, transforms) are streamed from disk using seek-based IO and loaded lazily at runtime, with a single dynamic dispatch on model type followed by fully statically-typed execution. This design avoids platform-specific linker behavior, keeps builds fast, enables inspection and tooling, and cleanly separates compilation from packaging. ### Benefits - No more cargo-in-cargo - Lazy loading of large assets (e.g., model weights) ### Why manually specified file format instead of appended sections? We intentionally avoid embedding artifacts via custom linker sections (.data, extern "C" symbols). While simpler initially, this approach is platform-specific, eagerly loads large blobs into memory, and provides no clear boundary for versioning or inspection. The appended encoderfile container instead offers predictable cross-platform behavior, lazy loading, and a stable, versioned format. ## Testing To test, run: ```bash cargo build --bin encoderfile --release ``` Then run: ```bash cargo run -- build -f test_config.yml --base-binary-path target/release/encoderfile ``` To run encoderfile: ```bash chmod +x ./my-model-2.encoderfile ./my-model-2.encoderfile infer "hello" ``` ## Notes - Tokenizer config is now located in `TokenizerService` instead of `Config` - Base binaries are compiled from `encoderfile-runtime` - Base binary must be explicitly specified until #204 is closed. For testing, run `cargo build --bin encoderfile --release`, and use `target/release/encoderfile-runtime`. - Dockerfile now uses debian:bookworm-slim as a base image since we no longer need Cargo toolchain. - we're explicitly encoding everything in little-endian, BE is theoretically possible but not a priority given our prioritized arch targets ## Out of Scope - Integration tests. See related issue: #203 - Pulling base binaries from GH releases. See related issue: #204 - Docs updates. See related issue: #205 - Encoderfile image CLI refactor. See related issue: #206

Docs: post-link encoderfile format #205

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions