AtomicAssets Track B: end-to-end Docker stack (Jungle 4) + config segment writer#8
Conversation
Stand up the Rust+WormDB ("v3") AtomicAssets serving lifecycle on Docker, fed by the
Jungle 4 testnet: snapshot -> Mongo (snapshot-load --tables atomic) -> .wseg (aa-build)
-> served by wormdb-server (lightapi + atomicassets domains composed).
- deploy/atomicassets/docker-compose.yml: mongodb + aa-loader (one-shot) + wormdb-aa
- Dockerfile.tools (Rust: snapshot-load + aa-build), Dockerfile.wormdb (slim runtime),
build-wormdb.sh (the 4-repo Zig build, quic=false, into bin/wormdb)
- entrypoints, wormdb.json (trusted-network: require_auth=false), README
Chain source = the existing WSL Jungle 4 node (host.docker.internal:28888/:28080); no
nodeos container. Mongo is the aa-build bridge AND the parallel state oracle (state stays
dual-homed Mongo+WormDB while WormDB is proven; store-once is a later flip).
Validated on the jungle4-v8 snapshot (block 269541093): 1552 assets / 188 templates /
48 schemas / 31 collections decoded with 0 errors; GET /atomicassets/v1/assets?owner=...
returns assets newest-first, byte-matching the Mongo oracle.
Security review of the prior commit flagged the Mongo + WormDB host port mappings as off-host network exposure (Mongo has no auth; the gateway runs require_auth=false for this trusted-LOCAL proving deploy). Bind all host port mappings to 127.0.0.1 — inter-service traffic uses the aa-stack bridge (mongodb:27017 / aa-wormdb:6389) and is unaffected; only the host-side convenience mappings change. 6389 (WormWire) / 6390 (HTTP) documented inline. Deferred (matches the existing wormdb-server/docker/build-linux.sh pattern, not new here): pin the Zig tarball SHA-256 + a meshguard commit in build-wormdb.sh.
aa-build now streams atomicassets-config and writes a sentinel-keyed config record into a new .wseg table (TABLE_AA_CONFIG=22): encode_config/decode_config carry contract, version, collection_format (reusing the schema-format blob), and supported_tokens [(token_contract, symbol, precision)]. AtomicBuilder gains push_config + a finish() emit; the matching Zig decoder + /atomicassets/v1/config route land in wormdb-domain-atomicassets. (version is absent from the snapshot's config table — sourced from the tokenconfigs delta — so it is stored empty for now; a snapshot-load follow-up can fill it.) cargo test -p wseg-build green (config_round_trips, table_ids_are_unique incl. 22).
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request implements the serialization, deserialization, and integration of the AtomicAssets config singleton (TABLE_AA_CONFIG) into the segment builder, along with Docker deployment scripts and configurations for the end-to-end stack. The review feedback highlights critical improvements for robustness and safety, including preventing serialization desynchronization in encode_config when collection formats or supported tokens exceed length limits, adding bounds checking in decode_config to avoid panics on malformed input, and enhancing build script debuggability by removing silent redirections during package installation.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| pub fn encode_config( | ||
| contract: u64, | ||
| version: &str, | ||
| collection_format: &[(String, String)], | ||
| supported_tokens: &[(u64, String, i64)], | ||
| ) -> Vec<u8> { | ||
| let mut o = Vec::new(); | ||
| o.push(ASSET_VERSION); | ||
| pu64(&mut o, contract); | ||
| let vb = version.as_bytes(); | ||
| let vlen = vb.len().min(u16::MAX as usize); | ||
| pu16(&mut o, vlen as u16); | ||
| o.extend_from_slice(&vb[..vlen]); | ||
| let fmt = encode_schema_format(collection_format); | ||
| pu16(&mut o, fmt.len().min(u16::MAX as usize) as u16); | ||
| o.extend_from_slice(&fmt); | ||
| pu16(&mut o, supported_tokens.len() as u16); | ||
| for (tc, sym, prec) in supported_tokens { | ||
| pu64(&mut o, *tc); | ||
| let sb = sym.as_bytes(); | ||
| let sl = sb.len().min(255); | ||
| o.push(sl as u8); | ||
| o.extend_from_slice(&sb[..sl]); | ||
| o.push(*prec as u8); | ||
| } | ||
| o | ||
| } |
There was a problem hiding this comment.
There are two potential serialization/deserialization mismatch bugs in encode_config:
- The
collection_formatencoded blob (fmt) has its length prefix capped atu16::MAX, but the entirefmtvector is appended to the output buffer. Iffmt.len() > u16::MAX, this will cause the decoder to desynchronize and corrupt subsequent fields. - The
supported_tokenslength prefix is cast tou16without capping, but the entire list is serialized. Ifsupported_tokens.len() > u16::MAX, the length prefix will truncate (wrap around), but all tokens will still be written, leading to a decoder desynchronization.
We should cap both the length prefixes and the serialized slices/iterators to u16::MAX.
pub fn encode_config(
contract: u64,
version: &str,
collection_format: &[(String, String)],
supported_tokens: &[(u64, String, i64)],
) -> Vec<u8> {
let mut o = Vec::new();
o.push(ASSET_VERSION);
pu64(&mut o, contract);
let vb = version.as_bytes();
let vlen = vb.len().min(u16::MAX as usize);
pu16(&mut o, vlen as u16);
o.extend_from_slice(&vb[..vlen]);
let fmt = encode_schema_format(collection_format);
let flen = fmt.len().min(u16::MAX as usize);
pu16(&mut o, flen as u16);
o.extend_from_slice(&fmt[..flen]);
let num_tokens = supported_tokens.len().min(u16::MAX as usize);
pu16(&mut o, num_tokens as u16);
for (tc, sym, prec) in supported_tokens.iter().take(num_tokens) {
pu64(&mut o, *tc);
let sb = sym.as_bytes();
let sl = sb.len().min(255);
o.push(sl as u8);
o.extend_from_slice(&sb[..sl]);
o.push(*prec as u8);
}
o
}| /// Decode the config singleton: `(contract, version, collection_format, supported_tokens)`. | ||
| pub fn decode_config(b: &[u8]) -> (u64, String, Vec<(String, String)>, Vec<(u64, String, u8)>) { | ||
| let mut p = 1usize; | ||
| let contract = gu64(b, &mut p); | ||
| let vlen = gu16(b, &mut p) as usize; | ||
| let version = String::from_utf8_lossy(&b[p..p + vlen]).into_owned(); | ||
| p += vlen; | ||
| let fmt_len = gu16(b, &mut p) as usize; | ||
| let collection_format = decode_schema_format(&b[p..p + fmt_len]); | ||
| p += fmt_len; | ||
| let n = gu16(b, &mut p) as usize; | ||
| let mut tokens = Vec::with_capacity(n); | ||
| for _ in 0..n { | ||
| let tc = gu64(b, &mut p); | ||
| let sl = b[p] as usize; | ||
| p += 1; | ||
| let sym = String::from_utf8_lossy(&b[p..p + sl]).into_owned(); | ||
| p += sl; | ||
| let prec = b[p]; | ||
| p += 1; | ||
| tokens.push((tc, sym, prec)); | ||
| } | ||
| (contract, version, collection_format, tokens) | ||
| } |
There was a problem hiding this comment.
decode_config does not perform any bounds checking on the input slice b. If the input buffer is malformed, truncated, or corrupted, slicing or indexing out of bounds (e.g., &b[p..p + vlen], b[p]) will cause a panic, leading to a potential Denial of Service (DoS) or crash.
We should make the decoder robust by returning an Option and performing bounds checks before slicing or indexing.
/// Decode the config singleton: `(contract, version, collection_format, supported_tokens)`.
pub fn decode_config(b: &[u8]) -> Option<(u64, String, Vec<(String, String)>, Vec<(u64, String, u8)>)> {
if b.is_empty() {
return None;
}
let mut p = 1usize;
if b.len() < p + 8 { return None; }
let contract = gu64(b, &mut p);
if b.len() < p + 2 { return None; }
let vlen = gu16(b, &mut p) as usize;
if b.len() < p + vlen { return None; }
let version = String::from_utf8_lossy(&b[p..p + vlen]).into_owned();
p += vlen;
if b.len() < p + 2 { return None; }
let fmt_len = gu16(b, &mut p) as usize;
if b.len() < p + fmt_len { return None; }
let collection_format = decode_schema_format(&b[p..p + fmt_len]);
p += fmt_len;
if b.len() < p + 2 { return None; }
let n = gu16(b, &mut p) as usize;
let mut tokens = Vec::with_capacity(n);
for _ in 0..n {
if b.len() < p + 8 { return None; }
let tc = gu64(b, &mut p);
if p >= b.len() { return None; }
let sl = b[p] as usize;
p += 1;
if b.len() < p + sl { return None; }
let sym = String::from_utf8_lossy(&b[p..p + sl]).into_owned();
p += sl;
if p >= b.len() { return None; }
let prec = b[p];
p += 1;
tokens.push((tc, sym, prec));
}
Some((contract, version, collection_format, tokens))
}| fn config_round_trips() { | ||
| let fmt = vec![ | ||
| ("name".to_string(), "string".to_string()), | ||
| ("img".to_string(), "ipfs".to_string()), | ||
| ]; | ||
| let tokens = vec![(crate::name::encode("eosio.token"), "EOS".to_string(), 4i64)]; | ||
| let blob = encode_config(crate::name::encode("atomicassets"), "1.2.0", &fmt, &tokens); | ||
| let (c, v, f, t) = decode_config(&blob); | ||
| assert_eq!(c, crate::name::encode("atomicassets")); | ||
| assert_eq!(v, "1.2.0"); | ||
| assert_eq!(f, fmt); | ||
| assert_eq!(t, vec![(crate::name::encode("eosio.token"), "EOS".to_string(), 4u8)]); | ||
| } |
There was a problem hiding this comment.
Update the test to handle the new Option return type of decode_config.
| fn config_round_trips() { | |
| let fmt = vec![ | |
| ("name".to_string(), "string".to_string()), | |
| ("img".to_string(), "ipfs".to_string()), | |
| ]; | |
| let tokens = vec![(crate::name::encode("eosio.token"), "EOS".to_string(), 4i64)]; | |
| let blob = encode_config(crate::name::encode("atomicassets"), "1.2.0", &fmt, &tokens); | |
| let (c, v, f, t) = decode_config(&blob); | |
| assert_eq!(c, crate::name::encode("atomicassets")); | |
| assert_eq!(v, "1.2.0"); | |
| assert_eq!(f, fmt); | |
| assert_eq!(t, vec![(crate::name::encode("eosio.token"), "EOS".to_string(), 4u8)]); | |
| } | |
| fn config_round_trips() { | |
| let fmt = vec![ | |
| ("name".to_string(), "string".to_string()), | |
| ("img".to_string(), "ipfs".to_string()), | |
| ]; | |
| let tokens = vec![(crate::name::encode("eosio.token"), "EOS".to_string(), 4i64)]; | |
| let blob = encode_config(crate::name::encode("atomicassets"), "1.2.0", &fmt, &tokens); | |
| let (c, v, f, t) = decode_config(&blob).expect("failed to decode config"); | |
| assert_eq!(c, crate::name::encode("atomicassets")); | |
| assert_eq!(v, "1.2.0"); | |
| assert_eq!(f, fmt); | |
| assert_eq!(t, vec![(crate::name::encode("eosio.token"), "EOS".to_string(), 4u8)]); | |
| } |
| ZIG_VER=0.16.0 | ||
| echo "=== tools ===" | ||
| apt-get update -qq | ||
| apt-get install -y -qq curl xz-utils git build-essential cmake libsodium-dev ca-certificates pkg-config >/dev/null 2>&1 |
There was a problem hiding this comment.
Redirecting both stdout and stderr of apt-get install to /dev/null (>/dev/null 2>&1) makes it extremely difficult to troubleshoot build failures (e.g., network issues, missing repositories, or package conflicts). Since -qq is already used to keep the output quiet, we should remove the redirection so that any actual errors are printed to the console.
| apt-get install -y -qq curl xz-utils git build-essential cmake libsodium-dev ca-certificates pkg-config >/dev/null 2>&1 | |
| apt-get install -y -qq curl xz-utils git build-essential cmake libsodium-dev ca-certificates pkg-config |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 89d6c2a136
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if let Some(cfg) = self.config.take() { | ||
| tables.push(Table { |
There was a problem hiding this comment.
When a segment that contains the new TABLE_AA_CONFIG is later compacted, aa_live::compact_with rebuilds the output through AtomicBuilder::new() and only replays schemas, templates, and assets; there is no raw config setter/carry-over path, so self.config remains None and this branch omits the config table from the compacted segment. In deployments that use the live overlay/compaction path, /atomicassets/v1/config will work on the initial snapshot-built segment but disappear after the first compaction.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Pull request overview
This PR adds a runnable Docker-based “Track B” end-to-end stack for serving AtomicAssets from a snapshot-derived .wseg via WormDB, and extends aa-build to emit a config singleton into the segment for /v1/config support.
Changes:
- Add
deploy/atomicassets/Docker Compose stack (MongoDB + one-shot loader + WormDB server), including build scripts, entrypoints, and documentation for Jungle 4. - Extend
aa-build/AtomicBuilderto stream anatomicassets-configMongo collection and write a newTABLE_AA_CONFIG=22singleton table. - Add binary encode/decode support and a unit test for the config singleton blob format.
Reviewed changes
Copilot reviewed 14 out of 14 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| deploy/atomicassets/wormdb.json | Adds WormDB server/gateway/auth config used by the runtime container. |
| deploy/atomicassets/README.md | Documents the local proving deployment and workflow (snapshot → Mongo → .wseg → serve). |
| deploy/atomicassets/entrypoint-server.sh | Adds a server entrypoint that validates the segment and starts WormDB with the AtomicAssets segment attached. |
| deploy/atomicassets/entrypoint-loader.sh | Adds a loader entrypoint to snapshot-load into Mongo and run aa-build to produce aa.wseg. |
| deploy/atomicassets/Dockerfile.wormdb | Creates a slim runtime image that ships the prebuilt wormdb binary and configs. |
| deploy/atomicassets/Dockerfile.tools | Builds and packages the Rust tooling (snapshot-load, aa-build) into a tools image. |
| deploy/atomicassets/docker-compose.yml | Defines the end-to-end Compose stack (Mongo + loader + server), loopback-bound ports, and profiles. |
| deploy/atomicassets/build-wormdb.sh | Adds an out-of-band Dockerized Zig build script to produce deploy/atomicassets/bin/wormdb. |
| deploy/atomicassets/.gitignore | Ignores local artifacts (bin/, snapshots, .env, *.wseg). |
| deploy/atomicassets/.env.example | Provides environment defaults for snapshot dir, chain, ports, and Mongo cache sizing. |
| crates/wseg-build/src/bin/aa_build.rs | Streams the new atomicassets-config collection before schemas/templates/assets. |
| crates/wseg-build/src/aa_tables.rs | Introduces TABLE_AA_CONFIG=22 and updates table-ID uniqueness test coverage. |
| crates/wseg-build/src/aa_builder.rs | Captures config from Mongo and writes a sentinel-keyed config singleton table into the segment. |
| crates/wseg-build/src/aa_binfmt.rs | Adds encode_config/decode_config and a round-trip unit test for the config blob. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| let fmt = encode_schema_format(collection_format); | ||
| pu16(&mut o, fmt.len().min(u16::MAX as usize) as u16); | ||
| o.extend_from_slice(&fmt); | ||
| pu16(&mut o, supported_tokens.len() as u16); | ||
| for (tc, sym, prec) in supported_tokens { | ||
| pu64(&mut o, *tc); | ||
| let sb = sym.as_bytes(); | ||
| let sl = sb.len().min(255); | ||
| o.push(sl as u8); | ||
| o.extend_from_slice(&sb[..sl]); | ||
| o.push(*prec as u8); | ||
| } |
| BIN="$(find /snap -maxdepth 1 -name 'snapshot-*.bin' | head -1 || true)" | ||
| [[ -n "$BIN" ]] || BIN="$(find /snap -maxdepth 1 -name '*.bin' ! -name '*archive*' | head -1 || true)" |
| ZIG_VER=0.16.0 | ||
| echo "=== tools ===" | ||
| apt-get update -qq | ||
| apt-get install -y -qq curl xz-utils git build-essential cmake libsodium-dev ca-certificates pkg-config >/dev/null 2>&1 | ||
| echo "=== zig ${ZIG_VER} ===" | ||
| cd /opt && curl -fsSL "https://ziglang.org/download/${ZIG_VER}/zig-x86_64-linux-${ZIG_VER}.tar.xz" -o zig.tar.xz && tar xf zig.tar.xz | ||
| export PATH="/opt/zig-x86_64-linux-${ZIG_VER}:${PATH}" | ||
| zig version |
| echo "=== meshguard from github (replaces the windows symlink) ===" | ||
| rm -rf /work/wormdb/deps/meshguard | ||
| git clone -q https://github.com/igorls/meshguard /work/wormdb/deps/meshguard | ||
| echo "=== zig build (quic=false) ===" |
| // config singleton (one sentinel-keyed entry), if the `atomicassets-config` doc was seen. | ||
| if let Some(cfg) = self.config.take() { | ||
| tables.push(Table { | ||
| table_id: TABLE_AA_CONFIG, | ||
| index: vec![IndexEntry { | ||
| key: SENTINEL_KEY, | ||
| off: 0, | ||
| len: cfg.len() as u32, | ||
| }], | ||
| arena: cfg, | ||
| }); | ||
| } |
… (PR review) Addresses gemini/copilot review on #8: - aa_binfmt encode_config: cap the length prefix AND the appended bytes together (collection_format blob + supported_tokens iteration) so the header can never desync the body. (gemini HIGH / copilot) - aa_binfmt decode_config: fully bounds-checked, returns Option — a truncated/corrupt blob yields None instead of panicking (DoS). Test updated + malformed-input cases added. (gemini SECURITY-HIGH) - aa_builder: the tiny-segment test now also pushes an atomicassets-config doc, exercising push_config → finish() emit. (copilot) - build-wormdb.sh: stop hiding apt-get errors (-qq stays); SHA-256-pin the Zig tarball + commit-pin meshguard, for reproducible, verified builds. (gemini MED / copilot supply-chain) - entrypoint-loader.sh: deterministic snapshot selection (sort + lexically-newest, warn on multiple) instead of `find | head -1`. (copilot) Deferred (tracked): aa_live::compact_with does not yet carry the config table into a compacted segment — a follow-up for the live-serving daemon path (the frozen-segment Docker stack is unaffected). cargo test -p wseg-build green (27).
|
Addressed the gemini / copilot / codex review (pushed
Deferred (tracked): cargo test -p wseg-build green (27). |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: df72934abf
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| /// `atomicassets-config` (singleton) → the config blob served at `/v1/config`. | ||
| fn push_config(&mut self, d: &Document) { | ||
| let contract = doc_str(d, "contract").map(name::encode).unwrap_or(0); | ||
| let version = doc_str(d, "version").unwrap_or("").to_string(); |
There was a problem hiding this comment.
In the Docker/bootstrap path added here, aa-build reads atomicassets-config documents created by snapshot-load --tables atomic; I checked crates/snapshot-load/src/atomicassets.rs::map_config, and that producer writes contract, counters, collection_format, supported_tokens, and block_num, but never a version field. As a result every real segment built by this flow falls through to "", so /atomicassets/v1/config serves an empty version even though the binary format and API payload now claim to carry it; this should be sourced from server/contract config or deliberately omitted rather than silently encoding an invalid value.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed — sourced properly rather than emitting a blank.
653c71a: the snapshot-load schema-registry pre-pass now also reads atomicassets:tokenconfigs (the v2 {standard, version} table where the contract version actually lives — not the config row) and stashes it on SchemaRegistry; map_config emits version when present and omits the field on pre-v2 chains that lack tokenconfigs, rather than encoding "". 756ed87 (companion serve PR) makes /config drop the field when empty too.
Validated on Jungle 4: tokenconfigs.version = "1.3.1" → the Mongo config doc and /atomicassets/v1/config now report "1.3.1".
…onfigs (PR review)
/atomicassets/v1/config was serving version:"" because map_config never captured it — the
AtomicAssets `version` lives in the contract's `tokenconfigs` table ({standard, version}), not
the `config` row. The schema-registry pre-pass now also reads `atomicassets:tokenconfigs` and
stashes `version` on SchemaRegistry; map_config emits it when present and OMITS the field entirely
on pre-v2 chains that lack tokenconfigs (rather than encoding an invalid "").
Validated on Jungle 4: tokenconfigs.version = "1.3.1" → the Mongo config doc + the served
/config now report "1.3.1". cargo check/clippy/fmt green.
Stands up the Rust + WormDB ("v3") AtomicAssets serving lifecycle on Docker, fed by the Jungle 4 testnet, plus the config segment writer. Companion serve PR: igorls/wormdb-domain-atomicassets#1.
What's here
deploy/atomicassets/—docker-compose.yml(mongodb + aa-loader + wormdb-aa),Dockerfile.tools(Rust: snapshot-load + aa-build),Dockerfile.wormdb+build-wormdb.sh(the 4-repo Zig build → bin/wormdb), entrypoints,wormdb.json, README. Pipeline: snapshot → Mongo (snapshot-load --tables atomic) →.wseg(aa-build) → served bywormdb-server(lightapi + atomicassets domains composed).crates/wseg-build—aa-buildnow emits the config singleton into a new.wsegtable (TABLE_AA_CONFIG=22):encode_config/decode_configcarry contract / version / collection_format (reusing the schema-format blob) / supported_tokens. The Zig decoder +/v1/configroute are in the companion PR.Chain source
The existing WSL Jungle 4 node (
host.docker.internal:28888chain API /:28080SHiP); no nodeos container. State is intentionally dual-homed (Mongo + WormDB) for the proving phase — we diff WormDB against the trusted Mongo tier; "store-once" (blacklist AA deltas) is a later config flip once WormDB is proven.Validated (jungle4-v8 snapshot, block 269541093)
1552 assets / 188 templates / 48 schemas / 31 collections / config decoded with 0 errors; the served API matches the Mongo oracle. Host ports are bound to loopback (
require_auth:false= trusted-LOCAL proving deploy — surfaced by the commit security review and hardened).Notes
build-wormdb.shmirrors the existingwormdb-server/docker/build-linux.shflow (unverified Zig download + meshguard HEAD clone) — pinning those is a shared follow-up across both scripts.bin/,snapshot/,.env,*.wsegare gitignored.Part of Phase 1 of the Track-B AtomicAssets plan.