Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 16 additions & 6 deletions .claude/CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,34 +8,44 @@ for tamper-evident responses, and a policy gate for access control.

## Architecture

Estate architecture law (day 1, non-negotiable): **ABI = Idris2, FFI = Zig,
API = Zig**. Never V-lang, Rust, or C for these layers.

- **ABI**: Idris2 (`src/abi/`) — formal type definitions with proofs
- **FFI**: Zig (`ffi/zig/`) — C-compatible implementation layer
- **API Gateway**: V-lang (`src/api/v/`) — GraphQL + REST server on port 4000
- **API Gateway**: Zig (`src/api/zig/`) — GraphQL + REST server on port 4000
- **Probes**: LibreSpeed (speed), Hyperglass (BGP), SmokePing (jitter)
- **Data**: Redis (cache/audit), VerisimDB (bitemporal, future)
- **Container**: Podman Compose with Chainguard base images

V-lang (`src/api/v/`) was removed 2026-05-16 (estate-wide V ban). The
`src/api/rust/` crate and the old `MIGRATION.adoc` "→ Rust" text are
off-policy drift — Rust is **not** an API language here; do not build,
extend, or migrate to it. Canonical = the Zig gateway.

## Allowed Languages

| Language | Use Case |
|----------|----------|
| **V-lang** | API gateway, HTTP server, service clients |
| **Idris2** | ABI definitions, type proofs |
| **Zig** | FFI implementation, C ABI bridge |
| **Idris2** | ABI definitions, type proofs (`src/abi/`) |
| **Zig** | API gateway, HTTP server, service clients, FFI / C ABI bridge |
| **Nickel** | K9 spec assembly |
| **Guile Scheme** | .machine_readable/6a2/STATE.a2ml, .machine_readable/6a2/META.a2ml, .machine_readable/6a2/ECOSYSTEM.a2ml |
| **Bash** | Scripts, automation |

Banned here: V-lang, Rust (for api/abi/ffi), TypeScript, Go, raw C.

## Build & Run

```bash
# Run all services (gateway + probes + redis)
podman-compose up -d

# Build gateway only
v src/api/v/main.v -o aerie-gateway
# Build gateway only (Idris2 ABI + Zig API/FFI)
zig build -Doptimize=ReleaseSafe # -> zig-out/bin/aerie-gateway

# Run tests
zig build test
cd ffi/zig && zig build test
cd ffi/zig && zig build test-integration
```
Expand Down
55 changes: 30 additions & 25 deletions Containerfile
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,16 @@
#
# Containerfile — Aerie Gateway (Triple-Mount API Server)
#
# Estate architecture law: ABI = Idris2 (src/abi/), API + FFI = Zig.
# The V-lang implementation (src/api/v/) was removed under the estate-wide
# V-lang ban (deprecated 2026-04-12, removed 2026-05-16). The canonical
# implementation is the Zig gateway src/api/zig/ built by ./build.zig.
# (The src/api/rust/ crate and the old V→Rust MIGRATION.adoc are off-policy
# drift — Rust is not an API language here. Tracked separately for removal.)
#
# Multi-stage build:
# Stage 1: Build the V-lang gateway binary using Alpine + V compiler
# Stage 2: Copy binary into a minimal Chainguard static image
# Stage 1: build libzig_api (developer-ecosystem/zig-api FFI) + aerie-gateway
# Stage 2: copy the static binary into a minimal Chainguard image
#
# Exposes:
# 4000 — HTTP (REST + GraphQL)
Expand All @@ -13,32 +20,30 @@
# Build: podman build -t aerie-gateway -f Containerfile .
# Run: podman run -p 4000:4000 -p 4001:4001 aerie-gateway

# --- Stage 1: Build ---
# --- Stage 1: Build (Zig) ---
FROM cgr.dev/chainguard/wolfi-base:latest AS builder

# Install build dependencies: V-lang compiler + C toolchain
RUN apk add --no-cache \
gcc \
glibc-dev \
git \
make \
wget

# Install V-lang compiler
WORKDIR /opt
RUN git clone --depth 1 https://github.com/vlang/v.git && \
cd v && \
make && \
ln -s /opt/v/v /usr/local/bin/v

# Copy gateway source code
# Zig toolchain + git (for the zig-api FFI dependency)
RUN apk add --no-cache zig git

# Build the external zig-api FFI dependency (sparse checkout — same org).
# build.zig accepts -Dzig-api-lib-path / -Dzig-api-include-path overrides;
# CI may instead inject a prebuilt libzig_api and skip this clone.
WORKDIR /deps
RUN git clone --depth 1 --filter=blob:none --sparse \
https://github.com/hyperpolymath/developer-ecosystem.git && \
cd developer-ecosystem && \
git sparse-checkout set zig-api && \
cd zig-api/ffi/zig && \
zig build -Doptimize=ReleaseSafe

# Build the aerie-gateway Zig binary against the freshly built zig-api
WORKDIR /app
COPY src/api/v/ ./src/api/v/
COPY src/api/graphql/ ./src/api/graphql/
COPY src/api/proto/ ./src/api/proto/

# Build the gateway binary (statically linked for Chainguard static image)
RUN v -prod -cc gcc -cflags '-static' -o /app/aerie-gateway src/api/v/main.v
COPY . .
RUN zig build -Doptimize=ReleaseSafe \
-Dzig-api-lib-path=/deps/developer-ecosystem/zig-api/ffi/zig/zig-out/lib \
-Dzig-api-include-path=/deps/developer-ecosystem/zig-api/ffi/zig/zig-out/include && \
cp zig-out/bin/aerie-gateway /app/aerie-gateway

# --- Stage 2: Runtime ---
FROM cgr.dev/chainguard/static:latest
Expand Down
40 changes: 20 additions & 20 deletions EXPLAINME.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,13 @@ The README makes claims. This file backs them up.

**Claim 1: "Zero-telemetry speedtest… BGP path visibility… Jitter persistence… Hardened access with rate-limiting"** (README, Key Features section)

Aerie integrates three network diagnostic tools behind a single V-lang gateway (port 4000):
Aerie integrates three network diagnostic tools behind a single Zig gateway (port 4000):

1. **LibreSpeed** (port 8080) — Zero-telemetry speedtest for bandwidth measurement
2. **Hyperglass** (port 8082) — BGP looking glass for routing path analysis
3. **SmokePing** (port 8081) — ICMP latency monitor for jitter tracking

The gateway in `src/api/v/main.v` acts as a reverse proxy and access control layer. All requests are logged to Redis and wrapped in `ProofEnvelope` structures (SHA-256 hash + query ID + timestamp). The policy gate in `src/api/v/policy.v` checks X-Api-Key headers and rate-limits by key.
The gateway in `src/api/zig/main.zig` acts as a reverse proxy and access control layer. All requests are logged to Redis and wrapped in `ProofEnvelope` structures (SHA-256 hash + query ID + timestamp). The policy gate in `src/api/zig/policy.zig` checks X-Api-Key headers and rate-limits by key. (Estate rule: no gatekeeperless gateways — every gateway fronts a policy gate.)

*How it works:* The gateway reads configuration from environment variables (`LIBRESPEED_URL`, `HYPERGLASS_URL`, `REDIS_URL`). Incoming requests hit the policy gate first. If authorized, the request is forwarded to the appropriate backend service. The response is hashed and wrapped in ProofEnvelope. The full request/response is logged to Redis for audit trail. Caveat: Phase 1 (current) uses SHA-256; Phase 2+ will add Ed448 signatures for cryptographic proof. Rate limiting is per-key; there is no global circuit breaker yet.

Expand All @@ -26,12 +26,12 @@ The architecture distinguishes between:
* **Passive forensics** — Network packet inspection via Zeek (IDS) or Suricata (threat detection)
* **Active forensics** — Measurement probes (LibreSpeed speedtest, SmokePing pings, Hyperglass BGP queries)

The clients in `src/api/v/` are implemented as V-lang modules:
- `librespeed_client.v` — Speed measurement wrapper
- `smokeping_client.v` — Jitter/latency monitoring
- `hyperglass_client.v` — BGP path lookup
The clients in `src/api/zig/` are implemented as Zig modules:
- `librespeed_client.zig` — Speed measurement wrapper
- `smokeping_client.zig` — Jitter/latency monitoring
- `hyperglass_client.zig` — BGP path lookup

Each client has a corresponding resolver in `src/api/v/resolvers.v` that fields GraphQL/REST requests.
Each client has a corresponding resolver in `src/api/zig/resolvers.zig` that fields GraphQL/REST requests.

*How it works:* Passive Zeek/Suricata instances listen on network interfaces and post alerts to the audit log (Redis). Active probes are on-demand: user requests trigger a new speed test or BGP query. Results are cached in Redis (TTL configurable per probe type). The frontend dashboard displays both real-time probe results and historical passive alerts. Caveat: Zeek/Suricata are not yet containerized in the compose stack (planned Phase 2); only the three primary probes are running currently.

Expand All @@ -51,9 +51,8 @@ The proof envelope concept (hash + metadata) is reused in all hyperpolymath API
|===
| Technology | Learn More | Why

| **V-lang** | https://vlang.io | API gateway, HTTP server, no-GC performance
| **Idris2 ABI** | https://www.idris-lang.org | Formal proofs of proof envelope correctness (Phase 2+)
| **Zig FFI** | https://ziglang.org | Cryptographic hash functions, C interop
| **Idris2** | https://www.idris-lang.org | ABI: formal proofs of proof-envelope correctness (estate ABI standard)
| **Zig** | https://ziglang.org | API gateway, HTTP server, service clients, FFI / C interop (estate API+FFI standard)
| **Nickel** | https://nickel-lang.org | K9 spec assembly for component definitions
| **Podman Compose** | https://podman.io | Container orchestration, Chainguard base images
| **Redis** | https://redis.io | Audit log, cache, real-time event stream
Expand All @@ -65,16 +64,17 @@ The proof envelope concept (hash + metadata) is reused in all hyperpolymath API
|===
| Path | What's There | Key Details

| `src/api/v/main.v` | Gateway entry point | Triple-mount API (GraphQL, REST, gRPC), protocol config, banner
| `src/api/v/policy.v` | Policy gate | X-Api-Key validation, rate limiting per key, request logging
| `src/api/v/proof.v` | Proof envelope | SHA-256 hashing, ProofEnvelope struct wrapping, response signing
| `src/api/v/librespeed_client.v` | Speed probe client | HTTP wrapper for LibreSpeed results, bandwidth metrics
| `src/api/v/smokeping_client.v` | Jitter probe client | Latency time series, ICMP RTT collection
| `src/api/v/hyperglass_client.v` | BGP client | AS path resolution, prefix origin validation
| `src/api/v/redis_client.v` | Cache/audit | Audit log persistence, request/response archival
| `src/api/v/resolvers.v` | GraphQL resolvers | Field resolvers for all three probe types
| `src/api/graphql/schema.gql.v` | GraphQL schema | Type definitions (Speed, Jitter, BGPPath, ProofEnvelope)
| `src/api/proto/aerie.pb.v` | Protobuf definitions | gRPC message definitions (Phase 1.1+)
| `src/api/zig/main.zig` | Gateway entry point | Triple-mount API (GraphQL, REST, gRPC), protocol config, banner
| `src/api/zig/policy.zig` | Policy gate | X-Api-Key validation, rate limiting per key, request logging
| `src/api/zig/proof.zig` | Proof envelope | SHA-256 hashing, ProofEnvelope struct wrapping, response signing
| `src/api/zig/librespeed_client.zig` | Speed probe client | HTTP wrapper for LibreSpeed results, bandwidth metrics
| `src/api/zig/smokeping_client.zig` | Jitter probe client | Latency time series, ICMP RTT collection
| `src/api/zig/hyperglass_client.zig` | BGP client | AS path resolution, prefix origin validation
| `src/api/zig/redis_client.zig` | Cache/audit | Audit log persistence, request/response archival
| `src/api/zig/resolvers.zig` | GraphQL resolvers | Field resolvers for all three probe types
| `src/abi/*.idr` | Idris2 ABI | Foreign/Layout/Types — formal interface contract
| `src/api/graphql/schema.graphql` | GraphQL schema | Type definitions (Speed, Jitter, BGPPath, ProofEnvelope)
| `src/api/proto/aerie.proto` | Protobuf definitions | gRPC message definitions (Phase 1.1+)
| `specs/` | K9 component specs | active-probe.adoc, alerting-retention.adoc, known-limitations.adoc
| `docker-compose.yml` | Container stack | LibreSpeed, Hyperglass, SmokePing, Redis services
|===
Expand Down
14 changes: 4 additions & 10 deletions Justfile
Original file line number Diff line number Diff line change
Expand Up @@ -18,17 +18,11 @@ specs-verify:
@./specs/tools/check_manifest.sh
@./specs/tools/install_hooks.sh

# --- API Generation ---
# --- API ---

# Generate V-lang stubs from Protobuf
specs-to-v:
@echo "Generating verified V-lang stubs (gRPC)..."
../developer-ecosystem/v-ecosystem/v-api-interfaces/v-grpc/bin/v-grpc-gen src/api/proto/aerie.proto

# Generate V-lang stubs from GraphQL
specs-to-v-gql:
@echo "Generating verified V-lang stubs (GraphQL)..."
../developer-ecosystem/v-ecosystem/v-api-interfaces/v-graphql/bin/v-graphql-gen src/api/graphql/schema.graphql
# No codegen: V-lang stub generation was removed with the V ban (2026-05-16).
# The canonical API is the hand-written Zig gateway in src/api/zig/ (Idris2 ABI
# in src/abi/). The .proto / .graphql schemas remain as the wire contract.



Expand Down
67 changes: 37 additions & 30 deletions MIGRATION.adoc
Original file line number Diff line number Diff line change
@@ -1,65 +1,70 @@
// SPDX-License-Identifier: PMPL-1.0-or-later
// Copyright (c) 2026 Jonathan D.A. Jewell (hyperpolymath) <j.d.a.jewell@open.ac.uk>
= Aerie — V-lang Deprecated (2026-04-12)
= Aerie — V-lang Removed (deprecated 2026-04-12, removed 2026-05-16)
:toc:

== Status

The V-lang implementation in `src/api/v/` is *deprecated* as of 2026-04-12
following the estate-wide V-lang ban (2026-04-10).
The V-lang implementation in `src/api/v/` is *removed* as of 2026-05-16
(deprecated 2026-04-12 following the estate-wide V-lang ban of 2026-04-10).

== Migration Target: Rust
Per the day-1 estate architecture law (ABI = Idris2, FFI = Zig, API = Zig),
the canonical implementation is the **Zig** gateway at `src/api/zig/`, built
by the repository's root `build.zig` (binary `aerie-gateway`). The
Containerfile and `stapeln.toml` build that Zig binary; no V toolchain is
fetched or used anywhere in this repo.

The canonical Rust rewrite is at `src/api/rust/` in this directory.
NOTE: the `src/api/rust/` crate and the earlier "Migration Target: Rust"
text in this document are *off-policy drift* — Rust is not an API language
in this estate. That crate is debt to be removed; it is NOT the migration
target. Retained below only as a historical module-name reference.

[source,toml]
----
# aerie/Cargo.toml workspace already includes this crate:
# "src/api/rust"
----
== Migration Target: Zig

Canonical: `src/api/zig/` (Idris2 ABI in `src/abi/`, Zig FFI in `ffi/zig/`).

=== Module Map

[cols="1,1"]
|===
| V module (deprecated) | Rust module

| `src/api/v/main.v` | `src/api/rust/src/main.rs`
| `src/api/v/policy.v` | `src/api/rust/src/policy.rs`
| `src/api/v/proof.v` | `src/api/rust/src/proof.rs`
| `src/api/v/redis_client.v` | `src/api/rust/src/redis_client.rs`
| `src/api/v/verb_governance.v` | `src/api/rust/src/verb_governance.rs`
| `src/api/v/resolvers.v` | `src/api/rust/src/resolvers.rs`
| `src/api/v/librespeed_client.v` | `src/api/rust/src/backends.rs` (LibreSpeedClient)
| `src/api/v/hyperglass_client.v` | `src/api/rust/src/backends.rs` (HyperglassClient)
| `src/api/v/smokeping_client.v` | `src/api/rust/src/backends.rs` (SmokePingClient)
| `src/api/v/verisim_client.v` | `src/api/rust/src/backends.rs` (VerisimDbClient)
| V module (removed) | Zig module

| `src/api/v/main.v` | `src/api/zig/main.zig`
| `src/api/v/policy.v` | `src/api/zig/policy.zig`
| `src/api/v/proof.v` | `src/api/zig/proof.zig`
| `src/api/v/redis_client.v` | `src/api/zig/redis_client.zig`
| `src/api/v/verb_governance.v` | `src/api/zig/verb_governance.zig`
| `src/api/v/resolvers.v` | `src/api/zig/resolvers.zig`
| `src/api/v/librespeed_client.v` | `src/api/zig/librespeed_client.zig`
| `src/api/v/hyperglass_client.v` | `src/api/zig/hyperglass_client.zig`
| `src/api/v/smokeping_client.v` | `src/api/zig/smokeping_client.zig`
| `src/api/v/verisim_client.v` | `src/api/zig/verisim_client.zig`
|===

== Build and Run

[source,bash]
----
# From aerie/ root
cargo build -p aerie-api
cargo test -p aerie-api
# From aerie/ root (Idris2 ABI + Zig API/FFI)
zig build -Doptimize=ReleaseSafe
zig build test

# Run (all mounts enabled, default ports)
./target/debug/aerie-api
./zig-out/bin/aerie-gateway

# Disable gRPC mount
ENABLE_GRPC=false ./target/debug/aerie-api
ENABLE_GRPC=false ./zig-out/bin/aerie-gateway

# Custom ports
AERIE_PORT=8080 AERIE_GRPC_PORT=8081 ./target/debug/aerie-api
AERIE_PORT=8080 AERIE_GRPC_PORT=8081 ./zig-out/bin/aerie-gateway

# Configure backends
LIBRESPEED_URL=http://localhost:8888 \
HYPERGLASS_URL=http://localhost:8001 \
SMOKEPING_URL=http://localhost:8080 \
REDIS_URL=redis://localhost:6379 \
VERISIMDB_URL=http://localhost:8084 \
./target/debug/aerie-api
./zig-out/bin/aerie-gateway
----

== Feature Parity
Expand All @@ -86,5 +91,7 @@ All V features are preserved:
. Start backends (Redis, LibreSpeed, Hyperglass, SmokePing, VerisimDB) and run
`./target/debug/aerie-api` against the container network.
. Verify `GET /health`, `GET /api/v1/telemetry`, and `POST /api/v1/graphql` against a valid key.
. Once confirmed healthy, delete `src/api/v/*.v`, `v.mod`, and `vpkg.json` (if present).
. [DONE 2026-05-16] `src/api/v/`, `v.mod`, `vpkg.json`, and the V generated
stubs (`src/api/proto/aerie.pb.v`, `src/api/graphql/schema.gql.v`) removed;
Containerfile + `stapeln.toml` repointed to the Rust build.
. Wire `aerie-api` binary into `compose.yml` to replace the V container.
2 changes: 1 addition & 1 deletion ROADMAP.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@

=== Phase 1 Implementation Details

* **Gateway** (V-lang): `src/api/v/main.v` — HTTP on port 4000, gRPC on port 4001
* **Gateway** (Zig, Idris2 ABI): `src/api/zig/main.zig` — HTTP on port 4000, gRPC on port 4001
* **Proof envelopes**: SHA-256 hash-based ("light" mode) on all responses
* **Policy gate**: Phase 1 permissive — all requests allowed, API keys validated and logged
* **Service clients**: LibreSpeed (telemetry), Hyperglass (BGP routes), Redis (cache + audit)
Expand Down
22 changes: 10 additions & 12 deletions TEST-NEEDS.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## CRG Grade: C — ACHIEVED 2026-04-04

**Project**: aerie (V-lang network diagnostic suite API)
**Project**: aerie (Zig network diagnostic suite API; Idris2 ABI)
**CRG Target**: C (comprehensive test coverage)
**Date**: 2026-04-04
**Status**: COMPLETE
Expand Down Expand Up @@ -256,16 +256,14 @@ Comprehensive test coverage for the aerie API gateway has been created, covering
- Ready for `v test aerie_test.v` (requires test harness)
- All assertions and properties explicitly coded

### Notes on V Testing
### Notes on Zig Testing

The aerie codebase uses V-lang modules in `src/api/v/`. Test files need to:
1. Be in same directory as modules OR
2. Include proper imports/dependencies OR
3. Run via `v test` with correct module resolution

**Recommendation**: For next phase, integrate tests into V build pipeline:
The aerie codebase uses Zig modules in `src/api/zig/` (Idris2 ABI in
`src/abi/`, Zig FFI in `ffi/zig/`). V-lang was removed 2026-05-16. Tests
run through the Zig build:
```bash
v test src/api/v/ # Tests all modules with _test.v files
zig build test # API gateway unit tests (src/api/zig/)
cd ffi/zig && zig build test test-integration
```

---
Expand Down Expand Up @@ -302,9 +300,9 @@ All requirements met:

## Next Steps (Phase 2+)

1. **Integrate V test harness**
- Place `_test.v` files in `src/api/v/`
- Run via CI/CD: `v test src/api/v/`
1. **Integrate Zig test harness**
- Place tests alongside `src/api/zig/` modules (Zig `test {}` blocks)
- Run via CI/CD: `zig build test`

2. **Add Phase 2 tests**
- Entitlement-based access control
Expand Down
Loading
Loading