KineTex

KineTex is a collaborative, visual LaTeX editor that bridges the gap between design tools and typesetting systems. It allows users to manipulate LaTeX documents through an intuitive "Block" interface while maintaining real-time collaboration and professional-grade compilation.

---

🏗️ System Architecture

KineTex is built with a decoupled, high-performance architecture designed for scale and isolation.

graph TD
    UserA[User A] -->|WebSocket| Server[Rust Server]
    UserB[User B] -->|WebSocket| Server
    
    subgraph ServerCore [Server: Sync & Coordination]
        Server -->|mpsc| Actors[Unified Actor Pool]
        Actors -->|Publish| Bus[Event Bus]
        Bus -->|Sidecar| LSP[LSP Handler]
    end

    subgraph Messaging [Infrastructure]
        Server -->|Kafka| Scaling[Horizontal Scaler]
        Server -->|AMQP| MQ[RabbitMQ]
    end

    subgraph Execution [Worker Cluster]
        MQ --> Worker[Rust Worker]
        Worker -->|Spawns| Docker[Tectonic Container]
        Docker -->|Upload| S3[AWS S3]
    end

    S3 -.->|Link| UserA

Loading

Infrastructure Breakdown

• Kafka: Used for horizontal scaling. Updates received on one server node are broadcast across the cluster so users on different nodes see edits in real-time.
• RabbitMQ: Acts as the reliable task queue for LaTeX compilation jobs, decoupling the real-time sync engine from heavy-lifting tasks.
• Docker: The Worker uses the Docker API (bollard) to spin up isolated Tectonic containers. This ensures that every compilation is sandboxed, reproducible, and secure.
• AWS S3: Storage for compiled PDFs and persistent assets.

---

🎨 Frontend: The Visual Bridge

The client is a Next.js application leveraging Effect for robust, functional reactive state management.

• Bridge Pattern: The UI components (Bridge) are decoupled from the synchronization logic. They merely subscribe to the state stream provided by the Adapter.
• Adapter State: Managed via yrs (Rust Yjs port compiled to WebAssembly) to ensure client-side CRDT consistency with near-native performance.
• Confident Optimism: The client buffers local edits in an internal Effect.Queue if the connection is unstable, ensuring a fluid user experience that converges eventually with the server's state.

---

🦀 Server: The Unified Actor Core

The server is a high-performance Rust gateway that prioritizes Zero Contention and deterministic state transitions.

1. Unified Actor Model

Instead of relying on fragile mutexes or read-write locks, the server core is built on a "Tell, Don't Ask" philosophy:

• Connection Actor: Manages all active WebSockets and message routing.
• Subscription Actor: Tracks which users are in which documents sequentially.
• Document Actor: Exclusively owns a document's yrs::Doc state. It processes inbound binary updates one-by-one, ensuring that CRDT merges are deterministic and thread-safe without ever needing a global lock.

2. Horizontal Scaling via Kafka

To facilitate collaboration across multiple server instances, we use Kafka as a high-throughput broadcast backbone:

• Document Sharding: Updates are published to a Kafka topic partitioned by document_id. This ensures that all updates for a specific document are processed in order by the cluster.
• Broadcast Pattern: When a server receives an update via WebSocket, it applies it locally to the DocumentActor and simultaneously publishes the binary update to Kafka.
• All-to-All Consumption: Every server node in the cluster consumes from the Kafka topic. If a node has active connections for a document, it forwards the Kafka update to those users, achieving seamless cross-server synchronization.

3. Message Bus (Sidecar Pattern)

LSP features (auto-complete, diagnostics) are provided via an internal Message Bus.

• Updates from the core sync engine are published to a local broadcast channel.
• An LSP Sidecar consumes these events and communicates with texlab processes without blocking the main collaboration loop.

---

💎 Core Design Principles

1. Protocol-Driven Contract

KineTex uses a strict, type-safe messaging protocol defined in both Rust and TypeScript.

• Binary-over-JSON: Large CRDT updates are transmitted as Base64-encoded binary strings within a standard JSON envelope (sync_update).
• Static Guarantees: Any message sent across the wire must conform to the ClientMessage or ServerMessage schemas, preventing runtime "ghost" errors in the collaboration layer.

2. Event Sourcing & Convergence

While the frontend is optimistic, the Document Actor in the server is the ultimate "Source of Truth."

• Deterministic Merging: Because all updates for a document are processed sequentially by a single actor, the final state is guaranteed to converge across all clients.
• Side-Effect Streams: Features like compilation status and PDF URLs are treated as observable streams, allowing the UI to react to asynchronous backend events in a purely functional manner.

3. Observability & Performance

The system is instrumented with the tracing ecosystem.

• Granular Logging: Every actor command and LSP transaction is logged with its associated connection_id and document_id.
• Zero-Copy Design: Large updates are passed via pointers (Arc) or channeled without deep cloning, maintaining high throughput even under heavy collaborative load.

---

👷 Worker: The Isolated Compiler

The worker handles the CPU-intensive and potentially risky task of LaTeX compilation.

1. Persistent Pool: The worker maintains a "warmed" pool of Docker containers to minimize startup latency.
2. Tectonic Engine: Uses Tectonic to avoid the "TeX Live install" bloat and fetch packages on-demand.
3. Security: Every compilation runs in a network: none container with limited memory and CPU, preventing malicious LaTeX code from accessing the host environment.

---

🛠️ Development Setup

Infrastructure

KineTex requires instances of Kafka, RabbitMQ, and an S3-compatible storage (like Minio for local dev).

1. Start Services:

   # Ensure your local Docker daemon is running
   # The worker will automatically pull dxjoke/tectonic-docker:latest

2. Start Server:

   cd server && cargo run

3. Start Worker:

   cd worker && cargo run

4. Start Frontend:

   cd client && npm run dev