Getting Started with CSILgen: The Complete API Definition Lifecycle

Welcome to CSILgen (CBOR Service Interface Language Generator), a tool that helps you define, validate, and generate code for APIs and data structures. This guide will take you through the entire software development lifecycle using CSILgen, from your first API definition to production-ready code generation and versioning.

CAVEAT: This is all new, and basically alpha. Don't start here if you aren't comfortable with experimental software. The community is going to polish this as we get it ready for various use cases of our own, and that might be more user friendly as a stable point to jump in. Join the Discord if you're interested in following along.

Why CSILgen?
Installation & Setup
Your First CSIL Definition
Generating Code Across Languages
Advanced Features: Field Metadata
Managing API Evolution
Data Modeling Best Practices
Multi-File Projects at Scale
Writing Custom WASM Generators
Production Workflow Integration

Why CSILgen?

Traditional API definition tools like OpenAPI or Protocol Buffers excel at specific tasks but often fall short when you need:

Comprehensive Definitions: Define your API contract, validation rules, and documentation in one place
Extended Metadata: Express field visibility (send-only, receive-only), conditional requirements, and validation constraints
Cross-Language Consistency: Generate idiomatic code (both client and server) for multiple languages from a single source of truth
Breaking Change Detection: Automatically detect when changes would break existing clients
Extensibility: Write custom generators for your specific frameworks or patterns using any language that can target WASM

CSILgen is intended as a superset of CDDL (Concise Data Definition Language) and extends it with service definitions, includes, and metadata, giving you the power to define APIs that are self-documenting, self-validating, and ready for code generation.

Installation & Setup

Let's start by installing CSILgen and setting up your first project:

# Clone and build from source (for now - package managers coming soon)
git clone https://github.com/catalystcommunity/csilgen.git
cd csilgen
cargo build --workspace --release

# Build the WASM generators (required for code generation)
cargo build --target wasm32-unknown-unknown --release \
  -p csilgen-json-generator \
  -p csilgen-rust-generator \
  -p csilgen-typescript-generator \
  -p csilgen-python \
  -p csilgen-openapi \
  -p csilgen-go

# Install the CLI tool globally
cargo install --path crates/csilgen-cli

# Verify installation
csilgen --version
csilgen --help

Your First CSIL Definition

Let's build a simple task management API to understand CSIL's core concepts. Create a file called tasks.csil:

;; tasks.csil - A simple task management API definition
;; CSIL extends CDDL with services and metadata

;; Basic type definitions using CDDL syntax
TaskID = text .size (10..20) .regex "^TASK-[A-Z0-9]+$"
UserID = text .size (10..20) .regex "^USER-[A-Z0-9]+$"

;; Enumeration using CDDL choice syntax
TaskStatus = text / "pending" / "in_progress" / "completed" / "cancelled"
TaskPriority = int .ge 1 .le 5  ;; 1=lowest, 5=highest

;; Main task structure with metadata
Task = {
    @description("Unique task identifier")
    @receive-only  ;; Generated by server, not sent by clients
    id: TaskID,
    
    @description("Task title")
    @min-length(1)
    @max-length(200)
    title: text,
    
    @description("Detailed task description")
    ? description: text .size (0..2000),
    
    @description("Current task status")
    status: TaskStatus,
    
    @description("Task priority level")
    priority: TaskPriority .default 3,
    
    @description("User who created the task")
    @receive-only
    created_by: UserID,
    
    @description("When the task was created")
    @receive-only
    created_at: int,  ;; Unix timestamp
    
    @description("User assigned to the task")
    ? assigned_to: UserID,
    
    @description("Task due date")
    ? due_date: int  ;; Unix timestamp
}

;; Service definition - this is where CSIL shines
service TaskAPI {
    ;; Simple CRUD operations
    create-task: CreateTaskRequest -> Task / APIError,
    get-task: TaskID -> Task / APIError,
    update-task: UpdateTaskRequest -> Task / APIError,
    delete-task: TaskID -> DeleteResponse / APIError,
    
    ;; Query operation returning multiple tasks
    list-tasks: ListTasksRequest -> TaskList / APIError,
    
    ;; Real-time updates (bidirectional for WebSocket/SSE)
    watch-tasks: WatchRequest <-> TaskUpdate / APIError
}

;; Request/Response types
CreateTaskRequest = {
    @send-only  ;; Only sent by client, not in responses
    title: text .size (1..200),
    
    @send-only
    ? description: text .size (0..2000),
    
    @send-only
    ? priority: TaskPriority .default 3,
    
    @send-only
    ? assigned_to: UserID,
    
    @send-only
    ? due_date: int
}

UpdateTaskRequest = {
    @send-only
    id: TaskID,
    
    @send-only
    ? title: text .size (1..200),
    
    @send-only
    ? description: text .size (0..2000),
    
    @send-only
    ? status: TaskStatus,
    
    @send-only
    ? priority: TaskPriority,
    
    @send-only
    ? assigned_to: UserID
}

ListTasksRequest = {
    @send-only
    ? status: TaskStatus,
    
    @send-only
    ? assigned_to: UserID,
    
    @send-only
    @min-value(0)
    ? offset: int .default 0,
    
    @send-only
    @min-value(1)
    @max-value(100)
    ? limit: int .default 20
}

TaskList = {
    @receive-only
    tasks: [* Task],
    
    @receive-only
    total_count: int .ge 0,
    
    @receive-only
    ? next_offset: int .ge 0
}

;; Real-time update types
WatchRequest = {
    @send-only
    ? filter_status: TaskStatus,
    
    @send-only
    ? filter_assigned_to: UserID
}

TaskUpdate = {
    @receive-only
    update_type: text / "created" / "updated" / "deleted",
    
    @receive-only
    task: Task,
    
    @receive-only
    timestamp: int
}

;; Common response types
DeleteResponse = {
    @receive-only
    success: bool,
    
    @receive-only
    message: text
}

APIError = {
    @receive-only
    code: int .ge 400 .le 599,
    
    @receive-only
    message: text,
    
    @receive-only
    ? details: any
}

Now let's validate our CSIL file:

# Validate the syntax and semantics
csilgen validate --input tasks.csil

# Expected output:
# ✅ Validation successful for tasks.csil
# 📊 Statistics:
#    - 1 service (TaskAPI)
#    - 6 operations
#    - 11 type definitions
#    - 23 fields with metadata

Generating Code Across Languages

Now comes the magic - generating production-ready code in multiple languages:

TypeScript Generation

# Generate TypeScript interfaces and client
csilgen generate --input tasks.csil --target typescript --output ./generated/ts/

# Check what was generated
ls generated/ts/
# Output: types.ts, TaskAPI.ts, client.ts, validations.ts

The generated TypeScript will include:

// generated/ts/types.ts
export interface Task {
  id: string;           // TaskID - received from server only
  title: string;        // min: 1, max: 200 characters
  description?: string; // max: 2000 characters
  status: TaskStatus;
  priority: number;     // 1-5, defaults to 3
  created_by: string;   // UserID - received only
  created_at: number;   // Unix timestamp - received only
  assigned_to?: string; // UserID
  due_date?: number;    // Unix timestamp
}

export type TaskStatus = "pending" | "in_progress" | "completed" | "cancelled";

// generated/ts/client.ts
export class TaskAPIClient {
  constructor(private baseUrl: string) {}
  
  async createTask(request: CreateTaskRequest): Promise<Task> {
    // Implementation with proper validation
  }
  
  async watchTasks(request: WatchRequest): AsyncIterable<TaskUpdate> {
    // WebSocket implementation for bidirectional communication
  }
}

Python Generation

# Generate Python dataclasses with type hints
csilgen generate --input tasks.csil --target python --output ./generated/py/

# Check generated files
ls generated/py/
# Output: types.py, task_api.py, client.py, validators.py

Generated Python code:

# generated/py/types.py
from dataclasses import dataclass, field
from typing import Optional, Literal
from datetime import datetime

TaskStatus = Literal["pending", "in_progress", "completed", "cancelled"]

@dataclass
class Task:
    """Task entity with validation"""
    id: str                     # TaskID - server-generated
    title: str                  # 1-200 characters
    status: TaskStatus
    priority: int = 3           # 1-5, default 3
    created_by: str            # UserID - server-set
    created_at: int            # Unix timestamp - server-set
    description: Optional[str] = None  # Max 2000 chars
    assigned_to: Optional[str] = None  # UserID
    due_date: Optional[int] = None     # Unix timestamp
    
    def __post_init__(self):
        # Auto-generated validation
        if not (1 <= len(self.title) <= 200):
            raise ValueError("title must be 1-200 characters")
        if not (1 <= self.priority <= 5):
            raise ValueError("priority must be 1-5")

Rust Generation

# Generate Rust structs with serde
csilgen generate --input tasks.csil --target rust --output ./generated/rust/

# Check generated files
ls generated/rust/
# Output: mod.rs, types.rs, api.rs, client.rs

Generated Rust code:

// generated/rust/types.rs
use serde::{Deserialize, Serialize};

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Task {
    #[serde(skip_serializing)]  // receive-only field
    pub id: TaskId,

    #[serde(deserialize_with = "validate_title")]
    pub title: String,  // 1-200 characters

    #[serde(skip_serializing_if = "Option::is_none")]
    pub description: Option<String>,  // Max 2000 characters

    pub status: TaskStatus,

    #[serde(default = "default_priority")]
    pub priority: u8,  // 1-5, default 3

    #[serde(skip_serializing)]
    pub created_by: UserId,

    #[serde(skip_serializing)]
    pub created_at: i64,  // Unix timestamp

    #[serde(skip_serializing_if = "Option::is_none")]
    pub assigned_to: Option<UserId>,

    #[serde(skip_serializing_if = "Option::is_none")]
    pub due_date: Option<i64>,
}

#[derive(Debug, Clone, Copy, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum TaskStatus {
    Pending,
    InProgress,
    Completed,
    Cancelled,
}

Go Generation

# Generate Go structs with JSON/YAML tags
csilgen generate --input tasks.csil --target go --output ./generated/go/

# Check generated files
ls generated/go/
# Output: types.gen.go

Generated Go code:

// generated/go/types.gen.go
// Code generated by csilgen; DO NOT EDIT.
package generated

import (
    "encoding/json"
    "fmt"
    "gopkg.in/yaml.v3"
)

type TaskStatus string

const (
    TaskStatusPending    TaskStatus = "pending"
    TaskStatusInProgress TaskStatus = "in_progress"
    TaskStatusCompleted  TaskStatus = "completed"
    TaskStatusCancelled  TaskStatus = "cancelled"
)

type Task struct {
    ID          string      `json:"id" yaml:"id"`
    Title       string      `json:"title" yaml:"title"`
    Description *string     `json:"description,omitempty" yaml:"description,omitempty"`
    Status      TaskStatus  `json:"status" yaml:"status"`
    Priority    uint        `json:"priority" yaml:"priority"`
    CreatedBy   string      `json:"created_by" yaml:"created_by"`
    CreatedAt   int64       `json:"created_at" yaml:"created_at"`
    AssignedTo  *string     `json:"assigned_to,omitempty" yaml:"assigned_to,omitempty"`
    DueDate     *int64      `json:"due_date,omitempty" yaml:"due_date,omitempty"`
}

// Validate performs validation on Task
func (t *Task) Validate() error {
    if len(t.Title) < 1 || len(t.Title) > 200 {
        return fmt.Errorf("field 'Title' must be between 1 and 200 characters")
    }
    if t.Priority < 1 || t.Priority > 5 {
        return fmt.Errorf("field 'Priority' must be between 1 and 5")
    }
    return nil
}

// NewTask creates a Task with default values
func NewTask() *Task {
    return &Task{
        Priority: 3,
    }
}

Advanced Features: Field Metadata

CSIL's field metadata is what sets it apart. Let's explore a more complex example with conditional fields and dependencies:

;; payment.csil - Demonstrating advanced field metadata

PaymentMethod = text / "credit_card" / "debit_card" / "payfriend" / "bank_transfer"

Payment = {
    @description("Payment amount in cents")
    @min-value(1)
    @max-value(999999999)  ;; Max $9,999,999.99
    amount: int,
    
    @description("Payment method")
    method: PaymentMethod,
    
    @description("Credit/debit card number")
    @send-only  ;; Never send card numbers in responses!
    @depends-on(method = "credit_card" / method = "debit_card")
    ? card_number: text .regex "^[0-9]{13,19}$",
    
    @description("Card CVV")
    @send-only
    @depends-on(method = "credit_card" / method = "debit_card")
    ? card_cvv: text .regex "^[0-9]{3,4}$",
    
    @description("Payfriend email")
    @send-only
    @depends-on(method = "payfriend")
    ? payfriend_email: text .regex "^[^@]+@[^@]+$",
    
    @description("Bank account number")
    @send-only
    @admin-only  ;; Only admins can see/set bank details
    @depends-on(method = "bank_transfer")
    ? bank_account: text,
    
    @description("Last 4 digits of payment method")
    @receive-only  ;; Server provides this for display
    ? last_four: text .size (4..4),
    
    @description("Payment processor transaction ID")
    @receive-only
    @admin-only
    ? transaction_id: text
}

The generators understand these metadata annotations:

@send-only: Field only appears in requests, stripped from responses
@receive-only: Field only appears in responses, not accepted in requests
@admin-only: Field only visible to admin users (generators can create separate types or extra logic in handlers, etc)
@depends-on: Field is required/relevant only when condition is met
@description: Becomes documentation in generated code

Managing API Evolution

One of CSILgen's killer features is automatic breaking change detection. Let's evolve our task API:

Version 1.0.0 (Original)

Save this as tasks-v1.csil:

TaskID = text .size (10..20)

Task = {
    id: TaskID,
    title: text,
    status: text / "pending" / "completed"
}

service TaskAPI {
    create-task: CreateTaskRequest -> Task,
    get-task: TaskID -> Task
}

CreateTaskRequest = {
    title: text
}

Version 2.0.0 (Evolution)

Save this as tasks-v2.csil:

;; Added regex validation - BREAKING (stricter validation)
TaskID = text .size (10..20) .regex "^TASK-[A-Z0-9]+$"

Task = {
    id: TaskID,
    title: text,
    status: text / "pending" / "in_progress" / "completed" / "cancelled",  ;; Added statuses - NON-BREAKING
    priority: int .default 3,  ;; Added field - NON-BREAKING (has default)
    created_at: int  ;; Added required field - BREAKING!
}

service TaskAPI {
    create-task: CreateTaskRequestV2 -> Task,  ;; Renamed type - BREAKING
    get-task: TaskID -> Task,
    ;; Removed update-task operation - BREAKING if it existed
    list-tasks: ListRequest -> TaskList  ;; Added operation - NON-BREAKING
}

CreateTaskRequestV2 = {  ;; Renamed from CreateTaskRequest
    title: text,
    priority: int  ;; Added required field - BREAKING
}

;; New types - NON-BREAKING
ListRequest = {
    ? limit: int .default 20
}

TaskList = {
    tasks: [* Task]
}

Detecting Breaking Changes

# Run breaking change detection
csilgen breaking --current tasks-v1.csil --new tasks-v2.csil

# Output:
# 🔍 Analyzing changes between tasks-v1.csil and tasks-v2.csil...
#
# ❌ BREAKING CHANGES DETECTED:
#
# Type Changes:
#   - TaskID: Added regex constraint (stricter validation)
#   - Task.created_at: Added required field without default
#   - CreateTaskRequest: Type renamed to CreateTaskRequestV2
#
# Service Changes:
#   - TaskAPI.create-task: Input type changed (CreateTaskRequest → CreateTaskRequestV2)
#
# ✅ NON-BREAKING CHANGES:
#
# Type Changes:
#   - Task.status: Added new enum values ("in_progress", "cancelled")
#   - Task.priority: Added optional field with default value
#
# Service Changes:
#   - TaskAPI.list-tasks: New operation added
#
# New Types:
#   - ListRequest (new type)
#   - TaskList (new type)
#
# RECOMMENDATION: These breaking changes require a major version bump (2.0.0)

Versioning Strategy

Based on the breaking changes detected, you should:

Major Version Bump (1.0.0 → 2.0.0) for breaking changes
Minor Version Bump (1.0.0 → 1.1.0) for non-breaking additions
Patch Version Bump (1.0.0 → 1.0.1) for documentation/metadata changes

Integrate this into your CI/CD by feeding your main and feature changes to the csilgen cli:

#!/bin/bash
# ci-check.sh - Add to your CI pipeline

# Check for breaking changes against main branch
if csilgen breaking --current main.csil --new feature.csil | grep "BREAKING CHANGES"; then
    echo "⚠️ Breaking changes detected!"
    echo "Options:"
    echo "1. Bump major version"
    echo "2. Make changes backward-compatible"
    echo "3. Create v2 endpoint alongside v1"
    exit 1
fi

echo "✅ No breaking changes - safe to merge!"

Data Modeling Best Practices

When designing CSIL schemas, follow these best practices to ensure compatibility across all target languages:

1. Map Key Types

❌ BAD: Using complex types as map keys

;; This won't translate well to many languages
BadMap = {* UserObject => ConfigObject}

✅ GOOD: Use only strings or integers as map keys

;; String keys work everywhere
UserConfigs = {* text => ConfigObject}

;; Or use an array of key-value pairs for complex keys
UserConfigList = [* UserConfigEntry]
UserConfigEntry = {
    user: UserObject,
    config: ConfigObject
}

2. Nullable vs Optional

❌ BAD: Ambiguous nullability

;; Is this null, undefined, or empty string?
MaybeTitle = text / null

✅ GOOD: Use CDDL optional syntax

Document = {
    title: text,           ;; Required, non-null
    ? subtitle: text,      ;; Optional (may be absent)
    ? description: text    ;; Optional
}

3. Enumerations

❌ BAD: Magic numbers

Status = int  ;; What does 1 mean? 2? 3?

✅ GOOD: Use text literals or documented integers

;; Text enums are self-documenting
Status = text / "active" / "inactive" / "suspended"

;; Or if you want to use integers, document them
@description("Status Codes are 0 = active, 1 = inactive, 2 = suspended")
StatusCode = int .ge 0 .le 2
;; 0 = active, 1 = inactive, 2 = suspended

4. Timestamps

❌ BAD: Ambiguous time formats

CreatedAt = text  ;; ISO? RFC3339? Custom format?

✅ GOOD: Use Unix timestamps or specify format

;; Unix timestamp (seconds since epoch) - universal
CreatedAt = int .ge 0

;; Or if you need subsecond precision
CreatedAtMillis = int .ge 0  ;; Milliseconds since epoch

;; If you must use strings, specify the format
ISODateTime = text .regex "^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}Z$"

5. Arrays and Size Constraints

❌ BAD: Unbounded arrays

;; Could be gigabytes of data!
Messages = [* text]

✅ GOOD: Always set reasonable limits

;; Bounded array with item constraints
Messages = [* text .size (1..1000)] .size (0..100)
;;         ^-- each message max 1000 chars
;;                                    ^-- max 100 messages

;; Required non-empty array
Tags = [+ text .size (1..50)] .size (1..20)
;;     ^-- at least one tag required

6. Recursive Structures

❌ BAD: Unbounded recursion

;; This could create infinite depth!
TreeNode = {
    value: any,
    ? children: [* TreeNode]
}

✅ GOOD: Limit recursion depth or use references

;; Use IDs to reference other nodes
TreeNode = {
    id: NodeID,
    value: any,
    ? parent_id: NodeID,
    ? child_ids: [* NodeID] .size (0..100)
}

;; Or limit depth in business logic with metadata
FileSystemEntry = {
    name: text,
    @max-depth(10)  ;; Metadata hint for generators
    ? children: [* FileSystemEntry]
}

7. Versioning Data Structures

✅ GOOD: Plan for evolution

;; Use versioned wrapper for future compatibility
APIResponse = {
    version: int .default 1,
    data: ResponseDataV1,
    ? extensions: {* text => any}  ;; For future additions
}

;; Version-specific data
ResponseDataV1 = {
    ;; V1 fields here
}

Multi-File Projects at Scale

As your API grows, organize it into multiple files:

Project Structure

api/
├── common/
│   ├── types.csil      # Shared basic types
│   └── errors.csil     # Common error definitions
├── services/
│   ├── user-api.csil   # User service (imports common/*)
│   ├── task-api.csil   # Task service (imports common/*)
│   └── admin-api.csil  # Admin service (imports all)
└── models/
    ├── user.csil       # User-related types
    └── task.csil       # Task-related types

Example: Modular API Definition

common/types.csil:

;; Shared primitive types
ID = text .regex "^[A-Z]{3}-[A-Z0-9]{8,16}$"
Timestamp = int .ge 0
Email = text .regex "^[^@]+@[^@]+\.[^@]+$"

models/user.csil:

;; Import shared types
;; @import "../common/types.csil"

UserID = ID
User = {
    id: UserID,
    email: Email,
    created_at: Timestamp
}

services/user-api.csil:

;; @import "../common/types.csil"
;; @import "../common/errors.csil"
;; @import "../models/user.csil"

service UserAPI {
    get-user: UserID -> User / APIError,
    create-user: CreateUserRequest -> User / APIError
}

CreateUserRequest = {
    email: Email,
    password: text .size (8..100)
}

Generating from Multi-File Projects

# CSILgen automatically handles dependencies!
csilgen generate --input ./api/services/ --target typescript --output ./generated/

# What happens:
# 1. Scans all .csil files in services/
# 2. Identifies entry points (user-api, task-api, admin-api)
# 3. Resolves imports automatically
# 4. Generates code without duplicates (within an entry point)

# Output:
# 📊 Dependency analysis completed:
#    Entry points: 3 files (user-api, task-api, admin-api)
#    Dependencies: 5 files (auto-included via imports)
#    Generating from entry points only to avoid duplicates.

Writing Custom WASM Generators

Want to generate code for a language/framework CSILgen doesn't support yet? Write your own generator! We have a Rust example, but anything that matches the WASM requirements will also work. That's up to the reader, though.

Generator Project Structure

Create a new Rust project for your generator:

cargo new --lib my-graphql-generator
cd my-graphql-generator

Cargo.toml:

[package]
name = "my-graphql-generator"
version = "0.1.0"
edition = "2021"

[lib]
crate-type = ["cdylib"]

[dependencies]
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
csilgen-common = "0.1"  # When published

[profile.release]
opt-level = "z"     # Optimize for size
lto = true          # Link-time optimization

src/lib.rs:

use serde::{Deserialize, Serialize};
use csilgen_common::*;

#[derive(Debug, Serialize)]
struct GeneratedFile {
    path: String,
    content: String,
}

// Main entry point - called by CSILgen runtime
#[no_mangle]
pub extern "C" fn generate(input_ptr: *const u8, input_len: usize) -> *mut u8 {
    // Parse input
    let input = parse_input(input_ptr, input_len);
    
    // Generate GraphQL schema
    let mut files = vec![];
    
    // Generate types
    let schema = generate_graphql_schema(&input.csil_spec);
    files.push(GeneratedFile {
        path: "schema.graphql".to_string(),
        content: schema,
    });
    
    // Generate resolvers
    let resolvers = generate_resolvers(&input.csil_spec);
    files.push(GeneratedFile {
        path: "resolvers.ts".to_string(),
        content: resolvers,
    });
    
    // Return generated files
    serialize_output(files)
}

fn generate_graphql_schema(spec: &CsilSpec) -> String {
    let mut schema = String::new();
    
    // Generate GraphQL types from CSIL types
    for rule in &spec.rules {
        match &rule.rule_type {
            CsilRuleType::GroupDef(group) => {
                schema.push_str(&format!("type {} {{\n", rule.name));
                
                for entry in &group.entries {
                    let field_name = entry.key.as_ref().unwrap();
                    let graphql_type = map_to_graphql_type(&entry.value_type);
                    
                    // Handle field metadata
                    let mut directives = vec![];
                    for metadata in &entry.metadata {
                        match metadata {
                            FieldMetadata::Visibility(vis) => {
                                if vis == "admin-only" {
                                    directives.push("@auth(requires: ADMIN)");
                                }
                            }
                            FieldMetadata::Description(desc) => {
                                // Add as GraphQL comment
                                schema.push_str(&format!("  # {}\n", desc));
                            }
                            _ => {}
                        }
                    }
                    
                    schema.push_str(&format!("  {}: {}", field_name, graphql_type));
                    for directive in directives {
                        schema.push_str(&format!(" {}", directive));
                    }
                    schema.push_str("\n");
                }
                
                schema.push_str("}\n\n");
            }
            CsilRuleType::ServiceDef(service) => {
                // Generate Query/Mutation types
                schema.push_str("type Query {\n");
                for op in &service.operations {
                    if op.name.starts_with("get-") || op.name.starts_with("list-") {
                        let gql_name = to_camel_case(&op.name);
                        schema.push_str(&format!("  {}: {}\n", gql_name, op.output_type));
                    }
                }
                schema.push_str("}\n\n");
                
                schema.push_str("type Mutation {\n");
                for op in &service.operations {
                    if !op.name.starts_with("get-") && !op.name.starts_with("list-") {
                        let gql_name = to_camel_case(&op.name);
                        schema.push_str(&format!("  {}: {}\n", gql_name, op.output_type));
                    }
                }
                schema.push_str("}\n\n");
            }
            _ => {}
        }
    }
    
    schema
}

Building Your Generator

# Build as WASM
cargo build --target wasm32-unknown-unknown --release

# The WASM file will be at:
# target/wasm32-unknown-unknown/release/my_graphql_generator.wasm

Using Your Custom Generator

# Use your custom generator
csilgen generate \
  --input api.csil \
  --target ./my_graphql_generator.wasm \
  --output ./generated/

# Your generator receives:
# - Parsed CSIL AST with all types, services, and metadata
# - Configuration options
# - Output directory path

# And returns:
# - Array of files to generate
# - Each with a path and content

Generator Best Practices

1. Structure Output for Clean Updates

❌ BAD: Mixing generated and manual code

// types.ts - DON'T mix generated and manual code
export interface User {
  id: string;  // Generated
  name: string; // Generated
  
  // Manual addition - will be lost on regeneration!
  getDisplayName() {
    return this.name.toUpperCase();
  }
}

✅ GOOD: Separate generated from manual code

// generated/types.ts - GENERATED, DO NOT EDIT
export interface UserBase {
  id: string;
  name: string;
}

// src/models/user.ts - Your extensions (not generated)
import { UserBase } from '../generated/types';

export class User implements UserBase {
  constructor(public id: string, public name: string) {}
  
  getDisplayName() {
    return this.name.toUpperCase();
  }
}

2. Generate Partial Classes/Interfaces

For languages that support it, generate partial definitions:

// Generated: User.generated.cs
public partial class User {
    public string Id { get; set; }
    public string Name { get; set; }
}

// Manual: User.cs
public partial class User {
    public string DisplayName => Name.ToUpper();
}

3. Include Generation Metadata

Always include metadata in generated files:

/**
 * GENERATED CODE - DO NOT EDIT
 * 
 * Generated from: api.csil
 * Generated at: 2024-03-15T10:30:00Z
 * Generator: my-graphql-generator v1.0.0
 * 
 * To regenerate: csilgen generate --input api.csil --target graphql
 */

Production Workflow Integration

CI/CD Pipeline Integration

.github/workflows/api-validation.yml:

name: API Validation

on:
  pull_request:
    paths:
      - 'api/**/*.csil'

jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      
      - name: Install CSILgen
        run: |
          cargo install csilgen-cli
          
      - name: Validate CSIL files
        run: |
          csilgen validate --input api/
          
      - name: Check breaking changes
        run: |
          git checkout main
          cp -r api/ api-main/
          git checkout -
          
          if csilgen breaking --current api-main/ --new api/ | grep "BREAKING"; then
            echo "::error::Breaking changes detected! Bump major version."
            exit 1
          fi
          
      - name: Generate code
        run: |
          csilgen generate --input api/ --target typescript --output generated/
          csilgen generate --input api/ --target python --output generated/
          
      - name: Test generated code
        run: |
          cd generated/typescript && npm test
          cd ../python && pytest

Pre-commit Hook

.git/hooks/pre-commit:

#!/bin/bash
# Validate and format CSIL files before commit

# Find all CSIL files
CSIL_FILES=$(git diff --cached --name-only --diff-filter=ACM | grep '\.csil$')

if [ -n "$CSIL_FILES" ]; then
    echo "Validating CSIL files..."
    
    for file in $CSIL_FILES; do
        # Validate
        if ! csilgen validate --input "$file"; then
            echo "❌ Validation failed for $file"
            exit 1
        fi
        
        # Format
        csilgen format "$file"
        git add "$file"
    done
    
    echo "✅ All CSIL files validated and formatted"
fi

Build System Integration

package.json (Node.js projects):

{
  "scripts": {
    "generate": "csilgen generate --input api/ --target typescript --output src/generated/",
    "validate": "csilgen validate --input api/",
    "check-breaking": "csilgen breaking --current api-stable/ --new api/",
    "prebuild": "npm run generate",
    "build": "tsc"
  }
}

Cargo.toml with build.rs (Rust projects):

// build.rs
use std::process::Command;

fn main() {
    // Generate Rust code from CSIL during build
    let output = Command::new("csilgen")
        .args(&["generate", "--input", "api/", "--target", "rust", "--output", "src/generated/"])
        .output()
        .expect("Failed to generate code from CSIL");
        
    if !output.status.success() {
        panic!("CSIL generation failed: {}", String::from_utf8_lossy(&output.stderr));
    }
    
    println!("cargo:rerun-if-changed=api/");
}

Documentation Generation

Generate API documentation from your CSIL files:

# Generate OpenAPI spec for Swagger UI
csilgen generate --input api/ --target openapi --output docs/

# Serve documentation
npx @redocly/cli preview-docs docs/openapi.yaml

Real-World Example: E-Commerce Platform

Let's put it all together with a real e-commerce API:

;; ecommerce.csil - Production-ready e-commerce API

;; ============ Common Types ============

Money = {
    @description("Amount in smallest currency unit")
    amount: int .ge 0,
    
    @description("ISO 4217 currency code")
    currency: text .size (3..3) .regex "^[A-Z]{3}$"
}

Address = {
    street: text .size (1..200),
    city: text .size (1..100),
    state: text .size (2..100),
    postal_code: text .size (3..20),
    country: text .size (2..3)  ;; ISO country code
}

;; ============ Product Domain ============

ProductID = text .regex "^PRD-[A-Z0-9]{10,20}$"
CategoryID = text .regex "^CAT-[A-Z0-9]{10,20}$"

Product = {
    @receive-only
    id: ProductID,
    
    name: text .size (1..200),
    description: text .size (0..5000),
    
    @description("Customer price")
    price: Money,
    
    @description("Cost to business - never expose")
    @admin-only
    @receive-only
    cost: Money,
    
    @description("Available inventory")
    @receive-only
    stock_quantity: int .ge 0,
    
    @description("Product is available for purchase")
    @receive-only
    @depends-on(stock_quantity > 0)
    available: bool,
    
    categories: [* CategoryID] .size (1..10),
    
    @receive-only
    created_at: int,
    
    @receive-only
    updated_at: int
}

;; ============ Order Domain ============

OrderID = text .regex "^ORD-[A-Z0-9]{10,20}$"
OrderStatus = text / "pending" / "confirmed" / "processing" / "shipped" / "delivered" / "cancelled"

Order = {
    @receive-only
    id: OrderID,
    
    @receive-only
    customer_id: CustomerID,
    
    items: [+ OrderItem] .size (1..100),  ;; At least 1 item
    
    @receive-only
    subtotal: Money,
    
    @receive-only
    tax: Money,
    
    @receive-only
    shipping: Money,
    
    @receive-only
    total: Money,
    
    @receive-only
    status: OrderStatus,
    
    shipping_address: Address,
    
    @admin-only
    payment_details: PaymentDetails,
    
    @receive-only
    created_at: int,
    
    @receive-only
    updated_at: int
}

OrderItem = {
    product_id: ProductID,
    quantity: int .ge 1 .le 100,
    
    @receive-only
    unit_price: Money,
    
    @receive-only
    total_price: Money
}

PaymentDetails = {
    @admin-only
    method: text / "card" / "payfriend" / "bank_transfer",
    
    @admin-only
    @receive-only
    last_four: text .size (4..4),
    
    @admin-only
    @receive-only
    processor_transaction_id: text
}

;; ============ Services ============

service ProductAPI {
    ;; Public endpoints
    list-products: ListProductsRequest -> ProductList / APIError,
    get-product: ProductID -> Product / APIError,
    search-products: SearchRequest -> ProductList / APIError,
    
    ;; Admin endpoints
    create-product: CreateProductRequest -> Product / APIError,
    update-product: UpdateProductRequest -> Product / APIError,
    delete-product: ProductID -> DeleteResponse / APIError,
    update-inventory: UpdateInventoryRequest -> Product / APIError
}

service OrderAPI {
    ;; Customer endpoints
    create-order: CreateOrderRequest -> Order / APIError,
    get-order: GetOrderRequest -> Order / APIError,
    list-my-orders: ListOrdersRequest -> OrderList / APIError,
    cancel-order: CancelOrderRequest -> Order / APIError,
    
    ;; Admin endpoints
    list-all-orders: ListOrdersRequest -> OrderList / APIError,
    update-order-status: UpdateOrderStatusRequest -> Order / APIError,
    
    ;; Real-time tracking
    track-order: OrderID <-> OrderTrackingUpdate / APIError
}

;; ============ Request/Response Types ============

ListProductsRequest = {
    @send-only
    ? category: CategoryID,
    
    @send-only
    ? min_price: Money,
    
    @send-only
    ? max_price: Money,
    
    @send-only
    ? in_stock_only: bool .default true,
    
    @send-only
    @min-value(0)
    ? offset: int .default 0,
    
    @send-only
    @min-value(1)
    @max-value(100)
    ? limit: int .default 20
}

ProductList = {
    @receive-only
    products: [* Product],
    
    @receive-only
    total: int .ge 0,
    
    @receive-only
    ? next_offset: int
}

CreateOrderRequest = {
    @send-only
    items: [+ OrderItemRequest] .size (1..100),
    
    @send-only
    shipping_address: Address,
    
    @send-only
    payment_method: text / "card" / "payfriend",
    
    @send-only
    @depends-on(payment_method = "card")
    ? card_token: text,  ;; From payment processor
    
    @send-only
    @depends-on(payment_method = "payfriend")
    ? payfriend_auth_code: text
}

OrderItemRequest = {
    @send-only
    product_id: ProductID,
    
    @send-only
    @min-value(1)
    @max-value(100)
    quantity: int
}

OrderTrackingUpdate = {
    @receive-only
    order_id: OrderID,
    
    @receive-only
    status: OrderStatus,
    
    @receive-only
    location: text,
    
    @receive-only
    message: text,
    
    @receive-only
    timestamp: int,
    
    @receive-only
    @depends-on(status = "shipped")
    ? tracking_number: text,
    
    @receive-only
    @depends-on(status = "shipped")
    ? carrier: text,
    
    @receive-only
    @depends-on(status = "delivered")
    ? signature: text
}

APIError = {
    @receive-only
    code: int .ge 400 .le 599,
    
    @receive-only
    message: text,
    
    @receive-only
    ? field_errors: {* text => [* text]},
    
    @receive-only
    @depends-on(code = 429)
    ? retry_after: int  ;; Seconds
}

Generate a complete e-commerce platform API spec client and server:

# Generate for multiple targets
for target in typescript python rust go openapi; do
    csilgen generate --input ecommerce.csil --target $target --output ./platform/$target/
done

# What you get:
# - TypeScript: Full type-safe client and server stubs
# - Python: FastAPI-ready models with Pydantic validation
# - Rust: Actix-web compatible handlers with serde
# - Go: gin-gonic ready handlers with validation
# - OpenAPI: Swagger documentation

# Each respects:
# - Field visibility (admin-only fields in separate types)
# - Validation constraints (automatic input validation)
# - Default values (applied in constructors)
# - Service operations (client methods and server interfaces)

Conclusion

CSILgen transforms API development from a manual, error-prone process into a systematic, validated workflow. You've learned how to:

Define APIs with rich metadata and validation rules
Generate code that respects visibility, constraints, and business logic
Detect breaking changes automatically before they reach production
Model data following best practices for cross-language compatibility
Scale projects with multi-file organization and dependency management
Extend CSILgen with custom WASM generators for any framework
Integrate into production CI/CD pipelines

The key insight: your API definition becomes the single source of truth, eliminating drift between documentation, validation, and implementation.

Next Steps

Explore examples: Check out /examples for more patterns
Join the community: The Catalyst Community has various ways to chat and work together
Share feedback: Help shape CSILgen's future at github.com/catalystcommunity/csilgen

Start with a simple API definition today, and experience the power of truly integrated API development. Your future self (and your API consumers) will thank you!

FilesExpand file tree

getting-started.md

Latest commit

History