Skip to content

kalamstack/KalamDB

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1,520 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

KalamDB logo

Give AI agents one SQL tool to your app data.

KalamDB is a SQL-first realtime backend for agent-native apps. Agents and frontends use the same backend directly: SQL over HTTP for reads and writes, live query rows over WebSocket for subscriptions. USER tables, auth, and EXECUTE AS USER keep access policy-scoped instead of exposing broad backend access.

Rust License CI Overall Tests Release Docker Pulls pg-kalam Docker Pulls

SDKs

TypeScript SDK Tests Dart SDK Tests TypeScript SDK Dart SDK

SDKs: TypeScript/JavaScript and Dart/Flutter.

PostgreSQL extension Docker image: jamals86/pg-kalam with pg_kalam preinstalled.

Frontend clients can execute SQL directly against KalamDB. You keep one SQL API plus live subscriptions instead of maintaining a custom REST or GraphQL CRUD layer for every agent workflow.

One SQL Tool, Live Rows

KalamDB is designed for both frontend and backend SQL execution.

  • Agents and workers use one SQL tool plus the current schema instead of dozens of narrow CRUD tools.
  • Frontends run the same backend directly for user-scoped reads, writes, and live row updates.
  • Queries and writes go through the HTTP SQL API; realtime subscriptions reuse the shared WebSocket connection.
  • Ordinary USER-table reads stay subject-scoped; privileged service, DBA, and system accounts may use explicit EXECUTE AS USER only through the role hierarchy.

This is what lets chat UIs, support agents, approval flows, and other product workflows share one data plane without separate polling, sync, or fanout infrastructure.

Why KalamDB

  • One SQL tool for agents, so app workflows can read and write allowed data without a custom CRUD surface for every action.
  • Live UI by default, with frontend-direct SQL and realtime subscriptions keeping web and mobile apps in sync without a separate sync layer.
  • Per-user isolation by default, with the same SQL returning user-scoped data safely across frontend and backend clients.
  • Built-in authentication and authorization with Basic auth, JWT, OAuth 2.0 (Google Workspace, GitHub, Azure AD), and RBAC.
  • Unified application primitives: SQL tables, live queries, pub/sub topics, consumer groups, and file attachments in one runtime.
  • Hybrid storage engine tuned for product workloads: RocksDB for fast active writes, Parquet for compressed historical storage and analytics.
  • Portable object storage support across filesystem, S3, GCS, and Azure backends.
  • Vector embeddings and vector search workflows for semantic retrieval, agent memory, and AI-powered product features.
  • Distributed clustering with Multi-Raft replication and failover for resilient production deployments.
  • First-party tooling for operators and app teams: Admin UI, kalam CLI, and official TypeScript and Dart SDKs.

Agent-Native Workflow

  • Read current app state with SELECT from the same backend your product already uses.
  • Write drafts, tasks, approvals, or replies into normal app tables with SQL.
  • Subscribe the frontend to pending rows live over WebSocket.
  • Commit final writes through the same backend path, including authorized EXECUTE AS USER when a trusted service acts for a user.
  • Let the UI update from row changes instead of a second sync system.

See examples/chat-with-ai for the base live agent loop already in the repo.

Feature & Status

Feature Status Feature Status
User Tables ✅ Available Shared Tables ✅ Available
Streams ✅ Available Pub/Sub Topics ✅ Available
Live Queries ✅ Available Consumer Groups ✅ Available
Cluster Replication ✅ Available Vector Embeddings ✅ Available
PostgreSQL Extension ✅ Available Admin UI ✅ Available
Kalam CLI ✅ Available TypeScript SDK ✅ Available
Dart/Flutter SDK ✅ Available Object Storage ✅ Available

60-Second Quick Start (Docker)

Single node

KALAMDB_JWT_SECRET="$(openssl rand -base64 32)" \
curl -sSL https://raw.githubusercontent.com/kalamstack/KalamDB/main/docker/run/single/docker-compose.yml | docker-compose -f - up -d

3-node cluster

KALAMDB_JWT_SECRET="$(openssl rand -base64 32)" \
curl -sSL https://raw.githubusercontent.com/kalamstack/KalamDB/main/docker/run/cluster/docker-compose.yml | docker-compose -f - up -d

Local run

git clone https://github.com/kalamstack/KalamDB.git
cd KalamDB/backend
cargo run --bin kalamdb-server

Browser/Frontend Client Example

import { createClient, Auth } from '@kalamdb/client';

const client = createClient({
  url: 'http://localhost:2900',
  authProvider: async () => Auth.jwt('<user-token>'),
});

const threadSql = `
  SELECT id, role, content, created_at
  FROM chat.messages
  WHERE thread_id = 'thread_42'
`;

await client.query(`
  INSERT INTO chat.messages (thread_id, role, content)
  VALUES ('thread_42', 'user', 'hello from frontend');
`);

const unsubscribe = await client.live(
  threadSql,
  (rows) => {
    // If `chat.messages` is a USER table, each signed-in user only sees
    // their own rows even though the SQL text is identical for everyone.
    console.log('live rows', rows);
  },
  {
    subscriptionOptions: { last_rows: 20 },
  },
);

// Later
await unsubscribe();
await client.disconnect();

AI Agent Example (Topic Subscription)

Subscribe a worker to a KalamDB topic and process each change with managed retries, backpressure, and at-least-once delivery via runConsumer.

import { createClient, Auth } from '@kalamdb/client';
import { runConsumer } from '@kalamdb/consumer';

const client = createClient({
  url: 'http://localhost:2900',
  authProvider: async () => Auth.basic('root', 'kalamdb123'),
});

const abort = new AbortController();
process.on('SIGINT', () => abort.abort());

await runConsumer<{ title: string; body: string }>({
  client,
  name: 'summarizer-agent',
  topic: 'blog.posts',       // KalamDB topic to consume
  groupId: 'summarizer-v1',  // consumer group — tracks per-agent offset
  start: 'earliest',
  batchSize: 20,
  timeoutSeconds: 30,
  stopSignal: abort.signal,

  // Called for every inserted, updated, or deleted topic change
  onChange: async (ctx, change) => {
    const row = change.data;
    const summary = await myLlm.summarize(row.body);
    console.log(`[${change.op ?? 'change'}:${row.title}] →`, summary);
  },

  onFailed: async (ctx, change) => {
    const row = change.data;
    console.error('failed row', row, ctx.error);
  },

  retry: {
    maxAttempts: 3,
    initialBackoffMs: 250,
    maxBackoffMs: 1500,
    multiplier: 2,
  },
  ackOnFailed: true,    // commit offset even on permanent failure
});

await client.disconnect();

See examples/summarizer-agent/ for a full working example with Gemini integration.

SQL Example

CREATE NAMESPACE IF NOT EXISTS chat;

CREATE TABLE chat.messages (
  id BIGINT PRIMARY KEY DEFAULT SNOWFLAKE_ID(),
  thread_id TEXT NOT NULL,
  role TEXT NOT NULL,
  content TEXT NOT NULL,
  attachment FILE,
  created_at TIMESTAMP DEFAULT NOW()
) WITH (TYPE = 'USER');

Architecture Snapshot

  • SQL parsing/classification: kalamdb-dialect; query execution: kalamdb-core + DataFusion/Arrow.
  • Hot path: kalamdb-store (RocksDB).
  • Cold path: kalamdb-filestore (Parquet/object storage).
  • Orchestration: kalamdb-core (DDL/DML, jobs, schema registry).
  • API surface: HTTP SQL API + WebSocket realtime + Admin UI + CLI.

Best-Fit Workloads

  • AI chat history and agent memory/state.
  • Tool-call logs and human-in-the-loop workflows.
  • Live dashboards, notifications, and collaborative feeds.
  • Multi-tenant SaaS that needs strong user-level isolation.

Docs and Links

  • Docs: https://kalamdb.org/docs
  • Quick start: docs/getting-started/quick-start.md
  • TypeScript SDK: link/sdks/typescript/client/
  • Docker deployment: docker/run/
  • PostgreSQL extension Docker image: jamals86/pg-kalam
  • Website: https://kalamdb.org

KalamDB is under active development and evolving quickly.

License

Licensed under the Apache License, Version 2.0 (Apache-2.0). See LICENSE.txt and NOTICE.

About

KalamDB — a lightweight, real-time, storage-efficient SQL database. Designed for per-user data isolation and scalable performance — ideal for the AI era.

Topics

Resources

License

Code of conduct

Contributing

Security policy

Stars

Watchers

Forks

Packages

 
 
 

Contributors