Skip to content

Feat/sqllite#57

Merged
nitesh32 merged 4 commits into
pre-releasefrom
feat/sqllite
May 25, 2026
Merged

Feat/sqllite#57
nitesh32 merged 4 commits into
pre-releasefrom
feat/sqllite

Conversation

@bharatsachya
Copy link
Copy Markdown
Collaborator

Pull Request Guide

## What changed

- Added `packages/sqlite` — a full SQLite implementation of `IDocumentDatabaseProvider` (knowledge, raw files, stats, activity, usage repositories + client + migrations)
- Introduced `packages/db-core` — shared TypeScript interfaces (`IDocumentDatabaseProvider` and its sub-repositories) that every document-DB provider must implement
- Introduced `packages/db` — a provider registry (`registerDbProvider` / `connectDb` / `getDb`) that decouples call-sites from any specific DB driver
- Introduced `packages/graph-core` — shared interfaces for the graph layer (`IGraphDatabaseProvider` and sub-repositories)
- Introduced `packages/graph-db` — a parallel provider registry for the graph layer (`registerGraphProvider` / `connectGraph` / `getGraph`)
- Refactored `packages/mongo` to implement `IDocumentDatabaseProvider` and self-register via `registerDbProvider("mongo", ...)`
- Refactored `packages/neo4j` to implement `IGraphDatabaseProvider` and self-register via `registerGraphProvider("neo4j", ...)`
- Updated `packages/server` and `packages/ingest-github` bootstrap to connect through the registry (`connectDb(dbProvider)` / `connectGraph(graphProvider)`) instead of hard-wiring Mongo/Neo4j imports
- Added three new config keys: `db_provider` (default `"sqlite"`), `graph_provider` (default `"neo4j"`), `sqlite_path`; `MONGO_URI` is no longer required when `db_provider !== "mongo"`
- Extracted all inline `readField` switch cases from `schema.ts` into a generated `schema-fields.ts` to reduce noise in the schema file
- Bumped `tsconfig.json` references across all affected packages

## Why

Mongo is a hosted dependency that adds friction for local development and self-hosted deployments. SQLite lets developers (and users running ByteBell on their own machines) spin up a fully functional instance with zero infrastructure. The provider registry pattern keeps every consumer (`server`, `queue`, `ingest-github`) DB-agnostic so future backends (Postgres, etc.) can be dropped in without touching call-sites.

## How to test

**SQLite path (new default)**

1. Make sure `db_provider` is unset or explicitly `sqlite` in your config:

bytebell set db_provider sqlite
bytebell set sqlite_path ~/.bytebell/data.sqlite # or leave blank for default

2. Start the server — no Mongo URI required:

bun run dev

3. Trigger a GitHub index job via `POST /github/index` and confirm it completes without errors
4. Hit `GET /health` — `db` ping should return `ok: true`
5. Open `sqlite3 ~/.bytebell/data.sqlite` and verify rows are present in the `knowledge`, `raw_files`, and `usage` tables

**Mongo path (regression)**

1. Set `db_provider mongo` and supply a valid `MONGO_URI`
2. Repeat steps 2–4 above — behaviour should be identical to pre-release

**Graph provider (Neo4j — no change expected)**

1. Confirm `graph_provider` defaults to `neo4j` and knowledge nodes still appear in Neo4j after an index run

**Config validation**

1. Start the server with `db_provider mongo` but no `MONGO_URI` — server should refuse to start with a missing-config error
2. Start with `db_provider sqlite` and no `MONGO_URI` — server should start cleanly

@bharatsachya bharatsachya added enhancement New feature or request Done Done and ready to be included in the next release labels May 21, 2026
@bharatsachya
Copy link
Copy Markdown
Collaborator Author

Against #56

@Dead-Bytes Dead-Bytes removed the Done Done and ready to be included in the next release label May 21, 2026
@bharatsachya bharatsachya added the Ready for Review Ready for review from reviewers label May 21, 2026
Comment thread packages/cli/src/DeleteCommand.ts Outdated
Comment thread packages/cli/src/repoSelectorPrompt.ts
Comment thread packages/db-core/src/index.ts Outdated
Comment thread packages/db-core/README.md
Comment thread packages/graph-core/src/index.ts
Comment thread packages/ingest-github/src/pipeline/pull.ts Outdated
Comment thread packages/ingest-github/src/pipeline/run.ts Outdated
Comment thread packages/server/src/deleteRoute.ts
@bharatsachya bharatsachya requested a review from Dead-Bytes May 21, 2026 15:59
@Dead-Bytes Dead-Bytes added the Flagged this has been marked flagged by one reveiwer for blocking label May 22, 2026
@bharatsachya bharatsachya removed the Flagged this has been marked flagged by one reveiwer for blocking label May 25, 2026
@nitesh32 nitesh32 merged commit 0e7e3ea into pre-release May 25, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request Ready for Review Ready for review from reviewers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants