Skip to content

Commit 8f6a9d7

Browse files
author
Leo Louvar
committed
Release v6.0.0: VectorDB backends (pgvector, redis, azure) and cloud resilience
- Bump version to 6.0.0 in mix.exs - Add docs/VECTORDB_BACKENDS.md: setup, pip install, 9 backends, resilience (pooling, backoff, circuit breaker, caching, health, metrics) - Update Zixir.VectorDB moduledoc and create/2 options for :pgvector, :redis, :azure - Add RELEASE_NOTES_v6.0.0.md - Update README and docs/INDEX.md with VectorDB doc links and implementation status
1 parent 6ce0c94 commit 8f6a9d7

6 files changed

Lines changed: 818 additions & 2 deletions

File tree

‎README.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -123,7 +123,7 @@ mix zig.get # after deps.get, for Zigler
123123
mix compile
124124
```
125125

126-
For Python specialist: ensure Python is on `PATH` or set in config; recommend a virtualenv.
126+
For Python specialist: ensure Python is on `PATH` or set in config; recommend a virtualenv. For **VectorDB** (nine backends: memory, chroma, pinecone, weaviate, qdrant, milvus, pgvector, redis, azure), see [docs/VECTORDB_BACKENDS.md](docs/VECTORDB_BACKENDS.md) for `pip install` and setup.
127127

128128
## Usage
129129

@@ -182,6 +182,7 @@ After Setup, run `mix zixir.run examples/hello.zixir`. Expected: `11.0`. For JIT
182182
| Portable CLI | Working | `zixir_run.sh` / `zixir_run.bat` from release; run from any path |
183183
| LSP Server | ✅ Ready | `mix zixir.lsp` + VS Code integration |
184184
| Package Manager | Complete | `Zixir.Package`: resolve, install (Git/path), list, cache; `zixir.toml` manifest |
185+
| VectorDB | Complete | Nine backends (memory, chroma, pinecone, weaviate, qdrant, milvus, pgvector, redis, azure); cloud resilience (pooling, backoff, circuit breaker, caching, health, metrics). See [docs/VECTORDB_BACKENDS.md](docs/VECTORDB_BACKENDS.md). |
185186

186187
### ⚠️ Partially Implemented
187188

‎RELEASE_NOTES_v6.0.0.md‎

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
# Zixir v6.0.0 — Major Release ⭐ RECOMMENDED
2+
3+
Major release with **VectorDB backend expansion** and **cloud resilience**. Nine backends supported; cloud backends use connection pooling, exponential backoff, circuit breaker, query caching, health checks, and request metrics. Follows semantic versioning (new backends + resilience = major).
4+
5+
---
6+
7+
## New Features & Improvements
8+
9+
### VectorDB: 3 new backends (9 total)
10+
11+
| Backend | Type | Best For |
12+
|-----------|--------|------------------------------------|
13+
| **:pgvector** | Python | PostgreSQL users, ACID compliance |
14+
| **:redis** | Python | Real-time apps, sub-ms latency |
15+
| **:azure** | Python | Microsoft ecosystems |
16+
17+
Existing: `:memory`, `:chroma`, `:pinecone`, `:weaviate`, `:qdrant`, `:milvus`.
18+
19+
### Cloud backends: real-world resilience
20+
21+
| Feature | Benefit |
22+
|-------------------|--------------------------------------|
23+
| Connection Pooling| Better resource utilization |
24+
| Exponential Backoff | Handles transient failures gracefully |
25+
| Circuit Breaker | Prevents cascade failures |
26+
| Query Caching | Reduces redundant DB calls |
27+
| Health Checks | Real-time monitoring |
28+
| Request Metrics | Track latency, success rates |
29+
30+
### Documentation
31+
32+
- **[docs/VECTORDB_BACKENDS.md](docs/VECTORDB_BACKENDS.md)** — Setup and `pip install` for each backend; resilience options; examples for create/health/metrics from Elixir.
33+
34+
---
35+
36+
## Complete Backend Support (9 Total)
37+
38+
| Backend | Type | Best For |
39+
|-----------|--------|------------------------------------|
40+
| :memory | Native | Prototyping (< 100K vectors) |
41+
| :chroma | Python | Simple local development |
42+
| :pinecone | Python | Production cloud |
43+
| :weaviate | Python | Self-hosted, GraphQL |
44+
| :qdrant | Python | High-performance |
45+
| :milvus | Python | Enterprise distributed |
46+
| :pgvector | Python | PostgreSQL users |
47+
| :redis | Python | Real-time apps |
48+
| :azure | Python | Microsoft ecosystems |
49+
50+
---
51+
52+
## Requirements
53+
54+
- **Elixir** 1.14+ / OTP 25+
55+
- **Zig** 0.15+ (build-time; run `mix zig.get` after `mix deps.get`)
56+
- **Python** 3.8+ *(optional)* for ML/specialist and VectorDB backends
57+
- **VectorDB** — Install backend-specific client, e.g. `pip install chromadb` or `pip install pgvector`. See [docs/VECTORDB_BACKENDS.md](docs/VECTORDB_BACKENDS.md).
58+
59+
## Quick start
60+
61+
```bash
62+
git clone https://github.com/Zixir-lang/Zixir.git
63+
cd Zixir
64+
git checkout v6.0.0
65+
mix deps.get
66+
mix zig.get
67+
mix compile
68+
```
69+
70+
**VectorDB (optional):** Pick a backend and install its client, then see [docs/VECTORDB_BACKENDS.md](docs/VECTORDB_BACKENDS.md) for create/health/metrics examples.
71+
72+
## License
73+
74+
**Apache-2.0** — see [LICENSE](LICENSE).

‎docs/INDEX.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@ Welcome to Zixir! This index will help you find the documentation you need.
1717
- **[USE_CASES.md](USE_CASES.md)** - Real-world use cases (pipelines, agentic, numeric services, ETL, edge)
1818
- **[CONFIG_AND_DEPLOY.md](CONFIG_AND_DEPLOY.md)** - Configuration options and production deployment
1919
- **[STANDARD_LIBRARY.md](STANDARD_LIBRARY.md)** - What's in the box (engine, CLI, Elixir API, workflow, cache)
20+
- **[VECTORDB_BACKENDS.md](VECTORDB_BACKENDS.md)** - VectorDB backend setup (pip install), resilience (pooling, backoff, circuit breaker), 9 backends (memory, chroma, pinecone, weaviate, qdrant, milvus, pgvector, redis, azure)
2021

2122
### Language Reference
2223
- **[LANGUAGE.md](LANGUAGE.md)** - Formal language specification

‎docs/VECTORDB_BACKENDS.md‎

Lines changed: 186 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,186 @@
1+
# VectorDB backends — setup and options
2+
3+
Zixir's VectorDB supports **9 backends**: one native in-memory backend and eight Python-based backends (Chroma, Pinecone, Weaviate, Qdrant, Milvus, pgvector, Redis, Azure). Cloud backends use **connection pooling**, **exponential backoff**, **circuit breaker**, **query caching**, **health checks**, and **request metrics** so they survive real-world failures.
4+
5+
---
6+
7+
## Cloud backends — resilience features
8+
9+
| Feature | Benefit |
10+
|---------|---------|
11+
| **Connection pooling** | Better resource utilization |
12+
| **Exponential backoff** | Handles transient failures gracefully |
13+
| **Circuit breaker** | Prevents cascade failures |
14+
| **Query caching** | Reduces redundant DB calls |
15+
| **Health checks** | Real-time monitoring |
16+
| **Request metrics** | Track latency, success rates |
17+
18+
Use `Zixir.VectorDB.health/1`, `Zixir.VectorDB.metrics/1`, `Zixir.VectorDB.cache_stats/1`, and `Zixir.VectorDB.circuit_breaker/1` to monitor cloud backends from Elixir.
19+
20+
---
21+
22+
## Complete backend support (9 total)
23+
24+
| Backend | Type | Best for |
25+
|---------|------|----------|
26+
| `:memory` | Native | Prototyping (&lt;100K vectors) |
27+
| `:chroma` | Python | Simple local development |
28+
| `:pinecone` | Python | Production cloud |
29+
| `:weaviate` | Python | Self-hosted, GraphQL |
30+
| `:qdrant` | Python | High-performance |
31+
| `:milvus` | Python | Enterprise distributed |
32+
| `:pgvector` | Python | PostgreSQL users needing ACID compliance |
33+
| `:redis` | Python | Real-time apps needing sub-ms latency |
34+
| `:azure` | Python | Microsoft ecosystems |
35+
36+
---
37+
38+
## 3 new backends (v6.0)
39+
40+
| Backend | For |
41+
|---------|-----|
42+
| **:pgvector** | PostgreSQL users needing ACID compliance |
43+
| **:redis** | Real-time apps needing sub-ms latency |
44+
| **:azure** | Microsoft ecosystems |
45+
46+
---
47+
48+
## How to set up backends
49+
50+
### Native (no Python install)
51+
52+
- **:memory** — Built in. No extra setup. Use for prototyping and small datasets.
53+
54+
### Python backends — install by backend
55+
56+
Install only the packages for the backends you use. Zixir’s Python bridge runs under your project’s Python (or the interpreter set in `config :zixir, :python_path`).
57+
58+
**Chroma (local dev):**
59+
```bash
60+
pip install chromadb
61+
```
62+
63+
**Pinecone (cloud):**
64+
```bash
65+
pip install pinecone-client
66+
```
67+
68+
**Weaviate (self-hosted):**
69+
```bash
70+
pip install weaviate-client
71+
```
72+
73+
**Qdrant (high-performance):**
74+
```bash
75+
pip install qdrant-client
76+
```
77+
78+
**Milvus (enterprise):**
79+
```bash
80+
pip install pymilvus
81+
```
82+
83+
**pgvector (PostgreSQL):**
84+
```bash
85+
pip install psycopg2-binary pgvector
86+
```
87+
You also need PostgreSQL with the [pgvector](https://github.com/pgvector/pgvector) extension enabled.
88+
89+
**Redis:**
90+
```bash
91+
pip install redis
92+
```
93+
Use a Redis instance with RediSearch/vector support (e.g. Redis Stack).
94+
95+
**Azure (AI Search / vector search):**
96+
```bash
97+
pip install azure-search-documents azure-identity
98+
```
99+
Configure Azure AI Search and credentials (e.g. `AZURE_SEARCH_ENDPOINT`, `AZURE_SEARCH_KEY` or managed identity).
100+
101+
### Install multiple backends
102+
103+
Example for Chroma + Pinecone + pgvector:
104+
```bash
105+
pip install chromadb pinecone-client psycopg2-binary pgvector
106+
```
107+
108+
---
109+
110+
## Creating a VectorDB in Zixir
111+
112+
```elixir
113+
# Memory (no Python)
114+
db = Zixir.VectorDB.create("my_db", dimensions: 384)
115+
116+
# Chroma (local)
117+
db = Zixir.VectorDB.create("local",
118+
backend: :chroma,
119+
collection: "docs",
120+
dimensions: 384
121+
)
122+
123+
# Pinecone (cloud)
124+
db = Zixir.VectorDB.create("prod",
125+
backend: :pinecone,
126+
api_key: System.get_env("PINECONE_API_KEY"),
127+
environment: "us-east-1",
128+
index_name: "my-index",
129+
dimensions: 384
130+
)
131+
132+
# pgvector (PostgreSQL)
133+
db = Zixir.VectorDB.create("pg",
134+
backend: :pgvector,
135+
connection_string: System.get_env("DATABASE_URL"),
136+
table_name: "embeddings",
137+
dimensions: 384
138+
)
139+
140+
# Redis
141+
db = Zixir.VectorDB.create("redis",
142+
backend: :redis,
143+
host: "localhost",
144+
port: 6379,
145+
index_name: "vec",
146+
dimensions: 384
147+
)
148+
149+
# Azure
150+
db = Zixir.VectorDB.create("azure",
151+
backend: :azure,
152+
endpoint: System.get_env("AZURE_SEARCH_ENDPOINT"),
153+
api_key: System.get_env("AZURE_SEARCH_KEY"),
154+
index_name: "my-index",
155+
dimensions: 384
156+
)
157+
```
158+
159+
---
160+
161+
## Health and resilience from Elixir
162+
163+
```elixir
164+
# Health (circuit breaker, pool, cache)
165+
health = Zixir.VectorDB.health(db)
166+
167+
# Request metrics (latency, success rate)
168+
metrics = Zixir.VectorDB.metrics(db)
169+
170+
# Cache stats (if query caching enabled)
171+
cache_stats = Zixir.VectorDB.cache_stats(db)
172+
173+
# Circuit breaker state (:closed | :open | :half_open)
174+
state = Zixir.VectorDB.circuit_breaker(db)
175+
176+
# Quick healthy check
177+
Zixir.VectorDB.healthy?(db)
178+
```
179+
180+
---
181+
182+
## See also
183+
184+
- [Zixir.VectorDB](https://github.com/Zixir-lang/Zixir) — module docs and `lib/zixir/vector_db.ex`
185+
- [SETUP_GUIDE.md](../SETUP_GUIDE.md) — Python and Elixir setup
186+
- [RELEASE_NOTES_v6.0.0.md](../RELEASE_NOTES_v6.0.0.md) — v6.0 VectorDB release notes

0 commit comments

Comments
 (0)