Multi tenancy 2#81
Open
JimA-cyborg wants to merge 8 commits into
Open
Conversation
There was a problem hiding this comment.
⚠️ API Contract Change Detected
This PR modifies the public API contract of the CyborgDB JS SDK.
Please provide an explanation for this change:
- Why is this change necessary?
- Is this a breaking change or backward compatible?
- Have you updated the documentation?
This review must be dismissed or addressed before the PR can be merged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Multi tenancy 2 (#81)
Summary
Adds per-index KMS / BYOK support to the TypeScript SDK, bringing it to parity
with the Python (
cyborgdb-py) and Go (cyborgdb-go) SDKs.index_keybecomesoptional across the API and a new
kmsNameparameter lets the service managethe encryption key, so an index can be created and used without the SDK ever
holding a key.
Three key-management modes are now supported:
indexKey, omitkmsName.kmsName(real provider:aws-kms/aws),omit
indexKey; the service generates and wraps the KEK.provider: none+ SDK KEK — pass both; the registry slot tracks theindex while the SDK supplies the KEK on each request.
Motivation
Tracks the cyborgdb-service per-index KMS / multi-tenancy work (BYOK). Customers
want index keys managed by a KMS in their own account rather than held by the
SDK. The SDK side of that is small but cross-cutting: every request that carried
an
index_keymust now be able to omit it, andcreateIndexmust accept a KMSslot reference. Keeping the JS SDK one-for-one with py/go avoids divergent
behavior across language clients against the same service.
Changes
cyborgdb-py):index_keyis optional on every request model andCreateIndexRequestgainskms_name.createIndex: added optionalkmsName, madeindexKeyoptional, added alocal guard ("requires indexKey, kmsName, or both") and shared 32-byte
key-length validation; absent fields are omitted from the wire.
loadIndex/describeIndex:indexKeyoptional; key omitted from therequest when absent; same key-length validation as
createIndex.EncryptedIndex: holds an optional key, hex-encoded once in the constructorand applied to every data-plane request via a
withKey()helper (matchespy's
_index_key_hex/ go's stored hex).EncryptedIndexmetadata caching to match py/go:getIndexName()returns thein-memory name with no API call;
getIndexType()/getIndexConfig()arefetched once and cached (immutable);
isTrained()always fetches fresh.handleApiError(+ type guards) andextractErrorDetailinto a sharedsrc/errors.ts; diagnostic logging is nowgated behind
CYBORGDB_DEBUG(off by default).indexKey+kmsNamepass-through._embeddingModelconstructor param.update-openapi-client.shclean-line (src/models/src/apis) and addedindex.tstosrc/.openapi-generator-ignoreso regenno longer clobbers the hand-written public exports.
kms.md: internal re-implementation guide — will be deleted before merge.Testing
src/__tests__/kms.test.ts:key-length validation on create + load,
CreateIndexRequestwire shape,keyless vs keyed request bodies across all data-plane ops including the
binary upsert/query paths, and the keyless
loadIndexdescribe shape.slot, real-provider + key rejected,
provider:nonemissing/wrong key), andmixed-mode concurrency.
kms.registryslots; run withCYBORGDB_KMS_NAME_{REAL,SM,NONE}set againstsuch a service (not exercised in standard CI).
build + lint + tsc all green. Live KMS suites not yet run against a
configured service — needs a deployment with the three registry slots.
Risk assessment
createIndex/loadIndex)and every data-plane request in
EncryptedIndex, plus regenerated models.However the change is additive/relaxing —
index_keywent from required tooptional, existing keyed call sites are unchanged, and the offline tests pin
the exact wire shapes for both keyed and keyless paths. The metadata-caching
change makes
getIndexName()stop hitting the server (it returns thein-memory name) — behavior-aligned with py/go but worth noting for anyone who
relied on it as an implicit existence probe.
multi-tenancy-2branch; revert thePR. The SDK is versioned/published independently, so reverting the package
release restores prior behavior with no service-side coordination. No data
migration and no persisted state change on the client.
Reviewers, please focus on:
EncryptedIndex(withKey()+ cached hex):confirm no data-plane path can leak or drop
index_keyincorrectly.getIndexName()behavior change (no longer an API call) — acceptable?errors.tsextraction (handleApiErrorconsolidation) and theCYBORGDB_DEBUGlogging gate.kms.mdis removed before merge.