Skip to content

Implement ingester database isolation#9

Merged
ian-ross merged 8 commits into
mainfrom
db-staging
May 9, 2026
Merged

Implement ingester database isolation#9
ian-ross merged 8 commits into
mainfrom
db-staging

Conversation

@ian-ross
Copy link
Copy Markdown
Member

@ian-ross ian-ross commented May 6, 2026

Summary

  • add required staging-directory config and path safety validation
  • publish clean public snapshots via scratch VACUUM INTO + hidden-copy atomic rename
  • rework DBCache to mutate only nursery/staging databases and import public snapshots into staging
  • add staging recovery, export throttling, idle finalization, and operator docs
  • constrain service/workspace Python to >=3.13,<3.14 while keeping API/common packages installable on >=3.12,<3.14

Python version policy

  • The receiver/server processes are constrained to Python 3.13 because uv-managed Python 3.12 hit an OpenSSL initialization failure when creating TLS client contexts, breaking HTTPS retrieval from the Contrails API.
  • Python 3.14 remains excluded for now.
  • The distributable client/API surface remains Python 3.12-compatible: api and feder-common use >=3.12,<3.14.
  • The top-level workspace stays at >=3.13,<3.14 so normal uv sync / service development does not select the broken 3.12 runtime.

Closes #3
Closes #8

Tests

  • uv lock --check
  • uv sync --all-packages --dev --python 3.13
  • uv run --python 3.13 pytest tests/feder_ingest/test_db_cache_paths.py tests/server_lib/test_config_paths.py -q
  • installed ./libs/common and ./api into a fresh Python 3.12 virtualenv successfully

ian-ross and others added 8 commits May 5, 2026 22:50
Co-authored-by: GPT-5.5 <openai-codex-gpt-5.5@pi.local>
Pi-Model: openai-codex/gpt-5.5
Co-authored-by: GPT-5.5 <openai-codex-gpt-5.5@pi.local>
Pi-Model: openai-codex/gpt-5.5
Co-authored-by: GPT-5.5 <openai-codex-gpt-5.5@pi.local>
Pi-Model: openai-codex/gpt-5.5
Co-authored-by: GPT-5.5 <openai-codex-gpt-5.5@pi.local>
Pi-Model: openai-codex/gpt-5.5
Co-authored-by: GPT-5.5 <openai-codex-gpt-5.5@pi.local>
Pi-Model: openai-codex/gpt-5.5
Co-authored-by: GPT-5.5 <openai-codex-gpt-5.5@pi.local>
Pi-Model: openai-codex/gpt-5.5
Co-authored-by: GPT-5.5 <openai-codex-gpt-5.5@pi.local>
Pi-Model: openai-codex/gpt-5.5
Co-authored-by: GPT-5.5 <openai-codex-gpt-5.5@pi.local>
Pi-Model: openai-codex/gpt-5.5
@ian-ross ian-ross merged commit 36f8160 into main May 9, 2026
1 check passed
@ian-ross ian-ross deleted the db-staging branch May 9, 2026 14:01
@ian-ross ian-ross restored the db-staging branch May 9, 2026 14:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Complete tests and operator docs for database isolation Database isolation is doubleplus ungood

1 participant