Skip to content

Add OpenEntropy entropy source and ECDF amplification#4

Closed
ereid7 wants to merge 2 commits intoalchemystack:mainfrom
ereid7:feat/openentropy-source
Closed

Add OpenEntropy entropy source and ECDF amplification#4
ereid7 wants to merge 2 commits intoalchemystack:mainfrom
ereid7:feat/openentropy-source

Conversation

@ereid7
Copy link
Copy Markdown
Contributor

@ereid7 ereid7 commented Mar 2, 2026

Summary

  • add an OpenEntropy entropy source implementation and register it via qr_sampler.entropy_sources
  • add an ECDF amplifier with calibration, config support, and processor wiring for calibrated amplification
  • add deployment/docs updates for OpenEntropy usage, configuration, and source catalog references
  • add comprehensive tests for ECDF behavior, OpenEntropy source behavior, and OpenEntropy config fields

Why ECDF Amplifier

  • z-score amplification assumes byte means follow the expected distribution for ideal uniform bytes; real hardware/QRNG sources can violate that assumption and introduce bias in u
  • ECDF is non-parametric: it calibrates on observed source behavior and maps ranks to uniform values, which makes token selection robust to source-specific skew
  • this gives a better research baseline: if source distributions drift, the sampler still targets uniform u without relying on fragile distributional assumptions

Configurability

  • yes, it is configurable
  • select amplifier via QR_SIGNAL_AMPLIFIER_TYPE (ecdf or zscore_mean)
  • tune calibration size via QR_ECDF_CALIBRATION_SAMPLES (default 2000, min 100)
  • per-request override supports qr_signal_amplifier_type; calibration sample count is configured at process/env config level

OpenEntropy Configuration Note

  • set QR_ENTROPY_SOURCE_TYPE=openentropy to enable the source
  • choose conditioning with QR_OE_CONDITIONING (raw, vonneumann, sha256)
  • optionally pin specific sources with QR_OE_SOURCES (for example: counter_beat)
  • for full setup and source catalog details, see:

Why Not gRPC for OpenEntropy

  • OpenEntropy is already a local in-process API; adding gRPC would add transport overhead and another failure surface without improving local functionality
  • source and conditioning controls are first-class in config (QR_OE_SOURCES, QR_OE_CONDITIONING); routing through a gRPC adapter would add extra config/plumbing complexity
  • gRPC remains appropriate for remote entropy providers (quantum_grpc), while native integration is simpler and lower-friction for local OpenEntropy research workflows

Validation

  • python3 -m ruff check src/ tests/
  • python3 -m pytest tests/ -q
  • python3 -m build --sdist --wheel

@alchemystack
Copy link
Copy Markdown
Owner

Hi, the PR failed some of the CI checks. I've put the error logs in a txt file for you to troubleshoot, CC should be able to handle it.
openentropy PR error logs.txt
Also plz let me know if you're able to run the checks, or if I need to se up a "develop" branch or something so people can do these checks themselves, before opening PRs for main.

@alchemystack
Copy link
Copy Markdown
Owner

I'll be moving forward with some larger restructuring of the codebase, so I went ahead and integrated your PR. Since OpenEntropy is Rust and not very compatible with the rest of the code, moved it to an optional addon, not part of the default build. Closing this as I made those edits in different branch.

alchemystack added a commit that referenced this pull request Mar 27, 2026
Integrate OpenEntropy hardware entropy source and ECDF signal amplifier
from ereid7's PR #4, with fixes for CI failures:

- Move openentropy to optional dependency (was breaking Python 3.10/3.11/3.13)
- Fix ruff formatting on openentropy.py and test_openentropy.py
- Remove bandit-flagged assert in ecdf.py (invariant enforced by _calibrated)
- Add ECDF calibration wiring in processor.py
- Optimize _to_numpy with is_cpu check to avoid unnecessary .cpu() calls
- Add oe_* config fields, entry point, mypy override, and tests

🤖 Generated with [Claude Code](https://claude.com/claude-code)
alchemystack added a commit that referenced this pull request Mar 27, 2026
feat: add OpenEntropy source and ECDF amplifier from PR #4
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants