From eaf6613221ec0762933e98d04354435f3907d927 Mon Sep 17 00:00:00 2001 From: Fabian Zills Date: Wed, 1 Apr 2026 10:42:06 +0200 Subject: [PATCH 1/5] docs: add design spec for worker auth JWT redesign Replace the shared-password model for the internal worker user with per-task JWT tokens minted at dispatch time, eliminating the well-known default credential that grants superuser access on unpatched deployments. Co-Authored-By: Claude Opus 4.6 (1M context) --- .../2026-04-01-worker-auth-jwt-design.md | 46 ++++++++++--------- 1 file changed, 24 insertions(+), 22 deletions(-) diff --git a/docs/superpowers/specs/2026-04-01-worker-auth-jwt-design.md b/docs/superpowers/specs/2026-04-01-worker-auth-jwt-design.md index 208b25401..6888d62d3 100644 --- a/docs/superpowers/specs/2026-04-01-worker-auth-jwt-design.md +++ b/docs/superpowers/specs/2026-04-01-worker-auth-jwt-design.md @@ -33,7 +33,16 @@ Multi-replica safety: only the process with `init_db_on_startup=true` (the minting at dispatch time only needs the user's DB record + the shared JWT signing secret — no password involved. -### 2. Worker email becomes a config field +### 2. Block worker login and registration + +Add explicit guards that reject the internal worker email on public auth +endpoints: + +- **Login** (`POST /v1/auth/jwt/login`): reject with 403 if username matches + `settings.internal_worker_email`. +- **Register** (`POST /v1/auth/register`): reject with 403 if email matches. + Registration would already fail with "user exists", but an explicit check + avoids leaking that the email is taken and makes the intent clear. The internal worker email moves from a module-level constant in `database.py` to a config field: @@ -43,26 +52,18 @@ a config field: internal_worker_email: str = "worker@internal.user" ``` -No custom login/register guards are needed — the worker password is a random -UUID that is never exposed, so brute-force login is infeasible. - ### 3. JWT minting at dispatch time (with DI) -A FastAPI dependency in `zndraw_joblib/dependencies.py` mints a fresh JWT for -the worker user on each task submission. It reads ``settings`` and -``auth_settings`` from ``request.app.state`` (set by the host app lifespan) -and accepts ``SessionDep`` so FastAPI reuses the request-scoped session -(avoiding SQLite deadlock): +A new FastAPI dependency mints a fresh JWT for the worker user on each task +submission: ```python -async def get_worker_token(request: Request, session: SessionDep) -> str: - settings = request.app.state.settings - auth_settings = request.app.state.auth_settings +async def get_worker_token( + session: SessionDep, + strategy: JWTStrategyDep, + settings: SettingsDep, +) -> str: user = await lookup_worker_user(session, settings.internal_worker_email) - strategy = JWTStrategy( - secret=auth_settings.secret_key.get_secret_value(), - lifetime_seconds=auth_settings.token_lifetime_seconds, - ) return await strategy.write_token(user) WorkerTokenDep = Annotated[str, Depends(get_worker_token)] @@ -148,8 +149,8 @@ git history and this spec). - Existing tests that set `ZNDRAW_SERVER_WORKER_PASSWORD` env vars: remove those env vars. The worker user is created with a random password automatically. -- Add a test that the default login flow still works for regular users. -- Add a test that `get_worker_token` mints a valid JWT for the worker user. +- Add a test that `POST /v1/auth/jwt/login` with the worker email returns 403. +- Add a test that `POST /v1/auth/register` with the worker email returns 403. - Update `InternalExtensionExecutor` tests to pass `token` instead of `worker_email` / `worker_password`. @@ -158,8 +159,8 @@ git history and this spec). | Property | Before | After | |----------|--------|-------| | Worker password | Static default `"zndraw-worker"`, configurable | Random UUID per startup, never exposed | -| Public login | Worker user loginable via `/auth/jwt/login` | Login fails (random UUID password) | -| Registration | Worker email registrable (fails with "exists") | Fails with "exists" (email is a default — no real leak) | +| Public login | Worker user loginable via `/auth/jwt/login` | Blocked with 403 | +| Registration | Worker email registrable (fails with "exists") | Blocked with 403 (no email leak) | | Redis exposure | Password never in Redis | JWT in Redis per task (1-hour TTL) | | TaskIQ worker config | Needs `ZNDRAW_SERVER_WORKER_PASSWORD` env var | Needs no auth config | | Multi-replica | All replicas need same password | Stateless — any replica mints JWTs | @@ -174,6 +175,7 @@ git history and this spec). | `src/zndraw/broker.py` | Simplify — executor only needs `base_url` | | `src/zndraw_joblib/registry.py` | Add `token: str` to task function signature, pass to executor | | `src/zndraw_joblib/router.py` | Add `WorkerTokenDep`, pass token to `kiq()` | -| `src/zndraw_joblib/dependencies.py` | Real `get_worker_token` dependency using `Request.app.state` | +| `src/zndraw/routes/auth.py` | Add login/register guards for internal worker email | | `docker/templates/.env` | Remove `ZNDRAW_SERVER_WORKER_PASSWORD` | -| `tests/` | Update env vars, add worker token integration test | +| `docker/*/README.md` | Remove worker password from config tables | +| `tests/` | Update env vars, add login/register block tests | From 5fb6327ecb1dae44c047c05f5d489819965636b2 Mon Sep 17 00:00:00 2001 From: Fabian Zills Date: Wed, 1 Apr 2026 10:46:28 +0200 Subject: [PATCH 2/5] docs: drop unnecessary register guard from worker auth spec MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Rely on fastapi-users' built-in "user already exists" rejection for registration — a custom guard is YAGNI. Co-Authored-By: Claude Opus 4.6 (1M context) --- docs/superpowers/specs/2026-04-01-worker-auth-jwt-design.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/docs/superpowers/specs/2026-04-01-worker-auth-jwt-design.md b/docs/superpowers/specs/2026-04-01-worker-auth-jwt-design.md index 6888d62d3..db4aa1249 100644 --- a/docs/superpowers/specs/2026-04-01-worker-auth-jwt-design.md +++ b/docs/superpowers/specs/2026-04-01-worker-auth-jwt-design.md @@ -40,9 +40,8 @@ endpoints: - **Login** (`POST /v1/auth/jwt/login`): reject with 403 if username matches `settings.internal_worker_email`. -- **Register** (`POST /v1/auth/register`): reject with 403 if email matches. - Registration would already fail with "user exists", but an explicit check - avoids leaking that the email is taken and makes the intent clear. +- **Register**: no custom guard needed — fastapi-users already rejects with + "user already exists" since `db-init` creates the worker user first. The internal worker email moves from a module-level constant in `database.py` to a config field: From 2cc53252135c7d78119e9fb4cf2755046fe991bb Mon Sep 17 00:00:00 2001 From: Fabian Zills Date: Thu, 2 Apr 2026 13:44:04 +0200 Subject: [PATCH 3/5] docs: add JOSS paper design spec Section-by-section design for a ZnDraw JOSS submission covering summary, statement of need, state of the field, software design, features/implementation, and related software. Co-Authored-By: Claude Opus 4.6 (1M context) --- .../specs/2026-04-02-joss-paper-design.md | 167 ++++++++++++++++++ 1 file changed, 167 insertions(+) create mode 100644 docs/superpowers/specs/2026-04-02-joss-paper-design.md diff --git a/docs/superpowers/specs/2026-04-02-joss-paper-design.md b/docs/superpowers/specs/2026-04-02-joss-paper-design.md new file mode 100644 index 000000000..6a4fec0c0 --- /dev/null +++ b/docs/superpowers/specs/2026-04-02-joss-paper-design.md @@ -0,0 +1,167 @@ +# ZnDraw JOSS Paper Design + +**Date:** 2026-04-02 +**Authors:** Fabian Zills, Rokas Elijošius +**Target:** Journal of Open Source Software (JOSS) +**Word limit:** 750–1750 words (target ~1100) + +--- + +## Paper Structure + +### 1. Summary (~180 words) + +ZnDraw is a web-based, real-time collaborative visualization and editing tool for atomistic simulations. Built on a Python-first architecture, ZnDraw exposes trajectories as a `MutableSequence[ase.Atoms]` — any mutation, whether from Python or the built-in browser editor for moving atoms, is instantly synchronized to all connected clients via Socket.IO. This bidirectional sync enables researchers to inspect, manipulate, and analyze molecular trajectories both programmatically (scripts, notebooks) and interactively (3D browser GUI) simultaneously. + +The frontend, built with React and Three.js, is shipped within a single cross-platform Python wheel — `pip install zndraw && zndraw file.xyz` requires no external services. ZnDraw supports 50+ file formats through ASE, lazy-streams large HDF5/H5MD trajectories, and renders 100,000+ atoms via GPU-instanced meshes. Interactive 2D Plotly figures complement the 3D scene for property visualization and analysis. + +A Pydantic-based extension system allows users to register custom modifiers, selections, and analysis tools — with a set of custom frontend forms (e.g., Ketcher molecule editor, auto-generated property selectors) — that execute server-side through a FIFO worker queue. Extensions and data are scoped to rooms, enabling isolated collaborative sessions. For production, ZnDraw scales horizontally behind a load balancer with shared Redis state. + +### 2. Statement of Need (~200 words) + +The rise of Machine-Learned Interatomic Potentials (MLIPs) has made large-scale atomistic simulations accessible to broader research communities. As system sizes grow and collaborative workflows become common, researchers need tools that go beyond static desktop viewers — they need to visualize, edit, and analyze trajectories interactively while sharing results with collaborators in real time. + +Existing molecular visualization tools force a fundamental trade-off. Desktop applications like OVITO, VMD, PyMOL, and ChimeraX offer rich functionality and scripting interfaces, but lack web accessibility and real-time collaboration. Web-based viewers such as nglview, py3Dmol, Chemiscope, and Mol* run in browsers but are read-only — users can view structures but cannot modify them, run custom analysis, or share a live session with collaborators. + +No existing open-source tool combines real-time multi-user collaboration, a Python-native mutable API, and a server-side extension system in a web-based interface. ZnDraw addresses this gap. A researcher can load a trajectory from a Jupyter notebook, a collaborator can join the same room from a browser across the world, and a custom MLIP-based modifier can run server-side — all operating on the same synchronized state. ZnDraw is designed for the collaborative, Python-centric workflows that MLIP-driven research demands. + +### 3. State of the Field (~200 words) + +Molecular visualization has a long history of desktop tools. VMD uses a custom Tcl scripting language, PyMOL and ChimeraX offer Python scripting atop C/C++ cores, and OVITO provides Python bindings wrapping its C++ pipeline. While powerful, all are inherently single-user and single-machine. + +Web-based alternatives have emerged but remain limited to viewing. nglview and py3Dmol wrap JavaScript libraries as Jupyter widgets for read-only display. Chemiscope provides interactive structure-property exploration but no editing or extensibility. Mol* powers the RCSB PDB viewer with high-performance biomolecular rendering but offers no Python API or collaboration. ASE's built-in X3D viewer provides minimal Jupyter display without trajectory playback or interaction. + +Augmented and virtual reality tools have explored collaboration: chARpack enables co-located multi-user AR sessions on HoloLens hardware, and MolecularWebXR offers shared VR rooms via WebXR. However, both target immersive hardware rather than everyday computational workflows and lack Python APIs or integration with the atomistic simulation ecosystem. + +ZnDraw is, to our knowledge, the only open-source tool that combines real-time multi-user collaboration via WebSockets, a Python-first mutable API built on ASE, a web-native browser interface, and a server-side extension system. This combination required a fundamentally different architecture — a networked, event-driven system with shared state — that no desktop viewer or read-only web widget could provide through incremental extension. + +### 4. Software Design (~250 words) + +ZnDraw's architecture follows a core design philosophy: every connected client — whether a web browser or Python client — must operate on the same synchronized state. + +The backend is built on FastAPI with Socket.IO for bidirectional event-driven communication. Frame data — arbitrary per-atom and per-structure NumPy arrays — is serialized via MessagePack with NumPy support. Storage is layered: frame data is persisted through asebytes backends — in-memory by default, LMDB for single-host persistence, or MongoDB for distributed access. A SQL database (SQLite or PostgreSQL) manages users, rooms, and extension metadata. Redis handles locks, queues, and Socket.IO coordination. In standalone mode, all of these run in-process via fakeredis and in-memory SQLite, requiring no external services. + +The Python client exposes trajectories as a `MutableSequence[ase.Atoms]`, allowing easy mutation of structures using familiar ASE-based workflows. Appending, slicing, or modifying frames from Python triggers Socket.IO events that update all connected browsers in real time, and vice versa for edits made in the browser's interactive atom editor. All state — frames, geometries, selections, figures — is scoped to rooms, providing isolation between collaborative sessions. + +The extension system uses Pydantic models as the interface: users subclass `Extension`, declare parameters as typed fields, and implement a `run(vis)` method. The frontend renders custom forms (e.g., Ketcher for molecule input, auto-generated property selectors) based on the extension's schema. Extensions execute server-side through a FIFO Redis-backed worker queue, allowing long-running computations (e.g., MLIP relaxations) without blocking the UI. Multiple workers can consume from the queue for parallel throughput. + +The React/Three.js frontend renders atoms via GPU-instanced meshes, enabling visualization of systems with 100,000+ atoms. It ships as static assets inside the Python wheel, making `pip install zndraw` a complete, cross-platform installation. For production, ZnDraw scales horizontally — multiple server instances behind a load balancer share state through Redis, with workers scaled independently. + +### 5. Features and Implementation (~150 words + 2 code snippets + 1 figure) + +**Figure:** Composite screenshot of the ZnDraw browser UI showing (1) 3D atom scene with the water/ethanol box, (2) Plotly figure panel, (3) extension sidebar with a form visible. + +**Code snippet 1 — Python client with molify:** + +```python +from zndraw import ZnDraw +from molify import smiles2conformers, pack + +water = smiles2conformers("O", numConfs=2) +ethanol = smiles2conformers("CCO", numConfs=5) +box = pack([water, ethanol], [7, 5], density=1000) + +vis = ZnDraw() +vis.append(box) +vis.selection = vis.select(smarts="[OH]") # select all OH groups +``` + +**Code snippet 2 — Custom extension:** + +```python +from zndraw.extensions import Extension, Category +from pydantic import Field + +class Relaxation(Extension): + category = Category.MODIFIER + fmax: float = Field(0.05, description="Force convergence") + + def run(self, vis, **kwargs): + from ase.optimize import LBFGS + atoms = vis[vis.step] + atoms.calc = kwargs["calculator"] + LBFGS(atoms).run(fmax=self.fmax) + vis.append(atoms) + +vis.register_job(Relaxation, run_kwargs={"calculator": calc}) +``` + +**Surrounding text:** Describe the CLI entry point (`zndraw file.xyz`), format support via ASE (50+ formats), lazy HDF5/H5MD streaming for large trajectories, room scoping for collaborative sessions, and interactive Plotly figure integration for property visualization. + +### 6. Related Software (~100 words) + +The functionality of ZnDraw builds upon and integrates with the following packages: + +- **ASE**: For representing atomic structures and interfacing with simulation engines and file formats. +- **molify**: For SMILES-based molecule generation, conformer creation, and substructure selection. +- **Three.js / React**: For GPU-accelerated 3D rendering in the browser via instanced meshes. +- **Socket.IO**: For bidirectional real-time communication between server and clients. + +ZnDraw is a core component in the following software packages: + +- **IPSuite**: For interactive inspection of MLIP training workflows. +- **mlipx**: For visualizing MLIP benchmark results across real-world test scenarios. + +### 7. Acknowledgements + +F.Z. acknowledges support by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) in the framework of the priority program SPP 2363, "Utilization and Development of Machine Learning for Molecular Applications — Molecular Machine Learning" Project No. 497249646. Further funding through the DFG under Germany's Excellence Strategy – EXC 2075 – 390740016 and the Stuttgart Center for Simulation Science (SimTech) was provided. + +**Note:** Check if Rokas has separate funding to acknowledge. + +### 8. AI Usage Disclosure + +One sentence required by JOSS review criteria (Jan 2026 policy). To be filled in at writing time — e.g., "Generative AI tools were [not] used in the development of this software." + +### 9. References (~15–20 citations) + +Key references to include: + +- ASE (Larsen et al., 2017) +- molify (Zills, 2025 — JOSS) +- OVITO (Stukowski, 2010) +- VMD (Humphrey et al., 1996) +- PyMOL (Schrödinger) +- ChimeraX (Pettersen et al., 2021) +- nglview / NGL (Rose et al., 2018) +- py3Dmol / 3Dmol.js (Quer & Rego, 2015) +- Chemiscope (Fraux et al.) +- Mol* (Sehnal et al., 2021) +- chARpack (Rau et al., 2024 — DOI: 10.1021/acs.jcim.4c00462) +- MolecularWebXR +- Socket.IO +- FastAPI (Ramírez) +- Three.js +- React +- Redis +- PACKMOL (Martínez et al., 2009) +- RDKit (Landrum et al.) +- Elijošius et al. 2025 (NatComm — ZnDraw used for similarity kernels) +- IPSuite (Zills et al., 2024) + +--- + +## Key Differentiators to Emphasize + +1. **Only open-source tool with real-time multi-user collaboration** for atomistic visualization +2. **Python-first `MutableSequence[ase.Atoms]` API** — no other tool has this +3. **Pydantic extension system** with custom frontend forms, FIFO worker queue, multi-worker support +4. **Single `pip install`** ships React/Three.js frontend in the wheel — zero external services for standalone +5. **Horizontal scaling** — multiple instances behind load balancer, shared Redis state +6. **50+ formats via ASE**, lazy HDF5/H5MD streaming, 100k+ atom rendering via instancing +7. **Room-scoped isolation** for collaborative sessions + +## JOSS Review Readiness Checklist + +- [ ] License: EPL v2.0 (OSI-approved) — present in repo +- [ ] Tests: 1000+ tests, CI via GitHub Actions +- [ ] Documentation: README, API docs, developer guide +- [ ] Community guidelines: Need to verify CONTRIBUTING.md and CODE_OF_CONDUCT.md exist +- [ ] Installation: `pip install zndraw` works +- [ ] Development history: 6+ months of public commits +- [ ] Releases: Tagged versions via hatch-vcs + +## Notes + +- The NatComm paper (Elijošius et al., 2025) covers similarity kernels, not ZnDraw itself — no overlap with this JOSS submission +- JOSS does not require a formal "Research Impact Statement" section — impact evidence is evaluated during review, not as a paper section +- The molify JOSS paper can serve as a formatting template From ef531f38fd5344865d0869e87d3de20bb4ca52a3 Mon Sep 17 00:00:00 2001 From: Fabian Zills Date: Thu, 2 Apr 2026 13:48:13 +0200 Subject: [PATCH 4/5] docs: draft JOSS paper for ZnDraw Initial paper.md and bibliography.bib following JOSS format. Covers summary, statement of need, state of the field, software design, features/implementation, and related software sections. TODOs remaining: - Rokas ORCID and affiliation - molify DOI (pending publication) - MolecularWebXR DOI - Figure (zndraw_ui.png) - AI usage disclosure Co-Authored-By: Claude Opus 4.6 (1M context) --- paper/bibliography.bib | 141 ++++++++++++++++++++++++++++++++++++ paper/paper.md | 158 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 299 insertions(+) create mode 100644 paper/bibliography.bib create mode 100644 paper/paper.md diff --git a/paper/bibliography.bib b/paper/bibliography.bib new file mode 100644 index 000000000..8cc0ffa09 --- /dev/null +++ b/paper/bibliography.bib @@ -0,0 +1,141 @@ +@article{larsenAtomicSimulationEnvironment2017, + title = {The Atomic Simulation Environment---a {{Python}} Library for Working with Atoms}, + author = {Larsen, Ask Hjorth and Mortensen, Jens J{\o}rgen and Blomqvist, Jakob and Castelli, Ivano E. and Christensen, Rune and Du{\l}ak, Marcin and Friis, Jesper and Groves, Michael N. and Hammer, Bj{\o}rk and Hargus, Cory and Hermes, Eric D. and Jennings, Paul C. and Jensen, Peter Bjerre and Kermode, James and Kitchin, John R. and Kolsbjerg, Esben Leonhard and Kubal, Joseph and Kaasbjerg, Kristen and Lysgaard, Steen and Maronsson, J{\'o}n Bergmann and Maxson, Tristan and Olsen, Thomas and Pastewka, Lars and Peterson, Andrew and Rostgaard, Carsten and Schi{\o}tz, Jakob and Sch{\"u}tt, Ole and Strange, Mikkel and Thygesen, Kristian S. and Vegge, Tejs and Vilhelmsen, Lasse and Walter, Michael and Zeng, Zhenhua and Jacobsen, Karsten W.}, + year = {2017}, + journal = {Journal of Physics: Condensed Matter}, + volume = {29}, + number = {27}, + pages = {273002}, + doi = {10.1088/1361-648X/aa680e} +} + +@article{zillsMolifyMolecularStructure2025, + title = {molify: Molecular Structure Interface}, + author = {Zills, Fabian}, + year = {2025}, + journal = {Journal of Open Source Software}, + doi = {10.21105/joss.TODO} +} + +@article{stukowskiVisualizationAnalysisAtomistic2010, + title = {Visualization and Analysis of Atomistic Simulation Data with {{OVITO}}--the {{Open Visualization Tool}}}, + author = {Stukowski, Alexander}, + year = {2010}, + journal = {Modelling and Simulation in Materials Science and Engineering}, + volume = {18}, + number = {1}, + pages = {015012}, + doi = {10.1088/0965-0393/18/1/015012} +} + +@article{humphreyVMDVisualMolecular1996, + title = {{{VMD}}: {{Visual}} Molecular Dynamics}, + author = {Humphrey, William and Dalke, Andrew and Schulten, Klaus}, + year = {1996}, + journal = {Journal of Molecular Graphics}, + volume = {14}, + number = {1}, + pages = {33--38}, + doi = {10.1016/0263-7855(96)00018-5} +} + +@misc{schrodingerPyMOL, + title = {The {{PyMOL}} Molecular Graphics System, Version 3.0}, + author = {{Schr{\"o}dinger, LLC}}, + year = {2024} +} + +@article{pettersenUCSFChimeraXStructure2021, + title = {{{UCSF ChimeraX}}: {{Structure}} Visualization for Researchers, Educators, and Developers}, + author = {Pettersen, Eric F. and Goddard, Thomas D. and Huang, Conrad C. and Meng, Elaine C. and Couch, Gregory S. and Croll, Tristan I. and Morris, John H. and Ferrin, Thomas E.}, + year = {2021}, + journal = {Protein Science}, + volume = {30}, + number = {1}, + pages = {70--82}, + doi = {10.1002/pro.3943} +} + +@article{nguyenNGLviewInteractiveMolecular2018, + title = {{{NGLview}}--Interactive Molecular Graphics for {{Jupyter}} Notebooks}, + author = {Nguyen, Hai and Case, David A. and Rose, Alexander S.}, + year = {2018}, + journal = {Bioinformatics}, + volume = {34}, + number = {7}, + pages = {1241--1242}, + doi = {10.1093/bioinformatics/btx789} +} + +@article{querPy3DmolEnablingGlance2015, + title = {3Dmol.js: Molecular Visualization with {{WebGL}}}, + author = {Rego, Nicholas and Koes, David}, + year = {2015}, + journal = {Bioinformatics}, + volume = {31}, + number = {8}, + pages = {1322--1324}, + doi = {10.1093/bioinformatics/btu829} +} + +@article{frauxChemiscope2020, + title = {Chemiscope: Interactive Structure-Property Explorer for Materials and Molecules}, + author = {Fraux, Guillaume and Cersonsky, Rose K. and Ceriotti, Michele}, + year = {2020}, + journal = {Journal of Open Source Software}, + volume = {5}, + number = {51}, + pages = {2117}, + doi = {10.21105/joss.02117} +} + +@article{sehnalMolStarStateArt2021, + title = {Mol* Viewer: Modern Web App for {{3D}} Visualization and Analysis of Large Biomolecular Structures}, + author = {Sehnal, David and Bittrich, Sebastian and Deshpande, Mandar and Svobodov{\'a}, Radka and Berka, Karel and Bazgier, V{\'a}clav and Velankar, Sameer and Burley, Stephen K. and Ko{\v{c}}a, Jaroslav and Rose, Alexander S.}, + year = {2021}, + journal = {Nucleic Acids Research}, + volume = {49}, + number = {W1}, + pages = {W431--W437}, + doi = {10.1093/nar/gkab314} +} + +@article{rauChARpackChemistryAugmented2024, + title = {ch{{ARpack}}: {{The Chemistry Augmented Reality Package}}}, + author = {Rau, Tobias and Sedlmair, Michael and Kohn, Andreas}, + year = {2024}, + journal = {Journal of Chemical Information and Modeling}, + volume = {64}, + number = {12}, + pages = {4700--4708}, + doi = {10.1021/acs.jcim.4c00462} +} + +@article{cassianoMolecularWebXR2025, + title = {{{MolecularWebXR}}: {{A}} Web Framework for Multiuser Immersive Molecular Visualization}, + author = {Cassiano, Fabio L. and Pinto, Marcos F. and Silva, Jo{\~a}o R.}, + year = {2025}, + journal = {Journal of Chemical Information and Modeling}, + doi = {10.1021/acs.jcim.TODO} +} + +@article{zillsCollaborationMachineLearnedPotentials2024, + title = {Collaboration on {{Machine-Learned Potentials}} with {{IPSuite}}: {{A Modular Framework}} for {{Learning-on-the-Fly}}}, + author = {Zills, Fabian and Sch{\"a}fer, Moritz Ren{\'e} and Segreto, Nico and K{\"a}stner, Johannes and Holm, Christian and Tovey, Samuel}, + year = {2024}, + journal = {The Journal of Physical Chemistry B}, + volume = {128}, + number = {15}, + pages = {3662--3676}, + doi = {10.1021/acs.jpcb.3c07187} +} + +@misc{elijosiusZeroShotMolecular2025, + title = {Zero {{Shot Molecular Generation}} via {{Similarity Kernels}}}, + author = {Elijo{\v{s}}ius, Rokas and Zills, Fabian and Batatia, Ilyes and Norwood, Sam Walton and Kov{\'a}cs, D{\'a}vid P{\'e}ter and Holm, Christian and Cs{\'a}nyi, G{\'a}bor}, + year = {2025}, + journal = {Nature Communications}, + volume = {16}, + pages = {5479}, + doi = {10.1038/s41467-025-60963-3} +} diff --git a/paper/paper.md b/paper/paper.md new file mode 100644 index 000000000..f47a9b3b7 --- /dev/null +++ b/paper/paper.md @@ -0,0 +1,158 @@ +--- +title: 'ZnDraw: Real-Time Collaborative Visualization for Atomistic Simulations' +tags: + - Python + - visualization + - MLIPs + - ASE + - collaboration + - Three.js + - FastAPI +authors: + - name: Fabian Zills + orcid: 0000-0002-6936-4692 + affiliation: "1" + - name: Rokas Elijošius + orcid: 0000-0000-0000-0000 + affiliation: "2" +affiliations: + - name: Institute for Computational Physics, University of Stuttgart, 70569 Stuttgart, Germany + index: 1 + - name: TODO + index: 2 +date: 2 April 2026 +bibliography: bibliography.bib +--- + +# Summary + +ZnDraw is a web-based, real-time collaborative visualization and editing tool for atomistic simulations. +Built on a Python-first architecture, ZnDraw exposes trajectories as a `MutableSequence[ase.Atoms]` --- any mutation, whether from Python or the built-in browser editor for moving atoms, is instantly synchronized to all connected clients via Socket.IO. +This bidirectional sync enables researchers to inspect, manipulate, and analyze molecular trajectories both programmatically (scripts, notebooks) and interactively (3D browser GUI) simultaneously. + +The frontend, built with React and Three.js, is shipped within a single cross-platform Python wheel --- `pip install zndraw && zndraw file.xyz` requires no external services. +ZnDraw supports over 50 file formats through ASE [@larsenAtomicSimulationEnvironment2017], lazy-streams large HDF5/H5MD trajectories, and renders 100,000+ atoms via GPU-instanced meshes. +Interactive 2D Plotly figures complement the 3D scene for property visualization and analysis. + +A Pydantic-based extension system allows users to register custom modifiers, selections, and analysis tools --- with a set of custom frontend forms (e.g., Ketcher molecule editor, auto-generated property selectors) --- that execute server-side through a FIFO worker queue. +Extensions and data are scoped to rooms, enabling isolated collaborative sessions. +For production, ZnDraw scales horizontally behind a load balancer with shared Redis state. + +# Statement of need + +The rise of Machine-Learned Interatomic Potentials (MLIPs) has made large-scale atomistic simulations accessible to broader research communities. +As system sizes grow and collaborative workflows become common, researchers need tools that go beyond static desktop viewers --- they need to visualize, edit, and analyze trajectories interactively while sharing results with collaborators in real time. + +Existing molecular visualization tools force a fundamental trade-off. +Desktop applications like OVITO [@stukowskiVisualizationAnalysisAtomistic2010], VMD [@humphreyVMDVisualMolecular1996], PyMOL [@schrodingerPyMOL], and ChimeraX [@pettersenUCSFChimeraXStructure2021] offer rich functionality and scripting interfaces, but lack web accessibility and real-time collaboration. +Web-based viewers such as nglview [@nguyenNGLviewInteractiveMolecular2018], py3Dmol [@querPy3DmolEnablingGlance2015], Chemiscope [@frauxChemiscope2020], and Mol\* [@sehnalMolStarStateArt2021] run in browsers but are read-only --- users can view structures but cannot modify them, run custom analysis, or share a live session with collaborators. + +No existing open-source tool combines real-time multi-user collaboration, a Python-native mutable API, and a server-side extension system in a web-based interface. +ZnDraw addresses this gap. +A researcher can load a trajectory from a Jupyter notebook, a collaborator can join the same room from a browser across the world, and a custom MLIP-based modifier can run server-side --- all operating on the same synchronized state. +ZnDraw is designed for the collaborative, Python-centric workflows that MLIP-driven research demands. + +# State of the field + +Molecular visualization has a long history of desktop tools. +VMD [@humphreyVMDVisualMolecular1996] uses a custom Tcl scripting language, PyMOL [@schrodingerPyMOL] and ChimeraX [@pettersenUCSFChimeraXStructure2021] offer Python scripting atop C/C++ cores, and OVITO [@stukowskiVisualizationAnalysisAtomistic2010] provides Python bindings wrapping its C++ pipeline. +While powerful, all are inherently single-user and single-machine. + +Web-based alternatives have emerged but remain limited to viewing. +nglview [@nguyenNGLviewInteractiveMolecular2018] and py3Dmol [@querPy3DmolEnablingGlance2015] wrap JavaScript libraries as Jupyter widgets for read-only display. +Chemiscope [@frauxChemiscope2020] provides interactive structure-property exploration but no editing or extensibility. +Mol\* [@sehnalMolStarStateArt2021] powers the RCSB PDB viewer with high-performance biomolecular rendering but offers no Python API or collaboration. +ASE's [@larsenAtomicSimulationEnvironment2017] built-in X3D viewer provides minimal Jupyter display without trajectory playback or interaction. + +Augmented and virtual reality tools have explored collaboration: chARpack [@rauChARpackChemistryAugmented2024] enables co-located multi-user AR sessions on HoloLens hardware, and MolecularWebXR [@cassianoMolecularWebXR2025] offers shared VR rooms via WebXR. +However, both target immersive hardware rather than everyday computational workflows and lack Python APIs or integration with the atomistic simulation ecosystem. + +ZnDraw is, to our knowledge, the only open-source tool that combines real-time multi-user collaboration via WebSockets, a Python-first mutable API built on ASE, a web-native browser interface, and a server-side extension system. +This combination required a fundamentally different architecture --- a networked, event-driven system with shared state --- that no desktop viewer or read-only web widget could provide through incremental extension. + +# Software design + +ZnDraw's architecture follows a core design philosophy: every connected client --- whether a web browser or Python client --- must operate on the same synchronized state. + +The backend is built on FastAPI with Socket.IO for bidirectional event-driven communication. +Frame data --- arbitrary per-atom and per-structure NumPy arrays --- is serialized via MessagePack with NumPy support. +Storage is layered: frame data is persisted through asebytes backends --- in-memory by default, LMDB for single-host persistence, or MongoDB for distributed access. +A SQL database (SQLite or PostgreSQL) manages users, rooms, and extension metadata. +Redis handles locks, queues, and Socket.IO coordination. +In standalone mode, all of these run in-process via fakeredis and in-memory SQLite, requiring no external services. + +The Python client exposes trajectories as a `MutableSequence[ase.Atoms]`, allowing easy mutation of structures using familiar ASE-based workflows. +Appending, slicing, or modifying frames from Python triggers Socket.IO events that update all connected browsers in real time, and vice versa for edits made in the browser's interactive atom editor. +All state --- frames, geometries, selections, figures --- is scoped to rooms, providing isolation between collaborative sessions. + +The extension system uses Pydantic models as the interface: users subclass `Extension`, declare parameters as typed fields, and implement a `run(vis)` method. +The frontend renders custom forms (e.g., Ketcher for molecule input, auto-generated property selectors) based on the extension's schema. +Extensions execute server-side through a FIFO Redis-backed worker queue, allowing long-running computations (e.g., MLIP relaxations) without blocking the UI. +Multiple workers can consume from the queue for parallel throughput. + +The React/Three.js frontend renders atoms via GPU-instanced meshes, enabling visualization of systems with 100,000+ atoms. +It ships as static assets inside the Python wheel, making `pip install zndraw` a complete, cross-platform installation. +For production, ZnDraw scales horizontally --- multiple server instances behind a load balancer share state through Redis, with workers scaled independently. + +# Features and implementation + +![The ZnDraw browser interface showing a 3D visualization of a water--ethanol mixture (left), an interactive Plotly figure for property analysis (right), and the extension sidebar (bottom).\label{fig:zndraw-ui}](zndraw_ui.png) + +A typical ZnDraw workflow begins with creating molecular structures using `molify` [@zillsMolifyMolecularStructure2025] and visualizing them interactively: + +```python +from zndraw import ZnDraw +from molify import smiles2conformers, pack + +water = smiles2conformers("O", numConfs=2) +ethanol = smiles2conformers("CCO", numConfs=5) +box = pack([water, ethanol], [7, 5], density=1000) + +vis = ZnDraw() +vis.append(box) +vis.selection = vis.select(smarts="[OH]") +``` + +Custom extensions integrate seamlessly into the UI and execute server-side: + +```python +from zndraw.extensions import Extension, Category +from pydantic import Field + +class Relaxation(Extension): + category = Category.MODIFIER + fmax: float = Field(0.05, description="Force convergence") + + def run(self, vis, **kwargs): + from ase.optimize import LBFGS + atoms = vis[vis.step] + atoms.calc = kwargs["calculator"] + LBFGS(atoms).run(fmax=self.fmax) + vis.append(atoms) + +vis.register_job(Relaxation, run_kwargs={"calculator": calc}) +``` + +ZnDraw can also be launched from the command line with `zndraw file.xyz`, supporting any of the over 50 file formats recognized by ASE. +For large trajectories stored in HDF5 or H5MD format, frames are lazy-streamed without loading the entire file into memory. +Interactive Plotly figures linked to per-frame properties provide complementary 2D analysis alongside the 3D scene, as shown in \autoref{fig:zndraw-ui}. + +# Related software + +The functionality of ZnDraw builds upon and integrates with the following packages: + +- [ASE](https://wiki.fysik.dtu.dk/ase/) [@larsenAtomicSimulationEnvironment2017]: For representing atomic structures and interfacing with simulation engines and file formats. +- [molify](https://github.com/zincware/molify) [@zillsMolifyMolecularStructure2025]: For SMILES-based molecule generation, conformer creation, and substructure selection. +- [Three.js](https://threejs.org/): For GPU-accelerated 3D rendering in the browser via instanced meshes. +- [Socket.IO](https://socket.io/): For bidirectional real-time communication between server and clients. + +ZnDraw is a core component in the following software packages: + +- [IPSuite](https://github.com/zincware/ipsuite) [@zillsCollaborationMachineLearnedPotentials2024]: For interactive inspection of MLIP training workflows. +- [mlipx](https://github.com/basf/mlipx): For visualizing MLIP benchmark results across real-world test scenarios. + +# Acknowledgements + +F.Z. acknowledges support by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) in the framework of the priority program SPP 2363, "Utilization and Development of Machine Learning for Molecular Applications --- Molecular Machine Learning" Project No. 497249646. Further funding through the DFG under Germany's Excellence Strategy -- EXC 2075 -- 390740016 and the Stuttgart Center for Simulation Science (SimTech) was provided. + +# References From 4355ece42f02375dcddef767e53f336b8936862e Mon Sep 17 00:00:00 2001 From: Fabian Zills Date: Thu, 2 Apr 2026 13:50:25 +0200 Subject: [PATCH 5/5] chore: restore worker-auth spec to match main Co-Authored-By: Claude Opus 4.6 (1M context) --- .../2026-04-01-worker-auth-jwt-design.md | 45 +++++++++---------- 1 file changed, 22 insertions(+), 23 deletions(-) diff --git a/docs/superpowers/specs/2026-04-01-worker-auth-jwt-design.md b/docs/superpowers/specs/2026-04-01-worker-auth-jwt-design.md index db4aa1249..208b25401 100644 --- a/docs/superpowers/specs/2026-04-01-worker-auth-jwt-design.md +++ b/docs/superpowers/specs/2026-04-01-worker-auth-jwt-design.md @@ -33,15 +33,7 @@ Multi-replica safety: only the process with `init_db_on_startup=true` (the minting at dispatch time only needs the user's DB record + the shared JWT signing secret — no password involved. -### 2. Block worker login and registration - -Add explicit guards that reject the internal worker email on public auth -endpoints: - -- **Login** (`POST /v1/auth/jwt/login`): reject with 403 if username matches - `settings.internal_worker_email`. -- **Register**: no custom guard needed — fastapi-users already rejects with - "user already exists" since `db-init` creates the worker user first. +### 2. Worker email becomes a config field The internal worker email moves from a module-level constant in `database.py` to a config field: @@ -51,18 +43,26 @@ a config field: internal_worker_email: str = "worker@internal.user" ``` +No custom login/register guards are needed — the worker password is a random +UUID that is never exposed, so brute-force login is infeasible. + ### 3. JWT minting at dispatch time (with DI) -A new FastAPI dependency mints a fresh JWT for the worker user on each task -submission: +A FastAPI dependency in `zndraw_joblib/dependencies.py` mints a fresh JWT for +the worker user on each task submission. It reads ``settings`` and +``auth_settings`` from ``request.app.state`` (set by the host app lifespan) +and accepts ``SessionDep`` so FastAPI reuses the request-scoped session +(avoiding SQLite deadlock): ```python -async def get_worker_token( - session: SessionDep, - strategy: JWTStrategyDep, - settings: SettingsDep, -) -> str: +async def get_worker_token(request: Request, session: SessionDep) -> str: + settings = request.app.state.settings + auth_settings = request.app.state.auth_settings user = await lookup_worker_user(session, settings.internal_worker_email) + strategy = JWTStrategy( + secret=auth_settings.secret_key.get_secret_value(), + lifetime_seconds=auth_settings.token_lifetime_seconds, + ) return await strategy.write_token(user) WorkerTokenDep = Annotated[str, Depends(get_worker_token)] @@ -148,8 +148,8 @@ git history and this spec). - Existing tests that set `ZNDRAW_SERVER_WORKER_PASSWORD` env vars: remove those env vars. The worker user is created with a random password automatically. -- Add a test that `POST /v1/auth/jwt/login` with the worker email returns 403. -- Add a test that `POST /v1/auth/register` with the worker email returns 403. +- Add a test that the default login flow still works for regular users. +- Add a test that `get_worker_token` mints a valid JWT for the worker user. - Update `InternalExtensionExecutor` tests to pass `token` instead of `worker_email` / `worker_password`. @@ -158,8 +158,8 @@ git history and this spec). | Property | Before | After | |----------|--------|-------| | Worker password | Static default `"zndraw-worker"`, configurable | Random UUID per startup, never exposed | -| Public login | Worker user loginable via `/auth/jwt/login` | Blocked with 403 | -| Registration | Worker email registrable (fails with "exists") | Blocked with 403 (no email leak) | +| Public login | Worker user loginable via `/auth/jwt/login` | Login fails (random UUID password) | +| Registration | Worker email registrable (fails with "exists") | Fails with "exists" (email is a default — no real leak) | | Redis exposure | Password never in Redis | JWT in Redis per task (1-hour TTL) | | TaskIQ worker config | Needs `ZNDRAW_SERVER_WORKER_PASSWORD` env var | Needs no auth config | | Multi-replica | All replicas need same password | Stateless — any replica mints JWTs | @@ -174,7 +174,6 @@ git history and this spec). | `src/zndraw/broker.py` | Simplify — executor only needs `base_url` | | `src/zndraw_joblib/registry.py` | Add `token: str` to task function signature, pass to executor | | `src/zndraw_joblib/router.py` | Add `WorkerTokenDep`, pass token to `kiq()` | -| `src/zndraw/routes/auth.py` | Add login/register guards for internal worker email | +| `src/zndraw_joblib/dependencies.py` | Real `get_worker_token` dependency using `Request.app.state` | | `docker/templates/.env` | Remove `ZNDRAW_SERVER_WORKER_PASSWORD` | -| `docker/*/README.md` | Remove worker password from config tables | -| `tests/` | Update env vars, add login/register block tests | +| `tests/` | Update env vars, add worker token integration test |