Skip to content

Add offline replay and staged DSA workflows - #5

Merged
zhouaihui merged 1 commit into
mainfrom
feat/offline-dsa-workflows
Nov 6, 2026
Merged

zhouaihui merged 1 commit into
mainfrom
feat/offline-dsa-workflows

Conversation

@zhouaihui

Copy link
Copy Markdown
Collaborator

Outcome

Add runnable Qwen3-8B workflows for offline feature generation/replay and dense→indexer→sparse DSA training, including evaluation, recovery and native export. Include example documentation in source distributions.

Fix offline cleanup to release the placement group handle after source teardown succeeds. Passing the placement metadata tuple to Ray previously left cleanup incomplete.

Verification

  • 149 startup-cleanup, offline replay, DSA and planning tests passed; lint, shell syntax, documentation links and wheel/sdist audits passed in isolated Linux environments.
  • GPU functional acceptance covered materialization, offline training/resume, the three DSA stages, stage-local recovery and strict native loading of both exports.
  • Exact resume restores saved state and committed source work. Asynchronous packing can regroup uncommitted samples, so later document counts and numerical trajectories need not match uninterrupted training. Quality, throughput and downstream serving are outside this validation.

Compatibility and provenance

The examples compose existing runtime and stage configurations. No checkpoint or dataset format changes. The TorchSpec-derived offline implementation retains its source header, pinned revision and MIT attribution; source and license records remain in the repository.

Provide offline feature generation and replay configs, plus a dense-to-indexer-to-sparse launcher with recovery and native export guides. Include example documentation in source distributions.

Release the offline placement group handle after source cleanup and cover cleanup ordering and failure retention.
@zhouaihui
zhouaihui merged commit 35e9abc into main Nov 6, 2026
2 checks passed
@zhouaihui
zhouaihui deleted the feat/offline-dsa-workflows branch November 6, 2026 17:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant