Skip to content

Conversation

@DragonFive
Copy link
Collaborator

@DragonFive DragonFive commented Dec 18, 2025

This PR is a follow-up to #565 and continues the generative recommendation work by introducing the runtime rec worker
layer for the OneRec execution path.

What’s included

  • Runtime worker implementation for rec requests:
    • RecWorkerImpl (abstract base): shared model init + rec input preparation via
      Batch::prepare_rec_forward_input(...)
    • OneRecWorkerImpl (concrete): executes the OneRec prefill flow
  • WorkerType::REC wiring: Worker now instantiates OneRecWorkerImpl (removes the temporary LLMWorkerImpl fallback)
  • Two-stage first-prefill execution:
    • encoder forward (from rec_params.encoder_sparse_embedding or rec_params.encoder_token_ids)
    • decoder forward
    • subsequent steps run decoder-only
  • Driver-only sampling/output: non-driver ranks return std::nullopt (same contract as the LLM path)
  • Tunable input-builder thread pool via XLLM_REC_INPUT_BUILDER_THREADS (driver only)

Out of scope (planned follow-ups)

  • Input/output processors (request ingestion and result emission)
  • Concrete kLlmRec implementation (enum value reserved only)
  • ValidPathFilter / rec decode constraint integration for the runtime worker (if needed)
  • OneRec model implementation + ATB graph building/integration
  • Pure NPU inference path

Roadmap checklist

@DragonFive
Copy link
Collaborator Author

DragonFive commented Dec 18, 2025

I will rebase this pr after #565 and #557 merged.

@DragonFive DragonFive changed the title feat: add onerec worker impl for rec framework[5/6]. feat: add onerec worker impl for rec framework[5/8]. Dec 18, 2025
@DragonFive DragonFive force-pushed the feature/add_rec_worker branch 2 times, most recently from 3485946 to 5f30825 Compare December 19, 2025 08:37
@DragonFive DragonFive force-pushed the feature/add_rec_worker branch 4 times, most recently from c5a76f0 to d0e2673 Compare December 23, 2025 02:29
@DragonFive DragonFive force-pushed the feature/add_rec_worker branch from d0e2673 to 19449dd Compare December 23, 2025 09:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant