Skip to content

[plugin][full oot test] change the trigger behavior to nightly + manually#388

Open
zejunchen-zejun wants to merge 1 commit intomainfrom
zejun/enable_nightly_for_full_test
Open

[plugin][full oot test] change the trigger behavior to nightly + manually#388
zejunchen-zejun wants to merge 1 commit intomainfrom
zejun/enable_nightly_for_full_test

Conversation

@zejunchen-zejun
Copy link
Contributor

@zejunchen-zejun zejunchen-zejun commented Mar 23, 2026

We need the OOT full test to cover the accuracy check for OOT mode. It should support

  • nightly 02:00 AM trigger

Then we add below models into nightly accuracy check

  • DeepSeek FP4 TP8
  • Kimi TP8
  • Qwen3.5-397B-A17B-FP8 TP8
  • GLM5 TP8

1. nightly 9:00 PM
2. manually trigger

Signed-off-by: zejunchen-zejun <zejun.chen@amd.com>
Copilot AI review requested due to automatic review settings March 23, 2026 07:31
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a nightly scheduled trigger to the existing ATOM vLLM OOT full validation GitHub Actions workflow so OOT accuracy coverage runs automatically, while keeping the manual trigger for on-demand runs.

Changes:

  • Add a schedule trigger to run nightly at 21:00 Beijing time (13:00 UTC).
  • Retain the existing workflow_dispatch manual trigger with inputs.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@zejunchen-zejun
Copy link
Contributor Author

Hi, @valarLip @gyohuangxin @wuhuikx
I change the FULL OOT model test(accuracy only) to be nightly triggered at 9:00 PM Beijing. Does it look good to you?
We badly need accuracy check for OOT mode. I think it can be dispatched to MI350 machines because it only tests acc.
Thank you.

@valarLip
Copy link
Collaborator

Hi, @valarLip @gyohuangxin @wuhuikx I change the FULL OOT model test(accuracy only) to be nightly triggered at 9:00 PM Beijing. Does it look good to you? We badly need accuracy check for OOT mode. I think it can be dispatched to MI350 machines because it only tests acc. Thank you.

looks good, but make sure you had runner updated to 350 before merge

@zejunchen-zejun
Copy link
Contributor Author

Hi, @valarLip @gyohuangxin @wuhuikx I change the FULL OOT model test(accuracy only) to be nightly triggered at 9:00 PM Beijing. Does it look good to you? We badly need accuracy check for OOT mode. I think it can be dispatched to MI350 machines because it only tests acc. Thank you.

looks good, but make sure you had runner updated to 350 before merge

yes, we need to establish 350 as runner and make full oot test choose this 350 runner pool

@wuhuikx
Copy link
Contributor

wuhuikx commented Mar 23, 2026

Hi, @valarLip @gyohuangxin @wuhuikx I change the FULL OOT model test(accuracy only) to be nightly triggered at 9:00 PM Beijing. Does it look good to you? We badly need accuracy check for OOT mode. I think it can be dispatched to MI350 machines because it only tests acc. Thank you.

looks good, but make sure you had runner updated to 350 before merge

yes, we need to establish 350 as runner and make full oot test choose this 350 runner pool

MI350 or AAC 355 (nightly). MI350 is better. We really need your help @gyohuangxin If we don't have accuracy check, we need to put precious human resource for manually test and triage.

cc @sunway513 @ChuanLi1101

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants