Add --test HF CLI path for 2-layer random model configs, olive run and ModelBuilder support, Qwen how-to/layer-types fix, and merge conflict resolution

[Copilot]

Describe your changes

Adds a CLI test path for Hugging Face models so generated config.json can carry a lightweight random-model definition instead of always using pretrained weights. When --test is passed, Olive now preserves the source architecture, instantiates a random model with 2 hidden layers, and can persist that test model for reuse.

• CLI/config support

  • Added --test to HF-backed CLI commands using shared input-model options.
  • --test now accepts an optional folder path where the generated test model is saved for reuse.
  • Emitted input_model.test_model_config into generated run configs, and now also emits input_model.test_model_path when a save folder is provided or derived.

    {
      "input_model": {
        "type": "HfModel",
        "model_path": "model-id",
        "test_model_config": { "hidden_layers": 2 },
        "test_model_path": "path/to/test_model"
      }
    }

  • When --test is used without an explicit folder, Olive uses <output_path>/test_model.
  • If --test is used in a context where no output path is available, Olive now fails clearly instead of silently skipping persistence.

• olive run support

  • Extended olive run --test so it can apply the same lightweight HF test-model override to an existing Hugging Face input_model already present in a workflow config.
  • When olive run --test is used without an explicit folder, it derives the saved test-model location from the effective workflow output path.
  • olive run --test now fails clearly when the workflow config does not contain a Hugging Face input_model.

• HF model loading

  • Extended HF config loading to accept test_model_config.
  • Derived a lightweight config from the original model config by overriding the architecture-specific hidden-layer field (num_hidden_layers, num_layers, n_layer, n_layers).
  • For Qwen-style configs that carry per-layer metadata, Olive now also trims layer_types to match the reduced hidden-layer count so the saved reduced config remains valid when reloaded.
  • Switched test-model loading to instantiate from config (from_config) so the model is random-initialized rather than loaded from pretrained weights.
  • Updated the test-model path to fail fast if the selected model class cannot be instantiated from the reduced config, instead of falling back to another candidate class that could produce a misleading larger model.
  • Refactored the from_config loading path to avoid nested try/except handling by only passing trust_remote_code when the model class signature supports it.
  • Added persistence/reuse support for test models: if test_model_path already contains a saved HF model, Olive loads that model instead of recreating it; otherwise it creates the reduced model once and saves it there.

• ModelBuilder support

  • Updated the ModelBuilder pass so --test workflows export from the saved reduced Hugging Face test checkpoint instead of still using the original full checkpoint.
  • When test_model_config is present, ModelBuilder now materializes or reuses test_model_path before export and passes that saved checkpoint to the builder.
  • This fixes the smoke-test flow so it avoids the original full-model dtype path instead of only deferring the same failure.

• IO config / dummy input propagation

  • Threaded test_model_config through HF IO-config and dummy-input generation so the reduced-layer model shape metadata stays consistent with the generated test model.

• Documentation

  • Added a new how-to page showing how to convert a Qwen LLM with a quick --test smoke check first, then rerun the full conversion.
  • Linked the new how-to page from the docs How Tos index.
  • Updated the how-to flow to use olive optimize --dry_run followed by olive run --test, matching the new CLI support.
  • Clarified the smoke-test commands so the generated ONNX artifacts are written to a dedicated output folder and are easy to find.
  • Updated the example model from Phi to Qwen/Qwen3-0.6B and renamed the how-to page and index entry to match.

• Merge conflict resolution

  • Merged origin/main into this PR branch and resolved the conflict in test/passes/onnx/test_model_builder.py.
  • Preserved both the upstream ModelBuilder fallback/multi-file output test coverage and this PR's saved test-model-path coverage.
  • Updated the upstream mock-based ModelBuilder tests to define the new Hugging Face test-model attributes used by this PR (test_model_config and test_model_path).

• Targeted coverage

  • Added focused tests for:
    • CLI config generation with --test
    • input-model config serialization of test_model_config
    • input-model config serialization of test_model_path
    • validation when --test needs an explicit folder
    • olive run --test overriding an existing HF input_model from a workflow config
    • validation when olive run --test is used on a non-HF workflow config
    • HF random-model instantiation for multiple config naming conventions
    • fail-fast behavior when test-model instantiation cannot use the expected model class
    • conditional trust_remote_code handling for supported, omitted, and unsupported from_config signatures
    • saving and reusing a persisted HF test model
    • ModelBuilder exporting from the saved reduced test-model checkpoint when test_model_config is active
    • a CLI smoke-flow test that follows the documented olive optimize --dry_run then olive run --test commands with hf-internal-testing/tiny-random-LlamaForCausalLM and verifies an ONNX artifact is produced
    • a Qwen3 regression test that verifies reduced test-model configs keep layer_types aligned with the reduced hidden-layer count and can be reloaded successfully
    • conflict-resolved ModelBuilder tests covering saved test-model reuse, single-file annotation fallback, and multi-file output component naming

Examples:

olive optimize \
  -m Qwen/Qwen3-0.6B \
  --test out/test_model \
  --dry_run \
  -o out

olive run \
  --config out/config.json \
  --test out/test_model \
  --output_path out/qwen-smoke-run

Checklist before requesting a review

• Add unit tests for this change.
• Make sure all tests can pass.
• Update documents if necessary.
• Lint and apply fixes to your code by running lintrunner -a
• Is this a user-facing change? If yes, give a description of this change to be included in the release notes.

Release notes: Added a --test option for Hugging Face CLI workflows that can take a folder path, writes a lightweight 2-layer random-model config, saves and reuses the generated HF test model from that folder, fails fast if the reduced test model cannot be instantiated from the expected model class, only passes trust_remote_code when the target from_config supports it, and now keeps Qwen-style layer_types metadata aligned with reduced test-model layer counts so saved reduced configs reload cleanly. Also added olive run --test support for workflow configs with Hugging Face input models, updated ModelBuilder to export from the saved reduced test checkpoint for --test flows, and added a Qwen3 0.6B how-to page for running a quick smoke test before the full conversion with an explicit output path for the generated ONNX files.

(Optional) Issue link

[Copilot]

Just as a heads up, I was blocked by some firewall rules while working on your feedback. Expand below for details.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• huggingface.co
  • Triggering command: /usr/bin/python python - (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

[Copilot]

Just as a heads up, I was blocked by some firewall rules while working on your feedback. Expand below for details.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• huggingface.co
  • Triggering command: /home/REDACTED/work/_temp/ghcca-node/node/bin/node /home/REDACTED/work/_temp/ghcca-node/node/bin/node --enable-source-maps /home/REDACTED/work/_temp/copilot-developer-action-main/dist/index.js (dns block)
  • Triggering command: /usr/bin/python python - (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

[xadupre]

@copilot resolve the merge conflicts in this pull request

[Copilot]

@copilot resolve the merge conflicts in this pull request

Resolved in 5daba5d. I merged origin/main, kept both the upstream ModelBuilder test additions and this PR's saved-test-model coverage, and finalized it as a true two-parent merge commit.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• huggingface.co
  • Triggering command: /usr/bin/python python -m pytest test/passes/onnx/test_model_builder.py (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

[Copilot]

Pull request overview

This PR adds a --test mode for Hugging Face-backed Olive CLI workflows and olive run, enabling generation and reuse of a lightweight random-initialized 2-layer “test model” (including persistence via test_model_path) to support fast smoke checks before running full conversions. It also threads test_model_config through HF config/IO/dummy-input generation, updates ModelBuilder to export from the saved reduced checkpoint when present, and adds a Qwen-focused how-to plus targeted tests.

Changes:

• Add CLI + config support for --test to emit input_model.test_model_config and input_model.test_model_path, including olive run --test override behavior.
• Extend HF loading utilities to apply reduced-layer config overrides, instantiate from from_config, and optionally persist/reuse a saved test model directory.
• Update ModelBuilder to use the saved test-model checkpoint when test_model_config is active; add tests and a new Qwen smoke-test how-to doc.

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
test/passes/onnx/test_model_builder.py	Adds coverage ensuring ModelBuilder uses a saved test_model_path when test_model_config is set.
test/common/test_hf.py	Adds unit tests for reduced-config application, trust_remote_code handling, persistence/reuse, and fail-fast behavior.
test/cli/test_cli.py	Adds CLI workflow tests for olive run --test overrides and optimize→run smoke flow.
test/cli/test_base.py	Adds tests for get_input_model_config emitting test_model_config/test_model_path and validation when output path isn’t available.
olive/passes/onnx/model_builder.py	Switches ModelBuilder input to test_model_path when test_model_config is present (materializing if needed).
olive/model/handler/mixin/hf.py	Threads test_model_config into HF config/IO/dummy-input helpers.
olive/model/handler/hf.py	Adds test_model_path resource + test_model_config JSON config plumbing and passes them into HF loader.
olive/common/hf/utils.py	Implements reduced-layer config application + random init via from_config, plus optional persistence/reuse.
olive/common/hf/model_io.py	Adds test_model_config parameter propagation into config-based IO/dummy-input generation.
olive/cli/run.py	Enables olive run --test to inject HF test-model settings into an existing HF input_model in a run config.
olive/cli/base.py	Adds shared helper and CLI option wiring for --test on HF input model options.
docs/source/how-to/index.md	Links the new Qwen --test smoke-check how-to from the How Tos index.
docs/source/how-to/cli/cli-convert-qwen-test.md	New how-to describing a Qwen smoke check using olive optimize --dry_run followed by olive run --test.