feat(perf): cache per-module builds and honor --rebuild/--ignore-cache#1004
Open
xieofxie wants to merge 1 commit into
Open
feat(perf): cache per-module builds and honor --rebuild/--ignore-cache#1004xieofxie wants to merge 1 commit into
xieofxie wants to merge 1 commit into
Conversation
Per-module benchmarking built every submodule into a throwaway tempfile.TemporaryDirectory with no cache_key and no rebuild flag, so artifacts were rebuilt on every run and --rebuild/--ignore-cache were silently ignored. The single-model path already honored these flags via WinMLAutoModel. Mirror auto.py's cache logic in _perf_modules: build into the model's persistent cache dir keyed per submodule (cfg.generate_cache_key() folds in loader.module_path and traced I/O shapes, so sibling instances get distinct keys and coexist in one model dir), and forward rebuild. With --ignore-cache, fall back to a temp dir + forced rebuild. Closes #918
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
winml perf --modulebuilt every submodule into a throwawaytempfile.TemporaryDirectorywith nocache_keyand norebuildflag, so per-module artifacts were rebuilt on every run and--rebuild/--ignore-cachewere silently ignored. The single-model path already honored these flags viaWinMLAutoModel→auto.py.This mirrors
auto.py's cache logic in_perf_modules:~/.cache/winml/artifacts/<model_slug>/) and is reused on the next run.--rebuild: reuses the cache dir but overwrites artifacts.--ignore-cache: falls back to a temp dir + forced rebuild (prior behavior, now opt-in).Why per-module caching is collision-free
Each submodule config carries a unique
loader.module_path(plus its traced I/O shapes).cfg.generate_cache_key()hashes the full config, so every instance gets a distinct key — even shape-identical siblings — andbuild_hf_model'scache_keyfilename prefixing keeps each instance'smodel.onnxseparate within one shared model dir.Behavior change worth noting
Per-module artifacts now persist by default (more disk use), which is the intended consequence of matching the single-model /
auto.pybehavior.Tests
Added
TestPerfModuleCache(default → persistent dir +rebuild=False;--rebuild→rebuild=Truein cache dir;--ignore-cache→ temp dir +rebuild=True). Fulltests/unit/commands/test_perf_module.py: 15 passed.Closes #918
🤖 Generated with Claude Code