mup: refinement pass on the merged plugin#17
Merged
Conversation
_coord_check/_mutransfer/_scaling now return (label, url_or_reason, ok) instead of appending into caller-owned produced/skipped accumulators; run_suite (renamed from _suite) collects outcomes into produced/skipped itself. _val_verb renamed to run_val to match: both are real verb bodies, not helpers. _curves folds into build_report, its only caller. Suite outcome ordering and content are unchanged.
fit_predictor was fitting L_inf + A*w^(-alpha) to as few as 3 width points, then publishing the fit and a "predicted" loss even when the fit was degenerate (alpha near 0, R2 meaningless, prediction circular on a handful of points). Drop the fit entirely: the scaling page now shows only the measured loss(w) points connected by a line. bigvit_width / --big-array only existed to feed the now-removed prediction, so it drops from scaling_report, the suite verb, and the CLI along with it.
…ages
None of the report pages named which dataset the runs they show were
trained on. Add TRAIN_DATASET_LABEL (the random-100k proxy list,
prune10m_random100k_seed0.txt) to spec.py and fold it into each page's
conditions sentence. mutransfer and scaling only ever read from the
vit_mup_w{width} sweep configs, which are hardcoded to this dataset, so
the clause there is a fixed fact. The val page evaluates arbitrary
checkpoints (config_registry.py already carries landed-but-unfired
configs for the random-1M and full-10M checkpoints), so its clause is
phrased as "runs to date" and points at the run label, which already
encodes each checkpoint's own training dataset via its directory stem.
coord_check trains on synthetic random tensors, not any dataset list --
its existing "random input" wording already states this truthfully, so
it is left unchanged rather than getting a misleading dataset clause.
SPECS declared convnext ready=True (the dataclass default), but no convnext_mup_w* config functions exist anywhere in the tree -- the convnext muP configs are a separate, not-yet-merged PR. Any routine verb run against convnext today would pass the ready gate and then fail deep in collect() with an opaque manifest-not-found error instead of the intended "no muP configs landed yet" message. fastvit already carries ready=False for the same situation; convnext should too.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
four follow-ups to #12: suite outcomes returned not mutated, degenerate fit removed from the scaling page, training dataset named in page conditions, convnext marked not-ready until its configs land.