[Test] Add dedicated perf test suite with entrypoint matrix#15
Open
cennn wants to merge 5 commits intoSandAI-org:mainfrom
Open
[Test] Add dedicated perf test suite with entrypoint matrix#15cennn wants to merge 5 commits intoSandAI-org:mainfrom
cennn wants to merge 5 commits intoSandAI-org:mainfrom
Conversation
Split perf benchmarks into tests/perf_tests with shared benchmarking helpers and add class/instance/function/method coverage plus torch-compile mode checks across MLP, norm-residual fusion, and pointwise chains. Made-with: Cursor
Document the known magi vs torch.compile gap in fusion-heavy perf suites so follow-up optimization work has explicit tracking context. Made-with: Cursor
Apply black-driven formatting updates for perf benchmark utilities and perf test files so repository hooks pass consistently in local and CI workflows. Made-with: Cursor
cennn
changed the title
Lower MLP, norm-residual, and pointwise speedup gates to reflect observed CI variance while preserving meaningful eager-baseline improvements across entrypoints. Made-with: Cursor
jiahy0825
reviewed
Apr 1, 2026
Move the repeated perf speedup assertion helper into tests/perf_tests/utils.py and reuse it across MLP, norm-residual fusion, and pointwise perf tests to reduce duplication and keep threshold checks consistent. Made-with: Cursor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🗂️ PR Category
📝 Description
Add a dedicated
tests/perf_testssuite and move performance benchmarking helpers there.This PR adds end-to-end perf coverage for MLP, norm+residual fusion, and pointwise fusion across class/instance/function/method entrypoints (including instance + TORCH_COMPILE mode), compares results against eager and raw
torch.compilebaselines, and documents the known fusion-gap follow-up withTODO(perf-fusion-gap).📊 Perf Snapshot (current run)
Measured with
pytest -q tests/perf_tests -son current branch.Notes
TORCH_COMPILEmode.TODO(perf-fusion-gap)is added for fusion-heavy workloads wheremagi_compilestill trails rawtorch.compile.