Skip to content

[fix] compile state with instance-level forward wrapping#12

Merged
cennn merged 1 commit intoSandAI-org:mainfrom
songh11:instance_compile
Mar 31, 2026
Merged

[fix] compile state with instance-level forward wrapping#12
cennn merged 1 commit intoSandAI-org:mainfrom
songh11:instance_compile

Conversation

@songh11
Copy link
Copy Markdown
Contributor

@songh11 songh11 commented Mar 31, 2026

🗂️ PR Category

  • ✨ New Feature
  • 🚀 Optimization (performance, memory, etc.)
  • 💥 Breaking Change
  • 🐛 Bug Fix
  • 🛠️ Development / Refactoring
  • 📚 Documentation
  • 🧹 Chore (Dependencies, CI/CD, Configuration, etc.)
  • 🧪 Testing

📝 Description

  • Align MagiCompileState with instance-level wrapping semantics

    • When compiling nn.Module instances, prefer _magi_original_forward (if present) as the target callable, ensuring the compiled graph reflects the original, pre-wrapped forward.
    • In JIT dispatch, temporarily rebind the instance’s forward to the class-level forward bound method while swapping bytecode, then restore the original binding after execution, so hooks/FSDP logic still flow through __call__ while keeping instance-level overrides consistent.
  • Refine instance-level compile orchestration path

    • In _magi_compile_instance, store module._magi_original_forward and change the orchestration entry from calling the unbound forward directly to going through module.__class__.__call__, preserving module __call__ behavior and hooks for instance-compiled models.
  • Add CUDA timing sanity test for multiple Magi entrypoints

    • Introduce a lightweight CUDA test that compares execution time across class, function, instance, and method @magi_compile entrypoints on the same SimpleModel, asserting their compiled timings stay within a small factor of each other to catch regressions or inconsistent fast-path behavior.

@cennn
Copy link
Copy Markdown
Collaborator

cennn commented Mar 31, 2026

Great catch! — clean fix for the instance-level compilation bypass. On our end, we'll be adding perf regression tests for all compile entry points, and exploring whether instance-level dispatch can avoid touching __class__.forward altogether to keep it fully isolated and multi-instance safe.

@cennn cennn merged commit c93a99d into SandAI-org:main Mar 31, 2026
2 of 8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants