Monitoring improvements#425

Draft

le1nux wants to merge 26 commits intomainfrom

monitoring_improvements

Commits on Dec 13, 2025

feat: added fused and foreach options to Adam and AdamW optimizers
le1nux
committed

Commits on Dec 21, 2025

Commits on Dec 22, 2025

feat: added memory profiling to kernel profiler
le1nux
committed

Commits on Dec 29, 2025

refactor: added experiments_root_path to warmstart API and improved error handling
le1nux
committed
refactor: refactored wamstart tutorial scripts
le1nux
committed

Commits on Dec 30, 2025

Commits on Jan 4, 2026

fix: HSDP was not applied at all due to wrong condition check
le1nux
committed

Commits on Jan 6, 2026

refactor: allow data_parallel_replicate_degree to be -1 for auto-calculation
le1nux
committed

Commits on Jan 16, 2026

Commits on Jan 17, 2026

fix: tp can now be used with dp_shard or dp_replicate
le1nux
committed

Commits on Jan 25, 2026

fix: fixed merge conflict bug
le1nux
committed