Skip to content

Add GraND and GREAT batch selection strategies#32

Merged
jainarchita merged 7 commits intomainfrom
batching_methods
Feb 26, 2026
Merged

Add GraND and GREAT batch selection strategies#32
jainarchita merged 7 commits intomainfrom
batching_methods

Conversation

@jainarchita
Copy link
Copy Markdown
Collaborator

Summary

  • Implement GraND (Gradient Normed Distance) batch selection strategy that selects training samples based on gradient norm magnitudes, combining exploration (random sampling) with exploitation (high gradient norm samples)
  • Implement GREAT (GREedy Approximation Taylor) batch selection strategy that greedily selects samples with maximum orthogonal gradient contribution to maximize expected loss reduction
  • Unify the batch sampler interface so all strategies (including existing ones) use a consistent **kwargs pattern for passing model, loss function, device, and loss history

Changes

  • New files: trainer/batching/vision_batching/grand.py, trainer/batching/vision_batching/great.py
  • Modified: trainer/pipelines/vision/vision.py : unified batch sampler call interface, added GPU support and device info logging
  • Modified: trainer/constants.py : increased default runs to 5, added automatic CUDA device detection
  • Modified: trainer/constants_batch_strategy.py : registered GraND and GREAT strategies
  • Both strategies use a candidate pool approach (similar to MILO/CORESET) for computational efficiency, with a reduced pool size (500) for GREAT given its higher per-sample cost

@jainarchita jainarchita merged commit 9f2a5f3 into main Feb 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant