Skip to content

Add step-aware LTDETR scale jitter control#675

Merged
yutong-xiang-97 merged 7 commits intomainfrom
yutong-trn-1873-update-scalejitter-augmentation-with-step-based-control
Mar 30, 2026
Merged

Add step-aware LTDETR scale jitter control#675
yutong-xiang-97 merged 7 commits intomainfrom
yutong-trn-1873-update-scalejitter-augmentation-with-step-based-control

Conversation

@yutong-xiang-97
Copy link
Copy Markdown
Contributor

@yutong-xiang-97 yutong-xiang-97 commented Mar 29, 2026

What has been changed and why?

Enable step-based augmentation control for LTDETR object detection with the following methods:

"Keep all LTDETR augmentations inside the regular transform and collate path, and
propagate the current training step by updating the main-process transform/collate
objects before each batch. When a worker-visible schedule change crosses an augmentation boundary, restart the dataloader iterator so the worker processes receive fresh copies of the transform and collate objects with the new step value."

As a start, this PR applies this approach to the existing ScaleJitter augmentation. We use a stop_step to control where this augmentations should be stopped. It is achieved by:

  1. In the respective ScaleJitterArgs, making LTDETR scale_jitter stop-aware via stop_step
  2. adding shared step-control hooks to collate functions
  • set_step: sets the current optimizer step
  • uses_step_dependent_worker_state determine whether the step-based control is enabled
  • requires_dataloader_reinitialization flags if dataloader reinit is needed
  1. Rewrite the above hooks in ObjectDetectionCollateFunction to determine if dataloader reset is needed.
  2. adding InfiniteCycleIterator.reset() and restart dataloader iteration when step-dependent worker state changes
  3. resetting the dataloader when stop_step is reached in the main loop.

Note that:

  1. We force persistent_workers=False for step-aware train dataloaders even if the users specify True and emit both a warning and log message when overriding user input.
  2. We don't reset cycles in the InfiniteCycleIterator.
  3. The hooks are also added to TaskTransform as a placeholder for MosaicTransform.
  4. Resize should be always on and applied before collate to match the behavior with original LTDETR.

How has it been tested?

  • Unit tests.
  • Manually.

Did you update CHANGELOG.md?

  • Yes
  • Not needed (internal change)

Did you update the documentation?

  • Yes
  • Not needed (internal change without effects for user)

Copilot AI review requested due to automatic review settings March 29, 2026 18:30
@chatgpt-codex-connector
Copy link
Copy Markdown

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces step-aware hooks for transforms/collate functions to enable step-windowed augmentations (notably LTDETR scale jitter) and safely refresh DataLoader workers when step-dependent worker state changes during training.

Changes:

  • Add step-control hooks (set_step, uses_step_dependent_worker_state, requires_dataloader_reinitialization) for task transforms and collate functions, and propagate the training step into them.
  • Add InfiniteCycleIterator.reset() and restart the DataLoader iterator mid-training when a step boundary requires refreshing worker state.
  • Make LTDETR scale_jitter stop-aware via stop_step, gate collate-time application by step, and force persistent_workers=False for step-aware training dataloaders (with warning/log).

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
tests/_transforms/test_object_detection_transform.py Adds coverage for step-windowed scale-jitter behavior triggering dataloader reinit.
tests/_data/test_infinite_cycle_iterator.py Adds reset-behavior test for InfiniteCycleIterator.
src/lightly_train/_transforms/task_transform.py Introduces step-aware hook methods on base transform/collate interfaces.
src/lightly_train/_transforms/object_detection_transform.py Implements step-gated scale jitter in the object-detection collate function plus reinit detection.
src/lightly_train/_task_models/dinov3_ltdetr_object_detection/transforms.py Adds stop_step to scale-jitter args and types the model’s scale_jitter accordingly.
src/lightly_train/_task_models/dinov2_ltdetr_object_detection/transforms.py Same as above for DINOv2 LTDETR.
src/lightly_train/_data/infinite_cycle_iterator.py Adds reset() to restart iteration over the wrapped iterable.
src/lightly_train/_commands/train_task_helpers.py Detects step-aware worker state and forces persistent_workers=False (with warning/log).
src/lightly_train/_commands/train_task.py Propagates step into transform/collate each step and resets the infinite dataloader iterator when needed.

Comment thread src/lightly_train/_commands/train_task.py
Comment thread src/lightly_train/_commands/train_task_helpers.py
@yutong-xiang-97 yutong-xiang-97 changed the title Add step-aware LTDETR scale jitter control and dataloader worker refresh Add step-aware LTDETR scale jitter control Mar 29, 2026
@yutong-xiang-97 yutong-xiang-97 force-pushed the yutong-trn-1873-update-scalejitter-augmentation-with-step-based-control branch from 3aa3b2c to 9aca65a Compare March 30, 2026 08:41
@yutong-xiang-97
Copy link
Copy Markdown
Contributor Author

/review

Comment thread src/lightly_train/_commands/train_task.py Outdated
Comment thread src/lightly_train/_commands/train_task.py Outdated
Comment thread src/lightly_train/_commands/train_task.py Outdated
Comment thread src/lightly_train/_commands/train_task_helpers.py Outdated
Comment thread src/lightly_train/_data/infinite_cycle_iterator.py Outdated
Comment thread src/lightly_train/_transforms/object_detection_transform.py
Comment thread src/lightly_train/_task_models/dinov3_ltdetr_object_detection/transforms.py Outdated
Comment thread src/lightly_train/_transforms/task_transform.py
Comment thread src/lightly_train/_task_models/dinov2_ltdetr_object_detection/transforms.py Outdated
@yutong-xiang-97 yutong-xiang-97 force-pushed the yutong-trn-1873-update-scalejitter-augmentation-with-step-based-control branch from 0eca2ca to f08694a Compare March 30, 2026 14:10
Comment thread src/lightly_train/_commands/train_task.py
Comment thread src/lightly_train/_task_models/dinov3_ltdetr_object_detection/transforms.py Outdated
Comment thread src/lightly_train/_transforms/object_detection_transform.py
@yutong-xiang-97 yutong-xiang-97 force-pushed the yutong-trn-1873-update-scalejitter-augmentation-with-step-based-control branch from 3f8bcd5 to e06df53 Compare March 30, 2026 15:37
@yutong-xiang-97 yutong-xiang-97 enabled auto-merge (squash) March 30, 2026 15:38
@yutong-xiang-97 yutong-xiang-97 merged commit 22be9af into main Mar 30, 2026
16 checks passed
@yutong-xiang-97 yutong-xiang-97 deleted the yutong-trn-1873-update-scalejitter-augmentation-with-step-based-control branch March 30, 2026 15:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants