[Feature] Adds HERReplayBuffer and HindsightStrategy to torchrl.data.

[theap06]

Description

Describe your changes in detail.

Motivation and Context

Fixes #3713

I have raised an issue to propose this change (required for new features and bug fixes)

##Summary

• Adds HERReplayBuffer — a TensorDictReplayBuffer subclass that applies goal relabeling at sample time, turning failed goal-conditioned trajectories into useful training signal (Andrychowicz et al., NeurIPS 2017)
• Adds HindsightStrategy enum with all four canonical strategies: FUTURE (recommended), FINAL, EPISODE, RANDOM
• Exports both symbols from torchrl.data and torchrl.data.replay_buffers

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3734

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

• Run pull request jobs on OSDC runners in shadow mode

⏳ No Failures, 16 Pending

As of commit b65a86d with merge base 0bde4b4 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

⚠️ PR Title Label Error

PR title must start with a label prefix in brackets (e.g., [BugFix]).

Current title: Adds HERReplayBuffer and HindsightStrategy to torchrl.data.

Supported Prefixes (case-sensitive)

Your PR title must start with exactly one of these prefixes:

Prefix	Label Applied	Example
[BugFix]	BugFix	[BugFix] Fix memory leak in collector
[Feature]	Feature	[Feature] Add new optimizer
[Doc] or [Docs]	Documentation	[Doc] Update installation guide
[Refactor]	Refactoring	[Refactor] Clean up module imports
[CI]	CI	[CI] Fix workflow permissions
[Test] or [Tests]	Tests	[Tests] Add unit tests for buffer
[Environment] or [Environments]	Environments	[Environments] Add Gymnasium support
[Data]	Data	[Data] Fix replay buffer sampling
[Performance] or [Perf]	Performance	[Performance] Optimize tensor ops
[BC-Breaking]	bc breaking	[BC-Breaking] Remove deprecated API
[Deprecation]	Deprecation	[Deprecation] Mark old function
[Quality]	Quality	[Quality] Fix typos and add codespell

Note: Common variations like singular/plural are supported (e.g., [Doc] or [Docs]).

[theap06]

@vmoens Results utilizing HER. Wrote up a quick script for DDPG

[vmoens]

Amazing, I love this.
I left a couple of high level comments can you have a look?

[vmoens]

Good progress, left a few more comments here

[vmoens]

I appreciate you going back and forth with me on this. Good calls, LGTM!

[vmoens]

This needs a conflict resolution