feat: Adding quantization aware training support with Model-Optimizer

[mxinO]

What does this PR do ?

Add quantization aware (QA) training, e.g. QA-GRPO, QA-On policy distillation.

Usage

Run a QARL by enabling simulated quantization with modelopt, adding the following config,

++policy.generation.quant_cfg="NVFP4_DEFAULT_CFG" \
++policy.quant_cfg="NVFP4_DEFAULT_CFG" \

Other quantization formats are available, but currently only tensor-wise quantization format are supported.

Before your PR is "Ready for review"

Pre checks:

• Make sure you read and followed Contributor guidelines
• Did you write any new necessary tests?
• Did you run the unit tests and functional tests locally? Visit our Testing Guide for how to run tests
• Did you add or update any necessary documentation? Visit our Document Development Guide for how to write, build and test the docs.

Additional Information

• ...

[github-actions]

⚠️ File Consistency Check

Check based on commit: 4f14851 (PR #1756 from mxin/qarl-3)

⚠️ DTensor Policy Worker Synchronization Warning

The file nemo_rl/models/policy/workers/dtensor_policy_worker_v2.py was modified in this PR, but nemo_rl/models/policy/workers/dtensor_policy_worker.py was not updated.

Why this matters:
These files contain related DTensor policy worker implementations that should be kept synchronized to ensure consistency across different versions.

Action required:

• Please review if the changes in nemo_rl/models/policy/workers/dtensor_policy_worker_v2.py should also be applied to nemo_rl/models/policy/workers/dtensor_policy_worker.py
• Update nemo_rl/models/policy/workers/dtensor_policy_worker.py if necessary to maintain consistency
• If the files are intentionally different, please add a comment in the PR explaining why

Files to check:

• Modified: nemo_rl/models/policy/workers/dtensor_policy_worker_v2.py
• Not modified: nemo_rl/models/policy/workers/dtensor_policy_worker.py

---

_{This check ensures that related file implementations remain synchronized across the codebase. If you believe this warning is incorrect or the files should intentionally differ, please add a comment explaining the reasoning.}

[realAsma]

@mxinO Is it possible to consolidate all ModelOpt related modifications to one folder (following the patch based integration ModelOpt adopted For MCore, Huggingface etc..)?