Skip to content

Commit f4aa17f

Browse files
authored
Updates Newton docs on main for 3.0 beta changes (isaac-sim#4934)
# Description Updates Newton docs on main for 3.0 beta changes ## Type of change - Documentation update ## Checklist - [x] I have read and understood the [contribution guidelines](https://isaac-sim.github.io/IsaacLab/main/source/refs/contributing.html) - [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with `./isaaclab.sh --format` - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] I have updated the changelog and the corresponding version in the extension's `config/extension.toml` file - [ ] I have added my name to the `CONTRIBUTORS.md` or my name already exists there <!-- As you go through the checklist above, you can mark something as done by putting an x character in it For example, - [x] I have done this task - [ ] I have not done this task -->
1 parent 2b78d92 commit f4aa17f

5 files changed

Lines changed: 62 additions & 68 deletions

File tree

docs/source/experimental-features/newton-physics-integration/index.rst

Lines changed: 6 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,34 +1,31 @@
11
Newton Physics Integration
22
===========================
33

4-
`Newton <https://newton-physics.github.io/newton/guide/overview.html>`_ is a GPU-accelerated, extensible, and differentiable physics simulation engine designed for robotics, research,
4+
`Newton <https://newton-physics.github.io/newton/latest/guide/overview.html>`_ is a GPU-accelerated, extensible, and differentiable physics simulation engine designed for robotics, research,
55
and advanced simulation workflows. Built on top of `NVIDIA Warp <https://nvidia.github.io/warp/>`_ and integrating MuJoCo Warp, Newton provides high-performance
66
simulation, modern Python APIs, and a flexible architecture for both users and developers.
77

88
Newton is an Open Source community-driven project with contributions from NVIDIA, Google Deep Mind, and Disney Research,
99
managed through the Linux Foundation.
1010

11-
This `experimental feature branch <https://github.com/isaac-sim/IsaacLab/tree/feature/newton>`_ of Isaac Lab provides an initial integration with the Newton Physics Engine, and is
11+
This integration is available on the `develop branch <https://github.com/isaac-sim/IsaacLab/tree/develop>`_ of Isaac Lab as part of Isaac Lab 3.0 Beta, and is
1212
under active development. Many features are not yet supported, and only a limited set of classic RL and flat terrain locomotion
1313
reinforcement learning examples are included at the moment.
1414

15-
Both this Isaac Lab integration branch and Newton itself are under heavy development. We intend to support additional
15+
Both this Isaac Lab integration and Newton itself are under heavy development. We intend to support additional
1616
features for other reinforcement learning and imitation learning workflows in the future, but the above tasks should be
1717
a good lens through which to understand how Newton integration works in Isaac Lab.
1818

1919
We have validated Newton simulation against PhysX by transferring learned policies from Newton to PhysX and vice versa
2020
Furthermore, we have also successfully deployed a Newton-trained locomotion policy to a G1 robot. Please see :ref:`here <sim2real>` for more information.
2121

22-
Newton can support `multiple solvers <https://newton-physics.github.io/newton/api/newton_solvers.html>`_ for handling different types of physics simulation, but for the moment, the Isaac
22+
Newton can support `multiple solvers <https://newton-physics.github.io/newton/latest/api/newton_solvers.html>`_ for handling different types of physics simulation, but for the moment, the Isaac
2323
Lab integration focuses primarily on the MuJoCo-Warp solver.
2424

25-
Future updates of this branch and Newton should include both ongoing improvements in performance as well as integration
25+
Future updates of Isaac Lab and Newton should include both ongoing improvements in performance as well as integration
2626
with additional solvers.
2727

28-
Note that this branch does not include support for the PhysX physics engine - only Newton is supported. We are considering
29-
several possible paths to continue to support PhysX within Lab, and feedback from users about their needs around that would be appreciated.
30-
31-
During the early development phase of both Newton and this Isaac Lab integration, you are likely to encounter breaking
28+
During the development phase of both Newton and this Isaac Lab integration, you are likely to encounter breaking
3229
changes as well as limited documentation. We do not expect to be able to provide official support or debugging assistance
3330
until the framework has reached an official release. We appreciate your understanding and patience as we work to deliver a robust and polished framework!
3431

docs/source/experimental-features/newton-physics-integration/installation.rst

Lines changed: 31 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -1,62 +1,68 @@
11
Installation
22
============
33

4-
Installing the Newton physics integration branch requires three things:
4+
Installing the Newton physics integration requires three things:
55

6-
1) The ``feature/newton`` branch of Isaac Lab
7-
2) Ubuntu 22.04 or 24.04 (Windows will be supported soon)
8-
3) [Optional] Isaac sim 5.1 (Isaac Sim is not required if the Omniverse visualizer is not used)
6+
1) The ``develop`` branch of Isaac Lab
7+
2) Ubuntu 22.04 or 24.04
8+
3) [Optional] Isaac Sim 6.0 (Isaac Sim is not required if the Omniverse visualizer is not used)
99

10-
To begin, verify the version of Isaac Sim by checking the title of the window created when launching the simulation app. Alternatively, you can
11-
find more explicit version information under the ``Help -> About`` menu within the app.
12-
If your version is less than 5.1, you must first `update or reinstall Isaac Sim <https://docs.isaacsim.omniverse.nvidia.com/latest/installation/quick-install.html>`_ before
13-
you can proceed further.
10+
To begin, navigate to the root directory of your local copy of the Isaac Lab repository and open a terminal.
1411

15-
Next, navigate to the root directory of your local copy of the Isaac Lab repository and open a terminal.
16-
17-
Make sure we are on the ``feature/newton`` branch by running the following command:
12+
Make sure we are on the ``develop`` branch by running the following command:
1813

1914
.. code-block:: bash
2015
21-
git checkout feature/newton
22-
23-
Below, we provide instructions for installing Isaac Sim through pip.
16+
git checkout develop
2417
2518
26-
Pip Installation
27-
----------------
19+
Installation
20+
------------
2821

29-
We recommend using conda for managing your python environments. Conda can be downloaded and installed from `here <https://docs.conda.io/en/latest/miniconda.html>`_.
22+
We recommend using **uv** as the package manager — it is significantly faster than pip and conda.
23+
To install ``uv``, please follow the instructions `here <https://docs.astral.sh/uv/getting-started/installation/>`__.
3024

3125
If you previously already have a virtual environment for Isaac Lab, please ensure to start from a fresh environment to avoid any dependency conflicts.
3226
If you have installed earlier versions of mujoco, mujoco-warp, or newton packages through pip, we recommend first
3327
cleaning your pip cache with ``pip cache purge`` to remove any cache of earlier versions that may be conflicting with the latest.
3428

35-
Create a new conda environment:
29+
Create a new virtual environment with Python 3.12:
3630

3731
.. code-block:: bash
3832
39-
conda create -n env_isaaclab python=3.11
33+
uv venv --python 3.12 --seed env_isaaclab
4034
4135
Activate the environment:
4236

4337
.. code-block:: bash
4438
45-
conda activate env_isaaclab
39+
source env_isaaclab/bin/activate
4640
47-
Install the correct version of torch and torchvision:
41+
.. note::
42+
43+
If you are using ``pip`` directly instead of ``uv pip``, replace ``uv pip`` with ``pip`` in the commands below.
44+
45+
46+
Ensure pip is up to date:
4847

4948
.. code-block:: bash
5049
51-
pip install -U torch==2.7.0 torchvision==0.22.0 --index-url https://download.pytorch.org/whl/cu128
50+
uv pip install --upgrade pip
5251
53-
[Optional] Install Isaac Sim 5.1:
52+
[Optional] Install Isaac Sim 6.0:
5453

5554
.. code-block:: bash
5655
57-
pip install "isaacsim[all,extscache]==5.1.0" --extra-index-url https://pypi.nvidia.com
56+
uv pip install "isaacsim[all,extscache]==6.0.0" --extra-index-url https://pypi.nvidia.com
5857
59-
Install Isaac Lab extensions and dependencies:
58+
59+
Install the correct version of torch and torchvision:
60+
61+
.. code-block:: bash
62+
63+
uv pip install -U torch==2.10.0 torchvision==0.25.0 --index-url https://download.pytorch.org/whl/cu128
64+
65+
Install Isaac Lab extensions and dependencies (this includes Newton 1.0):
6066

6167
.. code-block:: bash
6268
@@ -71,8 +77,3 @@ To verify that the installation was successful, run the following command from t
7177
.. code-block:: bash
7278
7379
./isaaclab.sh -p scripts/environments/zero_agent.py --task Isaac-Cartpole-Direct-v0 --num_envs 128
74-
75-
76-
Note that since Newton requires a more recent version of Warp than Isaac Sim 5.1, there may be some incompatibility issues
77-
that could result in errors such as ``ModuleNotFoundError: No module named 'warp.sim'``. These are ok to ignore and should not
78-
impact usability.

docs/source/experimental-features/newton-physics-integration/isaaclab_newton-beta-2.rst

Lines changed: 5 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,19 @@
1-
Isaac Lab - Newton Beta 2
2-
=========================
1+
Isaac Lab 3.0 Beta
2+
==================
33

4-
Isaac Lab - Newton Beta 2 (feature/newton branch) provides Newton physics engine integration for Isaac Lab. We refactored our code so that we can not only support PhysX and Newton, but
4+
Isaac Lab 3.0 Beta (develop branch) provides Newton physics engine integration for Isaac Lab. We refactored our code so that we can not only support PhysX and Newton, but
55
any other physics engine, enabling users to bring their own physics engine to Isaac Lab if they desire. To enable this, we introduce base implementations of
66
our ``simulation interfaces``, :class:`~isaaclab.assets.articulation.Articulation` or :class:`~isaaclab.sensors.ContactSensor` for instance. These provide a
77
set of abstract methods that all physics engines must implement. In turn this allows all of the default Isaac Lab environments to work with any physics engine.
8-
This also allows us to ensure that Isaac Lab - Newton Beta 2 is backwards compatible with Isaac Lab 2.X. For engine specific calls, users could get the underlying view of
8+
This also allows us to ensure that Isaac Lab 3.0 Beta is backwards compatible with Isaac Lab 2.X. For engine specific calls, users could get the underlying view of
99
the physics engine and call the engine specific APIs directly.
1010

1111
However, as we are refactoring the code, we are also looking at ways to limit the overhead of Isaac Lab's. In an effort to minimize the overhead, we are moving
1212
all our low level code away from torch, and instead will rely heavily on warp. This will allow us to write low level code that is more efficient, and also
1313
to take advantage of the cuda-graphing. However, this means that the ``data classes`` such as :class:`~isaaclab.assets.articulation.ArticulationData` or
1414
:class:`~isaaclab.sensors.ContactSensorData` will only return warp arrays. Users will hence have to call ``wp.to_torch`` to convert them to torch tensors if they desire.
1515
Our setters/writers will support both warp arrays and torch tensors, and will use the most optimal strategy to update the warp arrays under the hood. This minimizes the
16-
amount of changes required for users to migrate to Isaac Lab - Newton Beta 2.
16+
amount of changes required for users to migrate to Isaac Lab 3.0 Beta.
1717

1818
Another new feature of the writers and setters is the ability to provide them with masks and complete data (as opposed to indices and partial data in Isaac Lab 2.X).
1919
Note that this feature will be available along with the ability to provide indices and partial data, and that the default behavior will still be to provide indices and partial data.
@@ -41,12 +41,3 @@ we are introducing cuda-graphing support for direct rl tasks. This drastically r
4141
.. code-block:: bash
4242
4343
./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/train.py --task Isaac-Humanoid-Direct-Warp-v0 --num_envs 4096 --headless
44-
45-
46-
What's Next?
47-
============
48-
49-
Isaac Lab 3.0 is the upcoming release of Isaac Lab, which will be compatible with Isaac Sim 6.0, and at the same time will support the new Newton physics engine.
50-
This will allow users to train policies on the Newton physics engine, or PhysX. To accommodate this major code refactoring are required. In this section, we
51-
will go over some of the changes, how that will affect Isaac Lab 2.X users, and how to migrate to Isaac Lab 3.0. The current branch of ``feature/newton`` gives
52-
a glance of what is to come. While the changes to the internal code structure are significant, the changes to the user API are minimal.

docs/source/experimental-features/newton-physics-integration/training-environments.rst

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,16 +7,15 @@ The currently supported tasks are as follows:
77

88
* Isaac-Cartpole-Direct-v0
99
* Isaac-Cartpole-v0
10-
* Isaac-Cartpole-RGB-Camera-Direct-v0
11-
* Isaac-Cartpole-Depth-Camera-Direct-v0
10+
* Isaac-Cartpole-Camera-Presets-Direct-v0
1211
* Isaac-Ant-Direct-v0
1312
* Isaac-Ant-v0
13+
* Isaac-Dexsuite-Kuka-Allegro-Lift-v0
1414
* Isaac-Humanoid-Direct-v0
1515
* Isaac-Humanoid-v0
1616
* Isaac-Velocity-Flat-Anymal-B-v0
1717
* Isaac-Velocity-Flat-Anymal-C-v0
1818
* Isaac-Velocity-Flat-Anymal-D-v0
19-
* Isaac-Velocity-Flat-Cassie-v0
2019
* Isaac-Velocity-Flat-G1-v0
2120
* Isaac-Velocity-Flat-G1-v1 (Sim-to-Real tested)
2221
* Isaac-Velocity-Flat-H1-v0

docs/source/experimental-features/newton-physics-integration/visualization.rst

Lines changed: 18 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -59,8 +59,8 @@ Launch visualizers from the command line with ``--visualizer``:
5959

6060
.. code-block:: bash
6161
62-
# Launch all visualizers
63-
python scripts/reinforcement_learning/rsl_rl/train.py --task Isaac-Cartpole-v0 --visualizer omniverse newton rerun
62+
# Launch all visualizers (comma-delimited list, no spaces)
63+
python scripts/reinforcement_learning/rsl_rl/train.py --task Isaac-Cartpole-v0 --visualizer kit,newton,rerun
6464
6565
# Launch just newton visualizer
6666
python scripts/reinforcement_learning/rsl_rl/train.py --task Isaac-Cartpole-v0 --visualizer newton
@@ -77,18 +77,20 @@ If ``--headless`` is given, no visualizers will be launched.
7777
Configuration
7878
~~~~~~~~~~~~~
7979

80-
Launching visualizers with the command line will use default visualizer configurations. Default configs can be found and edited in ``source/isaaclab/isaaclab/visualizers``.
80+
Launching visualizers with the command line will use default visualizer configurations. Visualizer backends live in the ``isaaclab_visualizers`` package (e.g. ``source/isaaclab_visualizers/isaaclab_visualizers/kit``, ``newton``, ``rerun``).
8181

82-
You can also configure custom visualizers in the code by defining new ``VisualizerCfg`` instances for the ``SimulationCfg``, for example:
82+
You can also configure custom visualizers in the code by defining ``VisualizerCfg`` instances for the ``SimulationCfg``, for example:
8383

8484
.. code-block:: python
8585
8686
from isaaclab.sim import SimulationCfg
87-
from isaaclab.visualizers import NewtonVisualizerCfg, OVVisualizerCfg, RerunVisualizerCfg
87+
from isaaclab_visualizers.kit import KitVisualizerCfg
88+
from isaaclab_visualizers.newton import NewtonVisualizerCfg
89+
from isaaclab_visualizers.rerun import RerunVisualizerCfg
8890
8991
sim_cfg = SimulationCfg(
9092
visualizer_cfgs=[
91-
OVVisualizerCfg(
93+
KitVisualizerCfg(
9294
viewport_name="Visualizer Viewport",
9395
create_viewport=True,
9496
dock_position="SAME",
@@ -128,9 +130,9 @@ Omniverse Visualizer
128130

129131
.. code-block:: python
130132
131-
from isaaclab.visualizers import OVVisualizerCfg
133+
from isaaclab_visualizers.kit import KitVisualizerCfg
132134
133-
visualizer_cfg = OVVisualizerCfg(
135+
visualizer_cfg = KitVisualizerCfg(
134136
# Viewport settings
135137
viewport_name="Visualizer Viewport", # Viewport window name
136138
create_viewport=True, # Create new viewport vs. use existing
@@ -176,8 +178,6 @@ Newton Visualizer
176178
- Look around
177179
* - **Mouse Scroll**
178180
- Zoom in/out
179-
* - **Space**
180-
- Pause/resume rendering (physics continues)
181181
* - **H**
182182
- Toggle UI sidebar
183183
* - **ESC**
@@ -187,7 +187,7 @@ Newton Visualizer
187187

188188
.. code-block:: python
189189
190-
from isaaclab.visualizers import NewtonVisualizerCfg
190+
from isaaclab_visualizers.newton import NewtonVisualizerCfg
191191
192192
visualizer_cfg = NewtonVisualizerCfg(
193193
# Window settings
@@ -233,12 +233,14 @@ Rerun Visualizer
233233

234234
.. code-block:: python
235235
236-
from isaaclab.visualizers import RerunVisualizerCfg
236+
from isaaclab_visualizers.rerun import RerunVisualizerCfg
237237
238238
visualizer_cfg = RerunVisualizerCfg(
239239
# Server settings
240240
app_id="isaaclab-simulation", # Application identifier for viewer
241+
grpc_port=9876, # gRPC endpoint for logging SDK connection
241242
web_port=9090, # Port for local web viewer (launched in browser)
243+
bind_address="0.0.0.0", # Endpoint host formatting/reuse checks
242244
243245
# Camera settings
244246
camera_position=(8.0, 8.0, 3.0), # Initial camera position (x, y, z)
@@ -252,6 +254,10 @@ Rerun Visualizer
252254
record_to_rrd="recording.rrd", # Path to save .rrd file (None = no recording)
253255
)
254256
257+
Rerun startup uses the Python SDK through ``newton.viewer.ViewerRerun`` (no external ``rerun`` CLI process
258+
management). If ``grpc_port`` is already active, Isaac Lab reuses that server. If ``web_port`` is occupied while
259+
starting a new server, initialization fails with a clear port-conflict error.
260+
255261

256262
Performance Note
257263
----------------

0 commit comments

Comments
 (0)