You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[Newton] Adds policy transfer script for sim2sim transfer from Newton to physX (#3565)
# Description
This PR adds a play script to physX-based IsaacLab to make it possible
to play a Newton-based trained policy. Tested environments are
H1/G1/Anymal-D but other exisiting Newton environment should transfer as
well.
<!--
Thank you for your interest in sending a pull request. Please make sure
to check the contribution guidelines.
Link:
https://isaac-sim.github.io/IsaacLab/main/source/refs/contributing.html
💡 Please try to keep PRs small and focused. Large PRs are harder to
review and merge.
-->
Please include a summary of the change and which issue is fixed. Please
also include relevant motivation and context.
List any dependencies that are required for this change.
Fixes # (issue)
<!-- As a practice, it is recommended to open an issue to have
discussions on the proposed pull request.
This makes it easier for the community to keep track of what is being
developed or added, and if a given feature
is demanded by more than one party. -->
## Type of change
<!-- As you go through the list, delete the ones that are not
applicable. -->
- Bug fix (non-breaking change which fixes an issue)
- New feature (non-breaking change which adds functionality)
- Breaking change (existing functionality will not work without user
modification)
- Documentation update
## Screenshots
Please attach before and after screenshots of the change if applicable.
<!--
Example:
| Before | After |
| ------ | ----- |
| _gif/png before_ | _gif/png after_ |
To upload images to a PR -- simply drag and drop an image while in edit
mode and it should upload the image directly. You can then paste that
source into the above before/after sections.
-->
## Checklist
- [ ] I have read and understood the [contribution
guidelines](https://isaac-sim.github.io/IsaacLab/main/source/refs/contributing.html)
- [ ] I have run the [`pre-commit` checks](https://pre-commit.com/) with
`./isaaclab.sh --format`
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] I have updated the changelog and the corresponding version in the
extension's `config/extension.toml` file
- [ ] I have added my name to the `CONTRIBUTORS.md` or my name already
exists there
<!--
As you go through the checklist above, you can mark something as done by
putting an x character in it
For example,
- [x] I have done this task
- [ ] I have not done this task
-->
---------
Signed-off-by: Milad Rakhsha <[email protected]>
Copy file name to clipboardExpand all lines: docs/source/experimental-features/newton-physics-integration/sim-to-sim.rst
+91-23Lines changed: 91 additions & 23 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -2,53 +2,55 @@
2
2
3
3
Sim-to-Sim Policy Transfer
4
4
==========================
5
-
This section provides examples of sim-to-sim policy transfer using the Newton backend. Sim-to-sim transfer is an essential step before real robot deployment because it verifies that policies work across different simulators. Policies that pass sim-to-sim verification are much more likely to succeed on real robots.
5
+
This section provides examples of sim-to-sim policy transfer between PhysX and Newton backends. Sim-to-sim transfer is an essential step before real robot deployment because it verifies that policies work across different simulators. Policies that pass sim-to-sim verification are much more likely to succeed on real robots.
6
6
7
7
8
8
Overview
9
9
--------
10
10
11
-
This guide shows how to run a PhysX-trained policy on the Newton backend. While the method works for any robot and physics engine, it has only been tested with Unitree G1, Unitree H1, and ANYmal-D robots using PhysX-trained policies.
11
+
This guide shows how to transfer policies between PhysX and Newton backends in both directions. The main challenge is that different physics engines may parse the same robot model with different joint and link ordering.
12
12
13
-
PhysX-trained policies expect joints and links in a specific order determined by how PhysX parses the robot model. However, Newton may parse the same robot with different joint and link ordering.
13
+
Policies trained in one backend expect joints and links in a specific order determined by how that backend parses the robot model. When transferring to another backend, the joint ordering may be different, requiring remapping of observations and actions.
14
14
15
15
In the future, we plan to solve this using **robot schema** that standardizes joint and link ordering across different backends.
16
16
17
-
Currently, we solve this by remapping observations and actions using joint mappings defined in YAML files. These files specify joint names in both PhysX order (source) and Newton order (target). During policy execution, we use this mapping to reorder observations and actions so they work correctly with Newton.
17
+
Currently, we solve this by remapping observations and actions using joint mappings defined in YAML files. These files specify joint names in both source and target backend orders. During policy execution, we use this mapping to reorder observations and actions so they work correctly with the target backend.
18
+
19
+
The method has been tested with Unitree G1, Unitree Go2, Unitree H1, and ANYmal-D robots for both transfer directions.
18
20
19
21
20
22
What you need
21
23
~~~~~~~~~~~~~
22
24
23
-
- A policy checkpoint trained with PhysX (RSL-RL).
24
-
- A joint mapping YAML for your robot under ``scripts/newton_sim2sim/mappings/``.
25
-
- The provided player script: ``scripts/newton_sim2sim/rsl_rl_transfer.py``.
25
+
- A policy checkpoint trained with either PhysX or Newton (RSL-RL).
26
+
- A joint mapping YAML for your robot under ``scripts/sim2sim_transfer/config/``.
27
+
- The provided player script: ``scripts/sim2sim_transfer/rsl_rl_transfer.py``.
26
28
27
29
To add a new robot, create a YAML file with two lists where each joint name appears exactly once in both:
28
30
29
31
.. code-block:: yaml
30
32
31
33
# Example structure
32
-
source_joint_names: #PhysX joint order
34
+
source_joint_names: #Source backend joint order
33
35
- joint_1
34
36
- joint_2
35
37
# ...
36
-
target_joint_names: #Newton joint order
38
+
target_joint_names: #Target backend joint order
37
39
- joint_1
38
40
- joint_2
39
41
# ...
40
42
41
43
The script automatically computes the necessary mappings for locomotion tasks.
42
44
43
45
44
-
How to run
45
-
~~~~~~~~~~
46
+
PhysX-to-Newton Transfer
47
+
~~~~~~~~~~~~~~~~~~~~~~~~
46
48
47
-
Use this command template to run a PhysX-trained policy with Newton:
49
+
To run a PhysX-trained policy with the Newton backend, use this command template:
The key difference is using the ``newton_to_physx_*.yaml`` mapping files instead of ``physx_to_newton_*.yaml`` files. Also note that you need to checkout a PhysX-based IsaacLab branch such as ``main``.
93
159
94
-
Notes and limitations
160
+
Notes and Limitations
95
161
~~~~~~~~~~~~~~~~~~~~~
96
162
97
-
- This transfer method has only been tested with Unitree G1, Unitree H1, and ANYmal-D using PhysX-trained policies.
98
-
- The observation remapping assumes a locomotion layout with base observations followed by joint observations. For different observation layouts, you'll need to modify ``scripts/newton_sim2sim/policy_mapping.py``.
163
+
- Both transfer directions have been tested with Unitree G1, Unitree Go2, Unitree H1, and ANYmal-D robots.
164
+
- PhysX-to-Newton transfer uses ``physx_to_newton_*.yaml`` mapping files.
165
+
- Newton-to-PhysX transfer requires the corresponding ``newton_to_physx_*.yaml`` mapping files and the PhysX branch of IsaacLab.
166
+
- The observation remapping assumes a locomotion layout with base observations followed by joint observations. For different observation layouts, you'll need to modify the ``get_joint_mappings`` function in ``scripts/sim2sim_transfer/rsl_rl_transfer.py``.
99
167
- When adding new robots or backends, make sure both source and target have identical joint names, and that the YAML lists reflect how each backend orders these joints.
0 commit comments