add camera to g1 locomanipulation env and update docs with G1.5 training note #3549
Conversation
jaybdub
requested review from
jtigue-bdai,
kellyguo11 and
Mayankm96
as code owners
github-actions
bot
added
documentation
jaybdub
changed the title
|
Please add how you tested these changes. |
|
@ooctipus to review. |
|
@yanchangNvidia -- These changes were tested by
|
| you may convert the generated dataset to LeRobot format as expected by GR00T N1.5, and then run the finetuning script provided | ||
| in the GR00T N1.5 repository. An example closed-loop policy rollout is shown in the video below: | ||
|
|
||
| .. figure:: https://download.isaacsim.omniverse.nvidia.com/isaaclab/images/locomanipulation_sdg_disjoint_nav_groot_policy_4x.gif |
There was a problem hiding this comment.
Could we change this gif name to "locomanipulation_sdg_groot_policy_4x.gif"?
There was a problem hiding this comment.
We can but we need to ask for these images to be re-uploaded, I don't have direct access to do this.
There was a problem hiding this comment.
Got it. Who can help us to get these images re-uploaded?
| ## Usage | ||
|
|
||
| To run the locomanipulation SDG replay script execute the following command. | ||
|
|
There was a problem hiding this comment.
Do we need another step of running manipulation trajectory augmentation before the navigation trajectory augmentation?
There was a problem hiding this comment.
It is not needed, but desired. The script is generic, it can use the teleop dataset directly, or instead use the manipulation augmented trajectory.
There was a problem hiding this comment.
Got it. It would be great to add this information as a part of the documentation so that users know that they can use have another step to do the manipulation augmented trajectory before this step to scale up the data.
| width=256, | ||
| data_types=["rgb"], | ||
| spawn=sim_utils.PinholeCameraCfg(focal_length=8.0, clipping_range=(0.1, 20.0)), | ||
| offset=CameraCfg.OffsetCfg(pos=(0.0, 0.0, 0.0), rot=(0.9848078, 0.0, -0.1736482, 0.0), convention="world"), |
There was a problem hiding this comment.
How was the "rot" setting chosen?
There was a problem hiding this comment.
Empirically to give a meaningful field of view that shows hands and grasping area
There was a problem hiding this comment.
Got it. It might be good to add a comment on this setting.
Description
This PR adds the following changes
Type of change
Screenshots
Please attach before and after screenshots of the change if applicable.
Checklist
pre-commitchecks with./isaaclab.sh --formatconfig/extension.tomlfileCONTRIBUTORS.mdor my name already exists there