Skip to content

Commit

Permalink
Upgrades the repo to the latest version of IsaacLab (#32)
Browse files Browse the repository at this point in the history
# Description

This PR updates the code to the newer version of IsaacLab. Moreover, it removes the need for link the repo `orbit_envs`.
  • Loading branch information
pascal-roth authored Oct 22, 2024
1 parent 61016e8 commit 5fc8a3e
Show file tree
Hide file tree
Showing 47 changed files with 3,509 additions and 180 deletions.
54 changes: 54 additions & 0 deletions .github/ISSUE_TEMPLATE/bug.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
---
name: Bug Report
about: Submit a bug report
title: "[Bug Report] Bug title"

---

If you are submitting a bug report, please fill in the following details and use the tag [bug].

### Describe the bug

A clear and concise description of what the bug is.

### Steps to reproduce

Please try to provide a minimal example to reproduce the bug. Error messages and stack traces are also helpful.

<!-- Please post terminal logs, minimal example to reproduce, or command to run under three backticks (```) to allow code formatting.
```
Paste your error here
```
For more information on this, check: https://www.markdownguide.org/extended-syntax/#fenced-code-blocks
-->

### System Info

Describe the characteristic of your environment:

<!-- Please complete the following description. -->
- Commit: [e.g. 8f3b9ca]
- Isaac Sim Version: [e.g. 2022.2.0, this can be obtained by `cat ${ISAACSIM_PATH}/VERSION`]
- OS: [e.g. Ubuntu 20.04]
- GPU: [e.g. RTX 2060 Super]
- CUDA: [e.g. 11.4]
- GPU Driver: [e.g. 470.82.01, this can be seen by using `nvidia-smi` command.]

### Additional context

Add any other context about the problem here.

### Checklist

- [ ] I have checked that there is no similar issue in the repo (**required**)
- [ ] I have checked that the issue is not in running Isaac Sim itself and is related to the repo

### Acceptance Criteria

Add the criteria for which this task is considered **done**. If not known at issue creation time, you can add this once the issue is assigned.

- [ ] Criteria 1
- [ ] Criteria 2
37 changes: 37 additions & 0 deletions .github/ISSUE_TEMPLATE/proposal.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
---
name: Proposal
about: Propose changes that are not bug fixes
title: "[Proposal] Proposal title"
---


### Proposal

A clear and concise description of the proposal. In a few sentences, describe the feature and its core capabilities.

### Motivation

Please outline the motivation for the proposal. Summarize the core use cases and user problems and needs you are trying to solve.

Is your feature request related to a problem? e.g.,"I'm always frustrated when [...]".

If this is related to another GitHub issue, please link here too.

### Alternatives

A clear and concise description of any alternative solutions or features you've considered, if any.

### Additional context

Add any other context or screenshots about the feature request here.

### Checklist

- [ ] I have checked that there is no similar issue in the repo (**required**)

### Acceptance Criteria

Add the criteria for which this task is considered **done**. If not known at issue creation time, you can add this once the issue is assigned.

- [ ] Criteria 1
- [ ] Criteria 2
13 changes: 13 additions & 0 deletions .github/ISSUE_TEMPLATE/question.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
---
name: Question
about: Ask a question
title: "[Question] Question title"
---

### Question

Basic questions that are not bugs or feature requests will be closed without reply, because GitHub issues are not an appropriate venue for these.

Advanced/nontrivial questions, especially in areas where documentation is lacking, are very much welcome.

For questions that are related to running and understanding Isaac Sim, please post them at the official [Isaac Sim forums](https://docs.omniverse.nvidia.com/app_isaacsim/app_isaacsim/isaac_sim_forums.html).
47 changes: 47 additions & 0 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# Description

Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context.
List any dependencies that are required for this change.

Fixes # (issue)

<!-- As a practice, it is recommended to open an issue to have discussions on the proposed pull request.
This makes it easier for the community to keep track of what is being developed or added, and if a given feature
is demanded by more than one party. -->

## Type of change

<!-- As you go through the list, delete the ones that are not applicable. -->

- Bug fix (non-breaking change which fixes an issue)
- New feature (non-breaking change which adds functionality)
- Breaking change (fix or feature that would cause existing functionality to not work as expected)
- This change requires a documentation update

## Screenshots

Please attach before and after screenshots of the change if applicable.

<!--
Example:
| Before | After |
| ------ | ----- |
| _gif/png before_ | _gif/png after_ |
To upload images to a PR -- simply drag and drop an image while in edit mode and it should upload the image directly. You can then paste that source into the above before/after sections.
-->

## Checklist

- [ ] I have run the [`pre-commit` checks](https://pre-commit.com/) with `./formatter.sh`
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings

<!--
As you go through the checklist above, you can mark something as done by putting an x character in it
For example,
- [x] I have done this task
- [ ] I have not done this task
-->
14 changes: 14 additions & 0 deletions .github/workflows/pre-commit.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
name: Run linters using pre-commit

on:
pull_request:
push:
branches: [main]

jobs:
pre-commit:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v3
- uses: pre-commit/[email protected]
4 changes: 2 additions & 2 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ repos:
- id: black
args: ["--line-length", "120"]
- repo: https://github.com/pycqa/flake8
rev: 6.0.0
rev: 7.0.0
hooks:
- id: flake8
additional_dependencies: [flake8-simplify, flake8-return]
Expand Down Expand Up @@ -41,7 +41,7 @@ repos:
rev: v2.2.4
hooks:
- id: codespell
exclude: ^ros/joystick_drivers/
exclude: ^(ros/joystick_drivers/|omniverse/extension/omni.isaac.matterport/data/mappings)
additional_dependencies:
- tomli
- repo: https://github.com/Lucas-C/pre-commit-hooks
Expand Down
28 changes: 15 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@

ViPlanner is a robust learning-based local path planner based on semantic and depth images.
Fully trained in simulation, the planner can be applied in dynamic indoor as well outdoor environments.
We provide it as an extension for [NVIDIA Isaac-Sim](https://developer.nvidia.com/isaac-sim) within the [Orbit](https://isaac-orbit.github.io/) project (details [here](./omniverse/README.md)).
We provide it as an extension for [NVIDIA Isaac-Sim](https://developer.nvidia.com/isaac-sim) within the [IsaacLab](https://isaac-sim.github.io/IsaacLab/) project (details [here](./omniverse/README.md)).
Furthermore, a ready to use [ROS Noetic](http://wiki.ros.org/noetic) package is available within this repo for direct integration on any robot (tested and developed on ANYmal C and D).

**Keywords:** Visual Navigation, Local Planning, Imperative Learning
Expand Down Expand Up @@ -74,13 +74,25 @@ Note that for an editable install for packages without setup.py, PEP660 has to b
python3 -m pip install --upgrade setuptools
```


## Inference and Model Demo

1. Real-World <br>

ROS-Node is provided to run the planner on the LeggedRobot ANYmal, for details please see [ROS-Node-README](ros/README.md).

2. NVIDIA Isaac-Sim <br>

The planner can be executed within Nvidia Isaac Sim. It is implemented as part of the [IsaacLab Framework](https://isaac-sim.github.io/IsaacLab/) with an own extension. For details, please see [Omniverse Extension](./omniverse/README.md). This includes a **planner demo** in different environments with the trained model.


## Training

Here an overview of the steps involved in training the policy.
For more detailed instructions, please refer to [TRAINING.md](TRAINING.md).

0. Training Data Generation <br>
Training data is generated from the [Matterport 3D](https://github.com/niessner/Matterport), [Carla](https://carla.org/) and [NVIDIA Warehouse](https://docs.omniverse.nvidia.com/isaacsim/latest/tutorial_static_assets.html) using developed Isaac Sim Extension, that are open-sourced. Currently, the extensions are updated to the latest `Orbit` version and will be available soon, an intermediate solution is given [here](https://github.com/pascal-roth/orbit_envs).
Training data is generated from the [Matterport 3D](https://github.com/niessner/Matterport), [Carla](https://carla.org/) and [NVIDIA Warehouse](https://docs.omniverse.nvidia.com/isaacsim/latest/tutorial_static_assets.html) using developed Isaac Sim Extension, the extensions are part of a new internal project (``isaac-nav-suite``) and will be open sourced with that project. In the case that you require an earlier access, please contact us via mail.

1. Build Cost-Map <br>
The first step in training the policy is to build a cost-map from the available depth and semantic data. A cost-map is a representation of the environment where each cell is assigned a cost value indicating its traversability. The cost-map guides the optimization, therefore, is required to be differentiable. Cost-maps are built using the [cost-builder](viplanner/cost_builder.py) with configs [here](viplanner/config/costmap_cfg.py), given a pointcloud of the environment with semantic information (either from simultion or real-world information).
Expand All @@ -92,16 +104,6 @@ Once the cost-map is constructed, the next step is to train the policy. The poli
Performance assessment can be performed on simulation and real-world data. The policy will be evaluated regarding multiple metrics such as distance to goal, average and maximum cost, path length. In order to let the policy be executed on anymal in simulation, please refer to [Omniverse Extension](./omniverse/README.md)


## Inference

1. Real-World <br>

ROS-Node is provided to run the planner on the LeggedRobot ANYmal, for details please see [ROS-Node-README](ros/README.md).

2. NVIDIA Isaac-Sim <br>

The planner can be executed within Nvidia Isaac Sim. It is implemented as part of the [Orbit Framework](https://isaac-orbit.github.io/) with an own extension. For details, please see [Omniverse Extension](./omniverse/README.md).

### Model Download
The latest model is available to download: [[checkpoint](https://drive.google.com/file/d/1PY7XBkyIGESjdh1cMSiJgwwaIT0WaxIc/view?usp=sharing)] [[config](https://drive.google.com/file/d/1r1yhNQAJnjpn9-xpAQWGaQedwma5zokr/view?usp=sharing)]

Expand All @@ -121,7 +123,7 @@ The latest model is available to download: [[checkpoint](https://drive.google.co
This code belongs to Robotic Systems Lab, ETH Zurich.
All right reserved

**Authors: [Pascal Roth](https://github.com/pascal-roth), [Julian Nubert](https://juliannubert.com/), [Fan Yang](https://github.com/MichaelFYang), [Mayank Mittal](https://mayankm96.github.io/), and [Marco Hutter](https://rsl.ethz.ch/the-lab/people/person-detail.MTIxOTEx.TGlzdC8yNDQxLC0xNDI1MTk1NzM1.html)<br />
**Authors: [Pascal Roth](https://github.com/pascal-roth), [Julian Nubert](https://juliannubert.com/), [Fan Yang](https://github.com/MichaelFYang), [Mayank Mittal](https://mayankm96.github.io/), [Ziqi Fan](https://github.com/fan-ziqi), and [Marco Hutter](https://rsl.ethz.ch/the-lab/people/person-detail.MTIxOTEx.TGlzdC8yNDQxLC0xNDI1MTk1NzM1.html)<br />
Maintainer: Pascal Roth, [email protected]**

The ViPlanner package has been tested under ROS Noetic on Ubuntu 20.04.
Expand Down
Binary file added docs/example_carla.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/example_matterport.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion formatter.sh
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ if ! command -v pre-commit &>/dev/null; then
echo "[INFO] Installing pre-commit..."
pip install pre-commit
fi
# always execute inside the Orbit directory
# always execute inside the IsaacLab directory
echo "[INFO] Formatting the repository..."
cd ${VIPLANNER_PATH}
pre-commit run --all-files
Expand Down
Loading

0 comments on commit 5fc8a3e

Please sign in to comment.