Skip to content

Commit

Permalink
[setup] Rewrite Ubuntu install_prereqs from the ground up
Browse files Browse the repository at this point in the history
The existing from-source setup scripts are deprecated and will be
removed after 2025-06-01:
- setup/ubuntu/install_prereqs.sh
- setup/ubuntu/source_distribution/install_prereqs_user_environment.sh

Instead, users and developers should run setup/install_prereqs (not as
root).

Usability changes:
- Fine-tuned log output, with option for --verbose (e.g., in CI).
- Skip steps that are unnecessary; most runs are just a couple seconds.

Maintainability changes:
- Prefer using textfiles, not heredocs. (The pkg-config etc in a
  heredoc were documented as unwanted when using Drake's Debian
  packages but that was wrong; we always want those deps.)

Website changes:
- Major updates to from_source that explain how to install from CMake.
- Remove vestigial hope of supporting 'brew install drake'.

A few cleanups macOS come along for the ride:
- Remove useless binary_distribution_called_update shenanigans.
- Remove vestigial i386 rcfile.
  • Loading branch information
jwnimmer-tri committed Nov 4, 2024
1 parent d6ff6cc commit 709113b
Show file tree
Hide file tree
Showing 30 changed files with 791 additions and 559 deletions.
36 changes: 36 additions & 0 deletions doc/_pages/bazel.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,42 @@ Drake's primary build system is Bazel. For more information about Bazel, see
Drake also offers a CMake build system wrapper that invokes Bazel under the
hood.

# Getting Drake

Run:

```
git clone --filter=blob:none https://github.com/RobotLocomotion/drake.git
```

Note: we suggest you keep the default clone directory name (``drake``) and not
rename it (such as ``drake2``). The CLion integration will suffer if the
checkout directory is not named ``drake``. (See [CLion IDE setup](clion.html) for details.)

Note: the build process may encounter problems if you have unusual characters
like parentheses in the absolute path to the drake directory
(see [#394](https://github.com/RobotLocomotion/drake/issues/394)).

## Using a fork of Drake

The above ``git clone`` command will configure Drake's primary repository as a
remote called ``origin``. If you plan to fork Drake for development, we
recommend that you configure your fork of Drake's primary repository as the
``origin`` remote and Drake's primary repository as the ``upstream``
remote. This can be done by executing the following commands:

```
cd drake
git remote set-url origin [email protected]:[your github user name]/drake.git
git remote add upstream https://github.com/RobotLocomotion/drake.git
git remote set-url --push upstream no_push
```

We recommend that you
[setup SSH access to github.com](https://help.github.com/articles/adding-a-new-ssh-key-to-your-github-account/)
to avoid needing to type your password each time you access it.


# Bazel Installation

Follow Drake's
Expand Down
162 changes: 72 additions & 90 deletions doc/_pages/from_source.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,12 @@
title: Source Installation
---

# New Users

For first-time users, we strongly suggest using one of the pre-compiled binaries
described on our [installation](/installation.html) page. This page explains how
to build Drake form source, which is somewhat more challenging.

# Supported Configurations

The following table shows the configurations and platforms that Drake
Expand Down Expand Up @@ -39,113 +45,67 @@ setup steps.

⁽³⁾ Drake requires a compiler running in C++20 (or greater) mode.

# Getting Drake

Run:

```
git clone --filter=blob:none https://github.com/RobotLocomotion/drake.git
```

Note: we suggest you keep the default clone directory name (``drake``) and not
rename it (such as ``drake2``). The CLion integration will suffer if the
checkout directory is not named ``drake``. (See [CLion IDE setup](clion.html) for details.)

Note: the build process may encounter problems if you have unusual characters
like parentheses in the absolute path to the drake directory
(see [#394](https://github.com/RobotLocomotion/drake/issues/394)).

## Using a fork of Drake

The above ``git clone`` command will configure Drake's primary repository as a
remote called ``origin``. If you plan to fork Drake for development, we
recommend that you configure your fork of Drake's primary repository as the
``origin`` remote and Drake's primary repository as the ``upstream``
remote. This can be done by executing the following commands:

```
cd drake
git remote set-url origin [email protected]:[your github user name]/drake.git
git remote add upstream https://github.com/RobotLocomotion/drake.git
git remote set-url --push upstream no_push
```

We recommend that you
[setup SSH access to github.com](https://help.github.com/articles/adding-a-new-ssh-key-to-your-github-account/)
to avoid needing to type your password each time you access it.

# Mandatory platform-specific instructions

Before running the build, you must follow some one-time platform-specific
setup steps.

*Ubuntu:*

```
sudo ./setup/ubuntu/install_prereqs.sh
```

*macOS:*

We assume that you have already installed Xcode
([from the Mac App Store](https://itunes.apple.com/us/app/xcode/id497799835)).

After that, run:
# Building with CMake

```
./setup/mac/install_prereqs.sh
```

# Build with Bazel

For instructions, jump to
[Developing Drake using Bazel](/bazel.html#developing-drake-using-bazel),
or check out the full details at:
For sample projects that show how to import Drake as a CMake external project
(either by building Drake from source, or by downloading a pre-compiled Drake
release) please see our gallery of
[drake-external-examples](https://github.com/RobotLocomotion/drake-external-examples).

* [Bazel build system](/bazel.html)
Otherwise, you can run a build from source by hand like this:

## Building the Python Bindings
```bash
# Get the sources.
git clone --filter=blob:none https://github.com/RobotLocomotion/drake.git

To use the Python bindings from Drake externally, we recommend using CMake.
As an example:
# Install the build dependencies.
drake/setup/install_prereqs

```bash
git clone https://github.com/RobotLocomotion/drake.git
# Build and install using standard CMake commands.
mkdir drake-build
cd drake-build
cmake ../drake
make install
```

Note that a concurrency limit passed to `make` (e.g., `make -j 2`) has almost no
effect on the Drake build. You might need to add a bazel configuration dotfile
to your home directory if your build is running out of memory. See the
[troubleshooting](/troubleshooting.html#build-oom) page for details.
To change the build options, you can run one of the standard CMake GUIs (e.g.,
`ccmake` or `cmake-gui`) or specify command-line options with `-D`.

Be aware that repeatedly running `make install` will install the recompiled
version of Drake *on top of* the prior version. This will lead to disaster
unless the set of installed filenames is exactly the same (because old files
will be hanging around polluting your PYTHONPATH). It is safe if you are merely
tweaking a source code file and repeatedly installing, without any changes to
the build system. For any kind of larger change (e.g., upgrading to a newer
Drake), we strongly advise that you delete the prior tree (within the `install`
sub-directory) before running `make`.
## Drake-specific CMake options

Please note the additional CMake options which affect the Python bindings:
These options can be set using `-DFOO=bar` on the CMake command line, or in one
of the CMake GUIs.

* ``-DWITH_GUROBI={ON, [OFF]}`` - Build with Gurobi enabled.
* ``-DWITH_MOSEK={ON, [OFF]}`` - Build with MOSEK™ enabled.
* ``-DWITH_SNOPT={ON, [OFF]}`` - Build with SNOPT enabled.
Adjusting open-source dependencies:

``{...}`` means a list of options, and the option surrounded by ``[...]`` is
the default option. An example of building ``pydrake`` with both Gurobi and
MOSEK™, without building tests:
* WITH_USER_EIGEN (default OFF). When ON, uses `find_package(Eigen3)`
to locate a user-provided `Eigen3::Eigen` library
instead of hard-coding to the operating system version.
* WITH_USER_FMT (default OFF). When ON, uses `find_package(fmt)`
to locate a user-provided `fmt::fmt` library
instead of hard-coding to the operating system version.
* WITH_USER_SPDLOG (default OFF). When ON, uses `find_package(spdlog)`
to locate a user-provided `spdlog::spdlog` library
instead of hard-coding to the operating system version.

```bash
cmake -DWITH_GUROBI=ON -DWITH_MOSEK=ON ../drake
```
Adjusting closed-source (commercial) software dependencies:

* WITH_GUROBI (default OFF). When ON, enables the `GurobiSolver` in the build.
* If Gurobi is not installed to its standard location, you must also specify
`export GUROBI_HOME=${...GUROBI_UNZIP_PATH...}/linux64` in your terminal
so that `find_package(Gurobi)` will be able to find it.
* WITH_MOSEK (default OFF). When ON, enables the `MosekSolver` in the build.
* WITH_SNOPT (default OFF). When ON, enables the `SnoptSolver` in the build.
* SNOPT_PATH (no default). When WITH_SNOPT is ON, this must be set to a SNOPT
archive path, e.g. `/home/user/Downloads/snopt7.4.tar.gz`.
* WITH_ROBOTLOMOTION_SNOPT (default OFF). When ON, enables the `SnoptSolver`
in the build, using a hard-coded and access-controlled download of SNOPT.
This option is only valid for MIT- or TRI-affiliated Drake developers.

You will also need to have your ``PYTHONPATH`` configured correctly.
## Running the Python Bindings after a CMake install

To run the installed copy of `pydrake`, you will also need to have your
``PYTHONPATH`` configured correctly.

*Ubuntu 22.04 (Jammy):*

Expand All @@ -167,3 +127,25 @@ export PYTHONPATH=${PWD}/install/lib/python3.12/site-packages:${PYTHONPATH}
cd drake-build
export PYTHONPATH=${PWD}/install/lib/python3.12/site-packages:${PYTHONPATH}
```

## Caveats

Note that a concurrency limit passed to `make` (e.g., `make -j 2`) for a Drake
build has almost no effect. You might need to add a bazel configuration dotfile
to your home directory if your build is running out of memory. See the
[troubleshooting](/troubleshooting.html#build-oom) page for details.

Be aware that repeatedly running `make install` will install the recompiled
version of Drake *on top of* the prior version. This will lead to disaster
unless the set of installed filenames is exactly the same (because old files
will be hanging around, e.g., polluting your PYTHONPATH). It is safe if you are
merely tweaking a source code file and repeatedly installing, without any
changes to the build system. For any kind of larger change (e.g., upgrading to a
newer Drake), we strongly advise that you delete the prior tree (within the
`install` sub-directory) before running `make`.

# Making changes to Drake

Drake developers use Bazel (not CMake) for development. Refer to our [Bazel
instructions](/bazel.html) for details. Bazel is only used for development;
there is no way to _install_ Drake from Bazel.
5 changes: 0 additions & 5 deletions doc/_pages/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,11 +104,6 @@ following the instructions in [Source Installation](/from_source.html).
Drake's binary releases do not support the Gurobi solver.
To use Gurobi, you must build Drake from source.

We're considering adding macOS support for Homebrew, i.e., ``brew install
drake``. Please upvote or comment on
[#12782](https://github.com/RobotLocomotion/drake/issues/12782)
if you are interested.

# Quickstart

For Python, refer to
Expand Down
33 changes: 12 additions & 21 deletions setup/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -14,19 +14,9 @@ exports_files([
"mac/source_distribution/requirements.txt",
"mac/source_distribution/requirements-maintainer-only.txt",
"mac/source_distribution/requirements-test-only.txt",
"ubuntu/binary_distribution/packages-jammy.txt",
"ubuntu/source_distribution/packages-jammy.txt",
"ubuntu/source_distribution/packages-jammy-clang.txt",
"ubuntu/source_distribution/packages-jammy-doc-only.txt",
"ubuntu/source_distribution/packages-jammy-maintainer-only.txt",
"ubuntu/source_distribution/packages-jammy-test-only.txt",
"ubuntu/binary_distribution/packages-noble.txt",
"ubuntu/source_distribution/packages-noble.txt",
"ubuntu/source_distribution/packages-noble-clang.txt",
"ubuntu/source_distribution/packages-noble-doc-only.txt",
"ubuntu/source_distribution/packages-noble-maintainer-only.txt",
"ubuntu/source_distribution/packages-noble-test-only.txt",
])
] + glob([
"ubuntu/*.txt",
]))

filegroup(
name = "deepnote",
Expand All @@ -51,19 +41,20 @@ install_files(
"deepnote/install_xvfb",
"deepnote/nginx-meshcat-proxy.conf",
"deepnote/xvfb",
"ubuntu/binary_distribution/install_prereqs.sh",
"ubuntu/binary_distribution/packages-jammy.txt",
"ubuntu/binary_distribution/packages-noble.txt",
"ubuntu/install_prereqs",
"ubuntu/packages-jammy-binary.txt",
"ubuntu/packages-noble-binary.txt",
],
"//conditions:default": [],
}),
strip_prefix = [
"mac/binary_distribution",
"ubuntu/binary_distribution",
"ubuntu",
],
rename = {
"share/drake/setup/install_prereqs.sh": "install_prereqs",
},
)

add_lint_tests()
add_lint_tests(
python_lint_extra_srcs = [
"ubuntu/install_prereqs",
],
)
13 changes: 13 additions & 0 deletions setup/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,16 @@ documentation.
If you are a developer wishing to add dependencies, please see the
[Jenkins pages for Updating Installation Prerequisites](
https://drake.mit.edu/jenkins.html#updating-installation-prerequisites).

## Layout

The files at `drake/setup/*/binary_distribution/*` are coped into our binary
distribution packages (`drake-NNNN.tar.gz`).

XXX The order of precedence (stacking) is:
- binary
- source
- test
- clang (not relevant on macOS)
- doc (not relevant on macOS)
- maintainer (not relevant on macOS)
16 changes: 16 additions & 0 deletions setup/install_prereqs
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
#!/bin/bash

set -euo pipefail

case "$OSTYPE" in
darwin*)
exec "${BASH_SOURCE%/*}/mac/install_prereqs" "$@"
;;
linux*)
exec "${BASH_SOURCE%/*}/ubuntu/install_prereqs" "$@"
;;
*)
echo 'ERROR: Unsupported OS' >&2
exit 1
;;
esac
6 changes: 0 additions & 6 deletions setup/mac/binary_distribution/install_prereqs.sh
Original file line number Diff line number Diff line change
Expand Up @@ -53,16 +53,10 @@ export HOMEBREW_NO_AUTO_UPDATE=1
# completes or wait for the next automatic cleanup if necessary.
export HOMEBREW_NO_INSTALL_CLEANUP=1

binary_distribution_called_update=0

if [[ "${with_update}" -eq 1 ]]; then
# Note that brew update uses git, so HOMEBREW_CURL_RETRIES does not take
# effect.
brew update || (sleep 30; brew update)

# Do NOT call brew update again when installing prerequisites for source
# distributions.
binary_distribution_called_update=1
fi

brew bundle --file="${BASH_SOURCE%/*}/Brewfile" --no-lock
Expand Down
1 change: 0 additions & 1 deletion setup/mac/install_prereqs.sh
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,6 @@ while [ "${1:-}" != "" ]; do
# Do NOT call brew update during execution of this script.
--without-update)
binary_distribution_args+=(--without-update)
source_distribution_args+=(--without-update)
;;
*)
echo 'Invalid command line argument' >&2
Expand Down
9 changes: 0 additions & 9 deletions setup/mac/source_distribution/install_prereqs.sh
100755 → 100644
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@
set -euxo pipefail

with_test_only=1
with_update=1

while [ "${1:-}" != "" ]; do
case "$1" in
Expand All @@ -18,10 +17,6 @@ while [ "${1:-}" != "" ]; do
--without-test-only)
with_test_only=0
;;
# Do NOT call brew update during execution of this script.
--without-update)
with_update=0
;;
*)
echo 'Invalid command line argument' >&2
exit 5
Expand All @@ -39,10 +34,6 @@ if ! command -v brew &>/dev/null; then
exit 4
fi

if [[ "${with_update}" -eq 1 && "${binary_distribution_called_update:-0}" -ne 1 ]]; then
brew update || (sleep 30; brew update)
fi

brew bundle --file="${BASH_SOURCE%/*}/Brewfile" --no-lock

if [[ "${with_test_only}" -eq 1 ]]; then
Expand Down
Loading

0 comments on commit 709113b

Please sign in to comment.