Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new flexible parameters and new logic of using of nominatim-docker #394

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 53 additions & 0 deletions 4.1/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,8 @@ The following environment variables are available for configuration:
- `IMPORT_TIGER_ADDRESSES`: Whether to download and import the Tiger address data (`true`) or path to a preprocessed Tiger address set in the container. (default: `false`)
- `THREADS`: How many threads should be used to import (default: `16`)
- `NOMINATIM_PASSWORD`: The password to connect to the database with (default: `qaIACxO6wMR3`)
- `PGDATABASE`: Name of the database (default: `nominatim`)
- `SKIP_IMPORT`: If it `yes` it can skip import and create new database. It is used when the container is naive, i.e. it has not imported data, but is used only to update existing data.

The following run parameters are available for configuration:

Expand Down Expand Up @@ -206,6 +208,57 @@ docker run -it \
nominatim
```

## Using an external PostgreSQL database and setup with updating data container

If you want to safely update your data and set up regular updates, then you can do the following.

First of all you need build new container from Dockerfile from 4.1 folder:

```
cd nominatim-docker/4.1
docker build -t nominatim .
```

Then you need to import data into the new `nominatim_new` database so as not to touch the `nominatim` database.

```
sudo docker run -d --rm \
-e PBF_URL=http://download.geofabrik.de/europe/monaco-latest.osm.pbf \
-e NOMINATIM_TOKENIZER=icu \
-e NOMINATIM_DATABASE_DSN="pgsql:dbname=nominatim_new;hostaddr=192.168.0.1;user=postgres;password=SUPER_PASSWORD" \
-e PGHOSTADDR=192.168.0.1 \
-e PGDATABASE=nominatim_new \
-e PGUSER=postgres \
-e PGPASSWORD=SUPER_PASSWORD \
-e NOMINATIM_PASSWORD=very_secure_password \
-p 8080:8080 \
--name nominatim-import \
nominatim
```

Next, you make any adjustments and checks in `nominatim_new`. After making sure that everything is in order, we can rename the database to `nominatim`. And then safely update as follows:

```
sudo docker run -it \
-e PBF_URL=http://download.geofabrik.de/europe/monaco-latest.osm.pbf \
-e REPLICATION_URL=http://download.geofabrik.de/europe/monaco-updates/ \
-e UPDATE_MODE=once \
-e SKIP_IMPORT=true \
-e NOMINATIM_DATABASE_DSN="pgsql:dbname=nominatim;hostaddr=192.168.0.1;user=postgres;password=SUPER_PASSWORD" \
-e PGHOSTADDR=192.168.0.1 \
-e PGDATABASE=nominatim \
-e PGUSER=postgres \
-e PGPASSWORD=SUPER_PASSWORD \
-p 8080:8080 \
--name nominatim-update \
nominatim
```

After all, we can just restart the container for daily updates
```
sudo docker start -i nominatim-update
```

## Docker Compose

In addition, we also provide a basic `contrib/docker-compose.yml` template which you use as a starting point and adapt to your needs. Use this template to set the environment variables, mounts, etc. as needed.
Expand Down
8 changes: 6 additions & 2 deletions 4.1/init.sh
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,10 @@ if [ "$PBF_PATH" != "" ]; then
OSMFILE=$PBF_PATH
fi

if [ !-z $PGDATABASE ]; then
echo Make default PGDATABASE=nominatim
PGDATABASE=nominatim
fi

# if we use a bind mount then the PG directory is empty and we have to create it
if [ ! -f /var/lib/postgresql/14/main/PG_VERSION ]; then
Expand All @@ -66,7 +70,7 @@ sudo -E -u postgres psql postgres -tAc "SELECT 1 FROM pg_roles WHERE rolname='ww
sudo -E -u postgres psql postgres -tAc "ALTER USER nominatim WITH ENCRYPTED PASSWORD '$NOMINATIM_PASSWORD'" && \
sudo -E -u postgres psql postgres -tAc "ALTER USER \"www-data\" WITH ENCRYPTED PASSWORD '${NOMINATIM_PASSWORD}'" && \

sudo -E -u postgres psql postgres -c "DROP DATABASE IF EXISTS nominatim"
sudo -E -u postgres psql postgres -c "DROP DATABASE IF EXISTS $PGDATABASE"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering why we even drop the database in the first place. Seems quite destructive.

@philipkozeny do you have an idea?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@leonardehrenfried because the OG maintainer started doing that in the initial version he released, never questioned if we should continue with that behaviour to be honest ...

sudo -u postgres psql postgres -c "DROP DATABASE IF EXISTS nominatim" && \

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That because I try to make it safely.
It's OK, if you could change name, I think


chown -R nominatim:nominatim ${PROJECT_DIR}

Expand Down Expand Up @@ -102,7 +106,7 @@ sudo -E -u nominatim nominatim admin --warm
# gather statistics for query planner to potentially improve query performance
# see, https://github.com/osm-search/Nominatim/issues/1023
# and https://github.com/osm-search/Nominatim/issues/1139
sudo -E -u nominatim psql -d nominatim -c "ANALYZE VERBOSE"
sudo -E -u nominatim psql -d $PGDATABASE -c "ANALYZE VERBOSE"

sudo service postgresql stop

Expand Down
2 changes: 1 addition & 1 deletion 4.1/start.sh
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ fi

IMPORT_FINISHED=/var/lib/postgresql/14/main/import-finished

if [ ! -f ${IMPORT_FINISHED} ]; then
if [ ! -f ${IMPORT_FINISHED} ] && [ "$SKIP_IMPORT" != "true" ]; then
/app/init.sh
touch ${IMPORT_FINISHED}
else
Expand Down