From: Sarah Hoffmann Date: Thu, 14 Jan 2021 11:04:08 +0000 (+0100) Subject: adapt documentation to new nominatim cli tool X-Git-Tag: v3.7.0~57^2~6 X-Git-Url: https://git.openstreetmap.org/nominatim.git/commitdiff_plain/1c1e95182698b23372e87ccae9f535dcbb5f7c5a adapt documentation to new nominatim cli tool --- diff --git a/docs/admin/Advanced-Installations.md b/docs/admin/Advanced-Installations.md index 5701992c..4f59900d 100644 --- a/docs/admin/Advanced-Installations.md +++ b/docs/admin/Advanced-Installations.md @@ -47,15 +47,16 @@ The file `import_multiple_regions.sh` needs to be edited as per your requirement BASEURL="https://download.geofabrik.de" DOWNCOUNTRYPOSTFIX="-latest.osm.pbf" - + +### Setting up multiple regions + !!! tip - If your database already exists and you want to add more countries, replace the setting up part + If your database already exists and you want to add more countries, + replace the setting up part `${SETUPFILE} --osm-file ${UPDATEDIR}/tmp/combined.osm.pbf --all 2>&1` with `${UPDATEFILE} --import-file ${UPDATEDIR}/tmp/combined.osm.pbf --index --index-instances N 2>&1` where N is the numbers of CPUs in your system. -### Setting up multiple regions - Run the following command from your Nominatim directory after configuring the file. bash ./utils/import_multiple_regions.sh diff --git a/docs/admin/Faq.md b/docs/admin/Faq.md index 6203959d..b8b900d0 100644 --- a/docs/admin/Faq.md +++ b/docs/admin/Faq.md @@ -16,7 +16,7 @@ was killed. If it looks like this: then you can resume with the following command: ```sh -./utils/setup.php --index --create-search-indices --create-country-names +nominatim import --continue indexing ``` If the reported rank is 26 or higher, you can also safely add `--index-noanalyse`. @@ -31,7 +31,7 @@ list for hints. If it happened during index creation you can try rerunning the step with ```sh -./utils/setup.php --create-search-indices --ignore-errors +nominatim import --continue indexing ``` Otherwise it's best to start the full setup from the beginning. @@ -93,7 +93,7 @@ on a non-managed machine. ### I see the error: "function transliteration(text) does not exist" -Reinstall the nominatim functions with `setup.php --create--functions` +Reinstall the nominatim functions with `nominatim refresh --functions` and check for any errors, e.g. a missing `nominatim.so` file. ### I see the error: "ERROR: mmap (remap) failed" @@ -113,7 +113,8 @@ Double-check clang is installed. Instead of `make` try running `make CLANG=true` ### nominatim UPDATE failed: ERROR: buffer 179261 is not owned by resource owner Portal -Several users [reported this](https://github.com/openstreetmap/Nominatim/issues/1168) during the initial import of the database. It's +Several users [reported this](https://github.com/openstreetmap/Nominatim/issues/1168) +during the initial import of the database. It's something PostgreSQL internal Nominatim doesn't control. And PostgreSQL forums suggest it's threading related but definitely some kind of crash of a process. Users reported either rebooting the server, different hardware or just trying @@ -202,7 +203,7 @@ See the installation instructions for a full list of required packages. ### I forgot to delete the flatnodes file before starting an import. That's fine. For each import the flatnodes file get overwritten. -See [https://help.openstreetmap.org/questions/52419/nominatim-flatnode-storage]() +See [https://help.openstreetmap.org/questions/52419/nominatim-flatnode-storage](https://help.openstreetmap.org/questions/52419/nominatim-flatnode-storage) for more information. diff --git a/docs/admin/Import.md b/docs/admin/Import.md index 41942d6f..1fdeb0e3 100644 --- a/docs/admin/Import.md +++ b/docs/admin/Import.md @@ -47,8 +47,7 @@ The file is about 400MB and adds around 4GB to the Nominatim database. !!! tip If you forgot to download the wikipedia rankings, you can also add importances after the import. Download the files, then run - `./utils/setup.php --import-wikipedia-articles` - and `./utils/update.php --recompute-importance`. + `./nominatim refresh --wiki-data --importance`. ### Great Britain, USA postcodes @@ -85,11 +84,14 @@ that Nominatim cannot compute the areas for some administrative areas. About half of the data in Nominatim's database is not really used for serving the API. It is only there to allow the data to be updated from the latest changes from OSM. For many uses these dynamic updates are not really required. -If you don't plan to apply updates, the dynamic part of the database can be -safely dropped using the following command: +If you don't plan to apply updates, you can run the import with the +`--no-updates` parameter. This will drop the dynamic part of the database as +soon as it is not required anymore. + +You can also drop the dynamic part later using the following command: ``` -./utils/setup.php --drop +./nominatim freeze ``` Note that you still need to provide for sufficient disk space for the initial @@ -155,7 +157,7 @@ Download the data to import. Then issue the following command from the **build directory** to start the import: ```sh -./utils/setup.php --osm-file --all 2>&1 | tee setup.log +./nominatim import --osm-file 2>&1 | tee setup.log ``` ### Notes on full planet imports @@ -196,19 +198,10 @@ reduce the cache size or even consider using a flatnode file. Run this script to verify all required tables and indices got created successfully. ```sh -./utils/check_import_finished.php +./nominatim check-database ``` -### Setting up the website - -Run the following command to set up the configuration file for the API frontend -`settings/settings-frontend.php`. These settings are used in website/*.php files. - -```sh -./utils/setup.php --setup-website -``` -!!! Note - This step is not necessary if you use `--all` option while setting up the DB. +### Testing the installation Now you can try out your installation by running: @@ -231,7 +224,7 @@ planner to make the right decisions. Recomputing them can improve the performanc of forward geocoding in particular under high load. To recompute word counts run: ```sh -./utils/update.php --recompute-word-counts +./nominatim refresh --word-counts ``` This will take a couple of hours for a full planet installation. You can @@ -243,7 +236,7 @@ If you want to be able to search for places by their type through [special key phrases](https://wiki.openstreetmap.org/wiki/Nominatim/Special_Phrases) you also need to enable these key phrases like this: - ./utils/specialphrases.php --wiki-import > specialphrases.sql + ./nominatim special-phrases --from-wiki > specialphrases.sql psql -d nominatim -f specialphrases.sql Note that this command downloads the phrases from the wiki link above. You @@ -260,13 +253,12 @@ entire US adds about 10GB to your database. 1. Get preprocessed TIGER 2019 data and unpack it into the data directory in your Nominatim sources: - cd Nominatim/data wget https://nominatim.org/data/tiger2019-nominatim-preprocessed.tar.gz tar xf tiger2019-nominatim-preprocessed.tar.gz 2. Import the data into your Nominatim database: - ./utils/setup.php --import-tiger-data + ./nominatim add-data --tiger-data tiger 3. Enable use of the Tiger data in your `.env` by adding: @@ -275,7 +267,7 @@ entire US adds about 10GB to your database. 4. Apply the new settings: ```sh - ./utils/setup.php --create-functions --enable-diff-updates --create-partition-functions + ./nominatim refresh --functions ``` diff --git a/docs/admin/Migration.md b/docs/admin/Migration.md index 3f330bb9..753fb500 100644 --- a/docs/admin/Migration.md +++ b/docs/admin/Migration.md @@ -6,6 +6,28 @@ to newer versions of Nominatim. SQL statements should be executed from the PostgreSQL commandline. Execute `psql nominatim` to enter command line mode. +## 3.6.0 -> master + +### Introducing `nominatim` command line tool + +The various php utilities have been replaced with a single `nominatim` +command line tool. Make sure to adapt any scripts. There is no direct 1:1 +matching between the old utilities and the commands of nominatim CLI. The +following list gives you a list of nominatim sub-commands that contain +functionality of each script: + +* ./utils/setup.php: `import`, `freeze`, `refresh` +* ./utils/update.php: `replication`, `add-data`, `index`, `refresh` +* ./utils/specialphrases.php: `special-phrases` +* ./utils/check_import_finished.php: `check-database` +* ./utils/warm.php: `warm` +* ./utils/export.php: `export` + +Try `nominatim --help` for more information about each subcommand. + +`./utils/query.php` no longer exists in its old form. `nominatim search` +provides a replacement. + ## 3.5.0 -> 3.6.0 ### Change of layout of search_name_* tables diff --git a/docs/admin/Update.md b/docs/admin/Update.md index 98f5244a..4b1a2be7 100644 --- a/docs/admin/Update.md +++ b/docs/admin/Update.md @@ -1,8 +1,10 @@ # Updating the Database There are many different ways to update your Nominatim database. -The following section describes how to keep it up-to-date with Pyosmium. -For a list of other methods see the output of `./utils/update.php --help`. +The following section describes how to keep it up-to-date using +an [online replication service for OpenStreetMap data](https://wiki.openstreetmap.org/wiki/Planet.osm/diffs) +For a list of other methods to add or update data see the output of +`nominatim add-data --help`. !!! important If you have configured a flatnode file for the import, then you @@ -44,23 +46,19 @@ diffs for Ireland from Geofabrik add the following: To set up the update process now run the following command: - ./utils/update.php --init-updates + ./nominatim replication --init It outputs the date where updates will start. Recheck that this date is what you expect. -The `--init-updates` command needs to be rerun whenever the replication service -is changed. +The `replication --init` command needs to be rerun whenever the replication +service is changed. #### Updating Nominatim The following command will keep your database constantly up to date: - ./utils/update.php --import-osmosis-all - -(Note that even though the old name "import-osmosis-all" has been kept for -compatibility reasons, Osmosis is not required to run this - it uses pyosmium -behind the scenes.) + ./nominatim replication If you have imported multiple country extracts and want to keep them up-to-date, [Advanced installations section](Advanced-Installations.md) contains instructions diff --git a/docs/api/Faq.md b/docs/api/Faq.md index 86f0e783..e9d34a70 100644 --- a/docs/api/Faq.md +++ b/docs/api/Faq.md @@ -58,4 +58,4 @@ The [Overpass API](https://wiki.openstreetmap.org/wiki/Overpass_API) is more suited for these kinds of queries. That said if you installed your own Nominatim instance you can use the -`/utils/export.php` PHP script as basis to return such lists. +`nominatim export` PHP script as basis to return such lists. diff --git a/docs/develop/Postcodes.md b/docs/develop/Postcodes.md index ff36b0dd..343b8de3 100644 --- a/docs/develop/Postcodes.md +++ b/docs/develop/Postcodes.md @@ -14,7 +14,7 @@ country's format, e.g. if Swiss postcodes are 4 digits. ## Regular updating calculated postcodes The script to rerun the calculation is -`build/utils/update.php --calculate-postcodes` +`nominatim refresh --postcodes` and runs once per night on nominatim.openstreetmap.org.