From: Sarah Hoffmann Date: Sun, 1 Dec 2019 19:17:27 +0000 (+0100) Subject: Merge branch 'housenumber-zero' of https://github.com/mtmail/Nominatim into mtmail... X-Git-Tag: v3.5.0~122 X-Git-Url: https://git.openstreetmap.org/nominatim.git/commitdiff_plain/a4e514033df5fbe3f494d757892589772a690cce?hp=eeb26aaa6fb4010fcbe34decbf3efc8c42995d1c Merge branch 'housenumber-zero' of https://github.com/mtmail/Nominatim into mtmail-housenumber-zero --- diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index cd8d1cc1..89b76e3b 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -15,9 +15,9 @@ Please make sure to add the following information: * the result you are getting * the expected result, preferably a link to the OSM object you want to find, otherwise an address that is as precise as possible - - To get the link to the OSM object, you can try the following: - + +To get the link to the OSM object, you can try the following: + * go to https://openstreetmap.org * zoom to the area of the map where you expect the result and zoom in as much as possible @@ -26,7 +26,7 @@ Please make sure to add the following information: * find the object of interest in the list that appears on the left side * click on the object and report the URL back that the browser shows -### When Reporting Problems with your Installation... +### When Reporting Bugs... Please add the following information to your issue: @@ -38,6 +38,9 @@ Please add the following information to your issue: if you run from the git repo, the output of `git rev-parse HEAD`) * (if applicable) exact command line of the command that was causing the issue +Bug reports that do not include extensive information about your system, +about the problem and about what you have been trying to debug the problem +will be closed. ## Workflow for Pull Requests diff --git a/README.md b/README.md index a9a5a3e2..1ae0dab0 100644 --- a/README.md +++ b/README.md @@ -19,8 +19,16 @@ https://nominatim.org/release-docs/develop/ . Installation ============ +**Nominatim is a complex piece of software and runs in a complex environment. +Installing and running Nominatim is something for experienced system +administrators only who can do some trouble-shooting themselves. We are sorry, +but we can not provide installation support. We are all doing this in our free +time and there is just so much of that time to go around. Do not open issues in +our bug tracker if you need help. You can ask questions on the mailing list +(see below) or on [help.openstreetmap.org](https://help.openstreetmap.org/).** + The latest stable release can be downloaded from https://nominatim.org. -There you can also find [installation instructions for the release](https://nominatim.org/release-docs/latest/admin/Installation). +There you can also find [installation instructions for the release](https://nominatim.org/release-docs/latest/admin/Installation), as well as an extensive [Troubleshooting/FAQ section](https://nominatim.org/release-docs/latest/admin/Faq/). Detailed installation instructions for the development version can be found at [nominatim.org](https://nominatim.org/release-docs/develop/admin/Installation) diff --git a/docs/admin/Faq.md b/docs/admin/Faq.md index 485ba25b..2e5a7c43 100644 --- a/docs/admin/Faq.md +++ b/docs/admin/Faq.md @@ -26,14 +26,16 @@ If the reported rank is 26 or higher, you can also safely add `--index-noanalyse PHP Warning: file_get_contents(): open_basedir restriction in effect. -You need to adjust the [open_basedir](https://www.php.net/manual/en/ini.core.php#ini.open-basedir) setting -in your PHP configuration (`php.ini file`). By default this setting may look like this: +You need to adjust the +[open_basedir](https://www.php.net/manual/en/ini.core.php#ini.open-basedir) +setting in your PHP configuration (`php.ini` file). By default this setting may +look like this: open_basedir = /srv/http/:/home/:/tmp/:/usr/share/pear/ -Either add reported directories to the list or disable this setting temporarily by -dding ";" at the beginning of the line. Don't forget to enable this setting again -once you are done with the PHP command line operations. +Either add reported directories to the list or disable this setting temporarily +by adding ";" at the beginning of the line. Don't forget to enable this setting +again once you are done with the PHP command line operations. ### PHP timzeone warnings @@ -107,10 +109,11 @@ to get the full error message. `could not connect to server: No such file or directory` -On CentOS v7 the PostgreSQL server is started with `systemd`. -Check if `/usr/lib/systemd/system/httpd.service` contains a line `PrivateTmp=true`. -If so then Apache cannot see the `/tmp/.s.PGSQL.5432` file. It's a good security feature, -so use the [preferred solution](../appendix/Install-on-Centos-7/#adding-selinux-security-settings). +On CentOS v7 the PostgreSQL server is started with `systemd`. Check if +`/usr/lib/systemd/system/httpd.service` contains a line `PrivateTmp=true`. If +so then Apache cannot see the `/tmp/.s.PGSQL.5432` file. It's a good security +feature, so use the +[preferred solution](../appendix/Install-on-Centos-7/#adding-selinux-security-settings). However, you can solve this the quick and dirty way by commenting out that line and then run @@ -118,14 +121,12 @@ However, you can solve this the quick and dirty way by commenting out that line sudo systemctl restart httpd -### "must be an array or an object that implements Countable" warning in /usr/share/pear/DB.php - -The warning started with PHP 7.2. Make sure you have at least [version 1.9.3 of PEAR DB](https://github.com/pear/DB/releases) -installed. - ### Website reports "DB Error: insufficient permissions" -The user the webserver, e.g. Apache, runs under needs to have access to the Nominatim database. You can find the user like [this](https://serverfault.com/questions/125865/finding-out-what-user-apache-is-running-as), for default Ubuntu operating system for example it's `www-data`. +The user the webserver, e.g. Apache, runs under needs to have access to the +Nominatim database. You can find the user like +[this](https://serverfault.com/questions/125865/finding-out-what-user-apache-is-running-as), +for default Ubuntu operating system for example it's `www-data`. 1. Repeat the `createuser` step of the installation instructions. @@ -164,18 +165,8 @@ When running SELinux, make sure that the ### Setup.php fails with "DB Error: extension not found" Make sure you have the PostgreSQL extensions "hstore" and "postgis" installed. -See the installation instruction for a full list of required packages. - - -### Setup.php reports "Cannot redeclare getDB()" - -`Cannot redeclare getDB() (previously declared in /your/path/Nominatim/lib/db.php:4)` - -The message is a bit misleading as PHP needs to load the file `DB.php` and -instead re-loads Nominatim's `db.php`. To solve this make sure you -have the [Pear module 'DB'](https://pear.php.net/package/DB/) installed. +See the installation instructions for a full list of required packages. - sudo pear install DB ### I forgot to delete the flatnodes file before starting an import. diff --git a/docs/admin/Import-and-Update.md b/docs/admin/Import-and-Update.md index 757dab69..afda83b2 100644 --- a/docs/admin/Import-and-Update.md +++ b/docs/admin/Import-and-Update.md @@ -29,11 +29,11 @@ Add to your `settings/local.php`: @define('CONST_Osm2pgsql_Flatnode_File', '/path/to/flatnode.file'); Replace the second part with a suitable path on your system and make sure -the directory exists. There should be at least 40GB of free space. +the directory exists. There should be at least 64GB of free space. ## Downloading additional data -### Wikipedia rankings +### Wikipedia/Wikidata rankings Wikipedia can be used as an optional auxiliary data source to help indicate the importance of OSM features. Nominatim will work without this information @@ -41,15 +41,13 @@ but it will improve the quality of the results if this is installed. This data is available as a binary download: cd $NOMINATIM_SOURCE_DIR/data - wget https://www.nominatim.org/data/wikipedia_article.sql.bin - wget https://www.nominatim.org/data/wikipedia_redirect.sql.bin + wget https://www.nominatim.org/data/wikimedia-importance.sql.gz -Combined the 2 files are around 1.5GB and add around 30GB to the install -size of Nominatim. They also increase the install time by an hour or so. +The file is about 400MB and adds around 4GB to Nominatim database. -*NOTE:* you'll need to download the Wikipedia rankings before performing -the initial import of the data if you want the rankings applied to the -loaded data. +*NOTE:* if you forgot to download the wikipedia rankings, you can also add +them after the import by running `./utils/setup.php --import-wikipedia-articles` +and then `./utils/update.php --recompute-importance`. ### Great Britain, USA postcodes @@ -64,7 +62,7 @@ involve a GB or US postcode. This data can be optionally downloaded: In its default setup Nominatim is configured to import the full OSM data set for the entire planet. Such a setup requires a powerful machine with -at least 32GB of RAM and around 800GB of SSD hard disks. Depending on your +at least 64GB of RAM and around 800GB of SSD hard disks. Depending on your use case there are various ways to reduce the amount of data imported. This section discusses these methods. They can also be combined. @@ -126,7 +124,7 @@ The style can be changed with the configuration `CONST_Import_Style`. To give you an idea of the impact of using the different styles, the table below gives rough estimates of the final database size after import of a 2018 planet and after using the `--drop` option. It also shows the time -needed for the import on a machine with 32GB RAM, 4 CPUS and SSDs. Note that +needed for the import on a machine with 64GB RAM, 4 CPUS and SSDs. Note that the given sizes are just an estimate meant for comparison of style requirements. Your planet import is likely to be larger as the OSM data grows with time. @@ -137,7 +135,7 @@ street | 42h | 400 GB | 180 GB address | 59h | 500 GB | 260 GB full | 80h | 575 GB | 300 GB -You can also customize the styles further. For an description of the +You can also customize the styles further. For a description of the style format see [the development section](../develop/Import.md). ## Initial import of the data @@ -149,18 +147,44 @@ Download the data to import and load the data with the following command from the build directory: ```sh -./utils/setup.php --osm-file --all [--osm2pgsql-cache 28000] 2>&1 | tee setup.log +./utils/setup.php --osm-file --all 2>&1 | tee setup.log ``` -The `--osm2pgsql-cache` parameter is optional but strongly recommended for -planet imports. It sets the node cache size for the osm2pgsql import part -(see `-C` parameter in osm2pgsql help). As a rule of thumb, this should be -about the same size as the file you are importing but never more than -2/3 of RAM available. If your machine starts swapping reduce the size. - -Computing word frequency for search terms can improve the performance of -forward geocoding in particular under high load as it helps PostgreSQL's query -planner to make the right decisions. To recompute word counts run: +***Note for full planet imports:*** Even on a perfectly configured machine +the import of a full planet takes at least 2 days. Once you see messages +with `Rank .. ETA` appear, the indexing process has started. This part takes +the most time. There are 30 ranks to process. Rank 26 and 30 are the most complex. +They take each about a third of the total import time. If you have not reached +rank 26 after two days of import, it is worth revisiting your system +configuration as it may not be optimal for the import. + +### Notes on memory usage + +In the first step of the import Nominatim uses osm2pgsql to load the OSM data +into the PostgreSQL database. This step is very demanding in terms of RAM usage. +osm2pgsql and PostgreSQL are running in parallel at this point. PostgreSQL +blocks at least the part of RAM that has been configured with the +`shared_buffers` parameter during [PostgreSQL tuning](Installation#PostgreSQL_tuning) +and needs some memory on top of that. osm2pgsql needs at least 2GB of RAM for +its internal data structures, potentially more when it has to process very large +relations. In addition it needs to maintain a cache for node locations. The size +of this cache can be configured with the parameter `--osm2pgsql-cache`. + +When importing with a flatnode file, it is best to disable the node cache +completely and leave the memory for the flatnode file. Nominatim will do this +by default, so you do not need to configure anything in this case. + +For imports without a flatnode file, set `--osm2pgsql-cache` approximately to +the size of the OSM pbf file (in MB) you are importing. Make sure you leave +enough RAM for PostgreSQL and osm2pgsql as mentioned above. If the system starts +swapping or you are getting out-of-memory errors, reduce the cache size or +even consider using a flatnode file. + +## Tuning the database + +Accurate word frequency information for search terms helps PostgreSQL's query +planner to make the right decisions. Recomputing them can improve the performance +of forward geocoding in particular under high load. To recompute word counts run: ```sh ./utils/update.php --recompute-word-counts @@ -178,7 +202,8 @@ you also need to enable these key phrases like this: ./utils/specialphrases.php --wiki-import > specialphrases.sql psql -d nominatim -f specialphrases.sql -Note that this command downloads the phrases from the wiki link above. +Note that this command downloads the phrases from the wiki link above. You +need internet access for the step. ## Installing Tiger housenumber data for the US diff --git a/docs/admin/Installation.md b/docs/admin/Installation.md index d0724927..89afd2bc 100644 --- a/docs/admin/Installation.md +++ b/docs/admin/Installation.md @@ -29,7 +29,8 @@ For compiling: * a recent C++ compiler Nominatim comes with its own version of osm2pgsql. See the -osm2pgsql README for additional dependencies required for compiling osm2pgsql. +[osm2pgsql README](https://github.com/openstreetmap/osm2pgsql/blob/master/README.md#building) +for additional dependencies required for compiling osm2pgsql. For running tests: @@ -45,7 +46,6 @@ For running Nominatim: * [PHP](https://php.net) (7.0 or later) * PHP-pgsql * PHP-intl (bundled with PHP) - * [PEAR::DB](https://pear.php.net/package/DB) * a webserver (apache or nginx are recommended) For running continuous updates: @@ -55,15 +55,15 @@ For running continuous updates: ### Hardware A minimum of 2GB of RAM is required or installation will fail. For a full -planet import 32GB of RAM or more are strongly recommended. +planet import 64GB of RAM or more are strongly recommended. Do not report +out of memory problems if you have less than 64GB RAM. -For a full planet install you will need at least 700GB of hard disk space +For a full planet install you will need at least 800GB of hard disk space (take into account that the OSM database is growing fast). SSD disks will help considerably to speed up import and queries. -On a 6-core machine with 32GB RAM and SSDs the import of a full planet takes -a bit more than 2 days. Without SSDs 7-8 days are more realistic. - +Even on a well configured machine the import of a full planet takes +at least 2 days. Without SSDs 7-8 days are more realistic. ## Setup of the server @@ -73,17 +73,21 @@ You might want to tune your PostgreSQL installation so that the later steps make best use of your hardware. You should tune the following parameters in your `postgresql.conf` file. - shared_buffers (2GB) - maintenance_work_mem (10GB) - work_mem (50MB) - effective_cache_size (24GB) + shared_buffers = 2GB + maintenance_work_mem = (10GB) + autovacuum_work_mem = 2GB + work_mem = (50MB) + effective_cache_size = (24GB) synchronous_commit = off checkpoint_segments = 100 # only for postgresql <= 9.4 + max_wal_size = 1GB # postgresql > 9.4 checkpoint_timeout = 10min checkpoint_completion_target = 0.9 The numbers in brackets behind some parameters seem to work fine for -32GB RAM machine. Adjust to your setup. +64GB RAM machine. Adjust to your setup. A higher number for `max_wal_size` +means that PostgreSQL needs to run checkpoints less often but it does require +the additional space on your disk. For the initial import, you should also set: diff --git a/docs/admin/Migration.md b/docs/admin/Migration.md index f3668357..e6b6d102 100644 --- a/docs/admin/Migration.md +++ b/docs/admin/Migration.md @@ -6,6 +6,21 @@ to newer versions of Nominatim. SQL statements should be executed from the PostgreSQL commandline. Execute `psql nominatim` to enter command line mode. +## 3.4.0 -> master + +### New Wikipedia/Wikidata importance tables + +The `wikipedia_*` tables have a new format that also includes references to +Wikidata. You need to update the computation functions and the tables as +follows: + + * download the new Wikipedia tables as described in the import section + * reimport the tables: `./utils/setup.php --import-wikipedia-articles` + * update the functions: `./utils/setup.php --create-functions --enable-diff-updates` + * compute importance: `./utils/update.php --recompute-importance` + +The last step takes about 10 hours on the full planet. + ## 3.3.0 -> 3.4.0 ### Reorganisation of location_area_country table diff --git a/docs/api/Output.md b/docs/api/Output.md index df78ed7d..a0d85220 100644 --- a/docs/api/Output.md +++ b/docs/api/Output.md @@ -46,9 +46,9 @@ a single place (for reverse) of the following format: The possible fields are: - * `place_id` - reference to the Nominatim internal database ID (see notes below) + * `place_id` - reference to the Nominatim internal database ID ([see notes](#place_id-is-not-a-persistent-id)) * `osm_type`, `osm_id` - reference to the OSM object - * `boundingbox` - area of corner coordinates + * `boundingbox` - area of corner coordinates ([see notes](#boundingbox)) * `lat`, `lon` - latitude and longitude of the centroid of the object * `display_name` - full comma-separated address * `class`, `type` - key and value of the main OSM tag @@ -75,7 +75,7 @@ a bounding box (`bbox`). The feature list has the following fields: - * `place_id` - reference to the Nominatim internal database ID (see notes below) + * `place_id` - reference to the Nominatim internal database ID ([see notes](#place_id-is-not-a-persistent-id)) * `osm_type`, `osm_id` - reference to the OSM object * `category`, `type` - key and value of the main OSM tag * `display_name` - full comma-separated address @@ -148,11 +148,11 @@ attribution to OSM and the original querystring. The place information can be found in the `result` element. The attributes of that element contain: - * `place_id` - reference to the Nominatim internal database ID (see notes below) + * `place_id` - reference to the Nominatim internal database ID ([see notes](#place_id-is-not-a-persistent-id)) * `osm_type`, `osm_id` - reference to the OSM object * `ref` - content of `ref` tag if it exists * `lat`, `lon` - latitude and longitude of the centroid of the object - * `boundingbox` - comma-separated list of corner coordinates + * `boundingbox` - comma-separated list of corner coordinates ([see notes](#boundingbox)) The full address of the result can be found in the content of the `result` element as a comma-separated list. @@ -203,11 +203,11 @@ generic information about the query: The place information can be found in the `place` elements, of which there may be more than one. The attributes of that element contain: - * `place_id` - reference to the Nominatim internal database ID (see notes below) + * `place_id` - reference to the Nominatim internal database ID ([see notes](#place_id-is-not-a-persistent-id)) * `osm_type`, `osm_id` - reference to the OSM object * `ref` - content of `ref` tag if it exists * `lat`, `lon` - latitude and longitude of the centroid of the object - * `boundingbox` - comma-separated list of corner coordinates + * `boundingbox` - comma-separated list of corner coordinates ([see notes](#boundingbox)) * `place_rank` - class search rank * `display_name` - full comma-separated address * `class`, `type` - key and value of the main OSM tag @@ -244,3 +244,10 @@ relation) so `osm_type`+`osm_id`+`class_name` would be more unique. Comma separated list of min latitude, max latitude, min longitude, max longitude. The whole planet would be `-90,90,-180,180`. + +Can we used to pan and center the map on the result, for example with leafletjs +mapping library +`map.fitBounds([[bbox[0],bbox[2]],[bbox[1],bbox[3]]], {padding: [20, 20], maxzoom: 16});` + +Bounds crossing the antimeridian have a min latitude -180 and max latitude 180, +essentially covering the planet (See [issue 184](https://github.com/openstreetmap/Nominatim/issues/184)). diff --git a/docs/develop/Documentation.md b/docs/develop/Documentation.md index 6e792c25..df8d2b1a 100644 --- a/docs/develop/Documentation.md +++ b/docs/develop/Documentation.md @@ -1,36 +1,36 @@ # Documentation Pages -The [Nominatim documentation](https://nominatim.org/release-docs/develop/) is built using the [MkDocs](https://www.mkdocs.org/) static site generation framework. The master branch is automatically deployed every night on under [https://nominatim.org/release-docs/develop/]() +The [Nominatim documentation](https://nominatim.org/release-docs/develop/) is built using the [MkDocs](https://www.mkdocs.org/) static site generation framework. The master branch is automatically deployed every night on under [https://nominatim.org/release-docs/develop/](https://nominatim.org/release-docs/develop/) -To preview local changes: +To preview local changes, first install MkDocs -1. Install MkDocs +``` +pip3 install --user mkdocs +``` - ``` - pip3 install --user mkdocs - ``` +Then go to the build directory and run -2. In build directory run +``` +make doc +INFO - Cleaning site directory +INFO - Building documentation to directory: /home/vagrant/build/site-html +``` - ``` - make doc - INFO - Cleaning site directory - INFO - Building documentation to directory: /home/vagrant/build/site-html - ``` +This runs `mkdocs build` plus extra transformation of some files and adds +symlinks (see `CMakeLists.txt` for the exact steps). - This runs `mkdocs build` plus extra transformion of some files and adds symlinks (see `CMakeLists.txt` for the exact steps). +Now you can start webserver for local testing +``` +build> mkdocs serve +[server:296] Serving on http://127.0.0.1:8000 +[handlers:62] Start watching changes +``` -3. Start webserver for local testing +If you develop inside a Vagrant virtual machine: - ``` - mkdocs serve - [server:296] Serving on http://127.0.0.1:8000 - [handlers:62] Start watching changes - ``` - - If you develop inside a Vagrant virtual machine: - * add port forwarding to your Vagrantfile, e.g. `config.vm.network "forwarded_port", guest: 8000, host: 8000` - * use `mkdocs serve --dev-addr 0.0.0.0:8000` because the default localhost - IP does not get forwarded. + * add port forwarding to your Vagrantfile, + e.g. `config.vm.network "forwarded_port", guest: 8000, host: 8000` + * use `mkdocs serve --dev-addr 0.0.0.0:8000` because the default localhost + IP does not get forwarded. diff --git a/docs/develop/Postcodes.md b/docs/develop/Postcodes.md new file mode 100644 index 00000000..ff36b0dd --- /dev/null +++ b/docs/develop/Postcodes.md @@ -0,0 +1,45 @@ +# Postcodes in Nominatim + +The blog post +[Nominatim and Postcodes](https://www.openstreetmap.org/user/lonvia/diary/43143) +describes the handling implemented since Nominatim 3.1. + +Postcode centroids (aka 'calculated postcodes') are generated by looking at all +postcodes of a country, grouping them and calculating the geometric centroid. +There is currently no logic to deal with extreme outliers (typos or other +mistakes in OSM data). There is also no check if a postcodes adheres to a +country's format, e.g. if Swiss postcodes are 4 digits. + + +## Regular updating calculated postcodes + +The script to rerun the calculation is +`build/utils/update.php --calculate-postcodes` +and runs once per night on nominatim.openstreetmap.org. + + +## Finding places that share a specific postcode + +In the Nominatim database run + +```sql +SELECT address->'postcode' as pc, + osm_type, osm_id, class, type, + st_x(centroid) as lon, st_y(centroid) as lat +FROM placex +WHERE country_code='fr' + AND upper(trim (both ' ' from address->'postcode')) = '33210'; +``` + +Alternatively on [Overpass](https://overpass-turbo.eu/) run the following query + +``` +[out:json][timeout:250]; +area["name"="France"]->.boundaryarea; +( +nwr(area.boundaryarea)["addr:postcode"="33210"]; +); +out body; +>; +out skel qt; +``` diff --git a/docs/mkdocs.yml b/docs/mkdocs.yml index cb00e2e8..5cf63f83 100644 --- a/docs/mkdocs.yml +++ b/docs/mkdocs.yml @@ -22,6 +22,7 @@ pages: - 'Overview' : 'develop/overview.md' - 'OSM Data Import' : 'develop/Import.md' - 'Place Ranking' : 'develop/Ranking.md' + - 'Postcodes' : 'develop/Postcodes.md' - 'Documentation' : 'develop/Documentation.md' - 'External Data Sources': - 'Overview' : 'data-sources/overview.md' diff --git a/lib/DB.php b/lib/DB.php index fe2529b2..e4aa4349 100644 --- a/lib/DB.php +++ b/lib/DB.php @@ -284,7 +284,7 @@ class DB { // https://secure.php.net/manual/en/ref.pdo-pgsql.connection.php $aInfo = array(); - if (preg_match('/^pgsql:(.+)/', $sDSN, $aMatches)) { + if (preg_match('/^pgsql:(.+)$/', $sDSN, $aMatches)) { foreach (explode(';', $aMatches[1]) as $sKeyVal) { list($sKey, $sVal) = explode('=', $sKeyVal, 2); if ($sKey == 'host') $sKey = 'hostspec'; diff --git a/lib/cmd.php b/lib/cmd.php index 32fdc857..77878c15 100644 --- a/lib/cmd.php +++ b/lib/cmd.php @@ -148,12 +148,14 @@ function runSQLScript($sScript, $bfatal = true, $bVerbose = false, $bIgnoreError // Convert database DSN to psql parameters $aDSNInfo = \Nominatim\DB::parseDSN(CONST_Database_DSN); if (!isset($aDSNInfo['port']) || !$aDSNInfo['port']) $aDSNInfo['port'] = 5432; - $sCMD = 'psql -p '.$aDSNInfo['port'].' -d '.$aDSNInfo['database']; + $sCMD = 'psql' + .' -p '.escapeshellarg($aDSNInfo['port']) + .' -d '.escapeshellarg($aDSNInfo['database']); if (isset($aDSNInfo['hostspec']) && $aDSNInfo['hostspec']) { - $sCMD .= ' -h ' . $aDSNInfo['hostspec']; + $sCMD .= ' -h ' . escapeshellarg($aDSNInfo['hostspec']); } if (isset($aDSNInfo['username']) && $aDSNInfo['username']) { - $sCMD .= ' -U ' . $aDSNInfo['username']; + $sCMD .= ' -U ' . escapeshellarg($aDSNInfo['username']); } $aProcEnv = null; if (isset($aDSNInfo['password']) && $aDSNInfo['password']) { diff --git a/lib/setup/SetupClass.php b/lib/setup/SetupClass.php index fd6805fb..12a1bb0d 100755 --- a/lib/setup/SetupClass.php +++ b/lib/setup/SetupClass.php @@ -29,10 +29,13 @@ class SetupFunctions warn('resetting threads to '.$this->iInstances); } - // Assume we can steal all the cache memory in the box (unless told otherwise) if (isset($aCMDResult['osm2pgsql-cache'])) { $this->iCacheMemory = $aCMDResult['osm2pgsql-cache']; + } elseif (!is_null(CONST_Osm2pgsql_Flatnode_File)) { + // When flatnode files are enabled then disable cache per default. + $this->iCacheMemory = 0; } else { + // Otherwise: Assume we can steal all the cache memory in the box. $this->iCacheMemory = getCacheMemoryMB(); } @@ -80,13 +83,15 @@ class SetupFunctions fail('database already exists ('.CONST_Database_DSN.')'); } - $sCreateDBCmd = 'createdb -E UTF-8 -p '.$this->aDSNInfo['port'].' '.$this->aDSNInfo['database']; + $sCreateDBCmd = 'createdb -E UTF-8' + .' -p '.escapeshellarg($this->aDSNInfo['port']) + .' '.escapeshellarg($this->aDSNInfo['database']); if (isset($this->aDSNInfo['username'])) { - $sCreateDBCmd .= ' -U '.$this->aDSNInfo['username']; + $sCreateDBCmd .= ' -U '.escapeshellarg($this->aDSNInfo['username']); } if (isset($this->aDSNInfo['hostspec'])) { - $sCreateDBCmd .= ' -h '.$this->aDSNInfo['hostspec']; + $sCreateDBCmd .= ' -h '.escapeshellarg($this->aDSNInfo['hostspec']); } $result = $this->runWithPgEnv($sCreateDBCmd); @@ -158,13 +163,6 @@ class SetupFunctions if ($this->bNoPartitions) { $this->pgsqlRunScript('update country_name set partition = 0'); } - - // the following will be needed by createFunctions later but - // is only defined in the subsequently called createTables - // Create dummies here that will be overwritten by the proper - // versions in create-tables. - $this->pgsqlRunScript('CREATE TABLE IF NOT EXISTS place_boundingbox ()'); - $this->pgsqlRunScript('CREATE TYPE wikipedia_article_match AS ()', false); } public function importData($sOSMFile) @@ -178,30 +176,30 @@ class SetupFunctions fail("osm2pgsql not found in '$osm2pgsql'"); } - $osm2pgsql .= ' -S '.CONST_Import_Style; + $osm2pgsql .= ' -S '.escapeshellarg(CONST_Import_Style); if (!is_null(CONST_Osm2pgsql_Flatnode_File) && CONST_Osm2pgsql_Flatnode_File) { - $osm2pgsql .= ' --flat-nodes '.CONST_Osm2pgsql_Flatnode_File; + $osm2pgsql .= ' --flat-nodes '.escapeshellarg(CONST_Osm2pgsql_Flatnode_File); } if (CONST_Tablespace_Osm2pgsql_Data) - $osm2pgsql .= ' --tablespace-slim-data '.CONST_Tablespace_Osm2pgsql_Data; + $osm2pgsql .= ' --tablespace-slim-data '.escapeshellarg(CONST_Tablespace_Osm2pgsql_Data); if (CONST_Tablespace_Osm2pgsql_Index) - $osm2pgsql .= ' --tablespace-slim-index '.CONST_Tablespace_Osm2pgsql_Index; + $osm2pgsql .= ' --tablespace-slim-index '.escapeshellarg(CONST_Tablespace_Osm2pgsql_Index); if (CONST_Tablespace_Place_Data) - $osm2pgsql .= ' --tablespace-main-data '.CONST_Tablespace_Place_Data; + $osm2pgsql .= ' --tablespace-main-data '.escapeshellarg(CONST_Tablespace_Place_Data); if (CONST_Tablespace_Place_Index) - $osm2pgsql .= ' --tablespace-main-index '.CONST_Tablespace_Place_Index; + $osm2pgsql .= ' --tablespace-main-index '.escapeshellarg(CONST_Tablespace_Place_Index); $osm2pgsql .= ' -lsc -O gazetteer --hstore --number-processes 1'; - $osm2pgsql .= ' -C '.$this->iCacheMemory; - $osm2pgsql .= ' -P '.$this->aDSNInfo['port']; + $osm2pgsql .= ' -C '.escapeshellarg($this->iCacheMemory); + $osm2pgsql .= ' -P '.escapeshellarg($this->aDSNInfo['port']); if (isset($this->aDSNInfo['username'])) { - $osm2pgsql .= ' -U '.$this->aDSNInfo['username']; + $osm2pgsql .= ' -U '.escapeshellarg($this->aDSNInfo['username']); } if (isset($this->aDSNInfo['hostspec'])) { - $osm2pgsql .= ' -H '.$this->aDSNInfo['hostspec']; + $osm2pgsql .= ' -H '.escapeshellarg($this->aDSNInfo['hostspec']); } - $osm2pgsql .= ' -d '.$this->aDSNInfo['database'].' '.$sOSMFile; + $osm2pgsql .= ' -d '.escapeshellarg($this->aDSNInfo['database']).' '.escapeshellarg($sOSMFile); $this->runWithPgEnv($osm2pgsql); @@ -321,19 +319,14 @@ class SetupFunctions public function importWikipediaArticles() { - $sWikiArticlesFile = CONST_Wikipedia_Data_Path.'/wikipedia_article.sql.bin'; - $sWikiRedirectsFile = CONST_Wikipedia_Data_Path.'/wikipedia_redirect.sql.bin'; + $sWikiArticlesFile = CONST_Wikipedia_Data_Path.'/wikimedia-importance.sql.gz'; if (file_exists($sWikiArticlesFile)) { - info('Importing wikipedia articles'); - $this->pgsqlRunDropAndRestore($sWikiArticlesFile); - } else { - warn('wikipedia article dump file not found - places will have default importance'); - } - if (file_exists($sWikiRedirectsFile)) { - info('Importing wikipedia redirects'); - $this->pgsqlRunDropAndRestore($sWikiRedirectsFile); + info('Importing wikipedia articles and redirects'); + $this->pgExec('DROP TABLE IF EXISTS wikipedia_article'); + $this->pgExec('DROP TABLE IF EXISTS wikipedia_redirect'); + $this->pgsqlRunScriptFile($sWikiArticlesFile); } else { - warn('wikipedia redirect dump file not found - some place importance values may be missing'); + warn('wikipedia importance dump file not found - places will have default importance'); } } @@ -349,8 +342,6 @@ class SetupFunctions echo '.'; $this->pgExec('TRUNCATE place_addressline'); echo '.'; - $this->pgExec('TRUNCATE place_boundingbox'); - echo '.'; $this->pgExec('TRUNCATE location_area'); echo '.'; if (!$this->dbReverseOnly()) { @@ -468,6 +459,10 @@ class SetupFunctions { info('Import Tiger data'); + $aFilenames = glob(CONST_Tiger_Data_Path.'/*.sql'); + info('Found '.count($aFilenames).' SQL files in path '.CONST_Tiger_Data_Path); + if (empty($aFilenames)) return; + $sTemplate = file_get_contents(CONST_BasePath.'/sql/tiger_import_start.sql'); $sTemplate = str_replace('{www-user}', CONST_Database_Web_User, $sTemplate); $sTemplate = $this->replaceTablespace( @@ -492,7 +487,7 @@ class SetupFunctions pg_ping($aDBInstances[$i]); } - foreach (glob(CONST_Tiger_Data_Path.'/*.sql') as $sFile) { + foreach ($aFilenames as $sFile) { echo $sFile.': '; $hFile = fopen($sFile, 'r'); $sSQL = fgets($hFile, 100000); @@ -595,13 +590,15 @@ class SetupFunctions public function index($bIndexNoanalyse) { $sOutputFile = ''; - $sBaseCmd = CONST_InstallPath.'/nominatim/nominatim -i -d '.$this->aDSNInfo['database'].' -P ' - .$this->aDSNInfo['port'].' -t '.$this->iInstances.$sOutputFile; + $sBaseCmd = CONST_InstallPath.'/nominatim/nominatim -i' + .' -d '.escapeshellarg($this->aDSNInfo['database']) + .' -P '.escapeshellarg($this->aDSNInfo['port']) + .' -t '.escapeshellarg($this->iInstances.$sOutputFile); if (isset($this->aDSNInfo['hostspec'])) { - $sBaseCmd .= ' -H '.$this->aDSNInfo['hostspec']; + $sBaseCmd .= ' -H '.escapeshellarg($this->aDSNInfo['hostspec']); } if (isset($this->aDSNInfo['username'])) { - $sBaseCmd .= ' -U '.$this->aDSNInfo['username']; + $sBaseCmd .= ' -U '.escapeshellarg($this->aDSNInfo['username']); } info('Index ranks 0 - 4'); @@ -736,22 +733,6 @@ class SetupFunctions } } - private function pgsqlRunDropAndRestore($sDumpFile) - { - $sCMD = 'pg_restore -p '.$this->aDSNInfo['port'].' -d '.$this->aDSNInfo['database'].' --no-owner -Fc --clean '.$sDumpFile; - if ($this->oDB->getPostgresVersion() >= 9.04) { - $sCMD .= ' --if-exists'; - } - if (isset($this->aDSNInfo['hostspec'])) { - $sCMD .= ' -h '.$this->aDSNInfo['hostspec']; - } - if (isset($this->aDSNInfo['username'])) { - $sCMD .= ' -U '.$this->aDSNInfo['username']; - } - - $this->runWithPgEnv($sCMD); - } - private function pgsqlRunScript($sScript, $bfatal = true) { runSQLScript( @@ -810,15 +791,17 @@ class SetupFunctions { if (!file_exists($sFilename)) fail('unable to find '.$sFilename); - $sCMD = 'psql -p '.$this->aDSNInfo['port'].' -d '.$this->aDSNInfo['database']; + $sCMD = 'psql' + .' -p '.escapeshellarg($this->aDSNInfo['port']) + .' -d '.escapeshellarg($this->aDSNInfo['database']); if (!$this->bVerbose) { $sCMD .= ' -q'; } if (isset($this->aDSNInfo['hostspec'])) { - $sCMD .= ' -h '.$this->aDSNInfo['hostspec']; + $sCMD .= ' -h '.escapeshellarg($this->aDSNInfo['hostspec']); } if (isset($this->aDSNInfo['username'])) { - $sCMD .= ' -U '.$this->aDSNInfo['username']; + $sCMD .= ' -U '.escapeshellarg($this->aDSNInfo['username']); } $aProcEnv = null; if (isset($this->aDSNInfo['password'])) { @@ -831,12 +814,12 @@ class SetupFunctions 1 => array('pipe', 'w'), 2 => array('file', '/dev/null', 'a') ); - $hGzipProcess = proc_open('zcat '.$sFilename, $aDescriptors, $ahGzipPipes); + $hGzipProcess = proc_open('zcat '.escapeshellarg($sFilename), $aDescriptors, $ahGzipPipes); if (!is_resource($hGzipProcess)) fail('unable to start zcat'); $aReadPipe = $ahGzipPipes[1]; fclose($ahGzipPipes[0]); } else { - $sCMD .= ' -f '.$sFilename; + $sCMD .= ' -f '.escapeshellarg($sFilename); $aReadPipe = array('pipe', 'r'); } $aDescriptors = array( diff --git a/lib/template/details-index-html.php b/lib/template/details-index-html.php new file mode 100644 index 00000000..640a212c --- /dev/null +++ b/lib/template/details-index-html.php @@ -0,0 +1,55 @@ + + + + + + + + +
+
+
+ +

Show details for place

+ +
+

Search by place id

+ +
+ + +
+
+ +
+

Search by OSM type and OSM id

+ +
+ + + + +
+
+ +
+

Search by openstreetmap.org URL

+ +
+ + + + +
+
+ +
+
+
+ + + + + diff --git a/lib/template/includes/html-header.php b/lib/template/includes/html-header.php index 942af095..9a914bf5 100644 --- a/lib/template/includes/html-header.php +++ b/lib/template/includes/html-header.php @@ -6,7 +6,6 @@ - diff --git a/sql/functions.sql b/sql/functions.sql index f3bff89c..ad2007cb 100644 --- a/sql/functions.sql +++ b/sql/functions.sql @@ -1358,10 +1358,9 @@ BEGIN END LOOP; NEW.importance := null; - select language||':'||title,importance from get_wikipedia_match(NEW.extratags, NEW.country_code) INTO NEW.wikipedia,NEW.importance; - IF NEW.importance IS NULL THEN - select language||':'||title,importance from wikipedia_article where osm_type = NEW.osm_type and osm_id = NEW.osm_id order by importance desc limit 1 INTO NEW.wikipedia,NEW.importance; - END IF; + SELECT wikipedia, importance + FROM compute_importance(NEW.extratags, NEW.country_code, NEW.osm_type, NEW.osm_id) + INTO NEW.wikipedia,NEW.importance; --DEBUG: RAISE WARNING 'Importance computed from wikipedia: %', NEW.importance; @@ -1600,9 +1599,10 @@ BEGIN -- mark the linked place (excludes from search results) UPDATE placex set linked_place_id = NEW.place_id where place_id = linkedPlacex.place_id; - -- keep a note of the node id in case we need it for wikipedia in a bit - linked_node_id := linkedPlacex.osm_id; - select language||':'||title,importance from get_wikipedia_match(linkedPlacex.extratags, NEW.country_code) INTO linked_wikipedia,linked_importance; + select wikipedia, importance + FROM compute_importance(linkedPlacex.extratags, NEW.country_code, + 'N', linkedPlacex.osm_id) + INTO linked_wikipedia,linked_importance; --DEBUG: RAISE WARNING 'Linked label member'; END LOOP; @@ -1639,9 +1639,10 @@ BEGIN -- mark the linked place (excludes from search results) UPDATE placex set linked_place_id = NEW.place_id where place_id = linkedPlacex.place_id; - -- keep a note of the node id in case we need it for wikipedia in a bit - linked_node_id := linkedPlacex.osm_id; - select language||':'||title,importance from get_wikipedia_match(linkedPlacex.extratags, NEW.country_code) INTO linked_wikipedia,linked_importance; + select wikipedia, importance + FROM compute_importance(linkedPlacex.extratags, NEW.country_code, + 'N', linkedPlacex.osm_id) + INTO linked_wikipedia,linked_importance; --DEBUG: RAISE WARNING 'Linked admin_center'; END IF; @@ -1684,9 +1685,10 @@ BEGIN -- mark the linked place (excludes from search results) UPDATE placex set linked_place_id = NEW.place_id where place_id = linkedPlacex.place_id; - -- keep a note of the node id in case we need it for wikipedia in a bit - linked_node_id := linkedPlacex.osm_id; - select language||':'||title,importance from get_wikipedia_match(linkedPlacex.extratags, NEW.country_code) INTO linked_wikipedia,linked_importance; + select wikipedia, importance + FROM compute_importance(linkedPlacex.extratags, NEW.country_code, + 'N', linkedPlacex.osm_id) + INTO linked_wikipedia,linked_importance; --DEBUG: RAISE WARNING 'Linked named place'; END LOOP; END IF; @@ -1714,13 +1716,6 @@ BEGIN (NEW.importance is null or NEW.importance < linked_importance) THEN NEW.importance = linked_importance; END IF; - - -- Still null? how about looking it up by the node id - IF NEW.importance IS NULL THEN - --DEBUG: RAISE WARNING 'Looking up importance by linked node id'; - select language||':'||title,importance from wikipedia_article where osm_type = 'N'::char(1) and osm_id = linked_node_id order by importance desc limit 1 INTO NEW.wikipedia,NEW.importance; - END IF; - END IF; -- make sure all names are in the word table @@ -2627,7 +2622,7 @@ END; $$ LANGUAGE plpgsql IMMUTABLE; -DROP TYPE wikipedia_article_match CASCADE; +DROP TYPE IF EXISTS wikipedia_article_match CASCADE; create type wikipedia_article_match as ( language TEXT, title TEXT, @@ -2684,6 +2679,42 @@ END; $$ LANGUAGE plpgsql; +DROP TYPE IF EXISTS place_importance CASCADE; +create type place_importance as ( + importance FLOAT, + wikipedia TEXT +); + +CREATE OR REPLACE FUNCTION compute_importance(extratags HSTORE, country_code varchar(2), osm_type varchar(1), osm_id BIGINT) + RETURNS place_importance + AS $$ +DECLARE + match RECORD; + result place_importance; +BEGIN + FOR match IN SELECT * FROM get_wikipedia_match(extratags, country_code) + WHERE language is not NULL + LOOP + result.importance := match.importance; + result.wikipedia := match.language || ':' || match.title; + RETURN result; + END LOOP; + + IF extratags ? 'wikidata' THEN + FOR match IN SELECT * FROM wikipedia_article + WHERE wd_page_title = extratags->'wikidata' + ORDER BY language = 'en' DESC, langcount DESC LIMIT 1 LOOP + result.importance := match.importance; + result.wikipedia := match.language || ':' || match.title; + RETURN result; + END LOOP; + END IF; + + RETURN null; +END; +$$ +LANGUAGE plpgsql; + CREATE OR REPLACE FUNCTION quad_split_geometry(geometry GEOMETRY, maxarea FLOAT, maxdepth INTEGER) RETURNS SETOF GEOMETRY AS $$ diff --git a/sql/tables.sql b/sql/tables.sql index 0559abd4..0245e3c3 100644 --- a/sql/tables.sql +++ b/sql/tables.sql @@ -268,7 +268,9 @@ CREATE TABLE wikipedia_article ( lon double precision, importance double precision, osm_type character(1), - osm_id bigint + osm_id bigint, + wd_page_title text, + instance_of text ); ALTER TABLE ONLY wikipedia_article ADD CONSTRAINT wikipedia_article_pkey PRIMARY KEY (language, title); CREATE INDEX idx_wikipedia_article_osm_id ON wikipedia_article USING btree (osm_type, osm_id); diff --git a/test/bdd/api/search/params.feature b/test/bdd/api/search/params.feature index 23a86705..cdea3f69 100644 --- a/test/bdd/api/search/params.feature +++ b/test/bdd/api/search/params.feature @@ -119,13 +119,13 @@ Feature: Search queries | en | Then result addresses contain | ID | state | - | 0 | Salto | + | 0 | Florida | When sending json search query "25 de Mayo" with address | accept-language | viewbox | - | en | -56.35879,-34.18330,-56.31618,-34.20815 | + | en | -57.95468,-31.39261,-57.94741,-31.39490 | Then result addresses contain | ID | state | - | 0 | Florida | + | 0 | Salto | Scenario: viewboxes cannot be points When sending json search query "foo" diff --git a/test/bdd/environment.py b/test/bdd/environment.py index b12b6481..f0658c33 100644 --- a/test/bdd/environment.py +++ b/test/bdd/environment.py @@ -82,6 +82,7 @@ class NominatimEnvironment(object): (';password=' + self.db_pass) if self.db_pass else '' )) f.write("@define('CONST_Osm2pgsql_Flatnode_File', null);\n") + f.write("@define('CONST_Import_Style', CONST_BasePath.'/settings/import-full.style');\n") f.close() def cleanup(self): diff --git a/test/testdb/wikimedia-importance.sql.gz b/test/testdb/wikimedia-importance.sql.gz new file mode 100644 index 00000000..1024f725 Binary files /dev/null and b/test/testdb/wikimedia-importance.sql.gz differ diff --git a/test/testdb/wikipedia_article.sql.bin b/test/testdb/wikipedia_article.sql.bin deleted file mode 100644 index 628e2af4..00000000 Binary files a/test/testdb/wikipedia_article.sql.bin and /dev/null differ diff --git a/test/testdb/wikipedia_redirect.sql.bin b/test/testdb/wikipedia_redirect.sql.bin deleted file mode 100644 index 9c4b513d..00000000 Binary files a/test/testdb/wikipedia_redirect.sql.bin and /dev/null differ diff --git a/utils/update.php b/utils/update.php index c3620b06..4317367a 100644 --- a/utils/update.php +++ b/utils/update.php @@ -42,6 +42,7 @@ $aCMDOptions array('deduplicate', '', 0, 1, 0, 0, 'bool', 'Deduplicate tokens'), array('recompute-word-counts', '', 0, 1, 0, 0, 'bool', 'Compute frequency of full-word search terms'), array('update-address-levels', '', 0, 1, 0, 0, 'bool', 'Reimport address level configuration (EXPERT)'), + array('recompute-importance', '', 0, 1, 0, 0, 'bool', 'Recompute place importances'), array('no-npi', '', 0, 1, 0, 0, 'bool', '(obsolete)'), ); @@ -320,6 +321,23 @@ if ($aResult['update-address-levels']) { $oAlParser->createTable($oDB, 'address_levels'); } +if ($aResult['recompute-importance']) { + echo "Updating importance values for database.\n"; + $oDB = new Nominatim\DB(); + $oDB->connect(); + + $sSQL = 'ALTER TABLE placex DISABLE TRIGGER ALL;'; + $sSQL .= 'UPDATE placex SET (wikipedia, importance) ='; + $sSQL .= ' (SELECT wikipedia, importance'; + $sSQL .= ' FROM compute_importance(extratags, country_code, osm_type, osm_id));'; + $sSQL .= 'UPDATE placex s SET wikipedia = d.wikipedia, importance = d.importance'; + $sSQL .= ' FROM placex d'; + $sSQL .= ' WHERE s.place_id = d.linked_place_id and d.wikipedia is not null'; + $sSQL .= ' and (s.wikipedia is null or s.importance < d.importance);'; + $sSQL .= 'ALTER TABLE placex ENABLE TRIGGER ALL;'; + $oDB->exec($sSQL); +} + if ($aResult['import-osmosis'] || $aResult['import-osmosis-all']) { // if (strpos(CONST_Replication_Url, 'download.geofabrik.de') !== false && CONST_Replication_Update_Interval < 86400) { diff --git a/vagrant/Install-on-Centos-7.sh b/vagrant/Install-on-Centos-7.sh index 19b7ff84..b456fb74 100755 --- a/vagrant/Install-on-Centos-7.sh +++ b/vagrant/Install-on-Centos-7.sh @@ -17,12 +17,20 @@ sudo yum install -y epel-release +# More repositories for postgresql 11 (CentOS default 'postgresql' is 9.2), postgis +# and llvm-toolset (https://github.com/theory/pg-semver/issues/35) + + sudo yum install -y https://download.postgresql.org/pub/repos/yum/11/redhat/rhel-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm + sudo yum install -y centos-release-scl-rh + # Now you can install all packages needed for Nominatim: #DOCS: :::sh - sudo yum install -y postgresql-server postgresql-contrib postgresql-devel \ - postgis postgis-utils \ + + sudo yum install -y postgresql11-server postgresql11-contrib postgresql11-devel \ + postgis25_11 postgis25_11-utils \ wget git cmake make gcc gcc-c++ libtool policycoreutils-python \ + devtoolset-7 llvm-toolset-7 \ php-pgsql php php-intl libpqxx-devel \ proj-epsg bzip2-devel proj-devel libxml2-devel boost-devel \ expat-devel zlib-devel @@ -77,8 +85,11 @@ sudo chown vagrant /srv/nominatim #DOCS: # CentOS does not automatically create a database cluster. Therefore, start # with initializing the database, then enable the server to start at boot: - sudo postgresql-setup initdb - sudo systemctl enable postgresql + echo 'PATH=$PATH:/usr/pgsql-11/bin' > .bash_profile + source .bash_profile + + sudo /usr/pgsql-11/bin/postgresql-11-setup initdb + sudo systemctl enable postgresql-11 # # Next tune the postgresql configuration, which is located in @@ -88,7 +99,7 @@ sudo chown vagrant /srv/nominatim #DOCS: # # Now start the postgresql service after updating this config file. - sudo systemctl restart postgresql + sudo systemctl restart postgresql-11 # # Finally, we need to add two postgres users: one for the user that does @@ -150,7 +161,7 @@ fi #DOCS: # download the country grid: if [ ! -f data/country_osm_grid.sql.gz ]; then #DOCS: :::sh - wget -O data/country_osm_grid.sql.gz https://www.nominatim.org/data/country_grid.sql.gz + wget --no-verbose -O data/country_osm_grid.sql.gz https://www.nominatim.org/data/country_grid.sql.gz fi #DOCS: # The code must be built in a separate directory. Create this directory, diff --git a/website/css/details.css b/website/css/details.css index 171de42a..8f21c310 100644 --- a/website/css/details.css +++ b/website/css/details.css @@ -52,6 +52,17 @@ tr.all-columns td { margin: 10px 0; } +#details-index-page .search-form { + padding: 20px 10px; + margin: 2em 0; +} +#details-index-page .search-form h4 { + margin-top: 0; +} +#details-index-page .search-form .form-control{ + width: 30em; +} + footer { text-align: center; padding: 2em 0; diff --git a/website/details.php b/website/details.php index cb371e6b..44d4956b 100644 --- a/website/details.php +++ b/website/details.php @@ -30,6 +30,11 @@ $oDB->connect(); $sLanguagePrefArraySQL = $oDB->getArraySQL($oDB->getDBQuotedList($aLangPrefOrder)); +if ($sOutputFormat == 'html' && !$sPlaceId && !$sOsmType) { + include(CONST_BasePath.'/lib/template/details-index-html.php'); + exit; +} + if ($sOsmType && $iOsmId > 0) { $sSQL = 'SELECT place_id FROM placex WHERE osm_type = :type AND osm_id = :id'; // osm_type and osm_id are not unique enough diff --git a/website/js/nominatim-ui.js b/website/js/nominatim-ui.js index e96ebd25..d631e7aa 100644 --- a/website/js/nominatim-ui.js +++ b/website/js/nominatim-ui.js @@ -32,7 +32,6 @@ jQuery(document).ready(function(){ }); L.tileLayer(nominatim_map_init.tile_url, { - noWrap: true, // otherwise we end up with click coordinates like latitude -728 // moved to footer attribution: (nominatim_map_init.tile_attribution || null ) //'© OpenStreetMap contributors' }).addTo(map); @@ -78,6 +77,9 @@ jQuery(document).ready(function(){ function display_map_position(mouse_lat_lng){ + if (mouse_lat_lng) { + mouse_lat_lng = map.wrapLatLng(mouse_lat_lng); + } html_mouse = "mouse position " + (mouse_lat_lng ? [mouse_lat_lng.lat.toFixed(5), mouse_lat_lng.lng.toFixed(5)].join(',') : '-'); html_click = "last click: " + (last_click_latlng ? [last_click_latlng.lat.toFixed(5),last_click_latlng.lng.toFixed(5)].join(',') : '-'); @@ -92,9 +94,10 @@ jQuery(document).ready(function(){ $('#map-position-inner').html([html_center,html_zoom,html_viewbox,html_click,html_mouse].join('
')); + var center_lat_lng = map.wrapLatLng(map.getCenter()); var reverse_params = { - lat: map.getCenter().lat.toFixed(5), - lon: map.getCenter().lng.toFixed(5), + lat: center_lat_lng.lat.toFixed(5), + lon: center_lat_lng.lng.toFixed(5), zoom: map.getZoom(), format: 'html' } @@ -134,12 +137,22 @@ jQuery(document).ready(function(){ function map_viewbox_as_string() { - // since .toBBoxString() doesn't round numbers + var bounds = map.getBounds(); + var west = bounds.getWest(); + var east = bounds.getEast(); + + if ((east - west) >= 360) { // covers more than whole planet + west = map.getCenter().lng-179.999; + east = map.getCenter().lng+179.999; + } + east = L.latLng(77, east).wrap().lng; + west = L.latLng(77, west).wrap().lng; + return [ - map.getBounds().getSouthWest().lng.toFixed(5), // left - map.getBounds().getNorthEast().lat.toFixed(5), // top - map.getBounds().getNorthEast().lng.toFixed(5), // right - map.getBounds().getSouthWest().lat.toFixed(5) // bottom + west.toFixed(5), // left + bounds.getNorth().toFixed(5), // top + east.toFixed(5), // right + bounds.getSouth().toFixed(5) // bottom ].join(','); } function map_link_to_osm(){ @@ -236,7 +249,7 @@ jQuery(document).ready(function(){ if ( is_reverse_search ){ map.on('click', function(e){ $('form input[name=lat]').val( e.latlng.lat); - $('form input[name=lon]').val( e.latlng.lng); + $('form input[name=lon]').val( e.latlng.wrap().lng); $('form').submit(); }); @@ -273,6 +286,30 @@ jQuery(document).ready(function(){ }); +jQuery(document).ready(function(){ + + if ( !$('#details-index-page').length ){ return; } + + $('#form-by-type-and-id,#form-by-osm-url').on('submit', function(e){ + e.preventDefault(); + + var val = $(this).find('input[type=edit]').val(); + var matches = val.match(/^\s*([NWR])(\d+)\s*$/i); + + if (!matches) { + matches = val.match(/\/(relation|way|node)\/(\d+)\s*$/); + } + + if (matches) { + $(this).find('input[name=osmtype]').val(matches[1].charAt(0).toUpperCase()); + $(this).find('input[name=osmid]').val(matches[2]); + $(this).get(0).submit(); + } else { + alert('invalid input'); + } + }); +}); + jQuery(document).ready(function(){ if ( !$('#details-page').length ){ return; }