]> git.openstreetmap.org Git - nominatim.git/log
nominatim.git
2 years agoMerge remote-tracking branch 'upstream/master'
Sarah Hoffmann [Wed, 11 May 2022 14:26:35 +0000 (16:26 +0200)]
Merge remote-tracking branch 'upstream/master'

2 years agoMerge pull request #2709 from lonvia/less-strict-country-assignment
Sarah Hoffmann [Wed, 11 May 2022 14:24:47 +0000 (16:24 +0200)]
Merge pull request #2709 from lonvia/less-strict-country-assignment

Be more strict with country assignments

2 years agoMerge pull request #2708 from lonvia/use-format-literals
Sarah Hoffmann [Wed, 11 May 2022 12:29:56 +0000 (14:29 +0200)]
Merge pull request #2708 from lonvia/use-format-literals

Assorted fixes for new pylint warnings

2 years agono longer allow fuzzy assignment of country
Sarah Hoffmann [Wed, 11 May 2022 09:54:25 +0000 (11:54 +0200)]
no longer allow fuzzy assignment of country

The fallback country boundaries already contain a sufficiently large
part of the water area, so there is no need to extend the country
assignment even more. Features outside countries should not show a
country in their address.

2 years agofix spacing
Sarah Hoffmann [Wed, 11 May 2022 08:36:09 +0000 (10:36 +0200)]
fix spacing

Some versions of pylint are oddly picky.

2 years agopylint: disable no-self-use check
Sarah Hoffmann [Wed, 11 May 2022 08:25:00 +0000 (10:25 +0200)]
pylint: disable no-self-use check

This checker encourages bad behaviour (namely changing the static
status of a function during inheritence) and will be made optional
in upcoming versions of pylint.

2 years agosolve assorted issue with newer pylint versions
Sarah Hoffmann [Wed, 11 May 2022 08:22:14 +0000 (10:22 +0200)]
solve assorted issue with newer pylint versions

Includes more use of 'with', adding encodings to open statements
and a couple of issues with parameter renaming.

2 years agouse context management when processing Tiger data
Sarah Hoffmann [Wed, 11 May 2022 07:45:15 +0000 (09:45 +0200)]
use context management when processing Tiger data

2 years agoremove redundant 'u' prefixes for unicode strings
Sarah Hoffmann [Wed, 11 May 2022 07:06:32 +0000 (09:06 +0200)]
remove redundant 'u' prefixes for unicode strings

2 years agopylint: avoid explicit use of format() function
Sarah Hoffmann [Wed, 11 May 2022 06:59:28 +0000 (08:59 +0200)]
pylint: avoid explicit use of format() function

Use psycopg2 SQL formatters for SQL and formatted string literals
everywhere else.

2 years agoadd a function to return a formatted version
Sarah Hoffmann [Tue, 10 May 2022 21:00:18 +0000 (23:00 +0200)]
add a function to return a formatted version

Replaces the various repeated format strings throughout the code.

2 years agoMerge pull request #2707 from lonvia/make-icu-tokenizer-the-default
Sarah Hoffmann [Wed, 11 May 2022 06:52:49 +0000 (08:52 +0200)]
Merge pull request #2707 from lonvia/make-icu-tokenizer-the-default

Make ICU tokenizer the default

2 years agono longer need postgresql-server-dev packages
Sarah Hoffmann [Tue, 10 May 2022 15:34:58 +0000 (17:34 +0200)]
no longer need postgresql-server-dev packages

2 years agomake legacy tokenizer tests the extra on CI
Sarah Hoffmann [Tue, 10 May 2022 13:52:15 +0000 (15:52 +0200)]
make legacy tokenizer tests the extra on CI

2 years agoMerge pull request #2704 from mtmail/migrate-phpunit-xml-schema
Sarah Hoffmann [Tue, 10 May 2022 15:44:34 +0000 (17:44 +0200)]
Merge pull request #2704 from mtmail/migrate-phpunit-xml-schema

PHPUnit 9 changed configuration schema slightly

2 years agoalways state encoding when opening files in text mode
Sarah Hoffmann [Tue, 10 May 2022 13:36:29 +0000 (15:36 +0200)]
always state encoding when opening files in text mode

Also applies to Path.write_text().

2 years agoPHPUnit 9 changed configuration schema slightly
Marc Tobias [Tue, 10 May 2022 13:18:02 +0000 (15:18 +0200)]
PHPUnit 9 changed configuration schema slightly

2 years agoswitch tests to ICU tokenizer as default
Sarah Hoffmann [Tue, 10 May 2022 12:54:50 +0000 (14:54 +0200)]
switch tests to ICU tokenizer as default

2 years agoadd migration hint for the new tokenizer default
Sarah Hoffmann [Tue, 10 May 2022 10:07:21 +0000 (12:07 +0200)]
add migration hint for the new tokenizer default

2 years agomake ICU the default tokenizer
Sarah Hoffmann [Tue, 10 May 2022 10:02:50 +0000 (12:02 +0200)]
make ICU the default tokenizer

2 years agoMerge pull request #2702 from lonvia/move-country-names-into-includes
Sarah Hoffmann [Tue, 10 May 2022 07:21:16 +0000 (09:21 +0200)]
Merge pull request #2702 from lonvia/move-country-names-into-includes

Clean up country name settings

2 years agoMerge pull request #2695 from mtmail/git-commit-hash-to-version
Sarah Hoffmann [Tue, 10 May 2022 07:14:15 +0000 (09:14 +0200)]
Merge pull request #2695 from mtmail/git-commit-hash-to-version

add git commit hash to 'nominatim --version' output

2 years agoadd git commit hash to --version output
Marc Tobias [Wed, 4 May 2022 13:48:23 +0000 (15:48 +0200)]
add git commit hash to --version output

2 years agosupport arbitrary prefixes in country name list
Sarah Hoffmann [Thu, 5 May 2022 15:16:15 +0000 (17:16 +0200)]
support arbitrary prefixes in country name list

This means we can now get rid of the last special cases for names.

2 years agomove country names into separate include files
Sarah Hoffmann [Thu, 5 May 2022 10:03:36 +0000 (12:03 +0200)]
move country names into separate include files

2 years agoMerge pull request #2696 from mtmail/norminatyn-typos
Sarah Hoffmann [Thu, 5 May 2022 08:04:55 +0000 (10:04 +0200)]
Merge pull request #2696 from mtmail/norminatyn-typos

fix typos of name Nominatim

2 years agofix typos of name Nominatim
Marc Tobias [Wed, 4 May 2022 23:04:47 +0000 (01:04 +0200)]
fix typos of name Nominatim

2 years agoMerge pull request #2693 from mtmail/nominatim-cli-version
Sarah Hoffmann [Wed, 4 May 2022 07:14:35 +0000 (09:14 +0200)]
Merge pull request #2693 from mtmail/nominatim-cli-version

new "nominatim --version" global CLI argument

2 years agonew nominatim --version CLI argument
Marc Tobias [Tue, 3 May 2022 23:33:25 +0000 (01:33 +0200)]
new nominatim --version CLI argument

2 years agoMerge remote-tracking branch 'upstream/master'
Sarah Hoffmann [Mon, 2 May 2022 14:06:02 +0000 (16:06 +0200)]
Merge remote-tracking branch 'upstream/master'

2 years agoMerge pull request #2681 from lonvia/improve-geocodejson
Sarah Hoffmann [Mon, 2 May 2022 14:05:02 +0000 (16:05 +0200)]
Merge pull request #2681 from lonvia/improve-geocodejson

Fix 'type' field in the geocodejson response

2 years agoMerge pull request #2687 from lonvia/check-for-wikipedia
Sarah Hoffmann [Mon, 2 May 2022 14:04:32 +0000 (16:04 +0200)]
Merge pull request #2687 from lonvia/check-for-wikipedia

Add check for presence of wikipedia importance

2 years agoMerge remote-tracking branch 'upstream/master'
Sarah Hoffmann [Mon, 2 May 2022 12:13:43 +0000 (14:13 +0200)]
Merge remote-tracking branch 'upstream/master'

2 years agoexplicit cast for osm_type parameter in SQL needed
Sarah Hoffmann [Mon, 2 May 2022 12:12:17 +0000 (14:12 +0200)]
explicit cast for osm_type parameter in SQL needed

Otherwise PostgreSQL won't correctly pick up the index
condition.

2 years agoMerge remote-tracking branch 'upstream/master'
Sarah Hoffmann [Mon, 2 May 2022 09:44:50 +0000 (11:44 +0200)]
Merge remote-tracking branch 'upstream/master'

2 years agorespect exclude_place_ids for housenumber search
Sarah Hoffmann [Mon, 2 May 2022 09:01:54 +0000 (11:01 +0200)]
respect exclude_place_ids for housenumber search

2 years agoMerge pull request #2689 from lonvia/relations-in-associated-street
Sarah Hoffmann [Mon, 2 May 2022 09:42:34 +0000 (11:42 +0200)]
Merge pull request #2689 from lonvia/relations-in-associated-street

Accept any OSM type in street member of associatedStreet

2 years agoaccept any OSM type in street member of associatedStreet
Sarah Hoffmann [Mon, 2 May 2022 07:48:51 +0000 (09:48 +0200)]
accept any OSM type in street member of associatedStreet

This is needed for pedestrian areas mapped as multipolygons
and consequently as relations. The lookup in placex guarantees
that the referenced OSM object is indeed a street.

Fixes #2669.

2 years agoMerge pull request #2686 from mtmail/ubuntu20-php-fpm-version
Sarah Hoffmann [Fri, 29 Apr 2022 12:16:44 +0000 (14:16 +0200)]
Merge pull request #2686 from mtmail/ubuntu20-php-fpm-version

Install-on-Ubuntu-20.sh - correct php version

2 years agoskip wikipedia table test on reverse-only installations
Sarah Hoffmann [Fri, 29 Apr 2022 12:12:55 +0000 (14:12 +0200)]
skip wikipedia table test on reverse-only installations

Wikipedia importances are not imported on reverse-only imports.

2 years agoInstall-on-Ubuntu-20.sh - correct php-fpm version
Marc Tobias [Fri, 29 Apr 2022 11:24:15 +0000 (13:24 +0200)]
Install-on-Ubuntu-20.sh - correct php-fpm version

2 years agoMerge remote-tracking branch 'upstream/master'
Sarah Hoffmann [Fri, 29 Apr 2022 10:15:52 +0000 (12:15 +0200)]
Merge remote-tracking branch 'upstream/master'

2 years agoMerge pull request #2685 from lonvia/show-inherited-housenumber
Sarah Hoffmann [Fri, 29 Apr 2022 10:15:33 +0000 (12:15 +0200)]
Merge pull request #2685 from lonvia/show-inherited-housenumber

Keep address parts inherited from surrounding buildings after indexing

2 years agoadd check for wikipedia importance data
Sarah Hoffmann [Fri, 29 Apr 2022 10:11:39 +0000 (12:11 +0200)]
add check for wikipedia importance data

Adds a new check level WARNING because missing wikipedia importances
are not necessarily an error. If the database is run for reverse
requests only, then it is fine to go without them.

2 years agoMerge pull request #2684 from lonvia/translit-keep-spacing-marks
Sarah Hoffmann [Fri, 29 Apr 2022 08:38:28 +0000 (10:38 +0200)]
Merge pull request #2684 from lonvia/translit-keep-spacing-marks

ICU: better letter identification in normalization

2 years agokeep inherited address parts after indexing
Sarah Hoffmann [Thu, 28 Apr 2022 19:38:00 +0000 (21:38 +0200)]
keep inherited address parts after indexing

The inherited housenumber is needed for display output. We can't
take the one from the housenumber field because it is already
normalized. Remove the inherited address only when reindexing.

Fixes #2683.

2 years agoICU: better letter identification in normalization
Sarah Hoffmann [Thu, 28 Apr 2022 15:20:56 +0000 (17:20 +0200)]
ICU: better letter identification in normalization

The Letter class does not include non-spacing marks that can also
have a consonant or vowel meaning, especially in Indian languages.
Use the alnum propoerty instead which includes them all. Also
include the vowel-canceling Virama, which is not a letter by itself
but changes the transliteration.

2 years agomention the breaking API change in the migratioin docs
Sarah Hoffmann [Wed, 27 Apr 2022 09:52:53 +0000 (11:52 +0200)]
mention the breaking API change in the migratioin docs

2 years agoadapt docs to geocodejson changes
Sarah Hoffmann [Wed, 27 Apr 2022 09:50:12 +0000 (11:50 +0200)]
adapt docs to geocodejson changes

2 years agogeocodejson: add osm_key and osm_value fields
Sarah Hoffmann [Wed, 27 Apr 2022 08:58:25 +0000 (10:58 +0200)]
geocodejson: add osm_key and osm_value fields

Return OSM main tag information in geocodejson. This is not part
of the official spec but can be useful to get more detailed information
of the object type. Brings the Nominatim output closer to what
Photon produces.

2 years agogeocodejson: type should contain the general feature class
Sarah Hoffmann [Wed, 27 Apr 2022 08:53:12 +0000 (10:53 +0200)]
geocodejson: type should contain the general feature class

'type' so far contained the value of the OSM tag. That is rarely
helpful because it is not a restricted class of values. Change
this to contain the types as defined in the geocodejson spec,
which correspond to the address layer names.

2 years agoMerge remote-tracking branch 'upstream/master'
Sarah Hoffmann [Fri, 22 Apr 2022 18:56:23 +0000 (20:56 +0200)]
Merge remote-tracking branch 'upstream/master'

2 years agoMerge pull request #2678 from lonvia/address-part-order
Sarah Hoffmann [Fri, 22 Apr 2022 18:32:10 +0000 (20:32 +0200)]
Merge pull request #2678 from lonvia/address-part-order

Change selection of primary address part for ways that cross boundaries

2 years agofurther tweaking of address distance
Sarah Hoffmann [Fri, 22 Apr 2022 12:32:19 +0000 (14:32 +0200)]
further tweaking of address distance

For point features, keep using the distance to centroid.
For area features, add a tie breaker for the case where the
center point falls on the boundary.

2 years agochange distance computation between place and address part
Sarah Hoffmann [Thu, 21 Apr 2022 19:56:59 +0000 (21:56 +0200)]
change distance computation between place and address part

Instead of computing the distance to the centroid of the area
compute the distance of the area to the centroid of the feature.
This means we give preference to the area that covers the centroid.
It's still a heuristics but one that is a bit less random.

2 years agoMerge remote-tracking branch 'upstream/master'
Sarah Hoffmann [Thu, 14 Apr 2022 19:44:32 +0000 (21:44 +0200)]
Merge remote-tracking branch 'upstream/master'

2 years agoMerge pull request #2666 from lonvia/admin-command-for-forced-indexing
Sarah Hoffmann [Thu, 14 Apr 2022 19:44:08 +0000 (21:44 +0200)]
Merge pull request #2666 from lonvia/admin-command-for-forced-indexing

Admin command for forced indexing

2 years agoadd documentation for new refresh command
Sarah Hoffmann [Thu, 14 Apr 2022 13:10:24 +0000 (15:10 +0200)]
add documentation for new refresh command

2 years agoadd tests for new data invalidation functions
Sarah Hoffmann [Thu, 14 Apr 2022 12:52:13 +0000 (14:52 +0200)]
add tests for new data invalidation functions

2 years agoadd new commands for forced invalidation before indexing
Sarah Hoffmann [Wed, 13 Apr 2022 20:00:37 +0000 (22:00 +0200)]
add new commands for forced invalidation before indexing

2 years agorecommend PostgreSQL 13+
Sarah Hoffmann [Thu, 14 Apr 2022 07:21:25 +0000 (09:21 +0200)]
recommend PostgreSQL 13+

See https://github.com/osm-search/Nominatim/discussions/2659.

2 years agodocs: different default for format on osm.org
Sarah Hoffmann [Fri, 8 Apr 2022 15:13:42 +0000 (17:13 +0200)]
docs: different default for format on osm.org

Add a note that the format parameter is needed for
nominatim.openstreetmap.org for historical reasons.

2 years agoMerge pull request #2660 from lonvia/pyosmium-contextmanager
Sarah Hoffmann [Fri, 8 Apr 2022 15:07:33 +0000 (17:07 +0200)]
Merge pull request #2660 from lonvia/pyosmium-contextmanager

Support using ReplicationServer as contextmanager

2 years agosupport new ReplicationServer as contextmanager
Sarah Hoffmann [Thu, 7 Apr 2022 15:58:04 +0000 (17:58 +0200)]
support new ReplicationServer as contextmanager

2 years agoMerge remote-tracking branch 'upstream/master'
Sarah Hoffmann [Thu, 7 Apr 2022 14:42:46 +0000 (16:42 +0200)]
Merge remote-tracking branch 'upstream/master'

2 years agoMerge pull request #2466 from I70l0teN4ik/state-code
Sarah Hoffmann [Thu, 7 Apr 2022 14:39:50 +0000 (16:39 +0200)]
Merge pull request #2466 from I70l0teN4ik/state-code

add ISO3166-2-lvl<admin_level> field to response address details

2 years agofix bdd tests and docs
Artem Ziablytskyi [Thu, 7 Apr 2022 12:09:08 +0000 (14:09 +0200)]
fix bdd tests and docs

2 years agouse ISO3166-2-lvl<admin_level> instead of typeLabel prefix
Artem Ziablytskyi [Wed, 6 Apr 2022 17:24:39 +0000 (19:24 +0200)]
use ISO3166-2-lvl<admin_level> instead of typeLabel prefix

2 years agoproper instruction to import data
Artem Ziablytskyi [Wed, 23 Mar 2022 13:32:03 +0000 (14:32 +0100)]
proper instruction to import data

2 years agoFix API docs and Vagrant instructions to import data
Artem Ziablytskyi [Thu, 10 Mar 2022 09:57:26 +0000 (10:57 +0100)]
Fix API docs and Vagrant instructions to import data

2 years agoFix API docs and Vagrant instructions to import data
Artem Ziablytskyi [Thu, 10 Mar 2022 09:57:26 +0000 (10:57 +0100)]
Fix API docs and Vagrant instructions to import data

2 years agoChange the key to `<addresspart_type>-ISO3166-2` to support xml response correctly
Artem Ziablytskyi [Wed, 26 Jan 2022 16:40:39 +0000 (17:40 +0100)]
Change the key to `<addresspart_type>-ISO3166-2` to support xml response correctly

2 years agoadd `<addresspart_type>:ISO3166-2` field to response address details
Artem Ziablytskyi [Mon, 17 Jan 2022 16:20:06 +0000 (17:20 +0100)]
add `<addresspart_type>:ISO3166-2` field to response address details

2 years agoadd `state_code` field to response address details
Artem Ziablytskyi [Tue, 12 Oct 2021 15:35:15 +0000 (17:35 +0200)]
add `state_code` field to response address details

2 years agoMerge pull request #2629 from tareqpi/country-names-yaml-configuration
Sarah Hoffmann [Mon, 4 Apr 2022 07:04:25 +0000 (09:04 +0200)]
Merge pull request #2629 from tareqpi/country-names-yaml-configuration

Move default country names into yaml configuration

2 years agofix linting error
Tareq Al-Ahdal [Fri, 1 Apr 2022 16:14:18 +0000 (00:14 +0800)]
fix linting error

2 years agoremove the conversion to json for name
Tareq Al-Ahdal [Fri, 1 Apr 2022 14:54:14 +0000 (22:54 +0800)]
remove the conversion to json for name

2 years ago'read_config' is no longer a fixture
Tareq Al-Ahdal [Fri, 1 Apr 2022 14:52:17 +0000 (22:52 +0800)]
'read_config' is no longer a fixture

add 'read_config' to test cases that need it

2 years agotest for loading special characters from country_settings.yaml
Tareq Al-Ahdal [Fri, 1 Apr 2022 13:58:57 +0000 (21:58 +0800)]
test for loading special characters from country_settings.yaml

2 years agofix reset country info before loading custom data
Tareq Al-Ahdal [Fri, 1 Apr 2022 13:55:34 +0000 (21:55 +0800)]
fix reset country info before loading custom data

2 years agocustom country config loads correctly
Tareq Al-Ahdal [Fri, 1 Apr 2022 13:46:56 +0000 (21:46 +0800)]
custom country config loads correctly

2 years agoMerge remote-tracking branch 'upstream/master'
Sarah Hoffmann [Thu, 31 Mar 2022 16:05:24 +0000 (18:05 +0200)]
Merge remote-tracking branch 'upstream/master'

2 years agoMerge pull request #2655 from lonvia/migration-internal-country-name
Sarah Hoffmann [Thu, 31 Mar 2022 16:04:18 +0000 (18:04 +0200)]
Merge pull request #2655 from lonvia/migration-internal-country-name

Add migration for new country name handling in ICU tokenizer

2 years agoadd test for new table_has_column function
Sarah Hoffmann [Thu, 31 Mar 2022 13:50:02 +0000 (15:50 +0200)]
add test for new table_has_column function

2 years agoadd migration to mark internal country names
Sarah Hoffmann [Thu, 31 Mar 2022 13:43:46 +0000 (15:43 +0200)]
add migration to mark internal country names

2 years agoseparate unit test function into three functions
Tareq Al-Ahdal [Wed, 30 Mar 2022 14:06:59 +0000 (22:06 +0800)]
separate unit test function into three functions

2 years agoremove temporary file
Sarah Hoffmann [Wed, 30 Mar 2022 13:13:31 +0000 (15:13 +0200)]
remove temporary file

2 years agoMerge pull request #2650 from mtmail/update-lookup-examples
Sarah Hoffmann [Mon, 28 Mar 2022 18:21:45 +0000 (20:21 +0200)]
Merge pull request #2650 from mtmail/update-lookup-examples

documentation: update example output of lookup endpoint

2 years agodocumentation: update example output of lookup endpoint
Marc Tobias [Mon, 28 Mar 2022 14:41:10 +0000 (16:41 +0200)]
documentation: update example output of lookup endpoint

2 years agofix edge case handling when 'names' is not there
Tareq Al-Ahdal [Fri, 25 Mar 2022 14:25:55 +0000 (22:25 +0800)]
fix edge case handling when 'names' is not there

2 years agoAdded unit tests for loading country info from yaml file
Tareq Al-Ahdal [Fri, 25 Mar 2022 14:22:44 +0000 (22:22 +0800)]
Added unit tests for loading country info from yaml file

2 years agofix linting error
Tareq Al-Ahdal [Thu, 24 Mar 2022 05:27:24 +0000 (13:27 +0800)]
fix linting error

2 years agofix storing of escape sequences in database
Tareq Al-Ahdal [Thu, 24 Mar 2022 05:18:44 +0000 (13:18 +0800)]
fix storing of escape sequences in database

2 years agoclean up
Tareq Al-Ahdal [Thu, 24 Mar 2022 05:16:59 +0000 (13:16 +0800)]
clean up

2 years agomerge of insert query + modularity enhancements
Tareq Al-Ahdal [Thu, 24 Mar 2022 05:13:38 +0000 (13:13 +0800)]
merge of insert query + modularity enhancements

2 years agoMerge pull request #2641 from lonvia/reinit-tokenizer-dir
Sarah Hoffmann [Sun, 20 Mar 2022 20:46:07 +0000 (21:46 +0100)]
Merge pull request #2641 from lonvia/reinit-tokenizer-dir

Transparantly reinitialize tokenizer directory when necessary

2 years agoreinit the tokenizer directory on website refresh
Sarah Hoffmann [Sun, 20 Mar 2022 16:49:22 +0000 (17:49 +0100)]
reinit the tokenizer directory on website refresh

This means the project directory is usable again, once refresh --website
was run.

2 years agodocs: copying project dir no longer necessary
Sarah Hoffmann [Sun, 20 Mar 2022 15:01:27 +0000 (16:01 +0100)]
docs: copying project dir no longer necessary

2 years agorestore the tokenizer directory when missing
Sarah Hoffmann [Sun, 20 Mar 2022 10:31:42 +0000 (11:31 +0100)]
restore the tokenizer directory when missing

Automatically repopulate the tokenizer/ directory with the PHP stub
and the postgresql module, when the directory is missing. This allows
to switch working directories and in particular run the service
from a different maschine then where it was installed.
Users still need to make sure that .env files are set up correctly
or they will shoot themselves in the foot.

See #2515.

2 years agocache loaded configuration
Sarah Hoffmann [Sun, 20 Mar 2022 10:30:03 +0000 (11:30 +0100)]
cache loaded configuration

Reading the YAML files is fairly expensive and slows down the BDD tests
significantly. Therefore cache the results from reading the file.