]>
git.openstreetmap.org Git - nominatim.git/log
Sarah Hoffmann [Sun, 30 May 2021 14:15:22 +0000 (16:15 +0200)]
prepare release 3.7.2
Sarah Hoffmann [Wed, 26 May 2021 09:47:08 +0000 (11:47 +0200)]
commit changes to replication log table
Fixes #2350.
mogita [Wed, 19 May 2021 05:35:15 +0000 (13:35 +0800)]
fix: add the missing question mark
Frederik Ramm [Thu, 6 May 2021 18:44:04 +0000 (20:44 +0200)]
Add array_key_last function for PHP <7.3
This patch adds an array_key_last function if it doesn't yet exist, fixes #2316. It is tested on PHP 7.2.24 but not PHP 7.3.
Sarah Hoffmann [Thu, 6 May 2021 13:36:54 +0000 (15:36 +0200)]
correctly catch the exception when import date is missing
Sarah Hoffmann [Tue, 27 Apr 2021 08:14:26 +0000 (10:14 +0200)]
do not check for extra housenumber index for reverse-only
Also adds a database check for reverse only import to the CI.
Sarah Hoffmann [Mon, 26 Apr 2021 07:37:51 +0000 (09:37 +0200)]
prepare release 3.7.1
Sarah Hoffmann [Mon, 26 Apr 2021 07:37:36 +0000 (09:37 +0200)]
rebuild manpage
Sarah Hoffmann [Mon, 26 Apr 2021 08:16:05 +0000 (10:16 +0200)]
avoid Path in subprocess parameters
Not supported by Python 3.5.
Sarah Hoffmann [Mon, 26 Apr 2021 07:54:09 +0000 (09:54 +0200)]
add warming to CI import test
AntoJvlt [Sat, 17 Apr 2021 17:45:24 +0000 (19:45 +0200)]
Only log a warning if a wrong input is detected on the wiki while importing special phrases
Sarah Hoffmann [Sat, 24 Apr 2021 13:00:18 +0000 (15:00 +0200)]
docs: update deployment to use project directory
Fixes #2295.
Sarah Hoffmann [Fri, 23 Apr 2021 13:45:54 +0000 (15:45 +0200)]
actions: add import on ubuntu 18.04
This uses oldest possible dependencies where possible.
Sarah Hoffmann [Fri, 23 Apr 2021 20:18:55 +0000 (22:18 +0200)]
use group() for regex matches
Needed for compatibility with Python 3.5.
Sarah Hoffmann [Fri, 23 Apr 2021 19:57:05 +0000 (21:57 +0200)]
use pathlib version of open
Sarah Hoffmann [Fri, 23 Apr 2021 19:49:41 +0000 (21:49 +0200)]
subprocess needs string argument
Compatibility change for Python 3.5.
Sarah Hoffmann [Fri, 23 Apr 2021 19:42:24 +0000 (21:42 +0200)]
check for existance of custom .env before opening
Sarah Hoffmann [Fri, 23 Apr 2021 19:10:19 +0000 (21:10 +0200)]
use more generic ImportError to check for module
ModuleNotFoundError was only introduced in Python 3.6.
Sarah Hoffmann [Fri, 23 Apr 2021 18:53:00 +0000 (20:53 +0200)]
replace usages of fromisoformat() with strptime()
fromisoformat was only introduced with Python 3.7 while we
still support Python 3.5.
Fixes #2292.
Sarah Hoffmann [Fri, 23 Apr 2021 18:27:14 +0000 (20:27 +0200)]
remove argparse dependency for vagrant scripts
Users don't need to recreate the manpage.
RhinoDevel [Thu, 22 Apr 2021 13:40:22 +0000 (15:40 +0200)]
Replace "nominatim-update" with "nominatim".
If I am not mistaken, the correct command to index imported data via commandline is "nominatim index".
AntoJvlt [Sat, 17 Apr 2021 17:24:13 +0000 (19:24 +0200)]
Fix occurence regex
Channgping Chen [Mon, 19 Apr 2021 00:01:01 +0000 (00:01 +0000)]
fix index on location_property_tiger (parent_place_id)
Looks like
2af82975cd968ec09683ae5b16a9aa157a7f2176
accidentally renamed an index. Because of the added "if not
exists" clause, the index doesn't get created. This
significantly slows down reverse queries because they now
require full scans on location_property_tiger.
Without this fix, reverse queries can take 8s on a full
planet install on an r5.8xlarge instance in EC2.
Sarah Hoffmann [Sat, 17 Apr 2021 09:07:04 +0000 (11:07 +0200)]
add support index when continuing import at index phase
Indexing scans the placex table sequentially during indexing
on the initial import. That is okay because we know that all
rows need to be processed anywhere. When continuing the import,
however, a large part might already be indexed, so that the
process spends a lot of time going through rows that are no
longer of interest. Create a supporting index for all unindexed
rows to speed up the scan. This is the same index as used later
for updates.
Sarah Hoffmann [Wed, 14 Apr 2021 14:19:49 +0000 (16:19 +0200)]
github actions: reintroduce postgresql repo
Darkshredder [Tue, 13 Apr 2021 09:36:02 +0000 (15:06 +0530)]
Fix: Removed error if endstatement is wrong and improved tests
AntoJvlt [Sun, 11 Apr 2021 20:26:31 +0000 (22:26 +0200)]
Fix default languages loading
Sarah Hoffmann [Fri, 9 Apr 2021 15:48:28 +0000 (17:48 +0200)]
split LANGUAGES parameter before use
The user supplies the languages as a comma-separated list.
Sarah Hoffmann [Tue, 6 Apr 2021 19:23:29 +0000 (21:23 +0200)]
prepare 3.7.0 release
Sarah Hoffmann [Tue, 6 Apr 2021 14:09:53 +0000 (16:09 +0200)]
docs: minor spelling corrections
Sarah Hoffmann [Tue, 6 Apr 2021 13:56:08 +0000 (15:56 +0200)]
docs: unpacking tiger data is no longer necessary
Sarah Hoffmann [Mon, 5 Apr 2021 13:48:22 +0000 (15:48 +0200)]
Merge pull request #2250 from lonvia/save-transliterated-housenumbers
Switch to saving transliterated housenumbers in placex
Sarah Hoffmann [Fri, 2 Apr 2021 15:28:52 +0000 (17:28 +0200)]
add migration for transliterated housenumbers
Sarah Hoffmann [Thu, 1 Apr 2021 16:52:06 +0000 (18:52 +0200)]
use new transliteration in initial housenumber word computation
The new create_housenumber_id() function splits housenumber
lists correctly. Otherwise there is no difference.
Sarah Hoffmann [Mon, 29 Mar 2021 15:36:44 +0000 (17:36 +0200)]
correctly handle housenumber lists
Lists are now standardised to use a semicolon separator.
Sarah Hoffmann [Mon, 29 Mar 2021 14:45:09 +0000 (16:45 +0200)]
move transliteration of housenumbers into indexing
Housenumbers are now saved in transliterated form in the housenumber
column. This saves the transliteration step during lookup.
Sarah Hoffmann [Mon, 29 Mar 2021 14:36:45 +0000 (16:36 +0200)]
tests for finding non-ascii housenumbers
Sarah Hoffmann [Mon, 29 Mar 2021 07:36:56 +0000 (09:36 +0200)]
return housenumbers always from address field
This means that we can use normalized versions of the
housenumber in the housenumber field as it is no longer
a user visible field.
Sarah Hoffmann [Sat, 3 Apr 2021 16:31:01 +0000 (18:31 +0200)]
Merge pull request #2248 from darkshredder/special-term-test
Added Test for TokenSpecialTerm
Darkshredder [Thu, 1 Apr 2021 23:19:05 +0000 (04:49 +0530)]
Added Test for TokenSpecialTerm
Sarah Hoffmann [Thu, 1 Apr 2021 16:35:00 +0000 (18:35 +0200)]
Merge pull request #2247 from lonvia/index-for-housenumber-lookup
Index for housenumber lookup
Sarah Hoffmann [Thu, 1 Apr 2021 12:37:20 +0000 (14:37 +0200)]
add recommendation for Postgresql 11+
Sarah Hoffmann [Thu, 1 Apr 2021 12:29:34 +0000 (14:29 +0200)]
use non-key index to speed up housenumber search
On Postgresql versions 11+ add an index to speed up the lookup
of housenumbers for terms found in search_name. This is really
just a band-aid around the query planer's interpretation of the
query.
Sarah Hoffmann [Tue, 30 Mar 2021 07:19:01 +0000 (09:19 +0200)]
Merge pull request #2243 from darkshredder/XML-format-fix
Fixed: XML format: more_url points to localhost, not base URL
Sarah Hoffmann [Tue, 30 Mar 2021 07:17:27 +0000 (09:17 +0200)]
Merge pull request #2244 from AntoJvlt/import-special-phrases-tests-cleaning
Cleaned tests for special phrases.
Darkshredder [Mon, 29 Mar 2021 21:32:55 +0000 (03:02 +0530)]
Added HTTP_HOST to if statement
AntoJvlt [Mon, 29 Mar 2021 20:07:29 +0000 (22:07 +0200)]
Cleaned tests of exceptions and fix phrase_settings.json test file name.
Darkshredder [Mon, 29 Mar 2021 19:32:43 +0000 (01:02 +0530)]
fixed: XML format: more_url points to localhost, not base URL
Sarah Hoffmann [Mon, 29 Mar 2021 12:25:10 +0000 (14:25 +0200)]
Merge pull request #2234 from darkshredder/add-man-page
Added Manual page for Nominatim tool
Sarah Hoffmann [Mon, 29 Mar 2021 10:06:51 +0000 (12:06 +0200)]
allow sorting by housenumbers for rare street names
Usually we don't narrow down search results by house number when
only a street name is given because there may be a lot of rows
to cross check when the street name is very frequent. However,
when it is known to be rare, the housenumber check may be done
anyway.
Fixes #2238.
Darkshredder [Mon, 29 Mar 2021 08:57:06 +0000 (14:27 +0530)]
Update manpage after rebase
Darkshredder [Mon, 29 Mar 2021 08:46:41 +0000 (14:16 +0530)]
Nominatim/cli.py rebase fixes
Darkshredder [Mon, 29 Mar 2021 08:30:45 +0000 (14:00 +0530)]
Rebase with master
Darkshredder [Wed, 24 Mar 2021 15:08:14 +0000 (20:38 +0530)]
fixed indentation and used sed to remove AUTHORS section
Darkshredder [Tue, 23 Mar 2021 02:40:53 +0000 (08:10 +0530)]
Added Manual page and fixed documentation
Sarah Hoffmann [Mon, 29 Mar 2021 07:49:35 +0000 (09:49 +0200)]
Merge pull request #2228 from AntoJvlt/import-special-phrases-porting-python
Import special phrases porting python
AntoJvlt [Fri, 26 Mar 2021 00:57:51 +0000 (01:57 +0100)]
Change command 'import-special-phrases --from-wiki' to 'special-phrases --import-from-wiki'.
AntoJvlt [Fri, 26 Mar 2021 00:53:33 +0000 (01:53 +0100)]
Errors fixes, Cleaning code, Improvement and addition of tests
AntoJvlt [Wed, 24 Mar 2021 13:26:52 +0000 (14:26 +0100)]
Encapsulation of tools/special_phrases.py into SpecialPhrasesImporter class and add new tests.
AntoJvlt [Mon, 22 Mar 2021 22:56:24 +0000 (23:56 +0100)]
Code cleaning, tests simplification and use of python3-icu package
AntoJvlt [Mon, 22 Mar 2021 00:46:46 +0000 (01:46 +0100)]
Updated documentation for PyICU support
AntoJvlt [Sun, 21 Mar 2021 23:58:49 +0000 (00:58 +0100)]
Introduction of PyICU for transliteration in python. Reversed changes in normalization.sql.
AntoJvlt [Sun, 21 Mar 2021 23:16:39 +0000 (00:16 +0100)]
Added wrapper in specialphrases.php to call corresponding nominatim command.
AntoJvlt [Sun, 21 Mar 2021 23:07:55 +0000 (00:07 +0100)]
Changed phrase_settings.py to phrase-settings.json and added migration function for old php settings file.
AntoJvlt [Sun, 21 Mar 2021 23:07:55 +0000 (00:07 +0100)]
Changed phrase_settings.py to phrase-settings.json and added migration function for old php settings file.
Sarah Hoffmann [Tue, 23 Mar 2021 08:18:10 +0000 (09:18 +0100)]
Merge pull request #2233 from lonvia/index-for-postcode-ids
Create postcode id index earlier
Sarah Hoffmann [Mon, 22 Mar 2021 21:24:56 +0000 (22:24 +0100)]
create postcode id index earlier
Now that the indexer takes care of indexing the postcode tables,
the id index is needed to find the rows to index.
Sarah Hoffmann [Sun, 21 Mar 2021 15:52:20 +0000 (16:52 +0100)]
Merge pull request #2231 from mtmail/correct-cli-help-page
nominatim -h was printing wrong text for lookup and details
Sarah Hoffmann [Sun, 21 Mar 2021 15:47:22 +0000 (16:47 +0100)]
avoid division by zero in progress meter
On Windows systems the timer may not be accurate enough to measure
the time between init() and done(). Avoid computing statistics with
a diff time of 0 in such cases.
Fixes #2230.
marc tobias [Sun, 21 Mar 2021 15:06:41 +0000 (16:06 +0100)]
nominatim -h was priting wrong text for lookup and details
AntoJvlt [Sat, 20 Mar 2021 18:48:05 +0000 (19:48 +0100)]
Deleted specialphrases.php and phrase_settings.php
AntoJvlt [Sat, 20 Mar 2021 18:17:03 +0000 (19:17 +0100)]
Updated specialphrases_testdb.sql
AntoJvlt [Sat, 20 Mar 2021 17:55:08 +0000 (18:55 +0100)]
Ported functions for the import of special phrases from php to python.
- the command is now --import-special-phrases
- the output is not an sql file anymore, data are directly imported to the database.
- the little part on the documentation (section data import) has been modified.
Sarah Hoffmann [Sat, 20 Mar 2021 09:39:29 +0000 (10:39 +0100)]
bdd tests: make indexing less verbose
Do not print progress info for indexing when there is an error
in the BDD tests.
Sarah Hoffmann [Wed, 17 Mar 2021 10:40:34 +0000 (11:40 +0100)]
Merge pull request #2219 from lonvia/bdd-test-remove-php
BDD tests: run all setup via nominatim Python library
Sarah Hoffmann [Tue, 16 Mar 2021 21:13:33 +0000 (22:13 +0100)]
bdd: run all setup via nominatim Python library
Drops all calls to PHP utility functions. nominatim cli functions
are used where possible, to stay as close to the final code as
possible with the tests.
By removing the PHP calls, the test code now only uses osm2pgsql and
the database module from the build directory.
Sarah Hoffmann [Mon, 15 Mar 2021 13:08:54 +0000 (14:08 +0100)]
Merge pull request #2216 from lonvia/fix-reverse-interpolation
Reverse: do not prefer interpolations over closer housenumbers
Sarah Hoffmann [Mon, 15 Mar 2021 09:50:04 +0000 (10:50 +0100)]
reverse: do not prefer interpolations over closer housenumbers
Always look up the closest housenumber before looking up
interpolations. This ensures that closer housenumbers are
preferred over interpolations.
Fixes #2214.
Sarah Hoffmann [Mon, 15 Mar 2021 08:36:06 +0000 (09:36 +0100)]
Merge pull request #2212 from darkshredder/country-name
Ported createCountryNames() to python and Added tests
Darkshredder [Sun, 14 Mar 2021 09:32:30 +0000 (15:02 +0530)]
Add setup.php
Sarah Hoffmann [Sat, 13 Mar 2021 11:05:53 +0000 (12:05 +0100)]
fix template variable name
Sarah Hoffmann [Fri, 12 Mar 2021 14:47:36 +0000 (15:47 +0100)]
Merge pull request #2213 from lonvia/tweak-search-weights
Some more tweaking of the ranking of search interpretations
Darkshredder [Fri, 12 Mar 2021 12:58:47 +0000 (18:28 +0530)]
Linting fix
Darkshredder [Fri, 12 Mar 2021 12:53:20 +0000 (18:23 +0530)]
refactored tests and made changes to code for easy readibility
Darkshredder [Fri, 12 Mar 2021 04:58:41 +0000 (10:28 +0530)]
Ported createCountryNames() to python and added tests
Sarah Hoffmann [Thu, 11 Mar 2021 21:48:38 +0000 (22:48 +0100)]
Merge pull request #2204 from darkshredder/tiger-data
Ported tiger-data-import to Python and Added Tarball Support
Sarah Hoffmann [Thu, 11 Mar 2021 21:44:49 +0000 (22:44 +0100)]
do not mix partial names with other words
As soon as a housenumber, postcode, etc. appear, the name term
must obviously be closed and no further partial terms can be
appended.
Sarah Hoffmann [Thu, 11 Mar 2021 20:14:23 +0000 (21:14 +0100)]
make linter happy
Sarah Hoffmann [Thu, 11 Mar 2021 19:42:37 +0000 (20:42 +0100)]
avoid multi-term partials in names
Names are either full words or single-word partial names.
Searching for multi-word partials yields exactly the same
result as with full words.
Sarah Hoffmann [Thu, 11 Mar 2021 19:34:21 +0000 (20:34 +0100)]
higher penalty for special searches
Adds a general higher penalty for special search term and an
additional one if the term is anywhere but the beginning or the
end. Also housenumbers and special searches together are less
likely.
Sarah Hoffmann [Thu, 11 Mar 2021 19:22:54 +0000 (20:22 +0100)]
do not mix partial and full name terms
If NameNonSearch already contains a partial term, then a
full term must not be added to the Name list anymore.
Sarah Hoffmann [Thu, 11 Mar 2021 16:34:23 +0000 (17:34 +0100)]
add one-rank penalty for using partial search
Ensures that full matches are preferred over partial ones even when
the full word consists of only one term.
Sarah Hoffmann [Thu, 11 Mar 2021 16:14:46 +0000 (17:14 +0100)]
fix result splitting for last search group
When we are in the final iteration of the search groups, it is not
possible to further delay the results. Unconditionally use the
results with the best rank instead.
Sarah Hoffmann [Thu, 11 Mar 2021 14:03:36 +0000 (15:03 +0100)]
give preference to full words in address, too
Full word terms are already preferred for the name part. Adding
only one-word partials to the address, makes it impossible to
give a similar preference for the address part. Each term adds
a rank penalty. The problem here is that we interpret the query
forwards and backwards. Having different penalty systems for
name and address means that the same term ends up with different
penalties and that often leads to interpretations of the wrong
direction being in the way.
Darkshredder [Thu, 11 Mar 2021 10:09:17 +0000 (15:39 +0530)]
Added fixture for sql_preprocessor and fixed some issues
Darkshredder [Wed, 10 Mar 2021 12:44:17 +0000 (18:14 +0530)]
Added test for tarfile
Darkshredder [Wed, 10 Mar 2021 11:48:12 +0000 (17:18 +0530)]
Added test and removed runlegacyscript
Darkshredder [Wed, 10 Mar 2021 07:58:29 +0000 (13:28 +0530)]
fixed linting, refactored threaded sql handling and removed importTigerData() function
Darkshredder [Mon, 8 Mar 2021 19:30:56 +0000 (01:00 +0530)]
Test fixes
Darkshredder [Mon, 8 Mar 2021 17:40:49 +0000 (23:10 +0530)]
Linting fixes