]> git.openstreetmap.org Git - nominatim.git/commit
use analyser provided in the 'analyzer' property
authorSarah Hoffmann <lonvia@denofr.de>
Tue, 5 Oct 2021 12:10:32 +0000 (14:10 +0200)
committerSarah Hoffmann <lonvia@denofr.de>
Tue, 5 Oct 2021 12:10:32 +0000 (14:10 +0200)
commitd35400a7d7655e777224db9beddb44a68f4f2949
tree94ce4695cf66ac01a95354a70e95fa7fbce31624
parent92f6ec2328606666032f7e9adc0c3cdcd7c76804
use analyser provided in the 'analyzer' property

Implements per-name choice of analyzer. If a non-default
analyzer is choosen, then the 'word' identifier is extended
with the name of the ana;yzer, so that we still have unique
items.
nominatim/tokenizer/icu_rule_loader.py
nominatim/tokenizer/icu_token_analysis.py [new file with mode: 0644]
nominatim/tokenizer/icu_tokenizer.py
nominatim/tokenizer/sanitizers/split_name_list.py
nominatim/tokenizer/token_analysis/generic.py
test/python/test_tokenizer_icu.py