consider "modifier letter apostrophe" to be punctuation

While technically being a letter, the apostrophe is often replaced
with a normal apostrophe in writing which is a punctuation mark.
This makes sure that the modifier letter apostrophe yields the same
normalization results and thus is really interchangable.

Only has an effect after the next reimport.

Fixes #2569.

diff --git a/settings/icu_tokenizer.yaml b/settings/icu_tokenizer.yaml
index e5cbeb6f54ffcef3c3c0340e078c76f6084123e0..a3c62e67b8d2a500baee37223f56b151bce14324 100644 (file)
--- a/settings/icu_tokenizer.yaml
+++ b/settings/icu_tokenizer.yaml
@@ -7,7 +7,7 @@ normalization:
     - "'nº' > 'no'"
     - "ª > a"
     - "º > o"
-    - "[[:Punctuation:][:Symbol:]]  > ' '"
+    - "[[:Punctuation:][:Symbol:]\u02bc]  > ' '"
     - "ß > 'ss'" # German szet is unimbigiously equal to double ss
     - "[^[:Letter:] [:Number:] [:Space:]] >"
     - "[:Lm:] >"