Allow matching substrings in free text search? #28
Description
The FTS5 trigram tokenizer allows matching substrings in general (rather than only complete tokens or prefix tokens), but it does not match substrings shorter than 3 unicode characters. Supporting short tokens is required (e.g. general structure contains mainly short forms like V DO), so if substring matching is desired a different solution than SQLite FTS is needed.
Prefix tokens are supported by the (default) unicode61 tokenizer, which is used now.