Updated copyright notices to include the year 2018.

global: start relying on buffer_free(NULL) being a no-op Cleanup performed with the following semantic patch: @@ expression E; @@ - if (E != NULL) { - buffer_free(&E); - } + buffer_free(&E);

Updated copyright notices to include the year 2017.

global: freshen copyright git ls-files | xargs perl -p -i -e 's/(\d+)-201[0-5]/$1-2016/g;s/ (201[0-5]) Dovecot/ $1-2016 Dovecot/'

lib-fts: use NULL rather than 0 to shut up sparse Signed-off-by: Phil Carmody <phil@dovecot.fi>

lib-fts: fts_tokenizer_reset() didn't reset the entire state.

lib-fts: Added asserts to make sure we don't return empty tokens.

lib-fts: Changed fts_tokenizer_next/final() to return error string. The current tokenizers can't fail, but if we're doing tokenization via external services they could fail.

lib-fts: Added fts_tokenizer_final() as a convenience wrapper.

lib-fts: Minor code cleanups

lib-fts: Store pointers to fts_tokenizer classes instead of copying the data. The same reason as for doing this for fts-filter.

lib-fts: Added fts_tokenizer_reset()

fts: Change tokenizer API to be able to return errors Modify fts_tokenizer_next() to return integer status codes. It returns 1 if a token was returned in *token_r, 0 if more input is needed and -1 on error.

fts: Create tokenizers differently Create tokenizers earlier. Create separate tokenizers for search and indexing. Enable configuration of tokenizers. Add some helpers in fts-tokenizer.h api. Change tokenizer unit tests to match those changes. lib-fts: Refactor lib-fts settings a bit Turned address tokenizer settings into "boolean" values. Changed have_parent to "no_parent" and added "search" setting. Added documentation in fts-tokenizer.h. Change unit tests accordingly.

Initial import for lib-fts. Parts of what this code does was already implemented internally by fts-lucene. lib-fts is intended to be usable for all the FTS backends. The APIs are still going to change a bit, but hopefully not after v2.2.17 release. Mostly written by Teemu Huovila.