fts: Added fts_no_autofuzzy setting to require exact matches for found results. This is done by using the FTS search results as only filters on which the regular non-FTS search is done.

fts-lucene: Added mime_parts parameter to index MIME parts as separate documents. This is in preparation for being able to actually read such information and show it to clients via some new IMAP extension. Also it might be better to index some MIME parts together to avoid wasting disk space, but for now this is all or nothing. For example anything that is "message body" could probably be indexed without the part number at all.

Hash table API is now (mostly) type safe.

Marked functions parameters that are allowed to be NULL. Some APIs were also changed. The non-obvious APIs where NULL parameter was changed to "" are master_service_init() and auth_master_user_list_init(). These checks can currently be enabled only on a patched clang: http://llvm.org/bugs/show_bug.cgi?id=6786

Moved GUID code to liblib. Use guid_128_t type consistently everywhere.

fts-lucene: Added default_language setting and separated stemmer/textcat support.

fts-lucene: Free all Lucene's memory on plugin deinit.

fts-lucene: Automatically delete corrupted indexes and mark all mailboxes unindexed.

fts-lucene: Use fts expunge log instead of scanning the entire index for expunged mails.

fts-lucene: Added "doveadm dump" support for lucene index.

fts-lucene: Several fixes and improvements.

fts-lucene: Added initial support for language detection and stemming. This requires textcat and snowball (libstemmer) libraries. This can be enabled by setting: fts_lucene = textcat_dir=/usr/share/libtextcat \ textcat_conf=/etc/dovecot/textcat.conf I'm not yet sure how good of an idea it is to actually use this, so it needs some testing.. For example how bad is it if language is detected incorrectly?

fts: Redesigned/enhanced FTS API and how virtual plugin works with it. The changes include: - Only indexer process and "doveadm index" do FTS indexing now. Other processes connect to the indexer process via UNIX socket and request indexing a specific mailbox. - FTS backends can now index/search any search key. Current backends implement indexing for some specific header fields (Subject, From, To, Cc, Bcc), but it would also be possible to add indexing for e.g. message size or date. - CLucene support is fixed and fully functional. - Solr is split to "solr" and "solr_old" backends. The new "solr" backend supports the new header fields. It also uses mailbox GUIDs as mailbox identifiers, so that renaming a mailbox doesn't require reindexing. The "solr_old" uses the old Solr schema and doesn't support any new features.

Compiler warning fixes --HG-- branch : HEAD

Changed .h ifdef/defines to use <NAME>_H format. --HG-- branch : HEAD

Create a locks directory under the lucene-indexed directory. Detect and delete stale locks. Fixed expunges to work. --HG-- branch : HEAD

Separate headers and body in building and searching. Added support for Lucene to index/search them separately. --HG-- branch : HEAD

Fixed filtering to work with Lucene. Generic FTS code now implements the filtering if the backend doesn't. --HG-- branch : HEAD

Add support for expunges. --HG-- branch : HEAD

Fixes and cleanups --HG-- branch : HEAD

Create only a single index into INBOX's index dir. Did several fixes. However still a bit buggy. --HG-- branch : HEAD

--with-lucene now enables lucene full text search indexing. Note that using it breaks IMAP RFC. It also seems to have problems finding texts that have special characters in them, such as email addresses. You can anyway enable it by loading fts and fts_lucene plugins and setting fts=lucene in plugin section. --HG-- branch : HEAD