virtual: Contexts are now required or checked Satisfies static analyzers

virtual: Make sure sync_mailbox_idx isn't used uninitialized Changing its indexes to start from 1, we can add an assert to check that it's never the default 0 when we need to use it.

virtual: Make sure index is rewritten when new mailboxes are added Otherwise multiple bbox->sync_mailbox_idx point to 0 and they keep overwriting each others in the header. This causes the header to be constantly updated with wrong contents.

virtual: Stop notifications only if they were started Fixes Panic: file mail-storage.c: line 1950: assertion failed: (box->opened)

virtual: Use mailbox_notify_list API when mailbox_list_index is enabled

virtual: Generate GUID for virtual folder GUID is generated when index is first created or the rules change.

virtual: Fix saving to a virtual mailbox to work again Previous changes broke it.

global: unsigned int:1 -> bool:1 perl -i -pe 's/unsigned int ([^,:;]+):1;/bool $1:1;/' **/*.[ch]

virtual: Moved virtual_mailbox_vfuncs to lib-storage. This allows implementing other virtual storage backends.

virtual plugin: If we detect backend box to be deleted, mark it permanently deleted.

virtual plugin: Added support for filtering mailboxes by METADATA entries. Usage: <mailbox patterns as usual> [-]/<metadata-entry-name>:<value-wildcard> There can be multiple metadata entries. All the entries must match. For example: * /private/vendor/vendor.dovecot/virtual:* -/private/vendor/vendor.dovecot/virtual:ignore This matches all mailboxes, which contain a virtual METADATA entry that has any value except "ignore". Note that the current implementation requires still opening all the mailboxes before matching the METADATA entries. This could be avoided in v2.3 with some lib-storage API changes.

virtual plugin: Don't treat "+-box" as a negative entry. This was already documented by wiki to work as a mailbox named "-box", although it didn't actually work that way. There wasn't any practical difference between "+-box" and "-box", so this change is unlikely to break anyone's config. It was mainly done as code cleanup.

virtual: Fixed assert-crash when opening virtual mailbox triggered backend mailbox autocreation. Fixes assert: Panic: file virtual-storage.c: line 335 (virtual_mailbox_opened_hook): assertion failed: (!bbox->open_tracked)

virtual plugin: Auto-close mailboxes by hooking into backend's mailbox_open() This is a more reliable way of doing it. The earlier version had problems that all the backend mailboxes could be opened when doing a search on the virtual mailbox (for getting the GUID of the mailboxes).

virtual plugin: Fixed assert-crash when closing a virtual mailbox consisting of many real mailboxes.

virtual plugin: Read virtual index header when opening virtual mailbox. This allows doing some mailbox accessing without actually syncing the mailbox. For example MAIL_FETCH_MAILBOX_NAME can be used without crashing.

virtual: Never keep more than specified number of physical mailboxes open. This should make virtual mailboxes work for users who have a a ton of mailboxes with a ton of mails. Earlier code would likely have failed either with "Too many open files" or crashed with "Out of memory". You can change the max number of open mailboxes with: plugin { virtual_max_open_mailboxes = 64 } The default is 64.

virtual: Fixed assert-crash in IDLE caused by earlier changes. This change negates all the improvements of the earlier changes when IDLE is used. This could be fixed by using mailbox-list-notify.h API.

virtual: Delay initializing search args until the mailbox is actually opened. Fixes assert-crash caused by recent changes when virtual mailbox search args contained e.g. keywords.

virtual: Delay opening the backend mailboxes until it's really necessary. This should improve the performance a lot with mailbox_list_index=yes when opening virtual mailboxes whose backend mailboxes are mostly unchanged.

lib-storage: Added mailbox_status.have_save_guids.

lib-storage: Added mailbox_status.have_guids flag

Replaced *_INDEX_PREFIX macros with a common MAIL_INDEX_PREFIX.

Array API changed: ARRAY_DEFINE(name, type) -> ARRAY(type) name Easy way to update your existing code: perl -i -pe 's:ARRAY_DEFINE\(([^,]+), *([^)]+)\);:ARRAY($2) $1;:' **/*.[ch]

fts: Redesigned/enhanced FTS API and how virtual plugin works with it. The changes include: - Only indexer process and "doveadm index" do FTS indexing now. Other processes connect to the indexer process via UNIX socket and request indexing a specific mailbox. - FTS backends can now index/search any search key. Current backends implement indexing for some specific header fields (Subject, From, To, Cc, Bcc), but it would also be possible to add indexing for e.g. message size or date. - CLucene support is fixed and fully functional. - Solr is split to "solr" and "solr_old" backends. The new "solr" backend supports the new header fields. It also uses mailbox GUIDs as mailbox identifiers, so that renaming a mailbox doesn't require reindexing. The "solr_old" uses the old Solr schema and doesn't support any new features.

lib-storage: Changed mailbox_search_*() API to return pointer struct mail. This reduces the flexibility, because now different mailbox_search_next() calls can't have different mail parameters with possibly different wanted_fields settings. This flexibility was never used though, and removing it gives a couple of benefits: * it's now easier for backends to implement prefetch and parallel search * usage is easier since mail_alloc()/mail_free() is no longer needed

virtual: Support +mailbox entries that clear \Recent flag from messages.

Moved fts/virtual plugin communication functions away from lib-storage.

virtual: If non-matching messages weren't expunged within same session, they never got expunged. --HG-- branch : HEAD

Avoid including mailbox-list-private.h everywhere. --HG-- branch : HEAD

lib-storage: *_mailboxes don't descend from index_mailbox anymore, it's now a context. --HG-- branch : HEAD

mailbox_search_next*() API changed to return bool. If search fails, it'll be noticed anyway by mailbox_search_deinit(). --HG-- branch : HEAD

index-storage: Simplified the way transactions are are created/finished. Removed also mail_storage.class_init/deinit since they're no longer used for anything. --HG-- branch : HEAD

virtual: If another session adds a new mailbox to index, handle it without crashing. For now we'll just disconnect the session. --HG-- branch : HEAD

Mailbox opening and closing APIs changed. --HG-- branch : HEAD

Initial commit for making mail_storage:mailbox_list to be from 1:1 to n:n. This will allow namespaces to share the same storage and a single namespace to have multiple storages (multiple mailbox formats). Neither works currently, and this commit probably breaks some things. --HG-- branch : HEAD

virtual: Fixed crashes when simultaneous connections accessed the same virtual mailbox. --HG-- branch : HEAD

virtual: If we can't open a mailbox that was added using a wildcard, just skip it. --HG-- branch : HEAD

virtual: Added support for saving/copying messages to virtual mailboxes. --HG-- branch : HEAD

fts: Added support for handling multiple namespaces. --HG-- branch : HEAD

Finished making full text search indexes work fast with virtual mailboxes. --HG-- branch : HEAD

Initial support for getting full text search working nicely with virtual mailboxes. --HG-- branch : HEAD

Optimized searching with virtual mailboxes. Instead of going through the messages in the virtual mailbox order, go them through one mailbox at a time and in ascending message order within that mailbox. --HG-- branch : HEAD

Virtual mailboxes: Fixes to handling errors. --HG-- branch : HEAD

Virtual mailboxes: If CRC32 of the search strings changes, rebuild the mailbox. --HG-- branch : HEAD

virtual mailboxes: Detect loops in virtual mailbox definitions. --HG-- branch : HEAD

virtual mailboxes: Added support for using wildcards in mailbox names. The wildcards don't span namespaces so that "*" won't also match mailboxes in virtual namespace. --HG-- branch : HEAD

Virtual mailboxes: Speed up initial search result building using modseqs. --HG-- branch : HEAD

virtual mailboxes: Use saved search results to find out which messages were added/removed. Also several other fixes and optimizations. --HG-- branch : HEAD

Added initial support for virtual mailboxes. --HG-- branch : HEAD