lib-storage: Add tests for mailbox_verify_name

lib-storage: Remove mailbox_transaction_context.nontransactional_changes Nothing cares about them. It was only set for POP3 UIDL change with Maildir. In theory dsync replication would want to replicate such a change, but it doesn't actually support changing UIDLs for existing mails. Other mailbox formats don't support it anyway.

lib-storage: mail_storage_lock_create() - add support for dotlocks

lib-storage: Add mail_storage_lock_create() This is split off of mailbox_lock_file_create().

lib-storage: Add and use MAIL_STORAGE_CONTEXT_REQUIRE Satisfies static analyzers

lib-storage: Set keyword based on attachment presence when saving If attachment is detected, use $HasAttachment, if not use $HasNoAttachment

lib-storage: Add mail.event

lib-storage: Add mailbox.event

lib-storage: Add mail_user.event and give parent event in mail_user_alloc*()

lib-storage: Rebuild list index when doing doveadm force-resync

LAYOUT=index: Rebuild list index if INBOX is missing on inbox=yes namespace This avoids having to manually rebuild the list index afterwards if it becomes lost.

lib-storage: Rename mail_storage.list_index_corrupted() to list_index_rebuild() Also add a new parameter to it to specify the rebuild reason.

lib-storage: Add mailbox_set_critical() and mail_set_critical() These add the mailbox vname and mail UID to the log prefix and then call mail_storage_set_critical().

lib-storage: Implemented mailbox_is_autosubscribed(). Indicates whether the mailbox is automatically subscribed to when it is automatically created.

lib-storage: Remove obsolete mailbox_save_set_dest_mail()

lib-storage: Add reason to mailbox.transaction_begin()

lib-storage: Add mail_storage_set_index_error()

lib-storage: Panic if at mailbox_free() there are open attribute iterators.

lib-storage: mailbox_sync_init() - open mailbox immediately if it's not open yet This simplifies the work for plugins that want to hook into mailbox.sync_init() so they no longer have to handle the "mailbox isn't opened" case.

lib-storage: Move .vsize.lock creation to a generic mailbox_lock_file_create()

lib-storage: Make mailbox_is_autocreated() public This also removes duplicated code from acl plugin.

lib-storage: Fix mail_storage_last_error_push/pop() to work with internal errors

lib-storage: Add missing ATTR_FORMAT to mail_set_mail_cache_corrupted()

lib-storage: Use new mail_cache_set_seq_corrupted_reason

lib-storage: Add mail_storage.nonbody_access_fields This avoids index_mail_update_access_parts_pre() from opening the mail stream unnecessarily for fields that can be looked up via other methods by the storage.

lib-storage: Add mailbox_transaction_set_reason()

lib-storage: Add mailbox_set_reason() Use it to log a debug message when mailbox is being opened.

lib-storage: Indicate mail is being autoexpunged This way plugins, such as mail_log, can detect this.

lib-storage: Add mail_storage_get_last_internal_error() This returns the error given to mail_storage_set_critical().

lib-storage: Make sure mailbox undeletion won't go to infinite loop

lib-storage: Always create mail_save_context.dest_mail This allows removing similar mail_alloc() from storage backends and plugins that need it. As a side effect, this changes mbox code to always assign UIDs to saved mails. This shouldn't be much of a problem, since it happened practically always already.

lib-storage: Move index_mail.vsize_ext_id to mailbox.mail_vsize_ext_id It's mailbox-specific, so it doesn't have to be looked up every time. Also this is needed for the following fix.

index-storage: Update mailbox last_rename_stamp on rename

lib-storage: Add struct mail_save_context.copy_src_mail

lib-storage: Add mail_storage.list_index_corrupted() The actual implementation is in the next commit.

lib-storage: require MAIL_STORAGE_CLASS_FLAG_STUBS when caching If we are going to be using the storage for caching, we should check that the storage actually supports mail stubs.

lib-storage: Add support for storing "mail stubs" These allow treating a storage backend as a cache where the mail bodies don't necessarily exist.

lib-storage: Added struct mailbox.corrupted_mailbox_name This allows backends that are adding lost mailboxes to mailbox list index to indicate that they don't actually know the name of the mailbox, and the mailbox list index should try to figure it out and rename the mailbox to its proper name.

LAYOUT=index: Update mailbox name to mailbox index's header when opening. This way if the mailbox list index gets lost, a recent mailbox name is still available.

lib-storage: rename get_real_mail to get_backend_mail Also change the function signature so that it returns an int to indicate success or failure.

lib-storage: treat empty root_dir as an empty string

lib-storage: Don't reset mail_save_context.saving too early. If mailbox_save_using_mail() ended up in mail_storage_copy(), saving was set to FALSE before the copy() method was finished running. This caused e.g. notify plugin to think that this was a copy event instead of a save event. Added comments and asserts to clarify how the logic should work between all the different copying/moving/saving flags.

lib-storage: remove unused internal_attribute argument from attribute_{get,set} This removes an argument added in 9f37ef2a9192e7d47e3d7ac959080fd01120f2e9 but obsoleted (but not removed) in d1147c225f64596fc5eeb1cb2aab31b3c57d5215. Not only does this change simplify the codebase, it actually brings the API closer to what it was back in 2.2.

lib-storage: remove set_cache_corrupted_reason from mail_vfuncs Change set_cache_corrupted signature to what set_cache_corrupted_reason used to be.

lib-storage: Change mailbox_attribute_get* API lib-storage/mailbox-attribute.h: mailbox_attribute_get() and mailbox_attribute_get_stream() now has struct mailbox_transaction_context as their first parameter. This isn't good for performance, because opening a transaction requires opening mailbox indexes first. But reading attributes wouldn't actually require opening the indexes, so this is just wasteful.

lib-storage: Added mail_get_stream_because() and mail_get_hdr_stream_because() With mail_debug=yes each mail access is now logged with a reason. This can be helpful when figuring out why something isn't in dovecot.index.cache.

lib-storage: Added quick parameter to list_index_has_changed()

dbox: Optimize POP3 MAIL_FETCH_UIDL_BACKEND. We keep track of the highest UID known to have POP3 UIDL in index's header. If saving adds a newer message, it'll also update the header. When fetching UIDL_BACKEND, we can need to check only mails with lower UIDs. There are some race conditions here, but normally UIDLs are set only once during migration so it shouldn't matter.

global: unsigned int:1 -> bool:1 perl -i -pe 's/unsigned int ([^,:;]+):1;/bool $1:1;/' **/*.[ch]

virtual: Moved virtual_mailbox_vfuncs to lib-storage. This allows implementing other virtual storage backends.

lib-storage: Added MAIL_STORAGE_CLASS_FLAG_NO_LIST_DELETES and MAILBOX_LIST_FLAG_NO_DELETES

lib-storage: Added mail_set_cache_corrupted_reason()

dsync: Don't restrict what mailbox names can be created (except basic sanity checks) The mailbox names already exist on the other side, so we don't want to add arbitrary limits to them.

lib-storage: Don't add first_saved to mailbox list index unless autoexpunge is set.

Added mailbox { autoexpunge } setting. This can be used to automatically expunge mails from specified mailboxes after they're old enough. The expunges are done when the user is being deinitialized. mailbox_list_index=yes should be enabled to have the best performance with this setting. Example: namespace inbox { mailbox Spam { auto = create special_use = \Junk autoexpunge = 30d } }

lib-storage: Allow set/get for Dovecot-private attributes via internal attributes. This allows registering attributes with MAILBOX_ATTRIBUTE_PREFIX_DOVECOT_PVT prefix and having them be get/set via dict, instead of failing them.

lib-storage: Moved index/index-mailbox-check.c code to mailbox-watch.c This changes the API, but adds backwards compatibility macros.

lib-storage: Moved most of the \Recent flag handling code to mailbox-recent-flags.c There are also some API changes, because functions were renamed and the recent_* fields were moved to struct mailbox. I'm not aware of any plugins using these though, except for index_mailbox_set_recent_seq() which for now is kept as a backwards compatibility macro. No changes were made to the actual code logic.

lib-storage: Moved vsize_hdr_ext_id to struct mailbox for more global access.

lib-storage: Renamed struct index_vsize_header to struct mailbox_index_vsize Also moved the struct to mail-storage-private.h for more global access.

mail-storage: Moved mailbox attributes API to separate module.

lib-storage: Added mailbox_get_index_path() and caching of the index path. This solves some potential assert-crashes on race conditions when opening a mailbox while it's being deleted.

lib-index, lib-storage: Fixed race conditions with deleting mailbox. Now only one process can successfully finish mailbox_mark_index_deleted().

lib-storage: Added mail_storage_last_error_push/pop()

lib-storage: Move "shared" storage name to mail-storage-private.h where it can be used.

lib-storage: mail_never_cache_fields=* means now to disable all caching.

lib-storage: Partially reverted the mail.get_real_mail() API change for backwards compatibility. I hadn't realized Pigeonhole was also using it. (I thought it was using only the mail_get_real_mail() public API.) This fixes Pigeonhole v0.4.2 to work again.

lib-storage: Fixed crash if mailbox_save_cancel() was called in save_finish()/copy() method.

lib-storage: Replaced mail_get_real_mail() with mail_get_backend_mail() that can fail. For now the mail_get_real_mail() can still be used for backwards compatibility.

dsync: Support syncing storages with 128bit GUIDs <-> string GUIDs.

lib-storage: Added MAIL_STORAGE_CLASS_FLAG_BINARY_DATA flag for classes.

imapc: Fixed mailbox deletion.

lib-storage: Added mailbox_status.have_save_guids.

lib-storage: Added a way to create plugins that forcibly hook into mail storage. Mostly meant for doveadm_sieve plugin.

lib-storage: mailbox_attribute_set() now uses struct mail_attribute_value. This allows settig the last_change value, as well as using streams and setting other flags in future.

lib-storage: Changed mailbox_attribute_get/set() APIs to have a transaction and update index. The _get() doesn't currently actually use the transaction. It was mainly put there for future use where it could be useful. It's also possible that _iter() will need a transaction also. For now these decisions seem good enough.

lib-storage: Saving/copying no longer discards private message flags.

lib-storage: Assert-crash if mailbox_save_context isn't properly freed before reusing.

lib-storage: Added mailbox_status.have_guids flag

lib-storage: Changed mailbox_attribute_get() to return a struct for value. This allows returning flags for the values in future. Mainly readonly-flag for internal attributes and when restricted by ACL plugin.

lib-storage: Bypass virtual storage when copying mail to allow optimizations. As a side effect this also causes all plugins to see the backend mail instead of the virtual mail. This probably isn't bad in most situations, but for those where it's unwanted mail_private.vmail can be used to access the virtual mail. mail_log plugin does this to log the original virtual mailbox name for copy operations.

lib-storage: Added support for "private modseqs" for INDEXPVT index. Used by dsync (next commit).

lib-storage: Comment update for mail_prefetch()

lib-storage: Renamed mail_storage_set_index_error() to mailbox_set_index_error()

Replaced *_INDEX_PREFIX macros with a common MAIL_INDEX_PREFIX.

lib-storage: Fixes to handling separation of private/shared attributes.

lib-storage: Moved mail attribute dict to struct mail_storage. This also means that index_storage_destroy() must always be called, so removed now unnecessary mail_storage.destroy=NULL checks.

imap: URLFETCH BINARY BODYPARTSTRUCTURE returns binary-decoded line counts.

lib-storage: Added support to get/set/iterate mailbox attributes. The attributes are stored in a dict specified by mail_attribute_dict setting. The idea is to use this as storage for extensions that require per-mailbox key=value pairs, such as METADATA.

Backported parts of normalizer_func_t changes from v2.2 tree.

lib-storage: mailbox_get_path() and friends can now return an error. An upcoming "index" mailbox_list backend uses mailbox GUIDs as the directory names. Doing a name => GUID lookup may fail, so the path lookup may fail. This is a somewhat annoying change and perhaps it can be removed in future, but for now it looks like it can't be avoided.

lib-storage: Cleanups to mailbox mkdir() related functions.

lib-storage: Added mailbox_get_path_to() wrapper for mailbox_list_get_path(). Changed the code to use as much of mailbox_get_path_to() and mailbox_get_path() as possible.

lib-storage: Added mail_storage_copy_error()

lib-storage: Moved index_mail.data_pool to mail_private.data_pool

Replaced "decomposed titlecase" conversions with more generic normalizer function. Plugins can now change mail_user.default_normalizer. Specific searches can also use different normalizers by changing mail_search_context.normalizer.

lib-storage API change to allow using it from C++ code

lib-storage: Moved mail_storage_settings_to_index_flags to mail-storage-private.h This is mainly to allow #including mail-storage-settings.h from C++ code without having to #include mail-index.h from mail-storage-settings.h

lib-storage: Removed rename_children parameter from mailbox_rename() The children are now always renamed. The only reason this feature was added in the first place was to make dsync's job easier, but this is no longer needed.

lib-storage: Moved per-mail data in struct mail_save_context to separate struct. This fixes previous save leaking metadata to the next save.

Array API changed: ARRAY_DEFINE(name, type) -> ARRAY(type) name Easy way to update your existing code: perl -i -pe 's:ARRAY_DEFINE\(([^,]+), *([^)]+)\);:ARRAY($2) $1;:' **/*.[ch]

IMAP BINARY extension supports now FETCH BINARY command.

Make sure we check all the functions' return values. Minor API changes to simplify this. Checked using a patched clang that adds attribute(warn_unused_result) to all functions. This commit fixes several error handling mistakes.

lib-imap-storage: imap-msgpart rewrite and API change. The new API allows first parsing the validity of section strings and later relying on them being valid without having to re-parse it. The implementation also fixes a few things and adds "partial fetch cache".

lib-storage: Added mailbox_move(), which is basically copy+expunge.

shared mailboxes: Per-user flags can now be stored in private index files. This can be enabled by adding e.g.: mail_location = mdbox:/var/shared/mdbox:INDEXPVT=~/mdbox/shared

lib-storage: Added a mailbox.synced flag, which is set when mailbox sync has succeeded.

lib-storage: Allocate MAILBOX_METADATA_CACHE_FIELDS from a new metadata mempool.

lib-storage: Added mailbox_delete_empty().

lib-storage: Added mailbox_save_set_pop3_order() and implemented for Maildir.

quota: Added a kludge to avoid recalculating quota.

lib-storage: mail_transaction_commit_changes.changed=TRUE only when something actually changed.

lib-storage: Removed min_timeout parameter from mailbox_notify_changes() It's now internally taken from mailbox_idle_check_interval setting.

lib-storage: Added mailbox_save_using_mail()

lib-storage: Renamed mail_save_context.copying to copying_via_save.

lib-storage: Added struct mailbox.set_subscribed() virtual method.

Moved autocreate plugin functionality into lib-storage. The autocreate plugin is still used for backwards compatibility. Mailboxes can be configured like: mailbox Sent { auto = subscribe } mailbox Spam { auto = create }

lib-storage: Track storage's all mailboxes to make it easier to debug if one isn't closed.

lib-storage: Added mail_log_update_wanted_fields()

lib-storage: Added mail_get_hdr_stream() and use it where possible. This makes it clearer for backends when it needs a message body instead of only message header.

lib-storage: Mailbox list index's ext_id shouldn't be cached in mail_storage. It's mailbox_list-specific, not mail_storage-specific. The easiest fix was to just cache it into mailbox rather than create a new index_mailbox_list.

lib-storage: Added mailbox.disallow_new_keywords and use it for mailbox_status.

Redesigned mail precaching APIs. There's no longer a one monolithic mailbox_sync(MAILBOX_SYNC_FLAG_PRECACHE) call, but rather one mail_precache() call for each mail to be precached. This allows the callers to show the progress and in general is cleaner.

lib-storage: Moved all transaction stats_* to struct mailbox_transaction_stats.

lib-storage: Moved stats_* from mail to transaction. With the API changes to search, having them in mail doesn't make sense anymore, since mailbox_search_next() can return multiple different mails with different stats.

lib-storage: mailbox_list_get_permissions() now returns struct mailbox_permissions. Also added new file_uid and file_gid fields to the mailbox_permissions.

lib-storage: Added mailbox_alloc_guid() for opening mailbox by its GUID.

lib-storage: Added struct mailbox.generation_sequence. This can be used for caching struct mailbox pointers and checking later if two struct mailboxes + their generation sequences match. If they do, it's the same struct mailbox.

lib-storage: Added auto_boxes parameter to mailbox_exists().

lib-storage: mail->saving was set too late, which could have caused crashes.

Moved the main functionality from "doveadm index" to MAILBOX_SYNC_FLAG_PRECACHE This also allows plugins to hook into the sync and implement their own precaching easily. fts indexing is now done this way rather than kludging.

lib-storage: Export a list of permanent flags via mailbox_status(). Removed mailbox_allow_new_keywords(), since it's now included in the status structure. ACL plugin may limit what flags are permanent according to ACLs.

lib-storage: Added mail_parse().

lib-storage: Added mail_is_cached()

lib-storage: Removed struct mail_private.extra_* fields. There isn't much point in keeping them separated.

Search supports now prefetching data for returned mails. Dropped imapc's own prefetching. mail_prefetch_count specifies how many mails can be kept open and issue a prefetch. This works using posix_fadvise(POSIX_FADV_WILLNEED) for maildir, sdbox and cydir backends. Apparently only Linux supports this. imapc backend also implements this internally by sending wanted IMAP commands to remote server. The command pipelining helps with latency. This change also makes it actually possible for imapc backend to first check if wanted data is already cached in local index and avoid sending unnecessary IMAP commands to remote server.

lib-storage: Changed mailbox_search_*() API to return pointer struct mail. This reduces the flexibility, because now different mailbox_search_next() calls can't have different mail parameters with possibly different wanted_fields settings. This flexibility was never used though, and removing it gives a couple of benefits: * it's now easier for backends to implement prefetch and parallel search * usage is easier since mail_alloc()/mail_free() is no longer needed

lib-storage: Fixed mailbox_get_expunges() to return GUIDs with all mailbox formats. Also added mailbox_get_expunged_uids() for use cases that don't really need GUIDs.

lib-storage: Moved index_mailbox_context.save_*() methods to mailbox_vfuncs.

sdbox: Fixed creating mailbox with given GUID.

sdbox: Fixed creating mailbox with given GUID.

lib-storage: mailbox_exists() can now return if it's selectable or non-selectable.

lib-storage: Return STATUS replies from mailbox list index only when it's fully up-to-date.

imapc: Don't force a root directory to exist.

lib-storage: Moved setting "wanted fields/headers" fields for sort program from imap-specific code. Also separate between fields that are actually wanted to be fetched from the fields that sorting wants to temporarily use.

lib-storage: mailbox_alloc() now takes a virtual mailbox name and other related API changes. All storage_name <-> vname conversions now go through the same two mailbox_list methods. This has many benefits, such as: * listescape plugin is now much simpler and bugfree * allows changing lib-storage API to use UTF-8 mailbox names in future * allows creation of "mailbox aliases" plugin

lib-storage: Replaced mailbox_list_get_name_status() with mailbox_exists().

lib-storage: Moved some items from mailbox_get_status() to a new mailbox_get_metadata(). The idea is now that all status items are tracked all the time after mailbox is opened and they can always be looked up without failure. The metadata items are looked up lazily and the lookups may fail at any time. mailbox_get_status() can be used after mailbox_alloc() to indicate that the mailbox doesn't necessarily have to be opened, just that the status fields get returned. If mailbox is already known to be open, mailbox_get_open_status() can be used. It never fails.

lib-storage: mailbox_get_status() can now fail. It might have failed earlier also, but the errors were hidden from caller.

lib-storage: Moved mailbox.save_is_attachment() to mail_save_context.part_is_attachment()

lib-storage: Unvirtualized mailbox_header_lookup_*()

lib-storage: Unvirtualized some methods from struct mailbox. There was never any point in overriding or hooking into them.

Moved fts/virtual plugin communication functions away from lib-storage.

lib-storage: Simplified mailbox_keyword*() APIs. They are no longer struct mailbox methods, because they're always implemented as being wrappers to lib-index APIs anyway. Also mailbox_keywords_ref/unref() no longer take mailbox parameter.

lib-storage: Removed mailbox.backend_readonly. Backends set it internally lazily now.

lib-storage: Moved box->private_flags_mask to mailbox_get_private_flags_mask() This allows getting it lazily only when it's actually needed.

lib-storage: Don't allocate index until mailbox_open()

lib-storage: Lazily look up mailbox path and permissions. Added functions to get them.

lib-storage: Removed struct index_transaction_context. All of its contents are now in struct mailbox_transaction_context.

lib-storage: If disable_reflink_copy_to=TRUE, disable hardlink/refcount copying.

virtual: Set MAIL_STORAGE_CLASS_FLAG_NOQUOTA to storage's class_flags.

lib-storage: Added istream_opened() method to struct mail. This can be hooked into by plugins that want to do something with the message's stream in the message file, rather than the virtual resulting stream visible to client. For example when using dbox attachments, this stream contains only the data in the dbox files without any external attachments.

lib-storage: Added support for saving mail attachments separately via filesystem API. Currently this works only with sdbox and mdbox backends.

lib-storage: Allow mailbox_mark_index_deleted() to be called multiple times within same mailbox alloc.

Fixes to INBOX sharing. NAMESPACE_FLAG_INBOX was split to two flags: _FLAG_INBOX_USER (this namespace has the user's own INBOX) and _FLAG_INBOX_ANY (namespace has INBOX, either user's own or another user's). --HG-- branch : HEAD

Removed mail_update_uid() / mail_index_update_uid(). It was working properly only with mdbox and there wasn't really a need for it. --HG-- branch : HEAD

Added mail_max_lock_timeout setting. This could be useful inside lda/lmtp protocol sections to avoid them from spending too much time waiting for locks, since they can easily just return "try again later". --HG-- branch : HEAD

lib-storage: Plugin API changed to run plugin functions in correct order. Previously the hooks were run in correct order, but the functions they overrode were run in reverse order. This caused problems when multiple plugins were used. --HG-- branch : HEAD

Mailbox deletion: Delay marking its index deleted until ACL has had a chance to abort the deletion. --HG-- branch : HEAD

lib-storage: Removed unnecessary struct mail_save_context.saved_physical_size --HG-- branch : HEAD

Use IO_BLOCK_SIZE macro to specify how large read/write syscalls to use. --HG-- branch : HEAD

lib-storage: Removed struct mail.get_index_mail(). get_real_mail() is basically the same. --HG-- branch : HEAD

lib-storage: Added mail_get_real_mail() that returns physical mail in a virtual mailbox. --HG-- branch : HEAD

lib-storage: Added search query building from human-friendly input. --HG-- branch : HEAD

lib-storage: Rewrote search query building code to be more modular. --HG-- branch : HEAD

lib-storage: Panic if deiniting storage before closing its mailboxes. --HG-- branch : HEAD

lib-storage: mail_get_parts() no longer returns const pointer. The struct message_part contained context-pointer and other APIs were using it, so the pointer really couldn't be const without casting it away sometimes. --HG-- branch : HEAD

lib-storage: Moved index transaction/view from index_transaction to mailbox_transaction. --HG-- branch : HEAD

lib-storage: Added support for plugins to specify message's physical size when saving. Changed dbox to save this value to metadata, and use when present. --HG-- branch : HEAD

lib-storage: Removed INBOX auto-creating from backend code. Added bool box->inbox. --HG-- branch : HEAD

lib-storage: Added mailbox.free() method. Use it instead of close() where necessary. --HG-- branch : HEAD

lib-storage: Mailbox renaming API changed. --HG-- branch : HEAD

lib-storage: Removed input parameter from mailbox_alloc(), added mailbox_open_stream() --HG-- branch : HEAD

lib-storage: STATUS_GUID moved to mailbox_get_guid() that can fail. --HG-- branch : HEAD

Mailbox deletion: Fixed race condition where a mailbox couldn't get deleted. --HG-- branch : HEAD

lib-storage: Mailbox deletion API changed. Mailbox deletion should now be free of race conditions. The actual file deletion code is now responsibility of mailbox_list backend. --HG-- branch : HEAD

lib-storage: *_mailboxes don't descend from index_mailbox anymore, it's now a context. --HG-- branch : HEAD

lib-storage: Moved struct mail_index_* from index_mailbox to mailbox. We're relying more and more of all mailboxes being used via lib-index, and this change makes accessing the indexes easier. --HG-- branch : HEAD

lib-storage: Moved mailbox directory creation code to mailbox_list backend. --HG-- branch : HEAD

lib-storage: Changed mailbox_sync() and mailbox_sync_deinit() APIs. Although we're already in beta stage, this is simple enough of a change that it shouldn't matter much. Having syncing also return status information made the API ugly and the status information wasn't even wanted all that often. --HG-- branch : HEAD

lib-storage: Plugin hook API changed to support per-user plugins within a process. --HG-- branch : HEAD

mailbox_search_next*() API changed to return bool. If search fails, it'll be noticed anyway by mailbox_search_deinit(). --HG-- branch : HEAD

lib-storage: Moved mail_generate_*() to be public functions. --HG-- branch : HEAD

Added pop3_save_uidl setting. When UIDLs are sent to client, save them to dovecot-uidlist. This allows changing pop3_uidl_format without messages getting re-downloaded. It's also useful with virtual POP3 INBOX when UIDLs are based on IMAP UIDs that may not be as stable as in non-virtual INBOX. --HG-- branch : HEAD

mailbox_copy() now sets mail_save_context.copying=TRUE. --HG-- branch : HEAD

Use separate read block sizes when reading only message header vs. full message. --HG-- branch : HEAD

Removed mailbox_guid_is_empty(), use mail_guid_128_is_empty() directly instead. --HG-- branch : HEAD

index-storage: Simplified the way transactions are are created/finished. Removed also mail_storage.class_init/deinit since they're no longer used for anything. --HG-- branch : HEAD

Removed MAILBOX_GUID_SIZE macro, use MAIL_GUID_128_SIZE everywhere. They were required to be the same size anyway. --HG-- branch : HEAD

Added ability to specify message's minimum modseq value. --HG-- branch : HEAD

Added ability to change existing messages' UIDs with mail_[index_]update_uid(). --HG-- branch : HEAD

mailbox_transaction_commit_get_uids() changed to _get_changes() with more generic API. Saved UIDs are no longer necessarily in a simple x:y range. --HG-- branch : HEAD

mail_guid_128_is_empty() is now a public function. --HG-- branch : HEAD

mail_generate_guid_128_hash() is now a public function. --HG-- branch : HEAD

Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). The message GUIDs are stored in expunge records to transaction log. Before doing the final expunge, Maildir and dbox verify that the GUID in expunge request matches the current actual GUID. The GUID is stored in 128 bit field. If the real GUID isn't 128 bit, the bits are taken from SHA1 of the GUID. --HG-- branch : HEAD

lib-storage: Added mailbox_get_vname(). --HG-- branch : HEAD

Mailbox saving: Allow setting IMAP UID, POP3 UIDL and save date. Most of them aren't yet implemented by backends. --HG-- branch : HEAD

Whenever file's group changing fails, show the group origin in the error message. --HG-- branch : HEAD

Added support for creating/updating mailboxes with given metadata (guid, uid validity, etc). --HG-- branch : HEAD

Added reference counting to struct mail_keywords and related APIs. --HG-- branch : HEAD

Mailbox opening and closing APIs changed. --HG-- branch : HEAD

Added mailbox_keywords_create_from_indexes(). --HG-- branch : HEAD

mailbox_header_lookup*() private API changed. --HG-- branch : HEAD

struct mail_storage.callbacks is no longer a pointer. --HG-- branch : HEAD

mail storages can now be shared between namespaces. --HG-- branch : HEAD

Initial commit for making mail_storage:mailbox_list to be from 1:1 to n:n. This will allow namespaces to share the same storage and a single namespace to have multiple storages (multiple mailbox formats). Neither works currently, and this commit probably breaks some things. --HG-- branch : HEAD

Split struct mail_private.stats_dentry_lookup_count to open/stat counts. --HG-- branch : HEAD

struct mail_private now contains all kinds of statistics about file accesses. --HG-- branch : HEAD

Added struct mail.lookup_abort, which can be used to abort lookups that can't be done using cache. --HG-- branch : HEAD

Moved MAIL_READ_BLOCK_SIZE to mail-storage-private.h --HG-- branch : HEAD

Added mail_storage_purge() for dbox. dbox no longer calls it automatically at logout. --HG-- branch : HEAD

Maildir and Cydir: When saving mail, expose ostream to mail_save_context.output. This would allow plugins to hook into it. --HG-- branch : HEAD

Added mail_user_get_temp_prefix() and mail_storage_get_temp_prefix(). --HG-- branch : HEAD

Mail copying: Avoid updating cache file because of copying. The mail is being opened in any case, so it's unlikely to help anything. --HG-- branch : HEAD

dbox: Initial support for saving multiple messages per file. --HG-- branch : HEAD

Mail copying API changed to be based on save API. This allows extending it easily in future. --HG-- branch : HEAD

Initial commit for config rewrite. --HG-- branch : HEAD

Improved "Permission denied" error handling. It'll now show also the uid/gid name. Also moved the code to mail-error.c and used it when listing mailboxes. --HG-- branch : HEAD

fts: Hide "Searching .." notify while indexing. --HG-- branch : HEAD

Give more correct "* OK searched n%" notifications with fts and virtual mailboxes. --HG-- branch : HEAD

fts: Added support for handling multiple namespaces. --HG-- branch : HEAD

Finished making full text search indexes work fast with virtual mailboxes. --HG-- branch : HEAD

Initial support for getting full text search working nicely with virtual mailboxes. --HG-- branch : HEAD

Added wanted_fields and wanted_headers fields to struct mail_search_context. --HG-- branch : HEAD

mail_storage.search_next_update_seq returns now bool. It was never failing. --HG-- branch : HEAD

ACLs: Initial support for "post" right. --HG-- branch : HEAD

Implemented imap-response-codes draft. --HG-- branch : HEAD

Make mailbox_header_lookup_ctx's list of headers available. --HG-- branch : HEAD

Fixed shared mailboxes to work with plugins. --HG-- branch : HEAD

Added struct mail_user and fixed the code to support multiple users per process. --HG-- branch : HEAD

Added the concept of Global UIDs that are preserved across copies. They can be fetched, searched and specified for the save API. dbox format supports them correctly, but with maildir the base filename is used so maildir_copy_preserve_filename=yes is required to preserve GUIDs when copying. --HG-- branch : HEAD

mailbox_save_init() API was split to multiple functions. This allows adding more parameters easily in future. --HG-- branch : HEAD

Added mailbox_keyword_is_valid(). --HG-- branch : HEAD

mailbox.save_init() API now takes ** pointer to dest_mail, so multiple plugins can safely specify it. ctx->dest_mail also gets updated then. --HG-- branch : HEAD

struct mailbox_header_lookup_ctx can now be referenced/unreferenced so it can be shared among multiple struct mails. --HG-- branch : HEAD

CONDSTORE: STORE UNCHANGEDSINCE conflicts are now checked atomically. --HG-- branch : HEAD

Added hook_mailbox_index_opened. --HG-- branch : HEAD

Added mailbox_get_uid_range(). --HG-- branch : HEAD

Renamed mailbox_get_uids() to mailbox_get_seq_range(). --HG-- branch : HEAD

Implemented CONTEXT=SEARCH extension. --HG-- branch : HEAD

Make mail_search_args an independent structure that can be used for multiple separate searches. --HG-- branch : HEAD

Added QRESYNC support. --HG-- branch : HEAD

struct mail can't necessarily be casted to struct index_mail. Added a new get_index_mail() method to return it. --HG-- branch : HEAD

Added transaction flags to struct mailbox_transaction_context. --HG-- branch : HEAD

struct mail can't necessarily be casted to struct index_mail. Added a new get_index_mail() method to return it. --HG-- branch : HEAD

Added transaction flags to struct mailbox_transaction_context. --HG-- branch : HEAD

Initial CONDSTORE support. --HG-- branch : HEAD

Added mail_set_cache_corrupted() to rebuild the cache file and force recalculating the given field. --HG-- branch : HEAD

Added mail_set_cache_corrupted() to rebuild the cache file and force recalculating the given field. --HG-- branch : HEAD

If mail storage or mailbox can't be accessed because of EACCES, log an error message that clearly shows where the permission problem is. --HG-- branch : HEAD

Added mail_get_keyword_indexes() --HG-- branch : HEAD

Delay creating directories until we really need them. --HG-- branch : HEAD

Moved mailbox_deleted to struct mailbox. Added and used mailbox_set_deleted() which also sets an error message to storage. --HG-- branch : HEAD

Moved mail_create_mode/gid from struct maildir_mailbox to struct mailbox. --HG-- branch : HEAD

mailbox_transaction_commit*() doesn't sync the mailbox anymore, so it shouldn't take mailbox_sync_flags parameter either. --HG-- branch : HEAD

Renamed __attr_*__ to ATTR_*. Renamed __attrs_used__ to ATTRS_DEFINED. --HG-- branch : HEAD

Changed .h ifdef/defines to use <NAME>_H format. --HG-- branch : HEAD

Changed mail_keywords creation APIs to take mailbox/index instead of transaction. --HG-- branch : HEAD

mailbox_keywords_create() checks keyword validity now and returns failure if any of them are invalid. Added mailbox_keywords_create_valid() that only drops invalid keywords. Removed mbox and IMAP-specific keyword checks, they all use the same checking now. --HG-- branch : HEAD

Added mail.close() virtual function. --HG-- branch : HEAD

mail_*() APIs changed to return int and return the actual data as pointer. Changed some code to do error handling a bit better. --HG-- branch : HEAD

mail_index_view_sync_next() and mailbox_sync_next() returns now bool. Renamed void mailbox_index_view_sync_end() to int mailbox_index_view_sync_commit(). --HG-- branch : HEAD

mail_index_lookup*() can't fail anymore. Changed several APIs not to return failure anymore. --HG-- branch : HEAD

Removed explicit locking from views and maps. They were already locked all the time when they were used. Because of this change several functions can no longer fail, so they were changed to return void, and a lot of pointless error handling was removed. --HG-- branch : HEAD

mailbox_transaction_commit_get_uids() now returns also UIDVALIDITY. It's the only usable way to get it returned correctly. Removed maildir-specific mailbox_get_status() handling, it's supposed to return the current state of the mailbox, not the latest state. It was useful only for returning non-zero UIDVALIDITY for newly created mailboxes, and that's no longer needed. --HG-- branch : HEAD

Added mail_get_first_header_utf8() and mail_get_headers_utf8(). --HG-- branch : HEAD

Moved mailbox list index sync stamp checking and updating to storage-specific code. --HG-- branch : HEAD