lib-index: Add mail_index_transaction_commit_result.changes_mask This can be used to determine what type of changes were committed in a transaction.

lib-index: Unconstifty pointer array clang complained about the extra const.

lib-index: Add missing initialization for chained hooks

lib-index: Allow multiple hooks for transaction creation

lib-index: Add day_stamp parameter to mail_index_update_day_headers() This will help creating a unit test.

lib-index: Added mail_index_transaction_get_highest_modseq()

lib-index: Added internal mail_index_transaction_finish_so_far() This can be called multiple times before mail_index_transaction_finish(). It's going to perform all the work of reordering mails and dropping unnecessary changes that can be done before changing sequences to UIDs. This will be needed for the following patch that implements mail_index_transaction_get_highest_modseq().

global: unsigned int:1 -> bool:1 perl -i -pe 's/unsigned int ([^,:;]+):1;/bool $1:1;/' **/*.[ch]

lib-index: Don't update log_file_tail_offset unnecessarily. Update it only if we're already writing to transaction log anyway or if we're required to update the offset because mail_index_sync_commit() has increased it past non-external transactions (this is especially important with mdbox map index).

lib-index: Keep track of views and transactions in linked lists. This makes debugging easier.

lib-index: Add timestamps and value lengths to attribute change records in transaction log. The timestamps will be useful for dsync, and value lengths will be useful for metadata quota.

lib-index: mail_index_attribute_[un]set() adds changed attributes' keys to transaction log. This provides them both a modseq (so their changes become visible) as well as an efficient way to see what attributes have changed by reading the transaction log. The values themselves aren't written to the log, because they could be large.

Array API changed: ARRAY_DEFINE(name, type) -> ARRAY(type) name Easy way to update your existing code: perl -i -pe 's:ARRAY_DEFINE\(([^,]+), *([^)]+)\);:ARRAY($2) $1;:' **/*.[ch]

Make sure we check all the functions' return values. Minor API changes to simplify this. Checked using a patched clang that adds attribute(warn_unused_result) to all functions. This commit fixes several error handling mistakes.

shared mailboxes: Per-user flags can now be stored in private index files. This can be enabled by adding e.g.: mail_location = mdbox:/var/shared/mdbox:INDEXPVT=~/mdbox/shared

lib-index: Fixed MAIL_INDEX_SYNC_FLAG_AVOID_FLAG_UPDATES with concurrent keyword changes. If session 1 saw keywords="a", session 2 added keyword "b" and session 1 (without the client syncing session) replaced flags with "a", the "b" keyword wasn't removed.

Don't write "keyword reset" records to transaction log anymore. These are a bit problematic for dsync's keyword merging to handle.

lib-index: MAIL_INDEX_SYNC_FLAG_DELETING_INDEX now allows committing changes to deleted index. This fixes deleting mailboxes.

lib-index: Always update index map while committing a transaction. The previous code was doing this only in some situations, which was probably enough, except maybe in some special cases. The log file was also read until EOF always anyway, so the only thing this was saving was some CPU. With the new behavior the committing stage should be more reliable, and also makes it easier to do changes without accidentally breaking it.

lib-index: Cache offsets were sometimes written pointing to an older cache file.

Removed mail_update_uid() / mail_index_update_uid(). It was working properly only with mdbox and there wasn't really a need for it. --HG-- branch : HEAD

lib-index: Added support for undeleting a deleted index. --HG-- branch : HEAD

Added support for marking mailbox index deleted. Don't allow any changes after that. This is going to help with race conditions when deleting mailboxes. --HG-- branch : HEAD

Transaction commits can now track how many uid/modseq updates were ignored. --HG-- branch : HEAD

lib-index: Added mail_index_update_highest_modseq(). --HG-- branch : HEAD

Added ability to specify message's minimum modseq value. --HG-- branch : HEAD

Added ability to change existing messages' UIDs with mail_[index_]update_uid(). --HG-- branch : HEAD

mail index transactions: More code cleanups and unit tests. --HG-- branch : HEAD

Keep track of expunged messages' GUIDs and expose them via mailbox_get_expunges(). The message GUIDs are stored in expunge records to transaction log. Before doing the final expunge, Maildir and dbox verify that the GUID in expunge request matches the current actual GUID. The GUID is stored in 128 bit field. If the real GUID isn't 128 bit, the bits are taken from SHA1 of the GUID. --HG-- branch : HEAD

Moved around mail-index-transaction code and added initial unit tests. --HG-- branch : HEAD

mail_index_update_flags*() now does a better job of merging flag changes together. --HG-- branch : HEAD

Moved mail_index_seq_array_*() to mail-index-util.c and added its own .h file. --HG-- branch : HEAD

Isolated all cache transaction handling code to a single file. --HG-- branch : HEAD

Moved transaction commiting code to mail-index-transaction-*.c --HG-- branch : HEAD

indexes: Fixes to handling shrinking tail offsets. --HG-- branch : HEAD

indexes: Removed unnecessary code. --HG-- branch : HEAD

indexes: Fixes to handling shrinking tail offsets. --HG-- branch : HEAD

indexes: Removed unnecessary code. --HG-- branch : HEAD

Added mail_index_atomic_inc_ext() for atomically incrementing numbers in extensions. --HG-- branch : HEAD

mail_index_transaction_get_next_uid() shouldn't sort appends. They may still be modified! Fixes a dbox rebuild bug. --HG-- branch : HEAD

mail_get_flags/keywords() now returns updated values if they've been changed within transaction. --HG-- branch : HEAD

CONDSTORE: STORE UNCHANGEDSINCE conflicts are now checked atomically. --HG-- branch : HEAD

mail_index_update_header_ext() can now be called multiple times for the same extension. --HG-- branch : HEAD

Added mail_index_ext_reset_inc() to atomically increase extension's reset_id. Added clear_data parameter to mail_index_ext_reset*(). --HG-- branch : HEAD

Changed .h ifdef/defines to use <NAME>_H format. --HG-- branch : HEAD

Keep flag updates separately from transaction.log_updates flag, because the flag updates array can become empty. --HG-- branch : HEAD

Changed mail_transaction_begin() API to take flags parameter instead of two booleans. --HG-- branch : HEAD

Recent flag handling rewrite. Still not perfect with maildir. --HG-- branch : HEAD

If mail_index_ext_set_reset_id() is called, the transaction may go back to unchanged state and we don't want to write it to transaction log then. --HG-- branch : HEAD

Added mail_index_ext_set_reset_id(). --HG-- branch : HEAD

Implemented mail_index_update_header_ext() --HG-- branch : HEAD

Added support for resetting index. --HG-- branch : HEAD

Better type safety to module_contexts arrays. Already fixed some bugs. --HG-- branch : HEAD

When adding mails to index in non-sorted UID order and then using mail_index_get_header(), the next_uid field could have been wrong. --HG-- branch : HEAD

After saving messages the mailbox wasn't synced, which caused deliver to treat the mailbox as unsynced and not update the indexes. --HG-- branch : HEAD

Beginnings of joining mail-storage API more closely to mail-index, so that mail-index could be directly used if needed. Currently only transactions are joined. --HG-- branch : HEAD

Array API redesigned to work using unions. It now provides type safety without having to enable DEBUG, as long as the compiler supports typeof(). Its API changed a bit. It now allows directly accessing the array contents, although that's not necessarily recommended. Changed existing array usage to be type safe in a bit more places. Removed array_t completely. Also did s/modifyable/modifiable/. --HG-- branch : HEAD

Added some checks and better logging for figuring out why (*seq != 0) assert happens. --HG-- branch : HEAD

deinit, unref, destroy, close, free, etc. functions now take a pointer to their data pointer, and set it to NULL. This makes double-frees less likely to cause security holes. --HG-- branch : HEAD

Added "bool" type and changed all ints that were used as booleans to bool. --HG-- branch : HEAD

Allow appending mails in wrong UID order. Sort them internally when committing. --HG-- branch : HEAD

If UIDVALIDITY changes, don't invalidate the whole index. Just expunge all existing messages and update uidvalidity/nextuid fields. Now we don't have to re-login when this happens. --HG-- branch : HEAD

Keywords are now stored in X-Keywords headers in mbox. Did several related API changes to get better performance. --HG-- branch : HEAD

Changed many buffers to arrays. Cleans up the code a lot. --HG-- branch : HEAD

Internal changes in how keywords are handled. struct mail_keywords isn't automatically freed anymore, added *_keywords_free() for that. --HG-- branch : HEAD

Added mail_index_update_flags_range() and optimized the non-range version as well. --HG-- branch : HEAD

Keyword fixes. --HG-- branch : HEAD

Initial support for keywords. Syncing to mbox/maildir doesn't work yet. --HG-- branch : HEAD

Update view's header only after syncing the view. This and other changes fix several view syncing problems and crashes. --HG-- branch : HEAD

Allow specifying to transaction if it's external. Make mail saving transactions external to avoid problems with uids. --HG-- branch : HEAD

Removed cache_offset from mail_index_record and changed it to use extension instead. Added possibility to register sync and expunge handlers for extensions. Changed the way extension resets work: all extension updates which were committed without having seen the reset are ignored. --HG-- branch : HEAD

New way of handling extension introductions in transaction log. --HG-- branch : HEAD

Index extensions can now specify record alignment, and it's possible to resize header and records. Fixes mbox crashes with some 64bit systems. --HG-- branch : HEAD

Renamed "extra record info" and variations of it to "extension" or "ext" in short. --HG-- branch : HEAD

We didn't include extra_rec_updates at checking if transaction had changed anything, and so they didn't necessarily get written to transaction log. This broke mbox offsets sometimes. --HG-- branch : HEAD

Save extra record/header infos into index file permanently. --HG-- branch : HEAD

Make sure we don't set cache_offsets to old cache files. --HG-- branch : HEAD

Cache updating is done now by first reserving space where to write, and then writing to it whenever buffer gets full. There is no persistent cache file locks anymore because of this, but it also means that the same cached field may be written multiple times to the file by different processes. Also since we reserve more space than we actually need at first, it some space can be wasted if multiple processes are updating the cache. --HG-- branch : HEAD

UIDs for appended messages can now be assigned all at once. Maildir now updates indexes immediately while saving messages. --HG-- branch : HEAD

Record size is allowed to change between index files. This will allow adding extensions dynamically for existing indexes. --HG-- branch : HEAD

Added mail_index_transaction_get_updated_view() which can be used to access uncommitted messages. --HG-- branch : HEAD

Several fixes and cleanups to cache file code, still badly broken --HG-- branch : HEAD

Modifying extra_records should work now. --HG-- branch : HEAD

Index header changes now go through transaction log. Removed the kludgy parameters for mail_index_sync_end(). Removed code duplication of syncing index root mapping and view mapping. Some fixes to handling uidvalidity and nextuid in syncing. --HG-- branch : HEAD

Forced locking to be right with mprotect()ing index file. Support for disabling mmap for indexes, and disabling just mmap+write(). --HG-- branch : HEAD

importing new index code. mbox still broken. --HG-- branch : HEAD