lib-index: If mail_index_view is leaked, include in Panic the file:line where it was opened

global: unsigned int:1 -> bool:1 perl -i -pe 's/unsigned int ([^,:;]+):1;/bool $1:1;/' **/*.[ch]

lib-index: Keep track of views and transactions in linked lists. This makes debugging easier.

Array API changed: ARRAY_DEFINE(name, type) -> ARRAY(type) name Easy way to update your existing code: perl -i -pe 's:ARRAY_DEFINE\(([^,]+), *([^)]+)\);:ARRAY($2) $1;:' **/*.[ch]

lib-index: Added mail_index_view_dup_private()

mail_get_flags/keywords() now returns updated values if they've been changed within transaction. --HG-- branch : HEAD

View syncing: Keep track of highest modseq in views. If we lose transaction log, return all messages with higher modseq than the old highest-modseq as having changed. This works better than the previous code which compared the old modseq which may have already changed after the previous view sync. Return changes based on modseq being larger than --HG-- branch : HEAD

Keep view's map always at least up-to-date with syncing. If there have been no expunges, the map is immediately replaced with the latest map. Before the map may not have been updated until syncing was finished. This change is necessary to update modseqs correctly while they're being enabled. --HG-- branch : HEAD

Renamed mail_index_lookup_uid_range() to mail_index_lookup_seq_range(), made it return a bool and added mail_index_lookup_seq(). Cleaned up the code to use these functions. --HG-- branch : HEAD

Changed .h ifdef/defines to use <NAME>_H format. --HG-- branch : HEAD

fsck won't fail anymore with "corrupted index", all problems are fixed. Added mail_index_fsck_locked(). --HG-- branch : HEAD

mail_index_lookup*() can't fail anymore. Changed several APIs not to return failure anymore. --HG-- branch : HEAD

Removed explicit locking from views and maps. They were already locked all the time when they were used. Because of this change several functions can no longer fail, so they were changed to return void, and a lot of pointless error handling was removed. --HG-- branch : HEAD

mail_index_ext_get_reset_id() takes now map parameters. Fixed it to work properly with transaction views. --HG-- branch : HEAD

Added mail_index_ext_get_reset_id() which returns the latest reset_id when using transaction views. --HG-- branch : HEAD

Removed view->hdr. Use view->map->hdr directly. --HG-- branch : HEAD

Rewrite index lock handling. --HG-- branch : HEAD

mail_index_map now contains pointer to index, so all functions taking map parameter don't need index anymore. --HG-- branch : HEAD

View's counters can no longer be unreliable or broken. --HG-- branch : HEAD

Removed no-appends specific code that's unneeded now. --HG-- branch : HEAD

Removed syncs_done array. Instead track appends and expunges separately. Also fixed handling no-appends in some situations. --HG-- branch : HEAD

Initial commit for major index file code cleanup. - dovecot.index file isn't anymore required to be updated when syncing. - Getting the latest index file mapping is now done always by reading dovecot.index and then reading the latest changes from dovecot.index.log. - mmap()ing dovecot.index file is slower than reading it, so it's not currently done unless the file is at 256kB. This may change though. - Some things are still broken. --HG-- branch : HEAD

mail_index_view_add_hidden_transaction() takes now a whole area parameter so it has to be called only once for each transaction, instead of each for each transaction record. mail_index_view_add_synced_transaction() is no longer public. --HG-- branch : HEAD

Better type safety to module_contexts arrays. Already fixed some bugs. --HG-- branch : HEAD

After saving messages the mailbox wasn't synced, which caused deliver to treat the mailbox as unsynced and not update the indexes. --HG-- branch : HEAD

Beginnings of joining mail-storage API more closely to mail-index, so that mail-index could be directly used if needed. Currently only transactions are joined. --HG-- branch : HEAD

Array API redesigned to work using unions. It now provides type safety without having to enable DEBUG, as long as the compiler supports typeof(). Its API changed a bit. It now allows directly accessing the array contents, although that's not necessarily recommended. Changed existing array usage to be type safe in a bit more places. Removed array_t completely. Also did s/modifyable/modifiable/. --HG-- branch : HEAD

We used "already synced log positions in view" array to store locations of committed hidden transactions. They however weren't really synced, which caused some problems. Added a separate "hidden sync log positions" array for them which is handled differently. --HG-- branch : HEAD

Index header's seen/recent/deleted counters weren't always correct after syncing a view. This caused some problems with recent SEARCH optimizations, eg. EXPUNGE not always working. --HG-- branch : HEAD

Added "bool" type and changed all ints that were used as booleans to bool. --HG-- branch : HEAD

Comment updates, small cleanups and optimizations. --HG-- branch : HEAD

Converted some buffers to arrays. --HG-- branch : HEAD

Keep index pointer in dummy views. --HG-- branch : HEAD

Instead of using separate transaction sorting code for syncing, just put the data from transactions into a temporary transaction and read it from there. --HG-- branch : HEAD

Removed view->messages_count, view->hdr.messages_count is enough. Also fixes assert crash in mail_index_bsearch_uid(). Renamed .._get_message_count() to get_messages_count() to be more consistent. --HG-- branch : HEAD

Changed mail_index_get_header() to return the header as return value because it can't fail anymore. --HG-- branch : HEAD

Update view's header only after syncing the view. This and other changes fix several view syncing problems and crashes. --HG-- branch : HEAD

Allow specifying to transaction if it's external. Make mail saving transactions external to avoid problems with uids. --HG-- branch : HEAD

Removed cache_offset from mail_index_record and changed it to use extension instead. Added possibility to register sync and expunge handlers for extensions. Changed the way extension resets work: all extension updates which were committed without having seen the reset are ignored. --HG-- branch : HEAD

Removed mmap() memory area protections. They weren't always set correctly. Also grsec crashed with them anyway. --HG-- branch : HEAD

Renamed "extra record info" and variations of it to "extension" or "ext" in short. --HG-- branch : HEAD

Save extra record/header infos into index file permanently. --HG-- branch : HEAD

Refcounting fixes for index mappings. --HG-- branch : HEAD

Added mail_index_transaction_get_updated_view() which can be used to access uncommitted messages. --HG-- branch : HEAD

Index header changes now go through transaction log. Removed the kludgy parameters for mail_index_sync_end(). Removed code duplication of syncing index root mapping and view mapping. Some fixes to handling uidvalidity and nextuid in syncing. --HG-- branch : HEAD

View's message count shouldn't change until after view is synced. --HG-- branch : HEAD

fixes --HG-- branch : HEAD

cleanup --HG-- branch : HEAD

Forced locking to be right with mprotect()ing index file. Support for disabling mmap for indexes, and disabling just mmap+write(). --HG-- branch : HEAD

importing new index code. mbox still broken. --HG-- branch : HEAD