lib-index: Add mail_cache_open_or_create_path() This allows moving cache file to another directory.

lib-index: Add mail_cache_close_mail() to smartly drop cached data with INDEX=MEMORY Instead of reseting the entire transaction buffer when 256 kB is reached, just drop mails have have been marked closed with mail_cache_close_mail(). If that's not enough, continue deleting forcibly until the buffer is below 256 kB. This is especially important when mail_prefetch_count>0 and INDEX=MEMORY. In that case there can be multiple mails that are being added to cache and used later on. If they were dropped from cache too early, the work would have to be done all over again.

lib-index: Add mail_cache_set_seq_corrupted_reason Marks a seq in cache corrupted, and removes that instead of the whole cache.

lib-index: Add mail_cache_decisions_copy This allows priming cache decisions for a folder.

lib-index: Unconstify mail_cache_register_get_list This way the result is usable as input for mail_cache_register_fields.

lib-index: Added mail_cache_get_missing_reason()

lib-index: Minor code cleanup - removed duplicate function prototype.

lib-index: Delay unlocking cache compression until changes to transaction log are committed. This should fix race condition with two processes compressing the file at the same time with same file_seq and becoming confused.

lib-index: Fixed cache file creation race condition. If two processes are creating the index files at the same time, don't have one of them delete the dovecot.index.cache that the other one just created. This means we never should be calling mail_cache_create(), so it was removed entirely.

Replaced all -1U and (unsigned int)-1 with UINT_MAX. It's somewhat clearer this way. Also clang's -fsanitize=integer gives runtime errors about -1U (but not about explicit casts, so no need to change (type)-1 casts).

lib-index: mail_cache_open_and_verify() is now public.

lib-index: Allow updating cache's last_used field with mail_cache_register_fields()

lib-index: Changed mail_cache_view_close() API to take pointer-to-pointer.

lib-index: Added mail_cache_exists()

lib-index: Added mail_cache_view_update_cache_decisions()

lib-index: Added mail_cache_view_update_cache_decisions()

lib-index: Added mail_cache_field_exists_any()

lib-index: Added mail_cache_register_get_field().

lib-index: Added mail_cache_reset().

mail_cache_register_fields(): Update caching decisions from input fields. --HG-- branch : HEAD

More fixes to cache changes. --HG-- branch : HEAD

mail_cache_transaction_commit()/rollback() API is now public and takes a ** pointer. Don't break if transaction is committed/rollbacked multiple times for a cache view. --HG-- branch : HEAD

Renamed __attr_*__ to ATTR_*. Renamed __attrs_used__ to ATTRS_DEFINED. --HG-- branch : HEAD

Changed .h ifdef/defines to use <NAME>_H format. --HG-- branch : HEAD

s/field/field_idx/ --HG-- branch : HEAD

When compressing cache file, update the cache file offsets using the existing sync transaction. --HG-- branch : HEAD

If cache file contains broken field type or decision type mark the cache corrupted instead of assert-crashing later. --HG-- branch : HEAD

Added mail_cache_field_want_add() and mail_cache_field_can_add() --HG-- branch : HEAD

deinit, unref, destroy, close, free, etc. functions now take a pointer to their data pointer, and set it to NULL. This makes double-frees less likely to cause security holes. --HG-- branch : HEAD

Added "bool" type and changed all ints that were used as booleans to bool. --HG-- branch : HEAD

Added mail_cache_register_get_list() to get a list of all registered fields in cache file. --HG-- branch : HEAD

s/PREFIX/SUFFIX/ for .cache and .log filename suffixes.. --HG-- branch : HEAD

Several size_t -> unsigned int changes. Try to keep "unsigned int" for counters, size_t for actual memory range sizes. --HG-- branch : HEAD

Don't complain about cache indexid changing when recreating index. --HG-- branch : HEAD

Cache fixes. Decisions are saved again. --HG-- branch : HEAD

Header caching redesigned. New design allows caching decisions per field, so they can be divided to temporary/permanent. Cached headers are now always returned in original order, old code didn't guarantee it. Some other caching changes. (still missing code to store changes in caching decisions) --HG-- branch : HEAD

Cache file fixes, API changes, etc. It's still in somewhat ugly state, but getting better.. --HG-- branch : HEAD

Cache API updates. Don't return direct pointers to mmaped cache file anymore. --HG-- branch : HEAD

Cache updating is done now by first reserving space where to write, and then writing to it whenever buffer gets full. There is no persistent cache file locks anymore because of this, but it also means that the same cached field may be written multiple times to the file by different processes. Also since we reserve more space than we actually need at first, it some space can be wasted if multiple processes are updating the cache. --HG-- branch : HEAD

Caching fixes and optimizations. Removed all network byte ordering code - it's not worth the trouble and would require massive changes in indexing code as well to be useful. Changed next_offset to prev_offset which is updated while syncing index. --HG-- branch : HEAD

Cache file compression works now and compressed cache file is reopened. Several other cleanups related to opening - cache file isn't created immediately anymore. --HG-- branch : HEAD

fix --HG-- branch : HEAD

Added some smartness for deciding what to cache. Cache compression code compiles, but untested. --HG-- branch : HEAD

Added a framework for getting POP3 UIDL strings from mail-storage. The default still is uidvalidity.uid. Added beginnings of overriding it from X-UIDL header in mbox. --HG-- branch : HEAD

Several fixes and cleanups to cache file code, still badly broken --HG-- branch : HEAD

Beginnings of getting cache file working again. Easy to break currently, but basics seem to work. --HG-- branch : HEAD

Added support for setting dirty flags for messages (TODO: undirty..) s/mail_index_record_flag/mail_cache_record_flag/ --HG-- branch : HEAD

importing new index code. mbox still broken. --HG-- branch : HEAD

removed out of the way of new index code --HG-- branch : HEAD

mbox reading is kind of working again. Just don't try rewriting or expunging :) Changing headers are also hidden from clients so mbox messages are finally seen immutable as required by IMAP. --HG-- branch : HEAD

use int32_t instead of int in struct mail_sent_date --HG-- branch : HEAD

Removed some header/body size fields. We use only message_part now everywhere since it's small enough. virtual_full_size was left because we may know that by just reading maildir filename. --HG-- branch : HEAD

Removed NBO32_BIT* stuff --HG-- branch : HEAD

Index cache file rewrite. It's not finished yet and mbox support is completely broken. But it's getting difficult to maintain outside cvs :) --HG-- branch : HEAD