lib-storage: Remove unnecessary mail_save_context.dest_mail==NULL checks It can never be NULL after the previous change: "lib-storage: Always create mail_save_context.dest_mail". The code removal in maildir_transaction_save_commit_pre() seemed potentially dangerous, but I don't think such code path is possible anymore. Also even if it is, it's probably fine since the mail_free() is called even earlier than before (although that itself might have been a problem). This also removes last traces of code that made it possible to save mails to mbox without assigning UID to the mail. The previous commit already caused this, so this is just removing dead code.

lib-storage: Added quick parameter to list_index_has_changed()

global: unsigned int:1 -> bool:1 perl -i -pe 's/unsigned int ([^,:;]+):1;/bool $1:1;/' **/*.[ch]

mbox: Added more (and earlier) detection for errors in mbox istreams.

mbox: Fixed mailbox_list_index=yes to work with non-Dovecot mbox changes.

mbox: Handle broken Status: and X-Status: headers without sync errors.

Array API changed: ARRAY_DEFINE(name, type) -> ARRAY(type) name Easy way to update your existing code: perl -i -pe 's:ARRAY_DEFINE\(([^,]+), *([^)]+)\);:ARRAY($2) $1;:' **/*.[ch]

mbox: Generate mailbox GUID without syncing the mbox, locking it is enough.

mbox: Fixed getting header MD5 (GUID) if it wasn't already in index. --HG-- branch : HEAD

mbox: Removed unused MBOX_SYNC_LAST_COMMIT and related code. --HG-- branch : HEAD

index-storage: Code cleanup. --HG-- branch : HEAD

mbox: Added a new index header where dirtyness state is stored. This also fixes a bug where sync_size wasn't updated in header when mbox was marked dirty, causing other parts of code to assume wrong file size and return wrong size for the last message and/or cause "Next message unexpectedly lost" errors. --HG-- branch : HEAD

mbox_lazy_writes: If errors are noticed in mbox, write changes to fix them immediately to avoid constant errors in logs. --HG-- branch : HEAD

When saving to empty mbox files, sync them first since it doesn't take hardly any time anyway. --HG-- branch : HEAD

Renamed __attr_*__ to ATTR_*. Renamed __attrs_used__ to ATTRS_DEFINED. --HG-- branch : HEAD

Changed .h ifdef/defines to use <NAME>_H format. --HG-- branch : HEAD

mail_index_lookup*() can't fail anymore. Changed several APIs not to return failure anymore. --HG-- branch : HEAD

Fixes to recent handling. Now it should work properly. --HG-- branch : HEAD

Recent flag handling rewrite. Still not perfect with maildir. --HG-- branch : HEAD

Moved index sync record grouping from mbox to generic code. --HG-- branch : HEAD

Handle UIDVALIDITY changes by resetting index. --HG-- branch : HEAD

Preserve mbox files' atime when saving/syncing. --HG-- branch : HEAD

Added sync_notify() callback to struct mail_storage. It's now called for expunges and flag/keyword changes (except with cydir). --HG-- branch : HEAD

Give "mbox file was modified while we were syncing" error only if we detect some problems in the mbox file. The check can't be trusted with NFS. --HG-- branch : HEAD

Check if mbox has been modified externally before doing any dangerous writes. Also changed the old "space needed changed unexpectedly" assert to a more informative panic. --HG-- branch : HEAD

uint8_t:1 is GCC extension, changed back to using unsigned int:1 although it takes a bit more memory now. --HG-- branch : HEAD

Cleaned up the code a bit. Fixed sync rewrite crashing with pseudo mails. --HG-- branch : HEAD

Keep track of the mbox dirty state better. Also when moving mails inside an mbox without changing the file's size, make sure the file's mtime changes even if it means sleeping for a second. --HG-- branch : HEAD

If next_uid field gets to 2^31-1, handle it without crashing by creating a new uidvalidity and renumbering the UIDs (although it requires selecting the mailbox twice, first time to get index file deleted). --HG-- branch : HEAD

Rewriting still wasn't replacing broken X-UID headers correctly always, causing assert-crashes. --HG-- branch : HEAD

Fixed some wrong "uid-last unexpectedly lost" errors. --HG-- branch : HEAD

Restore sync_ctx->next_uid to original value when beginning a rewrite. This could fix some problems with broken UIDs causing crashes. --HG-- branch : HEAD

Last keyword pool clearing optimization broke the keywords if we had to move mails around in the file. --HG-- branch : HEAD

Fixes to handling CRLF linefeeds in mboxes. --HG-- branch : HEAD

Array API redesigned to work using unions. It now provides type safety without having to enable DEBUG, as long as the compiler supports typeof(). Its API changed a bit. It now allows directly accessing the array contents, although that's not necessarily recommended. Changed existing array usage to be type safe in a bit more places. Removed array_t completely. Also did s/modifyable/modifiable/. --HG-- branch : HEAD

Added "bool" type and changed all ints that were used as booleans to bool. --HG-- branch : HEAD

When rewriting mbox, don't parse the last message again. We already have its contents in mail_ctx, so pass it over to rewriter so it can reuse it. This fixes also a crash when last message's header ended unexpectedly and mbox file was grown (the added NULs were treated as the last message's headers). --HG-- branch : HEAD

Don't crash in some situations with broken X-UID headers. --HG-- branch : HEAD

If we see mbox offsets break, don't invalidate whole index but rather force resyncing. --HG-- branch : HEAD

If uid-last isn't actually updated while rewriting first message, don't assume it is. Fixes bogus "uid-last changed" errors. --HG-- branch : HEAD

If UIDVALIDITY changes, don't invalidate the whole index. Just expunge all existing messages and update uidvalidity/nextuid fields. Now we don't have to re-login when this happens. --HG-- branch : HEAD

Since we always parse the first message now, there's no need ever to do sync_restart. --HG-- branch : HEAD

Moved mbox/maildir-specific variables from struct index_mailbox to mbox_mailbox and maildir_mailbox. --HG-- branch : HEAD

Large mbox code cleanups, especially related to X-IMAP/X-IMAPbase header handling. uid-next field is updated now every time while syncing. If partial syncing failed, we weren't re-requesting all sync records from index, which could have caused some changes to be lost, and possibly caused some index corruption errors later on. Several other more or less possible problems fixed. --HG-- branch : HEAD

Keywords are now stored in X-Keywords headers in mbox. Did several related API changes to get better performance. --HG-- branch : HEAD

Recent flag handling cleanups. Added some comments. --HG-- branch : HEAD

i_stream_sync() replaced istream_raw_mbox_flush(). Use i_stream_stat() instead of fstat() directly. Now we have the possibility to use istreams as read-only mboxes. --HG-- branch : HEAD

Changed mails and syncs buffers to arrays. --HG-- branch : HEAD

Don't allow X-UID headers larger than current next-uid. --HG-- branch : HEAD

Initial support for keywords. Syncing to mbox/maildir doesn't work yet. --HG-- branch : HEAD

Removed X-UIDL header handling. It's not actually used. --HG-- branch : HEAD

Reorganized mbox md5 summing code so it doesn't require sync context anymore. --HG-- branch : HEAD

Moved mbox header MD5 summing code into separate file so it's easily changed. --HG-- branch : HEAD

cleanup --HG-- branch : HEAD

restart partial syncs correctly. and cleanups. --HG-- branch : HEAD

If rewriting began with expunged data areas, we didn't overwrite them which left garbage into mbox and broke things. --HG-- branch : HEAD

Bringing some sanity into mbox rewrites. Fixes expunge crashes and reduces writes. --HG-- branch : HEAD

Split sync_flag_full into sync_flag_full_read and _write. Closing mailbox now always writes changes. --HG-- branch : HEAD

Added mbox_lazy_writes setting. --HG-- branch : HEAD

Handle UIDVALIDITY changes. When partial syncing, make sure UIDs of new mails are larger than existing ones. If partial syncing fails, we need to rollback the transaction. --HG-- branch : HEAD

Last from offset sync fix broke everything. Another try. --HG-- branch : HEAD

cleanups --HG-- branch : HEAD

Wrong messages were set recent if another client had modified the mbox. --HG-- branch : HEAD

When mbox syncing is dirty, read the message headers to verify that we've found the correct message. --HG-- branch : HEAD

Added mbox_dirty_syncs setting which delays re-reading the whole mbox when it's changed. --HG-- branch : HEAD

Added struct message_header_line.middle and middle_len to contain the ':' in header and whitespace around it. --HG-- branch : HEAD

Renamed mail_index_sync_end() to _commit() and added _rollback(). Fixed mbox deadlocking issue. --HG-- branch : HEAD

Don't keep filling empty spaces which will be overwritten soon anyway. Major speedup for rewriting large mboxes. --HG-- branch : HEAD

some fixes for handling pseudo headers. --HG-- branch : HEAD

Several fixes for updating mbox. Recent changes also made it corrupt headers. --HG-- branch : HEAD

Save MD5 sums of some headers for readonly mboxes and use them for syncing --HG-- branch : HEAD

Added a framework for getting POP3 UIDL strings from mail-storage. The default still is uidvalidity.uid. Added beginnings of overriding it from X-UIDL header in mbox. --HG-- branch : HEAD

fixes for From-line offset updating --HG-- branch : HEAD

Don't die horribly when message with previously expunged UID is inserted into mbox. --HG-- branch : HEAD

mailbox_save() and mailbox_copy() functions can now return the saved mail so it can be immediately queried. Implemented UIDPLUS extension using it. Maildir implementation missing, so it crashes with it for now.. APPEND with mbox now doesn't require resyncing the mailbox since it updates indexes directly. --HG-- branch : HEAD

fixes --HG-- branch : HEAD

Added istream_raw_mbox_is_eof() and fixed updating UIDVALIDITY in index --HG-- branch : HEAD

fixes --HG-- branch : HEAD

Several fixes in space/offset logic. Should be much more robust now. --HG-- branch : HEAD

lock the mbox before reading it. --HG-- branch : HEAD

several fixes and optimizations. --HG-- branch : HEAD

major syncing code cleanups. the code finally looks almost readable. logic should have stayed same. --HG-- branch : HEAD

Save mbox offsets to index file using extra_records. Some other fixes. --HG-- branch : HEAD

some more fixes --HG-- branch : HEAD

expunging is somewhat working --HG-- branch : HEAD

mbox syncing fixes --HG-- branch : HEAD

mbox growing and locking works now --HG-- branch : HEAD

mbox rewriting is almost working - the hard part is done. --HG-- branch : HEAD

mbox code compiles again, but syncing is only partially implemented so accessing mboxes fails.. Also some cleanups to index-storage and maildir code. --HG-- branch : HEAD

s/custom flags/keywords/ --HG-- branch : HEAD

importing new index code. mbox still broken. --HG-- branch : HEAD