global: start relying on file_lock_free(NULL) being a no-op Cleanup performed with the following semantic patch: @@ expression E; @@ - if (E != NULL) { - file_lock_free(&E); - } + file_lock_free(&E);

lib-storage: Rename .vsize.lock file to dovecot-vsize.lock .vsize.lock can break maildir++ because it looks like a maildir folder, so rename it to dovecot-vsize.lock. Introduced in 9963bef6

Updated copyright notices to include the year 2018.

global: Use mail_set_critical() and mailbox_set_critical() if possible Replace calls to mail_storage_set_critical() with mail_set_critical() or mailbox_set_critical() in places where mailbox or mail are easily available.

lib-storage: Add reason to mailbox_transaction_begin() Remove mailbox_transaction_set_reason().

lib-storage: Move .vsize.lock creation to a generic mailbox_lock_file_create()

lib-storage: Move .vsize.lock creation to its own function

lib-storage: Use file_lock_set_*_on_free() for .vsize.lock This avoids storing the lock_path and lock_fd separately.

lib-storage: Fix error logging for failing to create .vsize.lock

lib-storage: Avoid unnecessary UIDNEXT lookups after saving a mail The vsize checking code was expecting that this would always be a cheap operation, but with imapc code it could trigger a remote STATUS (UIDNEXT) call. Do this only when we've checked that this mailbox is updating vsize header at all.

global: Add mailbox_transaction_set_reason() calls Added to the most important places.

global: Log internal storage error on failure

lib-storage, pop3-migration: Reset lookup_abort before continuing to search This doesn't really fix anything right now, but it'll allow adding the asserts in the following commit.

lib-storage: Add explicit MAIL_ERROR_LOOKUP_ABORT error The MAIL_ERROR_NOTPOSSIBLE was a bit too generic to assume to mean the same thing. It doesn't look like there are any external plugins that break because of this change.

lib-storage: Change vsize calculation to use the new mail_stream_opened This avoids calling mail_get_virtual_size() twice and is a bit cleaner approach.

lib-storage: Update vsize header after sync only if sizes are cached. The result isn't needed yet in that case, so if it's slow to get the sizes it might as well be delayd until later. This is especially useful when indexer-worker triggers FTS indexing. The vsizes can be added to index after the mail is already read for FTS. Without this change the vsize update would first open all the mails and then the FTS indexing would open all the mails a second time.

lib-storage: Add mail_vsize_bg_after_count setting. If folder vsize calculation requires opening more than this many mails from disk (i.e. mail sizes aren't in cache already), return failure and finish the calculation via indexer process. This should be used like: protocol !indexer-worker { mail_vsize_bg_after_count = 10 }

lib-storage: Don't stop vsize calculation on expunged mails. I don't know why I added such logic there in the first place. If we just skip the expunged mails, the end result should still be correct and usable when cached.

Updated copyright notices to include the year 2017.

global: Replaced all instances of memset(p, 0, sizeof(*p)) with the new i_zero() macro. Used the following script: C_FILES=`git ls-files *.c` H_FILES=`git ls-files *.h` for F in "$C_FILES $H_FILES"; do echo "$F" perl -p -i -e 's/safe_memset\(&\(?([^,]*)\)?,\s*0,\s*sizeof\(\g1\)\)/i_zero_safe(&$1)/g' $F perl -p -i -e 's/safe_memset\(([^,]*),\s*0,\s*sizeof\(\*\g1\)\)/i_zero_safe($1)/g' $F perl -p -i -e 's/memset\(&\(?([^,]*)\)?,\s*0,\s*sizeof\(\g1\)\)/i_zero(&$1)/g' $F perl -p -i -e 's/memset\(([^,]*),\s*0,\s*sizeof\(\*\g1\)\)/i_zero($1)/g' $F done

global: Code cleanup - avoid passing NULL to functions with non-null parameter

lib-storage: Add vsize hdr on empty mailbox

global: freshen copyright git ls-files | xargs perl -p -i -e 's/(\d+)-201[0-5]/$1-2016/g;s/ (201[0-5]) Dovecot/ $1-2016 Dovecot/'

lib-storage: Added assert - mailbox has to be open when starting vsize header update.

lib-storage: Update mailbox vsize header on save/copy/expunge. This allows always efficiently looking up maiboxes' vsizes after they're initially calculated. The expunge handling is unfortunately done currently in quota handling code, so it works only if quota is enabled. Ideally this would be solved in v2.3 with some lib-storage core changes.

lib-storage: Added MAILBOX_METADATA_PHYSICAL_SIZE If backend always uses the same virtual and physical sizes, this is implemented via the MAILBOX_METADATA_VIRTUAL_SIZE code. Otherwise it searches all the messages and sums up their physical sizes.

lib-storage: Moved vsize_hdr_ext_id to struct mailbox for more global access.

lib-storage: Renamed struct index_vsize_header to struct mailbox_index_vsize Also moved the struct to mail-storage-private.h for more global access.

lib-storage: Moved mailbox vsize calculation code to its own file. No functional differences.