imapc: Avoid duplicate FETCH BODY.PEEK[] when mail is already expunged When the first FETCH (e.g. as part of mail_prefetch()) indicated that the mail is already expunged, there's no need to do it again.

Updated copyright notices to include the year 2018.

global: Use mail_set_critical() and mailbox_set_critical() if possible Replace calls to mail_storage_set_critical() with mail_set_critical() or mailbox_set_critical() in places where mailbox or mail are easily available.

global: Use PRI* macros and %zu instead of casting

global: use i_close_fd{,_path}() instead of open-coding them

global: start relying on buffer_free(NULL) being a no-op Cleanup performed with the following semantic patch: @@ expression E; @@ - if (E != NULL) { - buffer_free(&E); - } + buffer_free(&E);

lib-storage: convert struct imapc_mail casts to container_of

lib-storage: convert index to use container_of

lib-storage: convert imapc to use container_of

imapc: Fix caching >= 32 kB mail bodies When closing the mail, only in-memory mail bodies were stored to cache.

imapc: Prefech - Initialize cached stream only if needed If the mail is immediately accessed for other purposes (e.g. fetching message flags), there's no need to go through all the trouble of initializing the mail stream.

imapc: include imapc-client.h in imapc-storage.h It was already required to be able to successfully include imapc-storage.h.

imapc: Add imapc_features=fetch-bodystructure This allows using the remote IMAP server's BODY and BODYSTRUCTURE replies.

imapc: Rename imapc_storage_has_modseqs() to imapc_mailbox_has_modseqs() Simplifies the following patch.

lib-storage: Allow mail_add_temp_wanted_fields() to be called before mail_set_seq*() Quota plugin was already doing this, but it didn't actually work. It was also crashing with imapc: Panic: file mail-cache-lookup.c: line 341 (mail_cache_field_exists): assertion failed: (seq > 0)

lib-imap: imap-envelope: Moved imap_envelope_headers to lib-mail/message-part-data as message_part_envelope_headers.

Updated copyright notices to include the year 2017.

lib-storage: rename get_real_mail to get_backend_mail Also change the function signature so that it returns an int to indicate success or failure.

imapc: mail_get_stream() shouldn't return cached hdr_size

Added asserts to make static analyzer happier.

imapc: Make sure imail.stream is not NULL Makes static analysers happy

lib-storage: remove set_cache_corrupted_reason from mail_vfuncs Change set_cache_corrupted signature to what set_cache_corrupted_reason used to be.

global: Fixed mismatched bool vs. int/pointer handling I don't think these fix any actual bugs.

lib: istream - migrate to i_stream_read_more() Scripted to find all the low-hanging-fruit (single line calls), but hand-checked: git grep i_stream_read_data.*,\ 0\) | sed s/:.*// | \ xargs sed -i -e 's/i_stream_read_data(\(.*\), \(.*\), \(.*\), 0)/i_stream_read_more(\1, \2, \3)/' Signed-off-by: Phil Carmody <phil@dovecot.fi>

imapc: Fixed crash caused by pool_unref() API change. Calling pool_unref() set the mail_private->pool=NULL, which crashed in index_mail_free(). Fixed this by removing the unnecessary ref/unref entirely. We're already waiting in imapc_mail_close() for all the existing FETCHes to finish, so the callback will always have only valid mails with valid pools.

imapc: Added support for imapc_features=modseq If the remote server supports CONDSTORE or QRESYNC extensions we'll use the remote's MODSEQ and HIGHESTMODSEQ counts. There are some situations where the HIGHESTMODSEQ isn't updated exactly correctly on an open mailbox, so this feature shouldn't be fully relied on. It was primarily implemented for dsync+imapc support - both for preserving modseqs and also for HIGHESTMODSEQ lookups.

lib-storage: Added mail_set_cache_corrupted_reason()

imapc: Handle disconnection when NOOP is sent to verify if mail is expunged.

imapc: If FETCH didn't send our wanted data, log the tagged FETCH reply text.

imapc: Don't use fetch-fix-broken-mails for OK and NO [SERVERBUG] FETCH replies Hopefully no servers are actually returning OK without returning a FETCH reply. If they are, maybe we need another workaround.. In any case Yahoo IMAP sometimes loses state and starts returning OK without FETCH replies to all UID FETCH requests (and BADs to FETCHes). It also may return NO [UNAVAILABLE] for both permanent and temporary FETCH failures, so we can't do anything with that. I added NO [SERVERBUG] because that's what Dovecot responds to temporary failures. I'm not sure how useful that check is though.

imapc: Prefetching shouldn't send FETCH requests for headers that are already cached.

imapc: Added imapc_features=fetch-fix-broken-mails This allows ignoring a missing FETCH reply. It shouldn't be used normally, but if there's a mail that the IMAP server simply won't serve, this can be used to finish dsync successfully.

global: freshen copyright git ls-files | xargs perl -p -i -e 's/(\d+)-201[0-5]/$1-2016/g;s/ (201[0-5]) Dovecot/ $1-2016 Dovecot/'

imapc: Various optimization fixes to fetching messages' virtual size.

imapc: imapc-features=rfc822.size didn't work for fetching mail's virtual size. Physical and virtual sizes are the same for imapc.

imapc: Added imapc_features=zimbra-workarounds Zimbra (at least v5.0.18_GA_3011.RHEL4_64) can return different headers depending on whether the whole message body was fetched or only (partial) headers. This probably happens only for invalid characters that are translated into '?'. With this workaround enabled we don't use FETCH BODY.PEEK[], but we do FETCH (BODY.PEEK[HEADER] BODY.PEEK[TEXT]) and merge the results together. This way the results are always consistent and headers don't suddenly change.

imapc: If reconnect fails during syncing, don't treat all mails as expunged in error handling code.

imapc: Use GUIDs from cache.

imapc: Add prefetched GUIDs to cache.

imapc: Removed Exchange-workaround for ignoring missing BODY[] replies. This causes data loss if server was actually returning a NO temporary error that would have succeeded later on.

imapc: Merged gmail-pop3 and gmail-labels-keyword features into a single gmail-migration feature There's really no other reason to use them except for migration.

imapc: Implemented imapc_features=gmail-pop3 that also migrates pop3_deleted_flag. This also caused gmail-pop3-uidl feature to be renamed to simply gmail-pop3, which includes both the features.

imapc: Added imapc_features=gmail-pop3-uidl This allows imapc to generate GMail POP3 compatible UIDL so that pop3-migration plugin isn't needed.

global: freshen copyright Robomatically: git ls-files | xargs perl -p -i -e 's/(\d+)-201[0-4]/$1-2015/g;s/ (201[0-4]) Dovecot/ $1-2015 Dovecot/' Happy 2015 everyone! Signed-off-by: Phil Carmody <phil@dovecot.fi>

imapc: Fixed fetching specific headers without imapc_features=fetch-headers

imapc: Avoid hanging when running into server problems. imapc_mail_fetch_flush() caused imail->fetch_count to drop to 0, but imapc_mailbox_run() still started imapc_client_run() which was doing nothing.

Updated copyright notices to include year 2014.

imapc: Merge identical FETCH commands together (only updating UID range)

imapc: Fixes to imapc_feature fetch-headers

imapc: Avoid FETCH INTERNALDATE for getting save_date when we already know received_date.

imapc: Don't send FETCHes for data that is already in cache.

imapc: Added imapc_feature fetch-header. It uses FETCH BODY.PEEK[HEADER.FIELDS (...)] whenever possible instead of fetching the entire header.

Oops :) Update copyrights to 2013 without breaking all .c files.

Updated copyright notices to include year 2013.