imapc: Fix sending initial FETCH after reconnection SELECTs mailbox Move sending the FETCH when the SELECT returns tagged OK reply instead of delaying it until mailbox is next synced. Most importantly this allows sending the FETCH before any retried commands that are also sent after SELECT receives tagged reply.

imapc: Keep "selected" state TRUE only while mailbox is successfully SELECTed

Updated copyright notices to include the year 2018.

global: Use mail_set_critical() and mailbox_set_critical() if possible Replace calls to mail_storage_set_critical() with mail_set_critical() or mailbox_set_critical() in places where mailbox or mail are easily available.

lib-storage: imapc_mailbox_get_selected_status() can't fail - remove dead code

global: use i_close_fd{,_path}() instead of open-coding them

global: start relying on buffer_free(NULL) being a no-op Cleanup performed with the following semantic patch: @@ expression E; @@ - if (E != NULL) { - buffer_free(&E); - } + buffer_free(&E);

global: start relying on timeout_remove(NULL) being a no-op Cleanup performed with the following semantic patch: @@ expression E; @@ - if (E != NULL) { - timeout_remove(&E); - } + timeout_remove(&E);

lib-storage: convert imapc to use container_of

imapc: Add imapc_features=send-id to send ID x-session-ext-id

imapc: Don't reconnect on STATUS if auth failure was already seen.

imapc: Make sure storage error has the proper auth failure error string The first failed command always had the correct error string, but the following failed commands just returned -1 without updating storage error. The storage error could have been something completely different by then.

imapc: Cleanup - Rename auth_error to auth_failed_reason for consistency

imapc: Change auth_failed boolean to more specific auth_failed_state Since we now rely on auth_failed_state being initialized to zero, explicitly set IMAPC_COMMAND_STATE_OK to be defined as 0.

imapc: Make sure a valid UIDVALIDITY is returned by SELECT Fixes errors such as: Error: Synchronization corrupted index header (in-memory index): uidvalidity=0, but next_uid=9

imapc: Fix assert-crash if server doesn't send EXISTS reply to SELECT Fixes: Panic: file imapc-sync.c: line 477 (imapc_sync_index): assertion failed: (mbox->sync_fetch_first_uid == 1)

imapc: Fix infinite reconnection when server keeps sending corrupted state When corrupted state was found, imapc_client_mailbox_reconnect() is called to reconnect. This call skipped the normal "is it safe to reconnect?" checks causing potentially infinite reconnections.

imapc: If initial SELECT fails, don't resend it twice on reconnect.

imapc: Fix memory leak when closing mailbox with delayed changes

imapc: include imapc-client.h in imapc-storage.h It was already required to be able to successfully include imapc-storage.h.

imapc: Add imapc_features=fetch-bodystructure This allows using the remote IMAP server's BODY and BODYSTRUCTURE replies.

imapc: Fix crash in mailbox_exists() when LAYOUT isn't imapc. Especially breaks LAYOUT=none.

imapc: Make sure client is logged in when checking capabilities Especially with imapc_features=delay-login the capability could have been looked at before the client was even connected.

imapc: Add imapc_mailbox.capabilities Use it instead of imapc_client_get_capabilities(). Simplifies the following patch.

imapc: Rename imapc_storage_has_modseqs() to imapc_mailbox_has_modseqs() Simplifies the following patch.

imapc: Change imapc_connection_retry_interval setting to be in milliseconds.

lib-imap-client: Change connect_retry_interval_secs to _msecs This allows caller to provide better precision.

imapc: Remove "_sec" suffix from imapc_connection_retry_interval_secs Its type is SET_TIME, so the value doesn't have to be in seconds.

lib-storage: Do not clobber error with imapc

global: Fix imapc_client_login usage

lib-imap-client: Make retry count and interval configurable

imapc: Add imapc_features=delay-login The connection to imapc_host is delayed until it's actually needed to be performed.

imapc: Fail user creation if login to imapc_host fails. This causes imapc to actually wait for the login to succeed or fail. Such a wait was already done by the imap code, which will be removed by the next patch.

imapc: Use LOGOUT to cleanly disconnect from server. This makes it clearer in the remote server's logs whether the disconnection was intentional or not. Use a hardcoded 5 second timeout for LOGOUT. It should be enough time for the server to finish sending the tagged reply.

lib-storage: Add MAIL_ERROR_LIMIT

lib-imap-client: centralize authentication failed error logging

imapc: Allow using LAYOUT=none without crashing.

imapc: Fix assert-crash on some connect failures imapc_storage_has_modseqs() causes imapc_client_get_capabilities() to be called, which assert-crashes if there isn't a valid connection: Panic: file imapc-client.c: line 438 (imapc_client_get_capabilities): assertion failed: (conn != NULL)

Updated copyright notices to include the year 2017.

global: Replaced all instances of memset(p, 0, sizeof(*p)) with the new i_zero() macro. Used the following script: C_FILES=`git ls-files *.c` H_FILES=`git ls-files *.h` for F in "$C_FILES $H_FILES"; do echo "$F" perl -p -i -e 's/safe_memset\(&\(?([^,]*)\)?,\s*0,\s*sizeof\(\g1\)\)/i_zero_safe(&$1)/g' $F perl -p -i -e 's/safe_memset\(([^,]*),\s*0,\s*sizeof\(\*\g1\)\)/i_zero_safe($1)/g' $F perl -p -i -e 's/memset\(&\(?([^,]*)\)?,\s*0,\s*sizeof\(\g1\)\)/i_zero(&$1)/g' $F perl -p -i -e 's/memset\(([^,]*),\s*0,\s*sizeof\(\*\g1\)\)/i_zero($1)/g' $F done

imapc: Log server disconnection error only once.

global: Change string position/length from unsigned int to size_t Mainly to avoid truncating >4GB strings, which might potentially cause some security holes. Normally there are other limits, which prevent such excessive strings from being created in the first place. I'm sure this didn't find everything. Maybe everything could be found with compiler warnings. -Wconversion kind of does it, but it gives way too many unnecessary warnings. These were mainly found with: grep " = strlen" egrep "unsigned int.*(size|len)"

lib-storage: Add mail_storage.list_index_corrupted() The actual implementation is in the next commit.

imapc: %zu isn't standard, use PRIuSIZE_T instead.

lib-storage: Add mail_namespace_get_driver_settings() and use it. It obsoletes mail_storage_get_driver_settings().

imapc: Fix assert-crash when deleting mailbox triggers mailbox_get_open_status(). For example vsize header updating triggers this. Fixes: Panic: file mail-storage.c: line 1683: unreached

imapc: use serialized settings as unique_root_dir Without this, it is not possible to instantiate multiple imapc storages. Any attempts to will result in multiple namespaces sharing the same storage, regardless of if that was the intention.

imapc: Added imapc_max_line_length to limit maximum memory usage. The default is still unlimited, but this should be set to something smaller for untrusted servers.

imapc: Added support for imapc_features=modseq If the remote server supports CONDSTORE or QRESYNC extensions we'll use the remote's MODSEQ and HIGHESTMODSEQ counts. There are some situations where the HIGHESTMODSEQ isn't updated exactly correctly on an open mailbox, so this feature shouldn't be fully relied on. It was primarily implemented for dsync+imapc support - both for preserving modseqs and also for HIGHESTMODSEQ lookups.

imapc: Mark most of the commands retriable.

imapc: Fixed mailbox_exists() error handling.

imapc: If we get disconnected during SELECT/EXAMINE, retry it once. This seems to be happening especially with dsync migrations from IMAP servers with small timeouts. The initial dsync run opens imapc connection early to do a LIST + SELECT the first mailbox, but then dsync may spend a while creating all the local mailboxes before it continues using the imapc connection.

imapc: Compare INBOX in STATUS case-insensitively.

imapc: Added imapc_cmd_timeout setting to control the command wait timeouts.

imapc: Added support for PROXYAUTH command by setting imapc_features = proxyauth This can be useful when talking to Sun/Oracle IMAP server, which wants you to use LOGIN+PROXYAUTH to perform master user logins, instead of supporting the standard SASL PLAIN authentication.

imapc: Added imapc_features = no-examine

imapc: Fixed mailbox name escaping for STATUS

imapc: Fixed mailbox name escaping for CREATE, DELETE, RENAME, SUBSCRIBE, UNSUBSCRIBE

imapc: Fixed escaping/unescaping mailbox names. This fixes accessing mailbox names with '%' (the escape_char). It also fixes local namespace separator being different from the remote separator.

global: freshen copyright git ls-files | xargs perl -p -i -e 's/(\d+)-201[0-5]/$1-2016/g;s/ (201[0-5]) Dovecot/ $1-2016 Dovecot/'

imapc: Fixed handling escape-char The current code should now handle all kinds of mailbox names correctly, including: ~/foo %7e/bar

imapc: If login is aborted at deinit, don't log an error.

imapc: Don't automatically login to IMAP server for list=no namespaces.

imapc: If auth fails due to connection failure, don't treat it the same as failed login.

imapc: After auth failed, return failure immediately when opening a mailbox. Instead of trying to re-authenticate which again will most likely fail. Based on patch by Michael M Slusarz

imapc: Don't crash in mailbox_is_inconsistent() with unopened mailbox.

imapc: If authentication fails, preserve the error message as storage/list error. This isn't very helpful yet, because the mailbox list creation itself fails.

imapc: Added throttling settings to imapc_features=throttle:a:b:c This change could be reverted once good settings are found.

imapc: Fixed STATUS_FIRST_RECENT_UID to return the (mostly) correct UID.

imapc: Added imapc_features=search support for sending SEARCH commands. Currently requires the remote server to support ESEARCH (but this would be easy to avoid). This is only minimally tested for now, so bugs may exist (especially related to sub-queries).

imapc: Added imapc_sasl_mechanisms setting The first supported SASL mechanism is used, otherwise the login fails entirely.

global: freshen copyright Robomatically: git ls-files | xargs perl -p -i -e 's/(\d+)-201[0-4]/$1-2015/g;s/ (201[0-4]) Dovecot/ $1-2015 Dovecot/' Happy 2015 everyone! Signed-off-by: Phil Carmody <phil@dovecot.fi>

imapc: Fixed potential crash when trying to send NOOP before mailbox was fully opened.

Handle "out of disk space" and "out of user quota" as separate cases. "Out of disk space" is a temporary error that should be logged as error and the failure should be sent to user as "Internal server error". Obsolete the use of MAIL_ERROR_NOSPACE and MAIL_ERRSTR_NO_SPACE. Use the clearer MAIL_ERROR_NOQUOTA and MAIL_ERRSTR_NO_QUOTA instead.

imapc: Avoid hanging when running into server problems. imapc_mail_fetch_flush() caused imail->fetch_count to drop to 0, but imapc_mailbox_run() still started imapc_client_run() which was doing nothing.

Updated copyright notices to include year 2014.

imapc: Merge identical FETCH commands together (only updating UID range)

imapc: Added a Courier-workaround for not returning UIDNEXT on SELECT.

imapc: Abort pending commands before any deinitialization to avoid crashes.

imapc: Reorganize code so that imapc_list works without imapc_storage. Most importantly fixes crashes when imapc_list is trying to lookup hierarchy separator before storage is created.

imapc: Minor code cleanup.

imapc: imapc_storage no longer requires imapc_mailbox_list.

imapc: If imapc isn't the inbox=yes namespace, do the login and initial LIST in background.

imapc: Fixed mailbox deletion.

imapc: Empty imapc_user expands to namespace's owner, or with public namespaces to user itself. The main idea is that with shared namespaces it now expands to the shared username, allowing shared mailbox access via imapc.

imapc: If APPEND to selected mailbox doesn't send EXISTS, try if NOOP sends it. This makes Dovecot behave better with Courier.

Added ssl_client_ca_file to specify the CA certs as a file instead of as a dir. This is required for Redhat-based systems where there isn't a CA directory like in Debian/Ubuntu.

imapc: Give an early error message if imapc_list_prefix ends with separator.

Moved ssl_client_ca_dir and ssl_crypto_device to mail_storage_settings. Probably could also be moved to master_service_settings, but that makes accessing them a bit more tricky with the current code.

lib-storage: Replaced pop3c_ssl_ca_dir and imapc_ssl_ca_dir with generic ssl_client_ca_dir.

Replaced all -1U and (unsigned int)-1 with UINT_MAX. It's somewhat clearer this way. Also clang's -fsanitize=integer gives runtime errors about -1U (but not about explicit casts, so no need to change (type)-1 casts).

lib-storage: Added mailbox_status.have_guids flag

Oops :) Update copyrights to 2013 without breaking all .c files.