dsync: Add hashed_headers setting This makes it possible to configure them

dsync: Use header hashing version 3

dsync: Try to commit transactions every dsync_commit_msgs_interval messages This was first attempted to be implemented by ec0cc8fa647794e44a1afaa448f495a713048dc4, but it was later partially reverted by 5973d496b16721af6d2c1fa90b016aacddf13554. This current commit should fix its problems.

doveadm-sync: Add end-date support

doveadm sync/backup: Added -S <max size> parameter to skip too large mails.

dsync: When full resync is wanted in a stateful sync, output empty state. This continues 3d49dc64d, which didn't actually work because brain->require_full_resync was either cleared earlier or it was never even set in this brain.

dsync: When comparing headers' hashes to match messages, try to normalize the input. This is especially useful because some IMAP servers return different data depending on whether we're fetching only specific header fields, all headers or entire body. For now we're assuming that any non-ASCII is going to be replaced with '?', which helps at least with Zimbra and Yahoo. The header hashing algorithm is now versionable, so it can be modified more easily in future. This change should make imapc_features=zimbra-workarounds setting obsolete.

dsync: Added -T parameter to specify the I/O stall timeout.

dsync: Use storage's mail_error to choose the doveadm exit code. Instead of always assuming that all errors are EX_TEMPFAIL.

dsync: Added an extra "finish" state to allow slave-dsync to report error to master-dsync. I'm not sure if that's actually necessary, but just trying to follow the different possibilities on how dsync run can finish made me unsure about it. This should make it at least clear that if a slave-dsync has a failure flag set at deinit master-dsync will know about it before it returns success.

dsync: Added -F parameter to sync only mails with[out] specific flag.

dsync: If same GUID already exists in storage, try to copy it instead of recreating the mail. This way most mailbox backends can reduce disk space by only doing a reference count update. This feature isn't enabled by default. A virtual "All Mails" mailbox needs to be configured using the virtual plugin. Then you need to give this mailbox as -a parameter, e.g.: doveadm sync -a "Virtual/All Mails" ... Currently this is implemented by reading through all the GUIDs in the virtual mailbox. This of course isn't very efficient for things like incremental replication. An upcoming conversation plugin will keep track of all the mails' GUIDs, so in future replication should be able to have this functionality efficiently as well.

dsync: Added -t <timestamp> parameter to save only mails newer than <timestamp> If one side has old mails that don't exist on the other side, they are ignored (not synced and not deleted).

dsync: Support multiple -n parameters.

dsync: Added -x parameter to exclude mailboxes from sync. Multiple -x parameters can be added. Giving \flag as parameter means that the mailbox with the given SPECIAL-USE \flag is skipped. For example: doveadm sync -x '\All' -x '\Flagged' -x '\Important' mdbox:~/mdbox

dsync: Fixed talking to earlier dsync without mailbox attribute support. Most importantly it can now be used as an example how to add more features to dsync.

doveadm sync/backup: Added -g <guid> to sync only the specified mailbox (by GUID) Similar to -m <mailbox name>.

dsync: Added support for syncing mailbox attributes.

dsync: If I/O gets stalled, log the state in which it happened.

dsync: -l parameter locking is now done on the server with "lower" hostname. This allows running multi-master replication on two servers without two dsyncs mixing up changes by running at the same time.

dsync: Close mail streams earlier on failures to avoid assert-crashing

dsync: Fixes to handling output stream buffering.

dsync: Added back support for syncing only one mailbox (-m parameter)

dsync: Added debugging and rawlogging support.

doveadm: "backup" command is working again.

dsync: Renamed "slave" to "ibc" (= inter-brain communicator)