Updated copyright notices to include the year 2018.

lmtp: Changed message data handling to use iostream-temp. The existing code predates iostream-temp.

lmtp: Removed structural comments that are now useless.

lmtp: Ported to use lib-smtp/server. Implicitly fixes handling of multi-line replies from proxy backend. Implicitly adds support for mixing local and proxy recipients. Implicitly adds support for SMTP CHUNKING. RCPT failures are reported back to the client immediately, rather than waiting for the DATA command.

lmtp: commands: Created local variable for client->dot_input in client_input_data_handle().

lmtp: commands: Renamed client_input_add() to cmd_data_input_add().

lmtp: commands: Renamed client_input_add_file() to cmd_data_input_add_file().

lmtp: commands: Renamed client_get_input() to cmd_data_get_input().

lmtp: Split off local delivery in a struct separate from the client. This mimics proxy delivery, making the implementation better structured.

lmtp: commands: Moved RCPT command handling relating to local recipients to lmtp-local.c.

lmtp: local: Renamed client_input_data_write_local() to lmtp_local_data().

lmtp: proxy: Changed return type of lmtp_proxy_rcpt() from bool to int.

lmtp: proxy: Renamed client_proxy_rcpt() to lmtp_proxy_rcpt().

lmtp: Renamed struct mail_recipient to struct lmtp_recipient.

lmtp: commands: Added structural comments.

lmtp: Moved code relating to proxy from commands.c to lmtp-proxy.c.

lmtp: Moved client_input_data_write_local() from commands.c to lmtp-local.c.

lmtp: Moved client_open_raw_mail() from commands.c to lmtp-local.c.

lmtp: Moved client_rcpt_fail_all() from commands.c to lmtp-local.c.

lmtp: Moved client_deliver_to_rcpts() from commands.c to lmtp-local.c.

lmtp: Moved client_deliver() from commands.c to lmtp-local.c.

lmtp: Moved rcpt_anvil_lookup_callback() from commands.c to lmtp-local.c.

lmtp: Moved cmd_rcpt_finish() from commands.c to lmtp-local.c.

lmtp: Moved lmtp_anvil_init() from commands.c to main.c. It fits better there.

lmtp: proxy: Changed implemention to use lib-smtp/smtp-client.

lmtp: commands: Grouped deliver context field assignments by topic in client_deliver().

lib-lda: Added SMTP parameter fields to struct mail_deliver_context. This moves the original recipient field inside the SMTP parameters. This also affects both lda and lmtp. Changes are applied there accordingly.

lib-lda: Changed struct mail_deliver_context address fields to use the new struct smtp_address type.

lib-lda: Renamed struct mail_deliver_context fields for consistency with lib-smtp. New fields are added in subsequent commits.

lmtp: Removed lmtp_address_translate setting. It is assumed not to be used.

lmtp: Accept XCLIENT ADDR field IPv6 address both with and without "IPV6:" prefix. This is the format that Postfix actually prescribes. The new lib-smtp will add the prefix by default.

global: Replace o_stream_nfinish() with o_stream_flush()

lmtp: Fixed passing SMTP submit settings to plugins. This caused Sieve to crash upon SMTP message submission; e.g., during redirect.

lib-smtp: smtp-submit: Made submission_timeout a setting rather than a function parameter. This makes it configurable.

lda/lmtp: Parse SMTP submit settings.

lmtp: commands: client_deliver(): Put client->state.dest_user in a variable. Keeps code more compact, as this value is used several times.

lmtp: commands: client_deliver(): Put result from mail_user_var_expand_table() in a variable. Keeps code more compact, as this function is called several times.

lib-storage: Add reason to mailbox_transaction_begin() Remove mailbox_transaction_set_reason().

global: start relying on timeout_remove(NULL) being a no-op Cleanup performed with the following semantic patch: @@ expression E; @@ - if (E != NULL) { - timeout_remove(&E); - } + timeout_remove(&E);

lmtp proxy: Avoid DNS lookup for "host" if passdb also returns "hostip"

lmtp: client_input_data_write_local() refactored loops in client_input_data_write_local() and in client_deliver_next() are merged as they are essentially the same loop, client_deliver_next() renamed to client_deliver_to_rcpts().

lmtp: Fix for wrong session id of mail user when saving mail, quota checking on When quota is checked mail user is allocated with custom ":quota" session id suffix without incrementing service user session id counter

lmtp: Use message_detail_address_parse

lmtp: Fix assert-crash when proxy overrides mail_max_lock_timeout Time unit is required or the call fails.

global: Add mailbox_set_reason() calls Added to the most important places.

lmtp: Trigger autoexpunging only for the last RCPT TO. Otherwise if the autoexpunging takes a long time, the LMTP client could disconnect due to a timeout. The mails would still eventually get delivered though, so it would result in duplicate mails being delivered. An alternative to this would be to keep all the mail_users referenced until the delivery is finished and then autoexpunge all of them at the end. It increases memory usage though and complicates the code, so at least for now it's not implemented.

lib-storage: Replace MAIL_STORAGE_SERVICE_FLAG_AUTOEXPUNGE with explicit mail_user_autoexpunge() This allows better control of which users are being autoexpunged. This patch changes behavior at least in two ways now: 1) After shared folder access, the owner user isn't autoexpunged at deinit. Although this is a bit questionable of whether it should be or not. 2) LMTP's quota check at RCPT TO stage doesn't trigger autoexpunging.

global: Log internal storage error on failure

lmtp: Log internal error on STATUS_CHECK_OVER_QUOTA failure

global: Replace mail_storage_service_user_free() with _unref()

lmtp: Fix lmtp_user_concurrency_limit when userdb changes the username Anvil LOOKUP was done with the original username, while CONNECT/DISCONNECT was done with the changed username. So LOOKUP always thought that there were zero concurrent sessions for the user.

lmtp: Don't deliver truncated email when client disconnects before "." line This didn't happen always, because the EOF was handled in two different places in different ways.

Updated copyright notices to include the year 2017.

global: Replaced all instances of memset(p, 0, sizeof(*p)) with the new i_zero() macro. Used the following script: C_FILES=`git ls-files *.c` H_FILES=`git ls-files *.h` for F in "$C_FILES $H_FILES"; do echo "$F" perl -p -i -e 's/safe_memset\(&\(?([^,]*)\)?,\s*0,\s*sizeof\(\g1\)\)/i_zero_safe(&$1)/g' $F perl -p -i -e 's/safe_memset\(([^,]*),\s*0,\s*sizeof\(\*\g1\)\)/i_zero_safe($1)/g' $F perl -p -i -e 's/memset\(&\(?([^,]*)\)?,\s*0,\s*sizeof\(\g1\)\)/i_zero(&$1)/g' $F perl -p -i -e 's/memset\(([^,]*),\s*0,\s*sizeof\(\*\g1\)\)/i_zero($1)/g' $F done

global: Change string position/length from unsigned int to size_t Mainly to avoid truncating >4GB strings, which might potentially cause some security holes. Normally there are other limits, which prevent such excessive strings from being created in the first place. I'm sure this didn't find everything. Maybe everything could be found with compiler warnings. -Wconversion kind of does it, but it gives way too many unnecessary warnings. These were mainly found with: grep " = strlen" egrep "unsigned int.*(size|len)"

lib: API change - var_expand*() now returns error string. This allows callers to fail properly if the format string is invalid.

lmtp: Handle lmtp_user_concurrency_limit before lmtp_rcpt_check_quota The quota lookup is more expensive, so it shouldn't be done unnecessarily.

lmtp: Support %{userdb:*} variables in mail_log_prefix

lmtp: Fixed using default port 24 Broken by dfe53c824.

lmtp: Use port 24 if no port has been provided This allows using the LMTP, IMAP and POP3 proxy on the same Dovecot installation without the userdb providing the port to connect to. TCP port 24 is registered at IANA as: "any private mail system" LMTP being the Local Mail Transfer Protocol classifies as a private mail system and thus justifies the usage of port 24. Prior to this patch the LTMP client would connect to TCP port 0 by default if the userdb did not provide a port to connect to.

lmtp: Don't send double-DISCONNECT to anvil. Happened with lmtp_user_concurrency_limit > 0. Fixes errors like: Error: connect limit: disconnection for unknown pid 123 + ident lmtp/username

lmtp: Removed assert, which triggers if anvil connect fails.

lmtp: Fixed pipelining of commands after RCPT TO We should stop while waiting for anvil reply. Broken by ced943b0a.

lmtp: If anvil lookup fails, ignore lmtp_user_concurrency_limit Previously it was randomly allowing or disallowing the client.

lmtp: Increase user's concurrency limit already after RCPT TO. This way it's not possible for a lot of mails to arrive to user concurrently and bypass the lmtp_user_concurrency_limit.

lmtp: If user has rached lmtp_user_concurrency_limit, fail at RCPT TO stage. Otherwise LMTP client would have to send the entire email body before getting the failure.

lib: remove autoclose parameter from [io]_stream_create_fd Use [io]_stream_create_fd_autoclose() for autoclose.

lib-storage: mail_storage_service_next() now returns error string.

lmtp, quota-status: Don't send mail_storage_service_lookup() error to client. This is in preparation for changing the returned error to contain the full error string. The current error string is pretty useless for any purpose.

lmtp: Fixed error logging when writing to temp file

lmtp: Removed (Dovecot) from Received header. Some people want to hide it, and I don't really see much benefit in including it anyway. So lets just hide it from everyone.

lmtp: Connect to anvil earlier while still running as root This was a problem only with lmtp_rcpt_check_quota=yes.

Support multiple recipient_delimiters The recipient_delimiter is treated as multiple one-character delimiters rather than one multi-character delimiter if more than one character is supplied. The address is split on the first character in recipient_delimiter found.

lmtp: Use separate session ID suffixes for each RCPT TO delivery. Otherwise each delivery will use the same session ID when talking to stats process, which results in errors like: Error: stats: FIFO input error: CONNECT: Duplicate session ID ME3ZHCi+A1dUDQAAvAUe3g for user foobar service lmtp Warning: stats: Couldn't find session ID: ME3ZHCi+A1dUDQAAvAUe3g (There was a DISCONNECT for the session ID between these two log lines.)

lmtp: Changed default LMTP proxy timeout to 125 seconds. The main problem with LMTP proxy timing out too early is that it causes duplicates if the backend actually finishes the mail delivery. The 30 seconds is bad, because there are various timeouts in backend set to 30 seconds also. 125 seconds is hopefully large enough to hit most of the 2 minute timeouts and we'll have a few extra seconds left to see the failure.

global: freshen copyright git ls-files | xargs perl -p -i -e 's/(\d+)-201[0-5]/$1-2016/g;s/ (201[0-5]) Dovecot/ $1-2016 Dovecot/'

lmtp: Memory leak fixes on handling some RCPT TO failures.

lmtp: quota_full_tempfail=yes didn't work with lmtp_rcpt_check_quota=yes

Replaced unlink() calls with i_unlink*() wherever possible.

Removed all invocations of atoi().

Changed type of internet port values to in_port_t everywhere. Created special SET_IN_PORT setting type for internet port values. Created net_str2port() for parsing internet port values. Removed several atoi() invocations in the process.

lmtp: Fixed assert-crash on anvil lookup failures. If anvil_client_query() fails, it immediately calls the callback and returns NULL. So we need to increase anvil_queries even before calling anvil_client_query()

lmtp: Earlier compiler warning fix broke compiling completely..

lmtp: chdir() to base_dir shouldn't fail - log an error if it does

lmtp: Added lmtp_hdr_delivery_address=final|original|none setting. This controls what username is used for logging Delivered-To: header and the "for user@domain" in Received: header. The default "final" is the regular username as earlier, "original" logs the RFC822 address received via ORCPT parameter or fallbacks to username, "none" logs nothing.

lib-lda: Assume that the caller sets delivery_time_started This is required because parts of the delivery time may be done at the time the user is being initialized, so it needs to be set before the user init is done.

lda, lmtp: Added delivery_time and session_time variables to deliver_log_format session_time is meaningful only with LMTP. The delivery_time is separate for each mail delivery. The results are in milliseconds now. If needed we could add a new %modifier that converts it into secs.millisecs.

lmtp: Fixed lmtp_user_concurrency_limit=0 to actually mean unlimited

lda/lmtp: Added a hardcoded LDA_SUBMISSION_TIMEOUT_SECS=30 timeout for SMTP client. Maybe we could have a new setting if needed later on, but we've been working pretty successfully without any timeout for a long time..

lmtp: Added lmtp_user_concurrency_limit setting. This allows limiting the number of concurrent deliveries to a single user. Useful in case one user is receiving a lot of mails and causing other deliveries to be delayed. If the limit is reached, Dovecot returns temporary failure error at DATA stage.

lmtp: Changed rcpt_to array to contain pointers to struct mail_recipient This allow storing the pointers so they don't change.

global: freshen copyright Robomatically: git ls-files | xargs perl -p -i -e 's/(\d+)-201[0-4]/$1-2015/g;s/ (201[0-4]) Dovecot/ $1-2015 Dovecot/' Happy 2015 everyone! Signed-off-by: Phil Carmody <phil@dovecot.fi>

lmtp: Added TLS security information to Received: header when STARTTLS was used.

lmtp: Added support for DSN ORCPT parameter. If the parameter is specified, it's used as the original recipient address in Sieve. The ORCPT parameter is also proxied to destination LMTP server if used (without checking if it supports it). We don't currently advertise DSN extension, so this feature won't be enabled automatically.

lmtp: Show more information in process title. Most importantly show the username while messages are being delivered.

lmtp: Added support for STARTTLS command.

lmtp: Use mail_log_prefix while mails are being delivered.

lmtp: Added missing session_id to mail_user

lda, lmtp: Use mail_deliver_context.tempfail_error first if it exists (instead of storage error).

lmtp: Put back the deduplication code. Too dangerous after all to change it in the middle of v2.2.x. It's also not causing that much trouble even when it breaks..

Handle "out of disk space" and "out of user quota" as separate cases. "Out of disk space" is a temporary error that should be logged as error and the failure should be sent to user as "Internal server error". Obsolete the use of MAIL_ERROR_NOSPACE and MAIL_ERRSTR_NO_SPACE. Use the clearer MAIL_ERROR_NOQUOTA and MAIL_ERRSTR_NO_QUOTA instead.

lmtp: Remove <> from Delivered-To: header. This annoyingly changes Dovecot behavior in the middle of v2.2.x series, but the earlier value was definitely wrong.. Perhaps we still need to provide a setting for this, but that's pretty annoying as well.

lmtp: Small code cleanup

lmtp: Removed code that attempts to deduplicate mail files by copying them between user mailboxes. This sometimes started failing if the mail that was being used for copying was deleted by the user. There's no good way for lmtp code to fix that situation. If deduplication is needed, it could be implemented in a more generic way inside mailbox_copy() where after initial copy it would store the destination struct mail to src_mail->last_copy_dest_mail. If another mail is copied, the last_copy_dest_mail could be attempted to be used for the copying and if that doesn't work it would fallback to regular copying. This should probably be attempted only for lda/lmtp processes as it would just cause extra overhead for others.

lmtp: Reduce corking so we get command replies more quickly even if they are pipelined. This may help avoid some LMTP client timeouts.

lmtp: Removed some unnecessary code

Updated copyright notices to include year 2014.