imap: support for FETCH SNIPPET

imap: Add debug logging for [un]hibernation The logs should now clearly say when a client was hibernated and when/why it was was unhibernated.

imap: Allow plugins to replace existing FETCH handlers They can use imap_fetch_handler_lookup() to get the old init() function. Then they can imap_fetch_handler_unregister() the existing handler and register a new handler, which can do its own work and call the old init() if needed.

global: Add mailbox_transaction_set_reason() calls Added to the most important places.

imap: Fix error message in failed FETCH's NO reply. This was broken only with the new imap_fetch_failure = no-after setting.

imap: Code cleanup - Remove duplicate flags from imap_fetch_state All the cur_flushed, line_partial and line_finished were about the same thing.

imap: Fix sending UID only when necessary on broken FETCHes. b748f91d0677fffaa2208b39ebb6db3aeb2e937b changed UID to be sent for most FETCH replies. There was also some existing code that attempted to do this, but didn't fully work. So now: 1) If there are no non-buffered replies, the entire FETCH response isn't sent. 2) If the buffer was already flushed and nothing else was sent, add UID to reply. The code paths for handling this are differently for imap_fetch_failure = disconnect-immediately vs others (depending on imap_fetch_cur_failed() return value).

imap: Add imap_fetch_failure setting This controls what happens when FETCH fails for some mails. The possible values are: disconnect-immediately: This is the original behavior. Whenever FETCH fails for a mail, the FETCH is aborted and client is disconnected. disconnect-after: The FETCH runs for all the requested mails, skipping any mails that returned failures, but at the end the client is still disconnected. no-after: The FETCH runs for all the requested mails, skipping any mails that returned failures. At the end tagged NO reply is returned. If the client attempts to FETCH the same failed mail more than once, the client is disconnected. This is to avoid clients from going into infinite loops trying to FETCH a broken mail.

global: unsigned int:1 -> bool:1 perl -i -pe 's/unsigned int ([^,:;]+):1;/bool $1:1;/' **/*.[ch]

global: Don't assume o_stream_send_istream() return bytes sent. This won't be true soon after its API changes.

Added %{fetch_hdr/body_count/bytes} variables to imap_logout_format

imap: Make sure we don't return empty "* n FETCH ()" reply, which violates RFC 3501 ABNF. It happened only during error conditions, and it would be a bit too much trouble to try to remove the reply entirely, so just append the message UID there.

imap: Fixed crashing in QRESYNC SELECT.

Removed CONTEXT_TYPE_SAFETY macro and reimplemented its functionality better. gcc/clang now gives a compiler error in many places if callback isn't exactly what was expected. It's also now much easier to add more of these checks.

Array API changed: ARRAY_DEFINE(name, type) -> ARRAY(type) name Easy way to update your existing code: perl -i -pe 's:ARRAY_DEFINE\(([^,]+), *([^)]+)\);:ARRAY($2) $1;:' **/*.[ch]

imap: Implemented NOTIFY extension. Requires mailbox_list_index=yes to work (and to show up in capabilities). SubscriptionChange event is still unimplemented.

imap: FETCH API updated to allow using the same parsed FETCH in multiple mailboxes. This is in preparation for NOTIFY extension.

imap: FETCH API cleanup: keep fetch state in a separate struct.

IMAP BINARY extension supports now FETCH BINARY command.

Marked functions parameters that are allowed to be NULL. Some APIs were also changed. The non-obvious APIs where NULL parameter was changed to "" are master_service_init() and auth_master_user_list_init(). These checks can currently be enabled only on a patched clang: http://llvm.org/bugs/show_bug.cgi?id=6786

imap: Rewrote FETCH command to use imap-msgpart API.

imap: Changed internal FETCH command handling API.

Removed unnecessary code.

lib-storage: Changed mailbox_search_*() API to return pointer struct mail. This reduces the flexibility, because now different mailbox_search_next() calls can't have different mail parameters with possibly different wanted_fields settings. This flexibility was never used though, and removing it gives a couple of benefits: * it's now easier for backends to implement prefetch and parallel search * usage is easier since mail_alloc()/mail_free() is no longer needed

IMAP: Register FETCH handlers at startup so we don't break if plugins add them. --HG-- branch : HEAD

imap: Code cleanup: Changed buffer to an array. --HG-- branch : HEAD

imap: Improve "FETCH .. got too little data" error messages. --HG-- branch : HEAD

imap: Code cleanup: Changed buffer to an array. --HG-- branch : HEAD

imap: Improve "FETCH .. got too little data" error messages. --HG-- branch : HEAD

QRESYNC: FETCH CHANGEDSINCE and SELECT with modseq shouldn't have returned changes with the same modseq. --HG-- branch : HEAD

Make mail_search_args an independent structure that can be used for multiple separate searches. --HG-- branch : HEAD

Added QRESYNC support. --HG-- branch : HEAD

Initial CONDSTORE support. --HG-- branch : HEAD

If stream ends unexpectedly, it was most likely because of a broken cached field, so mark the cache corrupted. --HG-- branch : HEAD

If stream ends unexpectedly, it was most likely because of a broken cached field, so mark the cache corrupted. --HG-- branch : HEAD

FLAGS/PERMENENTFLAGS weren't always sent to client early enough. Also optimized sending keywords with FETCH FLAGS. --HG-- branch : HEAD

Make sure we do a mailbox sync after flag updates (STORE, FETCH). Also added a new IMAP_SYNC_FLAG_SAFE which is used to figure out if appends and expunges are safe to send to client when delay-newmail workaround is enabled. --HG-- branch : HEAD

Changed .h ifdef/defines to use <NAME>_H format. --HG-- branch : HEAD

Added more consts to imap-parser API --HG-- branch : HEAD

When client fetches message data that has already been expunged, reply with dummy NIL and such values for the message. Don't disconnect the client anymore. --HG-- branch : HEAD

Dovecot is now able to execute multiple commands at the same time. Practically this means commands: FETCH, LIST, SEARCH and syncing output for all commands. For example it's possible that doing two FETCH commands at the same time makes their output mixed together. Non-blocking SEARCH is done by doing search for 20 mails at a time, and then checking if another command is pending. Also added X-CANCEL <tag> command to cancel running commands. --HG-- branch : HEAD

Replaced void *context from a lot of callbacks with the actual context type. Also added/fixed some context type checks. --HG-- branch : HEAD

Type safe callbacks weren't as easy as I thought. Only callback(void *context) can be handled generically. Others can be handled specially, but only if all the parameters are pointers, otherwise eg. int parameter can be replaced with long without compiler giving any warnings. --HG-- branch : HEAD

Added context parameter type safety checks for most callback APIs. --HG-- branch : HEAD

Array API redesigned to work using unions. It now provides type safety without having to enable DEBUG, as long as the compiler supports typeof(). Its API changed a bit. It now allows directly accessing the array contents, although that's not necessarily recommended. Changed existing array usage to be type safe in a bit more places. Removed array_t completely. Also did s/modifyable/modifiable/. --HG-- branch : HEAD

Memory leak fixes --HG-- branch : HEAD

Added "bool" type and changed all ints that were used as booleans to bool. --HG-- branch : HEAD

Send buffered fetch items to client first. Fixes kmail and Thunderbird issues.. --HG-- branch : HEAD

Major mail-storage API changes. It's now a bit cleaner and much more plugin friendly. Removed proxy_mailbox* stuff, they were difficult to use and there's now much easier way to replace them. --HG-- branch : HEAD

If UID FETCH notices in the middle of fetching that message is expunged, don't fail but just don't return all the data. --HG-- branch : HEAD

Moved command-specific variables from struct client to struct client_command_context and changed code to use it. --HG-- branch : HEAD

Initial support for keywords. Syncing to mbox/maildir doesn't work yet. --HG-- branch : HEAD

If fetching fails, finish sending the untagged FETCH reply correctly. --HG-- branch : HEAD

BODY.PEEK[HEADER.FIELDS (...)] list is allowed to contain strings and literals. We didn't handle them correctly before. --HG-- branch : HEAD

Added netscape-eoh workaround. --HG-- branch : HEAD

We never do blocking reads/writes to network anymore. Changed imap and pop3 processes to use a single I/O loop. Not much tested yet, and currently LIST/LSUB may eat too much memory and APPEND eats all CPU. --HG-- branch : HEAD

Header caching redesigned. New design allows caching decisions per field, so they can be divided to temporary/permanent. Cached headers are now always returned in original order, old code didn't guarantee it. Some other caching changes. (still missing code to store changes in caching decisions) --HG-- branch : HEAD

importing new index code. mbox still broken. --HG-- branch : HEAD

Removed fetch_init/fetch_next from mail-storage. search_* makes it unnecessary --HG-- branch : HEAD

Index cache file rewrite. It's not finished yet and mbox support is completely broken. But it's getting difficult to maintain outside cvs :) --HG-- branch : HEAD

API change for updating message flags. --HG-- branch : HEAD

Partial fetches are working fast again - faster than ever actually. --HG-- branch : HEAD

mail-storage.h interface changes, affects pretty much everything. FETCH, SEARCH, SORT and THREAD handling were pretty much moved from lib-storage/ to imap/ so adding non-index storages would be much easier now. Also POP3 server can now be easily implemented with lib-storage. Not too well tested, and at least one major problem: partial fetching is _slow_. --HG-- branch : HEAD