lib-storage: imapc - Ensure search ctx is not NULL Satisfies static analyzers

Updated copyright notices to include the year 2018.

global: Use mail_set_critical() and mailbox_set_critical() if possible Replace calls to mail_storage_set_critical() with mail_set_critical() or mailbox_set_critical() in places where mailbox or mail are easily available.

lib-storage: convert imapc to use container_of

lib-storage: rename IMAPC_CONTEXT to IMAPC_SEARCHCTX

global: Add/change switch case "fall through" comments These fix warnings with gcc 7's -Wimplicit-fallthrough

imapc: SEARCH - Don't add parenthesis around OR There's no need to write "(OR a b)" when "OR a b" works just the same.

imapc: Use UID SEARCH instead of SEARCH UIDs work better for some broken IMAP servers, so it's safer to use it. Keep using sequences for ESEARCH, since servers supporting it aren't at least yet known to be broken and the search result with sequences is smaller.

imapc: include imapc-client.h in imapc-storage.h It was already required to be able to successfully include imapc-storage.h.

imapc: Support imapc_features=search without ESEARCH extension

imapc: Add imapc_mailbox.capabilities Use it instead of imapc_client_get_capabilities(). Simplifies the following patch.

Partially implemented IMAP SEARCH=X-MIMEPART capability. This capability is currently Dovecot-specific.

Updated copyright notices to include the year 2017.

imapc: Log server disconnection error only once.

global: Change string position/length from unsigned int to size_t Mainly to avoid truncating >4GB strings, which might potentially cause some security holes. Normally there are other limits, which prevent such excessive strings from being created in the first place. I'm sure this didn't find everything. Maybe everything could be found with compiler warnings. -Wconversion kind of does it, but it gives way too many unnecessary warnings. These were mainly found with: grep " = strlen" egrep "unsigned int.*(size|len)"

imapc: Don't allow "*" in SEARCH replies Doesn't fix anything, but makes the parsing a bit more correct.

global: freshen copyright git ls-files | xargs perl -p -i -e 's/(\d+)-201[0-5]/$1-2016/g;s/ (201[0-5]) Dovecot/ $1-2016 Dovecot/'

imapc: Don't write SEARCH YOUNGER/OLDER queries if server doesn't support WITHIN extension

imapc: Use mail_search_arg_to_imap() to write IMAP SEARCH parameters. This also fixes a few bugs, since some of the search parameters weren't written entirely correctly.

imapc: Fixed modseq search for previous imapc_features=search change

imapc: Added imapc_features=search support for sending SEARCH commands. Currently requires the remote server to support ESEARCH (but this would be easy to avoid). This is only minimally tested for now, so bugs may exist (especially related to sub-queries).

Search supports now prefetching data for returned mails. Dropped imapc's own prefetching. mail_prefetch_count specifies how many mails can be kept open and issue a prefetch. This works using posix_fadvise(POSIX_FADV_WILLNEED) for maildir, sdbox and cydir backends. Apparently only Linux supports this. imapc backend also implements this internally by sending wanted IMAP commands to remote server. The command pipelining helps with latency. This change also makes it actually possible for imapc backend to first check if wanted data is already cached in local index and avoid sending unnecessary IMAP commands to remote server.

lib-storage: Changed mailbox_search_*() API to return pointer struct mail. This reduces the flexibility, because now different mailbox_search_next() calls can't have different mail parameters with possibly different wanted_fields settings. This flexibility was never used though, and removing it gives a couple of benefits: * it's now easier for backends to implement prefetch and parallel search * usage is easier since mail_alloc()/mail_free() is no longer needed

imapc: Use multiple client connections when necessary. Some other fixes.

imapc: Don't try to wait for FETCH to return messages that are known to be expunged.

imapc: Fixes to handling FETCH commands.

imapc: Write large message bodies to temp files rather than keeping in memory.

imapc: Fixed sending single FETCH queries.

imapc: Minor code cleanup

imapc: Send UID FETCH commands with larger uidset parameter if possible. This code assumes that server doesn't reorganize FETCH replies when it's given an increasing UID range, which in theory IMAP server would be allowed to do, but I doubt there are any.

imapc: If mail's input stream is changed, don't assume virtual size = physical size.

imapc: Added support for saving and copying messages.

imapc: Fixed mail_set_seq/uid() to work.

imapc: Call istream_opened() method if it's set for mail.

imapc: Don't send broken FETCH command when we don't actually have to fetch anything.

Added initial implementation of "imapc" storage. It can be used to create a "smart IMAP proxy" where Dovecot uses remote IMAP server as a mail storage. This is a very rough early implementation. Performance isn't good, many required features are missing, error handling is lacking and code needs de-uglification. Still, it should be enough for selecting INBOX and accessing mails in it.