lib-storage: convert struct imapc_mail casts to container_of

imapc: Prefech - Initialize cached stream only if needed If the mail is immediately accessed for other purposes (e.g. fetching message flags), there's no need to go through all the trouble of initializing the mail stream.

imapc: If FETCH didn't send our wanted data, log the tagged FETCH reply text.

imapc: Don't use fetch-fix-broken-mails for OK and NO [SERVERBUG] FETCH replies Hopefully no servers are actually returning OK without returning a FETCH reply. If they are, maybe we need another workaround.. In any case Yahoo IMAP sometimes loses state and starts returning OK without FETCH replies to all UID FETCH requests (and BADs to FETCHes). It also may return NO [UNAVAILABLE] for both permanent and temporary FETCH failures, so we can't do anything with that. I added NO [SERVERBUG] because that's what Dovecot responds to temporary failures. I'm not sure how useful that check is though.

imapc: Prefetching shouldn't send FETCH requests for headers that are already cached.

imapc: If FETCH commands fail, have imapc_mail_fetch() always return failure. Earlier it may have returned success and imapc would later complain about a missing FETCH field.

imapc: Added imapc_features=zimbra-workarounds Zimbra (at least v5.0.18_GA_3011.RHEL4_64) can return different headers depending on whether the whole message body was fetched or only (partial) headers. This probably happens only for invalid characters that are translated into '?'. With this workaround enabled we don't use FETCH BODY.PEEK[], but we do FETCH (BODY.PEEK[HEADER] BODY.PEEK[TEXT]) and merge the results together. This way the results are always consistent and headers don't suddenly change.

imapc: Make sure we don't flush prefetch FETCH command unneededly.

imapc: Merge identical FETCH commands together (only updating UID range)

imapc: Don't send FETCHes for data that is already in cache.

imapc: Added imapc_feature fetch-header. It uses FETCH BODY.PEEK[HEADER.FIELDS (...)] whenever possible instead of fetching the entire header.

imapc: Keep the last fetched message body cached until mailbox is closed. This primarily helps partial IMAP fetches so each partial fetch doesn't redownload the message body.

imapc: Allow accessing a mail that is being saved without crashing. This fixes crashes with LDA, LMTP, mail_log plugin, etc.

Search supports now prefetching data for returned mails. Dropped imapc's own prefetching. mail_prefetch_count specifies how many mails can be kept open and issue a prefetch. This works using posix_fadvise(POSIX_FADV_WILLNEED) for maildir, sdbox and cydir backends. Apparently only Linux supports this. imapc backend also implements this internally by sending wanted IMAP commands to remote server. The command pipelining helps with latency. This change also makes it actually possible for imapc backend to first check if wanted data is already cached in local index and avoid sending unnecessary IMAP commands to remote server.

imapc: Write large message bodies to temp files rather than keeping in memory.

imapc: Fixed sending single FETCH queries.

imapc: Send UID FETCH commands with larger uidset parameter if possible. This code assumes that server doesn't reorganize FETCH replies when it's given an increasing UID range, which in theory IMAP server would be allowed to do, but I doubt there are any.