lib-storage: convert maildir to use container_of

global: unsigned int:1 -> bool:1 perl -i -pe 's/unsigned int ([^,:;]+):1;/bool $1:1;/' **/*.[ch]

maildir: Fixed looking up physical size for a mail while it's being saved. Based on patch by Teemu Huovila.

Replaced *_INDEX_PREFIX macros with a common MAIL_INDEX_PREFIX.

Removed CONTEXT_TYPE_SAFETY macro and reimplemented its functionality better. gcc/clang now gives a compiler error in many places if callback isn't exactly what was expected. It's also now much easier to add more of these checks.

Marked functions parameters that are allowed to be NULL. Some APIs were also changed. The non-obvious APIs where NULL parameter was changed to "" are master_service_init() and auth_master_user_list_init(). These checks can currently be enabled only on a patched clang: http://llvm.org/bugs/show_bug.cgi?id=6786

Added mail_temp_scan_interval setting and changed its default from 8h -> 1w.

lib-storage: Mailbox list index's ext_id shouldn't be cached in mail_storage. It's mailbox_list-specific, not mail_storage-specific. The easiest fix was to just cache it into mailbox rather than create a new index_mailbox_list.

maildir: Avoid refreshing uidlist unnecessarily. Even if the uidlist itself wasn't read, it was still stat()ed.

lib-storage: Removed mailbox.backend_readonly. Backends set it internally lazily now.

lib-storage: Moved box->private_flags_mask to mailbox_get_private_flags_mask() This allows getting it lazily only when it's actually needed.

dbox, maildir: When copying messages, copy the cached fields also. Copy only fields whose caching decision is not "no" in the destination mailbox.

maildir: Avoid refreshing transaction log immediately on first sync.

maildir: If we see unwanted non-empty directories in new/ or cur/, move them to ../extra-*

maildir: Cleaned up filename/guid preserving code on save/copy.

Removed mail_update_uid() / mail_index_update_uid(). It was working properly only with mdbox and there wasn't really a need for it. --HG-- branch : HEAD

Avoid including mailbox-list-private.h everywhere. --HG-- branch : HEAD

Removed dead code. --HG-- branch : HEAD

maildir_copy_with_hardlinks=yes no longer has a race condition. --HG-- branch : HEAD

lib-storage: Mailbox deletion API changed. Mailbox deletion should now be free of race conditions. The actual file deletion code is now responsibility of mailbox_list backend. --HG-- branch : HEAD

lib-storage: *_mailboxes don't descend from index_mailbox anymore, it's now a context. --HG-- branch : HEAD

lib-storage: Removed mailbox_list_get_guid(). Directory GUIDs are pretty pointless. --HG-- branch : HEAD

maildir: Always add S=size to filenames. It's necessary for using zlib plugin and doesn't hurt much anyway. --HG-- branch : HEAD

Transaction commits can now track how many uid/modseq updates were ignored. --HG-- branch : HEAD

Added mailbox_list_get_guid() to get/create directory GUID. Directory GUIDs are mainly useful for dsync to identify \Noselect mailboxes. --HG-- branch : HEAD

maildir: mail_update_uid() now updates uidlist also. --HG-- branch : HEAD

index-storage: Simplified the way transactions are are created/finished. Removed also mail_storage.class_init/deinit since they're no longer used for anything. --HG-- branch : HEAD

maildir: Support saving messages with specified UIDs. --HG-- branch : HEAD

maildir: Cleaned up save/copy code. --HG-- branch : HEAD

maildir: Small code cleanup. --HG-- branch : HEAD

Mailbox opening and closing APIs changed. --HG-- branch : HEAD

Initial commit for making mail_storage:mailbox_list to be from 1:1 to n:n. This will allow namespaces to share the same storage and a single namespace to have multiple storages (multiple mailbox formats). Neither works currently, and this commit probably breaks some things. --HG-- branch : HEAD

Added maildir_very_dirty_syncs setting. --HG-- branch : HEAD

Mail copying API changed to be based on save API. This allows extending it easily in future. --HG-- branch : HEAD

Initial commit for config rewrite. --HG-- branch : HEAD

Maildir/dbox: Try harder to assign unique UIDVALIDITY values to mailboxes. --HG-- branch : HEAD

mailbox_save_init() API was split to multiple functions. This allows adding more parameters easily in future. --HG-- branch : HEAD

mailbox.save_init() API now takes ** pointer to dest_mail, so multiple plugins can safely specify it. ctx->dest_mail also gets updated then. --HG-- branch : HEAD

Keep track of dovecot-uidlist mtime+size in index file. If it's up-to-date (and unless indexes weren't updated at some point, it always is) and we're saving a new message, don't bother reading the uidlist contents. --HG-- branch : HEAD

Removed maildir_tmp_cleanup(). Use unlink_old_files() instead. --HG-- branch : HEAD

Delay creating directories until we really need them. --HG-- branch : HEAD

Moved mail_create_mode/gid from struct maildir_mailbox to struct mailbox. --HG-- branch : HEAD

Changed .h ifdef/defines to use <NAME>_H format. --HG-- branch : HEAD

Moved mailbox list index sync stamp checking and updating to storage-specific code. --HG-- branch : HEAD

Moved private_flags_mask to struct mailbox. --HG-- branch : HEAD

Store syncing information to maildir extension header instead of kludging them into base header. Also we're now using nanoseconds to check for changes if they're available. --HG-- branch : HEAD

Moved maildir_create_tmp() to maildir-save. It's the only place where it's used. --HG-- branch : HEAD

Moved maildir filename related functions to maildir-filename.c --HG-- branch : HEAD

Changed *_transaction_created() to static. --HG-- branch : HEAD

Removed unused macros --HG-- branch : HEAD

Removed struct index_storage abstraction. It's pointless. --HG-- branch : HEAD

Better type safety to module_contexts arrays. Already fixed some bugs. --HG-- branch : HEAD

Moved delete/rename operations to mailbox_list API. Fixed mbox/maildir to work with either fs/maildir++ directory layout. They can be changed by appending :LAYOUT=fs|maildir++ to mail_location. --HG-- branch : HEAD

When syncing huge maildirs check once in a while if we need to update dovecot-uidlist.lock so that another process won't decide that it's stale and overwrite it. Check more often when moving mails from new/ to cur/. --HG-- branch : HEAD

If pop3_lock_session=yes, update maildir/dbox uidlist lock file every 10 seconds so it won't be treated as a stale lock. --HG-- branch : HEAD

Saved mails and dovecot-keywords file didn't set the group from dovecot-shared file. --HG-- branch : HEAD

When opening maildir, check if tmp/'s atime is over 8h old. If it is, delete old files in it. However if atime - ctime > 36h, it means there's nothing to be deleted so the scanning isn't done then. We update atime if filesystem is mounted with noatime. --HG-- branch : HEAD

Added maildir_copy_preserve_filename setting. --HG-- branch : HEAD

maildir_file_do() takes uid, not seq.. --HG-- branch : HEAD

Type safe callbacks weren't as easy as I thought. Only callback(void *context) can be handled generically. Others can be handled specially, but only if all the parameters are pointers, otherwise eg. int parameter can be replaced with long without compiler giving any warnings. --HG-- branch : HEAD

Added context parameter type safety checks for most callback APIs. --HG-- branch : HEAD

Mailbox list indexing and related changes. Currently works only with maildir and mmap_disable=no. This allows doing STATUS to synced mailboxes without opening their index files at all. --HG-- branch : HEAD

Initial code for separation of mailbox accessing and directory layout handling. It's not yet possible to change the default layouts though. --HG-- branch : HEAD

Saving mails could have skipped over transactions, which caused different kinds of problems. --HG-- branch : HEAD

Beginnings of joining mail-storage API more closely to mail-index, so that mail-index could be directly used if needed. Currently only transactions are joined. --HG-- branch : HEAD

Array API redesigned to work using unions. It now provides type safety without having to enable DEBUG, as long as the compiler supports typeof(). Its API changed a bit. It now allows directly accessing the array contents, although that's not necessarily recommended. Changed existing array usage to be type safe in a bit more places. Removed array_t completely. Also did s/modifyable/modifiable/. --HG-- branch : HEAD

Add ",S=size" to maildir filenames when quota plugin is loaded with Maildir++ backend. --HG-- branch : HEAD

Changed mailbox_save_*() API a bit: Moved the struct mail *dest_mail to save_init() instead of being in save_finish(). This way you can request wanted fields from the mail while it's being saved. With maildir the message is being parsed at the same time as it's being saved, and the results are stored into cache file. --HG-- branch : HEAD

Adding mail to index while saving it had a race condition. Fixing it required a bit larger changes. Switched uidlist/index locking order so that uidlist is now locked first. --HG-- branch : HEAD

When deleting a maildir, rename it to ..DOVECOT-TRASHED, and assume only that directory to be safe to be deleted, instead of all ..dirs. When deleting maildir with absolute path, don't do renaming at all but delete it directly. --HG-- branch : HEAD

Support accessing saved mails that haven't been committed yet (fixes quota for maildir). --HG-- branch : HEAD

Merged save-copying and hardlink-copying code so that hardlink-copying updates indexes immediately. --HG-- branch : HEAD

mailbox_save_init() supports now returning failure. Quota plugin now checks if mail is too large in mailbox_save_init() hook. APPEND fails before giving "+ OK" reply if mailbox_save_init() failed. Also fixed some APPEND failure cases where the next command was eaten away. --HG-- branch : HEAD

Added "bool" type and changed all ints that were used as booleans to bool. --HG-- branch : HEAD

maildir_copy_with_hardlinks works again. --HG-- branch : HEAD

Added flags and keywords parameters to mailbox_copy() in case they're wanted to be changed while copying. --HG-- branch : HEAD

Store new/ directory's timestamp in sync_size header in index (kludgy..). Fixed saving message into empty and non-synced mailboxes. --HG-- branch : HEAD

Keywords are stored in maildir filename and maildir-keywords file (backwards compatible with 0.99.x's .customflags file) --HG-- branch : HEAD

Use macros when using W and S extras too. --HG-- branch : HEAD

Replaced ':' and ',' character usages with #defines, so they can be changed easily if needed. --HG-- branch : HEAD

Created mbox_storage and maildir_storage. --HG-- branch : HEAD

Moved mbox/maildir-specific variables from struct index_mailbox to mbox_mailbox and maildir_mailbox. --HG-- branch : HEAD

Major mail-storage API changes. It's now a bit cleaner and much more plugin friendly. Removed proxy_mailbox* stuff, they were difficult to use and there's now much easier way to replace them. --HG-- branch : HEAD

Keyword fixes. --HG-- branch : HEAD

Initial support for keywords. Syncing to mbox/maildir doesn't work yet. --HG-- branch : HEAD

Warning fix for cygwin. --HG-- branch : HEAD

Always protect maildir syncing with uidlist lock. Before we only tried to do it and if it failed, we went ahead and just didn't sync any new messages. That however could have caused files to be lost temporarily due to race conditions between readdir() and rename(). --HG-- branch : HEAD

Unlock uidlist file only after appends are written to transaction log. --HG-- branch : HEAD

Don't break if saving messages to maildir without index files. --HG-- branch : HEAD

Added sync flags parameter to mailbox_transaction_commit(), so EXPUNGE can do a full sync with dirty syncing enabled. --HG-- branch : HEAD

Changed mail saving API to be nonblocking. --HG-- branch : HEAD

Moved namespace and hierarchy separator handling to imap-specific code. LIST now shows non-hidden namespaces in the LIST reply. --HG-- branch : HEAD

Broke mailbox_sync() into iterator. --HG-- branch : HEAD

mailbox_save() and mailbox_copy() functions can now return the saved mail so it can be immediately queried. Implemented UIDPLUS extension using it. Maildir implementation missing, so it crashes with it for now.. APPEND with mbox now doesn't require resyncing the mailbox since it updates indexes directly. --HG-- branch : HEAD

"readonly sync" -> "forced sync" --HG-- branch : HEAD

mbox code compiles again, but syncing is only partially implemented so accessing mboxes fails.. Also some cleanups to index-storage and maildir code. --HG-- branch : HEAD

s/custom flags/keywords/ --HG-- branch : HEAD

importing new index code. mbox still broken. --HG-- branch : HEAD

Mail storages support now configurable namespace prefix and hierarchy separator. Subscription file handling needs some more thought. --HG-- branch : HEAD

API change: Mailbox list sorting must now always done by storage itself if it's needed. Maildir listing rewritten. --HG-- branch : HEAD

API change for expunging messages. Not exactly what I wanted, but good enough. --HG-- branch : HEAD

API change for copying messages. --HG-- branch : HEAD

Rewritten maildir syncing. Uses dovecot-uidlist file to store UIDs permanently. --HG-- branch : HEAD

Include microseconds in maildir filename base. --HG-- branch : HEAD

Rewrote LIST, LSUB and subscription file handling. LIST replies aren't sorted anymore by default, it can be enabled with client_workarounds = list-sort. --HG-- branch : HEAD

Support for MULTIAPPEND extension. COPY now behaves like spec says - if it fails, none of the messages are copied. maildir_copy_with_hardlinks didn't actually work. --HG-- branch : HEAD

mail-storage.h interface changes, affects pretty much everything. FETCH, SEARCH, SORT and THREAD handling were pretty much moved from lib-storage/ to imap/ so adding non-index storages would be much easier now. Also POP3 server can now be easily implemented with lib-storage. Not too well tested, and at least one major problem: partial fetching is _slow_. --HG-- branch : HEAD

Naming change for function typedefs. --HG-- branch : HEAD

Naming style changes, finally got tired of most of the typedefs. Also the previous enum -> macro change reverted so that we don't use the highest bit anymore, that's incompatible with old indexes so they will be rebuilt. --HG-- branch : HEAD

Renamed IBuffer and OBuffer to IStream and OStream which describes their functionality better. I tried to keep the variable names and comments also sensible. --HG-- branch : HEAD

Fixes to timezone handling which were handling quite badly. added ioloop_timezone which gets updated with ioloop_time. Changed some GMT references to UTC. Timezone offsets are in minutes now everywhere instead of seconds. Fixes for unsigned time_t. --HG-- branch : HEAD

Hardlink COPYing still copied the file name instead of generating a new one, which really broke things when mail was duplicated in the same mailbox. --HG-- branch : HEAD

MailStorage interface change. We now let storage call several sync-functions whenever it wants instead of forcing it through sync() function. This now allows us to send client FLAGS and PERMANENTFLAGS messages whenever new custom flags are added. Also EXPUNGE command now goes nicely through expunge() function (which CLOSE used) instead of through flag in sync(). "Out of disk space" alert is also sent now whenever it happens, before it was sent only when opening mailbox. Message set parser now complains when referencing mail sequences outside the possible range. Modify log before sent flags/expunges changes to client even if it didn't know about those messages yet. Also some other smaller cleanups. Beginnings of fixing APPEND and COPY when trying to store mails into selected mailbox. Before this didn't work too well or simply broke things. Still broken, needs a bit more changes.. --HG-- branch : HEAD

Split IOBuffer into mmaped IBuffer, file IBuffer, memory data IBuffer and file OBuffer. --HG-- branch : HEAD

move size_t fixes --HG-- branch : HEAD

s/user_data/context/ and some s/Data/Context/ --HG-- branch : HEAD

Initial revision --HG-- branch : HEAD