lib-storage: Add mailbox_list_[un]lock()

lib-storage: Allow LISTINDEX to point to a different directory Most importantly because the mailbox list index is only a cache, it could be safely pointed to e.g. tmpfs to save disk I/O.

lib-storage: Add mailbox_list_last_error_push/pop() These are similar to mail_storage_last_error_push/pop()

lib-storage: Add mailbox_permissions_copy()

lib-storage: Make mailbox_list_iter_init_autocreate() static again It's no longer needed elsewhere.

lib-storage: Fix mailbox_list_index=yes + ACLs + auto=subscribe mailboxes If the auto=subscribe mailbox didn't have a lookup ACL, it shouldn't have been listed. This didn't work properly, because ACL plugin initialized the autocreate_ctx only for the index iterator, while the autoboxes were listed using the backend iterator. Fixed by not creating index iterator at all when doing a passthrough iteration.

lib-storage: Implement mailbox_list_notify_flush() This flushes any pending notifications. For now it only checks if there's a pending notification waiting for a timeout. It could also stat() the files to make sure, but that's probably unnecessary.

lib-storage: Rename autocreate_iter_existing to mailbox_list_iter_autocreate_filter It makes next commit possible

lib-storage: Add and use default mailbox iterator The idea is to allow mail plugins to see also the non-existent autoboxes by feeding them thru the iterator. Fixes problem where autocreated boxes are not seen by ACL plugin.

lib-storage: Expose mailbox_list_init_autocreate It is needed when backend_ctx is created on the fly

lib-storage: Add mailbox_list_get_last_internal_error() This returns the error given to mailbox_list_set_critical().

lib-storage: Make dovecot.list.index's filename configurable. This is useful when there are multiple namespaces pointing to the same mail root directory. For example mdbox with lazy-expunge: namespace { prefix = Expunged/ location = mdbox:~/mdbox:MAILBOXDIR=expunged:LISTINDEX=expunged.list.index .. }

lib-storage: Deduplicate code into mailbox_list_settings_init_defaults()

lib-storage: Limit folder full name only Before we had limit of 16 levels and 255 bytes per name which is same as 4096 bytes. Now we limit only the total length of the name to MAILBOX_LIST_NAME_MAX_LENGTH. For compability reasons, we are restricting individual component names to 255 characters.

global: unsigned int:1 -> bool:1 perl -i -pe 's/unsigned int ([^,:;]+):1;/bool $1:1;/' **/*.[ch]

lib-storage: Added mailbox_list_[un]escape_name_params() This allows calling the mailbox_list_[un]escape_name() with wanted parameters instead of taking them from mailbox_list.

lib-storage: mailbox_list_[un]escape_name() can now be called globally

lib-storage: Added mailbox_list_fs_get_list() Using mailbox_list_init_fs() now sets the mailbox_list pointer to root fs, which allows the fs backends to lookup the list. Although this can't be done in the init() function since the list is set only afterwards.

lib-storage: Further changes to mailbox_alloc_guid() caching. We'll now cache the GUIDs forever and force invalidation after mailbox rename or when mailbox GUIDs have been updated. This is what dsync needs, especially for good performance, and currently there aren't any other users for this API.

lib-storage: mailbox_alloc_guid() shouldn't refresh GUID cache on every nonexistent GUID. Refresh it only if we know it has had some changes (made by our process) or timestamp has changed. This should handle the cases that are currently important (=single dsync run).

lib-storage: Added mailbox_list_init_fs() to easily initialize fs. Use it for dbox's attachment_fs.

"Mailbox doesn't exist" error mixed up using storage and virtual mailbox names. This could have allowed a user to figure out existence of a mailbox without having lookup ACL.

lib-storage: Fixes to LAYOUT=index backend.

lib-storage: Added optional mailbox_list.init(), which can check for failure. "index" backend now gives a nice error message instead of crashing when mailbox_list_index=no

lib-storage: MAILBOX_LIST_ITER_SKIP_ALIASES now skips also "alias symlinks" An "alias symlink" is a symlink that points to the same directory. These can safely be skipped when iterating through all mails in all mailboxes (unlike other symlinks that may point to external storages).

lib-storage: mailbox_get_path() and friends can now return an error. An upcoming "index" mailbox_list backend uses mailbox GUIDs as the directory names. Doing a name => GUID lookup may fail, so the path lookup may fail. This is a somewhat annoying change and perhaps it can be removed in future, but for now it looks like it can't be avoided.

lib-storage: Cleanups to mailbox mkdir() related functions.

lib-storage: mailbox_list_get_root_permissions() now returns struct mailbox_permissions.

lib-storage: Removed mailbox_list.create_mailbox_dir() virtual function. It really didn't need to be virtual. Some storages didn't care about it, others wanted it to work basically the same.

lib-storage: Removed unused MAILBOX_DIR_CREATE_TYPE_ONLY_NOSELECT

lib-storage: Removed mailbox list backend-specific mailbox name/pattern validation. The virtual name validity is independent of the backend. The physical name is always used for paths in filesystem, so it can also be verified with common code. Backend's mailbox_dir and storage's internal directories can also be checked. There's nothing else left really.

Removed all the "enum foo;" declarations. They didn't seem to be very standard. Old gcc versions warn about them and C++ code doesn't like them.

lib-storage: Allow #including mailbox-list-private.h from C++ code

lib-storage: Removed rename_children parameter from mailbox_rename() The children are now always renamed. The only reason this feature was added in the first place was to make dsync's job easier, but this is no longer needed.

NOTIFY: Implemented SubscriptionChange

Array API changed: ARRAY_DEFINE(name, type) -> ARRAY(type) name Easy way to update your existing code: perl -i -pe 's:ARRAY_DEFINE\(([^,]+), *([^)]+)\);:ARRAY($2) $1;:' **/*.[ch]

Hash table API is now (mostly) type safe.

lib-storage: Added mailbox-list-notify API for tracking changes in all mailboxes. Requires mailbox_list_index=yes to work.

Marked functions parameters that are allowed to be NULL. Some APIs were also changed. The non-obvious APIs where NULL parameter was changed to "" are master_service_init() and auth_master_user_list_init(). These checks can currently be enabled only on a patched clang: http://llvm.org/bugs/show_bug.cgi?id=6786

shared mailboxes: Per-user flags can now be stored in private index files. This can be enabled by adding e.g.: mail_location = mdbox:/var/shared/mdbox:INDEXPVT=~/mdbox/shared

lib-storage: mailbox_list_get_storage_name/_get_vname() are now public.

Implemented IMAP SPECIAL-USE extension.

Moved autocreate plugin functionality into lib-storage. The autocreate plugin is still used for backwards compatibility. Mailboxes can be configured like: mailbox Sent { auto = subscribe } mailbox Spam { auto = create }

lib-storage: Added pool to struct mailbox_list_iterate_context.

lib-storage: Removed unused struct stat parameter from mailbox_list.get_mailbox_flags()

Moved GUID code to liblib. Use guid_128_t type consistently everywhere.

lib-storage: Added mailbox_alloc_guid() for opening mailbox by its GUID.

lib-storage: Made mailbox_list_mkdir_root() public

lib-storage: mkdir APIs changed.

lib-storage: mailbox_delete() for a symlink now removes the symlink instead of expunging mails.

lib-storage: Changed mailbox_list.subscriptions_refresh() API to be more flexible. The new API makes it easier implement subscriptions=no to a backend that doesn't use the regular subscriptions file.

lib-storage: Cleaned up subscription listing internally. The subscription listing code is now mostly separated from the mailbox listing code.

lib-storage: Made struct mailbox_list_iterate_context usable for plugins.

lib-storage: mailbox_alloc() now takes a virtual mailbox name and other related API changes. All storage_name <-> vname conversions now go through the same two mailbox_list methods. This has many benefits, such as: * listescape plugin is now much simpler and bugfree * allows changing lib-storage API to use UTF-8 mailbox names in future * allows creation of "mailbox aliases" plugin

lib-storage: Replaced mailbox_list_get_name_status() with mailbox_exists().

lib-storage: Added mailbox_list_create_dir()

lib-storage: Make sure index directory is created early enough for mailbox.log For example if subscriptions were added before any mailbox was opened, the index dir may not have been created yet. --HG-- branch : HEAD

lib-storage: Plugin API changed to run plugin functions in correct order. Previously the hooks were run in correct order, but the functions they overrode were run in reverse order. This caused problems when multiple plugins were used. --HG-- branch : HEAD

When creating shared directories, preserve parent dir's permissions if it has setgid bit enabled. This works only if location path uses %variables. The directories up to last variable are created by preserving parent directory's modes, while the rest of the directories are created with 0700. For example with "/var/mail/%d/%2n/%n/Maildir", "/var/mail/domain/nn" preserves /var/mail's permissions, while the "username/Maildir" directories have 0700 mode. --HG-- branch : HEAD

lib-storage: Added mailbox_list_get_root_path() to reduce code duplication. --HG-- branch : HEAD

lib-storage: Code cleanup. --HG-- branch : HEAD

lib-storage: Changed mailbox_list.iter_is_mailbox() API. --HG-- branch : HEAD

lib-storage: Mailbox renaming API changed. --HG-- branch : HEAD

lib-storage: Mailbox deletion API changed. Mailbox deletion should now be free of race conditions. The actual file deletion code is now responsibility of mailbox_list backend. --HG-- branch : HEAD

lib-storage: Moved mailbox directory creation code to mailbox_list backend. --HG-- branch : HEAD

lib-storage: Added mailbox_list_delete_dir() to (try to) delete a \noselect mailbox. --HG-- branch : HEAD

lib-storage: Removed mailbox_list_get_guid(). Directory GUIDs are pretty pointless. --HG-- branch : HEAD

dsync: Write mailbox changelog records using original timestamps. --HG-- branch : HEAD

lib-storage: Plugin hook API changed to support per-user plugins within a process. --HG-- branch : HEAD

Added mailbox_list_set_changelog_writable(). --HG-- branch : HEAD

mbox: Added support for mailbox directory GUIDs. --HG-- branch : HEAD

lib-storage: Fill mailbox log with mailbox delete/rename/(un)subscribes. --HG-- branch : HEAD

Whenever file's group changing fails, show the group origin in the error message. --HG-- branch : HEAD

Mailbox opening and closing APIs changed. --HG-- branch : HEAD

Initial commit for making mail_storage:mailbox_list to be from 1:1 to n:n. This will allow namespaces to share the same storage and a single namespace to have multiple storages (multiple mailbox formats). Neither works currently, and this commit probably breaks some things. --HG-- branch : HEAD

Implemented initial support for renaming mailboxes across namespaces. --HG-- branch : HEAD

Shared mailboxes: Fix to previous mailbox_open() change. --HG-- branch : HEAD

Shared mailboxes: Find the final storage before calling mail_storage.mailbox_open(), not inside it. This fixes some crashes with plugins. --HG-- branch : HEAD

mailboxes: Copy root directory's +x modes for newly created dirs perfectly. --HG-- branch : HEAD

Initial commit for config rewrite. --HG-- branch : HEAD

Fixed dbox to work with Maildir++ layout. --HG-- branch : HEAD

Created mail_user_[try_]home_expand(). Used them for expanding mail directories. --HG-- branch : HEAD

Added mailbox_name parameter to mailbox_list.iter_is_mailbox(). --HG-- branch : HEAD

mailbox_list_iter_update() was always setting subscribed flags to processed nodes. Changed the API so the flags can be specified. --HG-- branch : HEAD

dbox: Fixes to creating, deleting and renaming mailboxes when using alt directories. --HG-- branch : HEAD

Memory leak fixes --HG-- branch : HEAD

Memory leak fixes --HG-- branch : HEAD

Added mailbox_list_get_global_temp_prefix() --HG-- branch : HEAD

Simplified mailbox_list_get_permissions() a bit. Now it returns only permissions for the root directory. Also UID isn't returned anymore. --HG-- branch : HEAD

Support for looking up dbox files from an alternative directory if they're not found from the primary directory. --HG-- branch : HEAD

Fixes to handling namespace prefixes and real!=virtual separator. Added a new MAILBOX_LIST_ITER_VIRTUAL_NAMES which specifies if mailbox listing should be using virtual or real mailbox names. --HG-- branch : HEAD

Moved mail_location data parsing to a common function. It supports now also specifying SUBSCRIPTIONS and DIRNAME. --HG-- branch : HEAD

Renamed __attr_*__ to ATTR_*. Renamed __attrs_used__ to ATTRS_DEFINED. --HG-- branch : HEAD

Changed .h ifdef/defines to use <NAME>_H format. --HG-- branch : HEAD

Added support for multiple mailbox list patterns. --HG-- branch : HEAD

s/mask/pattern/ for mailbox listing. --HG-- branch : HEAD

mailbox_list_iter_next() returns now const pointer. --HG-- branch : HEAD

Error handling rewrite. --HG-- branch : HEAD

Removed HAS_INBOX and SHARED_NAMESPACE flags from mail storage and mailbox list flags. They can be directly taken from struct mail_namespace. --HG-- branch : HEAD

Added mailbox_list_get_permissions() and MAILBOX_LIST_ITER_RAW_LIST. --HG-- branch : HEAD

Added mailbox_list to struct mail_namespace. Added a few functions to get private data out of mailbox_list. --HG-- branch : HEAD

Better type safety to module_contexts arrays. Already fixed some bugs. --HG-- branch : HEAD

Moved delete/rename operations to mailbox_list API. Fixed mbox/maildir to work with either fs/maildir++ directory layout. They can be changed by appending :LAYOUT=fs|maildir++ to mail_location. --HG-- branch : HEAD

Mailbox list indexing and related changes. Currently works only with maildir and mmap_disable=no. This allows doing STATUS to synced mailboxes without opening their index files at all. --HG-- branch : HEAD

Initial code for separation of mailbox accessing and directory layout handling. It's not yet possible to change the default layouts though. --HG-- branch : HEAD