director: Show number of incoming USERs/sec in ps title

director: Don't send USERs in handshake that were already sent between handshake If the user was refreshed since the handshake was started, it means that the same user was already sent to the other side (added to the stream immediately after it was received/handled). There's no need to send it again. This fixes a potentally infinite handshake when users are rapidly changing and the handshake iterator never sees the end of the list. (Each refreshed user is moved to the end of the list, so handshaking can keep sending the same user over and over again.)

director: Avoid USER loops when ring latency is over 1 second Do this by adding a timestamp parameter to USER events. This way if it takes over 1 second for the USER event to traverse the ring, it won't get into an infinite loop getting the user updated over and over again.

director: Replace USER command during handshake with "U" This clearly differentiates the two commands and allows extending the USER command with new parameters without mixing it up with the handshake-USER.

director: Add kick_callback to director This callback gets called whenever director has performed a kick.

director: Show in process title how many users are being kicked.

director: Show in process title how many requests are being delayed.

director: Track show long the last ring sync took.

director: Fix flush to kick the user also when all backends are down The user's host can't change, because there are no other hosts. So add a new parameter to force the kick anyway.

director: Fix crash when flush is run and all backends are down. Instead of moving the users elsewhere, just kill them and flush the backend.

director: Log info line for every incoming/outgoing connection This can help with debugging problems.

director: Log info line whenever a director is added/removed from ring This can help with debugging problems.

director: Add old host's up/down and vhost count parameters to director_flush_socket The "down" status allows the script to determine whether it should try to perform any work on the old host. The vhost count may be unnecessary, but might be useful for some purpose.

director: Keep per-tag directory

director: Code cleanup - moved username_hash generation to director.h It's not really user_directory specific. This is especially important in the following patches that add per-tag user_directories. It's also not always known which tag the username_hash refers to, so there can't be different tag-specific username_hash generators.

director: Code cleanup - added user director_iterate_users_*() wrappers. This will shrink the diff for the following changes.

director: Send user's old and new backend IPs to flush script. If user was created due to the user-move, don't execute the flush script since it wouldn't really have any work to do anyway.

director: Code cleanup - global director is no longer needed.

director: Moved all user killing state to struct director_kill_context This should make it a bit easier to understand the life time of user killing. It also simplifies code by removing struct director_user_kill_finish_ctx. Finally, this already reduces memory usage with 32bit systems, and would make it possible to reduce also on 64bit systems if timestamp is shrank to 31 bits and weak bit moved after it. I'm not sure if that would be better for performance though. In any case it would provide free space for 4 extra bytes if that were needed in future.

director: Code cleanup - move enum user_kill_state to director.h This will be needed by following commits

director: Show in process title how many users are being moved.

director: Call state_change_callback on user move timeout.

doveadm director kick: Added -f <passdb field> parameter. This works for all the user_* passdb fields.

director: Throttle user move/kill failure errors after 100/sec. If it happened to a lot of users, they can take a lot of space in logs.

global: unsigned int:1 -> bool:1 perl -i -pe 's/unsigned int ([^,:;]+):1;/bool $1:1;/' **/*.[ch]

director: Don't re-send SYNC unnecessarily often.

director: Fixed backend selection when multiple tags were used. The previous algorithm was causing an uneven load for backends. This change breaks compatibility with older director servers that were using tags because of the different selection algorithm. The new director code refuses to run within a cluster with old directors if tags are used.

director: Detect if directors' hosts have become desynced by sending hosts_hash in SYNC parameter. Also fix up such a situation by resending all HOSTs.

Changed type of internet port values to in_port_t everywhere. Created special SET_IN_PORT setting type for internet port values. Created net_str2port() for parsing internet port values. Removed several atoi() invocations in the process.

director: Added "up" vs "down" states and doveadm director up/down commands. These commands are intended to be used by automated watchdogs that detect if backends are up or down. This way the vhost count doesn't get forgotten after server goes down. It also means that admin can manually take down a server by setting its vhost count to 0 without the watchdog automatically bringing it back up.

director: Include useful statistics in process title.

director: Added support for backend cluster "tags". This allows using a single director ring for multiple backend clusters. By default everything has an empty tag. A passdb lookup can return "director_tag" field containing the wanted tag name. If there aren't any backend servers with the wanted tag, it's treated the same as if there aren't any backend servers available (= wait for 30 secs for a backend and then return temporary failure). Tags can be added to configuration by adding @tag suffix to IPs/hosts. For example: director_mail_servers = 10.0.0.100-10.0.0.110@name1 10.0.0.120@name2 "doveadm director add" can also add tags either with @tag suffix or with -t parameter. "doveadm director status user@domain" requires giving the user's correct tag with -t parameter or the results won't be correct (empty tag's results are shown). Tags can't currently be changed for an existing host without removing it first.

director: Added director_consistent_hashing setting. This should have been the default since the beginning. I didn't thik it was worth the trouble originally because in theory it shouldn't matter much if users jump between multiple backends. But this makes caching behavior worse, especially for systems which are using local cache files.

director: If we detect that a user is being proxied to multiple backends, disconnect wrong connections. Especially IMAP connections can otherwise stay alive for a long time and cause problems.

doveadm director kick command added. The kick gets sent to all the proxies within the director ring.

director: Fixed handling weak users when there were exactly 2 director servers in the ring.

director: Avoid infinite SYNC loops if the originating director goes away for a long time.

director: -D parameter now enables extensive debug logging.

director: Don't handle pending requests from all around the code. I'm not sure if this actually fixes any bugs, but it definitely makes the state cleaner.

Renamed network.[ch] to net.[ch]. The function prefixes already started with net_ instead of network_. And icecap wants to use network.h for other purpose. :)

director: Improved logging related to disconnections.

Array API changed: ARRAY_DEFINE(name, type) -> ARRAY(type) name Easy way to update your existing code: perl -i -pe 's:ARRAY_DEFINE\(([^,]+), *([^)]+)\);:ARRAY($2) $1;:' **/*.[ch]

Marked functions parameters that are allowed to be NULL. Some APIs were also changed. The non-obvious APIs where NULL parameter was changed to "" are master_service_init() and auth_master_user_list_init(). These checks can currently be enabled only on a patched clang: http://llvm.org/bugs/show_bug.cgi?id=6786

director: Implemented ability to remove directors from a running ring. Also added doveadm command for adding a new director to a running ring.

director: Try harder to connect to ring before thinking we're alone.

director: Don't communicate with directors that recently sent invalid input. Track network and protocol failures separately. If a director had sent invalid protocol data within last 60 seconds, don't try to connect to it and don't allow it to connect to us.

director: Redesigned connection handling and error handling. Director now accepts all connections from everywhere and syncs them until the handshaking is finished. At that point it finally decides if this is a connection that should be used as our left/right connection, or if it should be disconnected. This should make connecting more reliable, especially if one of the directors sends broken handshake or has other trouble.

director: List of director connections belongs to struct director.

director: Avoid user getting redirected to different servers near its expiration. Fixes a problem when user is logging in at the same time on director1 which thinks the user is expired, and on director2 which thinks the user expires only in 1 second.

director: Keep track of the highest supported protocol version in the ring.

director: Added timeout to syncing to make sure we don't hang if it somehow gets lost.

director: Find director's own IP earlier in init.

director: If request is timed out, log an error.

director: Added support for moving user to another server with "doveadm director move".

director: Lots of fixes. It should be pretty stable now. --HG-- branch : HEAD

director: A lot of fixes. --HG-- branch : HEAD

director: Added initial testing framework and some debugging output. --HG-- branch : HEAD

Added doveadm director flush command for dropping user associations from memory. This should probably mainly be used for testing. --HG-- branch : HEAD

doveadm director status user: Show more ways of what user's potential hosts are. --HG-- branch : HEAD

director: Code cleanup - keep mail hosts in a struct rather than in static variables. --HG-- branch : HEAD

director: If we logged that we're delaying connections, also log when we continue. --HG-- branch : HEAD

Added initial implementation of a director process (for NFS users). There are still some unimplemented features and bugs. Also changing mail server list doesn't yet make sure that other directors won't assign the same user to a different server at the same time. --HG-- branch : HEAD