maildir-sync.c revision 757726d9acbd04cf0d0d4be8ce14e11525476a0b
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen/* Copyright (C) 2004 Timo Sirainen */
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen/*
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen Here's a description of how we handle Maildir synchronization and
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen it's problems:
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen We want to be as efficient as we can. The most efficient way to
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen check if changes have occured is to stat() the new/ and cur/
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen directories and uidlist file - if their mtimes haven't changed,
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen there's no changes and we don't need to do anything.
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen Problem 1: Multiple changes can happen within a single second -
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen nothing guarantees that once we synced it, someone else didn't just
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen then make a modification. Such modifications wouldn't get noticed
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen until a new modification occured later.
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen Problem 2: Syncing cur/ directory is much more costly than syncing
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen new/. Moving mails from new/ to cur/ will always change mtime of
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen cur/ causing us to sync it as well.
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen Problem 3: We may not be able to move mail from new/ to cur/
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen because we're out of quota, or simply because we're accessing a
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen read-only mailbox.
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen MAILDIR_SYNC_SECS
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen -----------------
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen Several checks below use MAILDIR_SYNC_SECS, which should be maximum
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen clock drift between all computers accessing the maildir (eg. via
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen NFS), rounded up to next second. Our default is 1 second, since
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen everyone should be using NTP.
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen Note that setting it to 0 works only if there's only one computer
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen accessing the maildir. It's practically impossible to make two
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen clocks _exactly_ synchronized.
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen It might be possible to only use file server's clock by looking at
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen the atime field, but I don't know how well that would actually work.
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen cur directory
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen -------------
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen We have dirty_cur_time variable which is set to cur/ directory's
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen mtime when it's >= time() - MAILDIR_SYNC_SECS and we _think_ we have
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen synchronized the directory.
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen When dirty_cur_time is non-zero, we don't synchronize the cur/
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen directory until
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen a) cur/'s mtime changes
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen b) opening a mail fails with ENOENT
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen c) time() > dirty_cur_time + MAILDIR_SYNC_SECS
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen This allows us to modify the maildir multiple times without having
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen to sync it at every change. The sync will eventually be done to
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen make sure we didn't miss any external changes.
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen The dirty_cur_time is set when:
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen - we change message flags
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen - we expunge messages
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen - we move mail from new/ to cur/
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen - we sync cur/ directory and it's mtime is >= time() - MAILDIR_SYNC_SECS
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen It's unset when we do the final syncing, ie. when mtime is
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen older than time() - MAILDIR_SYNC_SECS.
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen new directory
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen -------------
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen If new/'s mtime is >= time() - MAILDIR_SYNC_SECS, always synchronize
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen it. dirty_cur_time-like feature might save us a few syncs, but
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen that might break a client which saves a mail in one connection and
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen tries to fetch it in another one. new/ directory is almost always
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen empty, so syncing it should be very fast anyway. Actually this can
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen still happen if we sync only new/ dir while another client is also
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen moving mails from it to cur/ - it takes us a while to see them.
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen That's pretty unlikely to happen however, and only way to fix it
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen would be to always synchronize cur/ after new/.
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen Normally we move all mails from new/ to cur/ whenever we sync it. If
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen it's not possible for some reason, we mark the mail with "probably
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen exists in new/ directory" flag.
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen If rename() still fails because of ENOSPC or EDQUOT, we still save
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen the flag changes in index with dirty-flag on. When moving the mail
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen to cur/ directory, or when we notice it's already moved there, we
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen apply the flag changes to the filename, rename it and remove the
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen dirty flag. If there's dirty flags, this should be tried every time
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen after expunge or when closing the mailbox.
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen uidlist
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen -------
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen This file contains UID <-> filename mappings. It's updated only when
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen new mail arrives, so it may contain filenames that have already been
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen deleted. Updating is done by getting uidlist.lock file, writing the
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen whole uidlist into it and rename()ing it over the old uidlist. This
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen means there's no need to lock the file for reading.
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen Whenever uidlist is rewritten, it's mtime must be larger than the old
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen one's. Use utime() before rename() if needed. Note that inode checking
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen wouldn't have been sufficient as inode numbers can be reused.
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen This file is usually read the first time you need to know filename for
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen given UID. After that it's not re-read unless new mails come that we
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen don't know about.
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen broken clients
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen --------------
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen Originally the middle identifier in Maildir filename was specified
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen only as <process id>_<delivery counter>. That however created a
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen problem with randomized PIDs which made it possible that the same
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen PID was reused within one second.
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen So if within one second a mail was delivered, MUA moved it to cur/
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen and another mail was delivered by a new process using same PID as
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen the first one, we likely ended up overwriting the first mail when
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen the second mail was moved over it.
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen Nowadays everyone should be giving a bit more specific identifier,
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen for example include microseconds in it which Dovecot does.
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen There's a simple way to prevent this from happening in some cases:
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen Don't move the mail from new/ to cur/ if it's mtime is >= time() -
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen MAILDIR_SYNC_SECS. The second delivery's link() call then fails
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen because the file is already in new/, and it will then use a
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen different filename. There's a few problems with this however:
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen - it requires extra stat() call which is unneeded extra I/O
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen - another MUA might still move the mail to cur/
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen - if first file's flags are modified by either Dovecot or another
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen MUA, it's moved to cur/ (you _could_ just do the dirty-flagging
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen but that'd be ugly)
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen Because this is useful only for very few people and it requires
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen extra I/O, I decided not to implement this. It should be however
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen quite easy to do since we need to be able to deal with files in new/
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen in any case.
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen It's also possible to never accidentally overwrite a mail by using
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen link() + unlink() rather than rename(). This however isn't very
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen good idea as it introduces potential race conditions when multiple
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen clients are accessing the mailbox:
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen Trying to move the same mail from new/ to cur/ at the same time:
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen a) Client 1 uses slightly different filename than client 2,
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen for example one sets read-flag on but the other doesn't.
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen You have the same mail duplicated now.
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen b) Client 3 sees the mail between Client 1's and 2's link() calls
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen and changes it's flag. You have the same mail duplicated now.
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen And it gets worse when they're unlink()ing in cur/ directory:
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen c) Client 1 changes mails's flag and client 2 changes it back
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen between 1's link() and unlink(). The mail is now expunged.
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen d) If you try to deal with the duplicates by unlink()ing another
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen one of them, you might end up unlinking both of them.
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen So, what should we do then if we notice a duplicate? First of all,
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen it might not be a duplicate at all, readdir() might have just
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen returned it twice because it was just renamed. What we should do is
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen create a completely new base name for it and rename() it to that.
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen If the call fails with ENOENT, it only means that it wasn't a
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen duplicate after all.
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen*/
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen#include "lib.h"
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen#include "ioloop.h"
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen#include "buffer.h"
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen#include "hash.h"
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen#include "str.h"
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen#include "maildir-storage.h"
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen#include "maildir-uidlist.h"
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen#include <stdio.h>
e06c0b65c16ccce69bbee009ead14d7d3d17a256Timo Sirainen#include <stddef.h>
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen#include <unistd.h>
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen#include <dirent.h>
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen#include <sys/stat.h>
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen#define MAILDIR_SYNC_SECS 1
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen#define MAILDIR_FILENAME_FLAG_FOUND 128
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainenstruct maildir_sync_context {
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen struct index_mailbox *ibox;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen const char *new_dir, *cur_dir;
b7b81543899e306c71e6152516d8698416162bcbTimo Sirainen int partial;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen struct maildir_uidlist_sync_ctx *uidlist_sync_ctx;
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen};
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainenstruct maildir_index_sync_context {
fd3d711f219fd6813492acbe051e04327f0ca0f0Timo Sirainen struct index_mailbox *ibox;
fd3d711f219fd6813492acbe051e04327f0ca0f0Timo Sirainen struct mail_index_view *view;
b35f7104715edee0cfac6d46ab0b342033867eb7Timo Sirainen struct mail_index_sync_ctx *sync_ctx;
fd3d711f219fd6813492acbe051e04327f0ca0f0Timo Sirainen struct mail_index_transaction *trans;
b35f7104715edee0cfac6d46ab0b342033867eb7Timo Sirainen
fd3d711f219fd6813492acbe051e04327f0ca0f0Timo Sirainen struct mail_index_sync_rec sync_rec;
fd3d711f219fd6813492acbe051e04327f0ca0f0Timo Sirainen uint32_t seq;
fd3d711f219fd6813492acbe051e04327f0ca0f0Timo Sirainen int dirty_state;
b92813e2f96d4b28f989528ed5dd6115da7d9bdbTimo Sirainen};
fd3d711f219fd6813492acbe051e04327f0ca0f0Timo Sirainen
fd3d711f219fd6813492acbe051e04327f0ca0f0Timo Sirainenstatic int maildir_expunge(struct index_mailbox *ibox, const char *path,
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen void *context __attr_unused__)
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen{
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen if (unlink(path) == 0) {
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen ibox->dirty_cur_time = ioloop_time;
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen return 1;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen }
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen if (errno == ENOENT)
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen return 0;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen mail_storage_set_critical(ibox->box.storage,
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen "unlink(%s) failed: %m", path);
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen return -1;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen}
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainenstatic int maildir_sync_flags(struct index_mailbox *ibox, const char *path,
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen void *context)
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen{
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen struct maildir_index_sync_context *ctx = context;
fd3d711f219fd6813492acbe051e04327f0ca0f0Timo Sirainen const char *newpath;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen enum mail_flags flags;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen uint8_t flags8;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen keywords_mask_t keywords;
d8b77aef97e89f1ccc5cbdaef77be9052279e35fTimo Sirainen
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen ctx->dirty_state = 0;
b92813e2f96d4b28f989528ed5dd6115da7d9bdbTimo Sirainen
b35f7104715edee0cfac6d46ab0b342033867eb7Timo Sirainen (void)maildir_filename_get_flags(path, &flags, keywords);
d8b77aef97e89f1ccc5cbdaef77be9052279e35fTimo Sirainen
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen flags8 = flags;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen mail_index_sync_flags_apply(&ctx->sync_rec, &flags8, keywords);
d8b77aef97e89f1ccc5cbdaef77be9052279e35fTimo Sirainen
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen newpath = maildir_filename_set_flags(path, flags8, keywords);
d8b77aef97e89f1ccc5cbdaef77be9052279e35fTimo Sirainen if (rename(path, newpath) == 0) {
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen if ((flags8 & MAIL_INDEX_MAIL_FLAG_DIRTY) != 0)
b92813e2f96d4b28f989528ed5dd6115da7d9bdbTimo Sirainen ctx->dirty_state = -1;
b92813e2f96d4b28f989528ed5dd6115da7d9bdbTimo Sirainen ibox->dirty_cur_time = ioloop_time;
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen return 1;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen }
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen if (errno == ENOENT)
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen return 0;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen if (ENOSPACE(errno) || errno == EACCES) {
b35f7104715edee0cfac6d46ab0b342033867eb7Timo Sirainen memset(keywords, 0, sizeof(keywords));
b35f7104715edee0cfac6d46ab0b342033867eb7Timo Sirainen mail_index_update_flags(ctx->trans, ctx->seq, MODIFY_ADD,
b35f7104715edee0cfac6d46ab0b342033867eb7Timo Sirainen MAIL_INDEX_MAIL_FLAG_DIRTY, keywords);
b35f7104715edee0cfac6d46ab0b342033867eb7Timo Sirainen ctx->dirty_state = 1;
b92813e2f96d4b28f989528ed5dd6115da7d9bdbTimo Sirainen return 1;
fd3d711f219fd6813492acbe051e04327f0ca0f0Timo Sirainen }
fd3d711f219fd6813492acbe051e04327f0ca0f0Timo Sirainen
fd3d711f219fd6813492acbe051e04327f0ca0f0Timo Sirainen mail_storage_set_critical(ibox->box.storage,
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen "rename(%s, %s) failed: %m", path, newpath);
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen return -1;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen}
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainenstatic int maildir_sync_record(struct index_mailbox *ibox,
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen struct maildir_index_sync_context *ctx)
fd3d711f219fd6813492acbe051e04327f0ca0f0Timo Sirainen{
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen struct mail_index_sync_rec *sync_rec = &ctx->sync_rec;
fd3d711f219fd6813492acbe051e04327f0ca0f0Timo Sirainen struct mail_index_view *view = ctx->view;
fd3d711f219fd6813492acbe051e04327f0ca0f0Timo Sirainen uint32_t seq, seq1, seq2, uid;
659fe5d24825b160cae512538088020d97a60239Timo Sirainen
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen switch (sync_rec->type) {
fd3d711f219fd6813492acbe051e04327f0ca0f0Timo Sirainen case MAIL_INDEX_SYNC_TYPE_APPEND:
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen break;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen case MAIL_INDEX_SYNC_TYPE_EXPUNGE:
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen /* make it go through sequences to avoid looping through huge
659fe5d24825b160cae512538088020d97a60239Timo Sirainen holes in UID range */
659fe5d24825b160cae512538088020d97a60239Timo Sirainen if (mail_index_lookup_uid_range(view, sync_rec->uid1,
659fe5d24825b160cae512538088020d97a60239Timo Sirainen sync_rec->uid2,
659fe5d24825b160cae512538088020d97a60239Timo Sirainen &seq1, &seq2) < 0)
659fe5d24825b160cae512538088020d97a60239Timo Sirainen return -1;
659fe5d24825b160cae512538088020d97a60239Timo Sirainen
659fe5d24825b160cae512538088020d97a60239Timo Sirainen if (seq1 == 0)
659fe5d24825b160cae512538088020d97a60239Timo Sirainen break;
659fe5d24825b160cae512538088020d97a60239Timo Sirainen
659fe5d24825b160cae512538088020d97a60239Timo Sirainen for (seq = seq1; seq <= seq2; seq++) {
659fe5d24825b160cae512538088020d97a60239Timo Sirainen if (mail_index_lookup_uid(view, seq, &uid) < 0)
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen return -1;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen if (maildir_file_do(ibox, uid, maildir_expunge,
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen NULL) < 0)
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen return -1;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen }
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen break;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen case MAIL_INDEX_SYNC_TYPE_FLAGS:
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen if (mail_index_lookup_uid_range(view, sync_rec->uid1,
659fe5d24825b160cae512538088020d97a60239Timo Sirainen sync_rec->uid2,
659fe5d24825b160cae512538088020d97a60239Timo Sirainen &seq1, &seq2) < 0)
659fe5d24825b160cae512538088020d97a60239Timo Sirainen return -1;
659fe5d24825b160cae512538088020d97a60239Timo Sirainen
659fe5d24825b160cae512538088020d97a60239Timo Sirainen if (seq1 == 0)
659fe5d24825b160cae512538088020d97a60239Timo Sirainen break;
659fe5d24825b160cae512538088020d97a60239Timo Sirainen
659fe5d24825b160cae512538088020d97a60239Timo Sirainen for (ctx->seq = seq1; ctx->seq <= seq2; ctx->seq++) {
659fe5d24825b160cae512538088020d97a60239Timo Sirainen if (mail_index_lookup_uid(view, ctx->seq, &uid) < 0)
fd3d711f219fd6813492acbe051e04327f0ca0f0Timo Sirainen return -1;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen if (maildir_file_do(ibox, uid,
b35f7104715edee0cfac6d46ab0b342033867eb7Timo Sirainen maildir_sync_flags, ctx) < 0)
b35f7104715edee0cfac6d46ab0b342033867eb7Timo Sirainen return -1;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen if (ctx->dirty_state < 0) {
b92813e2f96d4b28f989528ed5dd6115da7d9bdbTimo Sirainen /* flag isn't dirty anymore */
b92813e2f96d4b28f989528ed5dd6115da7d9bdbTimo Sirainen keywords_mask_t keywords;
b92813e2f96d4b28f989528ed5dd6115da7d9bdbTimo Sirainen
b92813e2f96d4b28f989528ed5dd6115da7d9bdbTimo Sirainen memset(keywords, 0, sizeof(keywords));
b92813e2f96d4b28f989528ed5dd6115da7d9bdbTimo Sirainen mail_index_update_flags(ctx->trans, ctx->seq,
b92813e2f96d4b28f989528ed5dd6115da7d9bdbTimo Sirainen MODIFY_REMOVE,
b92813e2f96d4b28f989528ed5dd6115da7d9bdbTimo Sirainen MAIL_INDEX_MAIL_FLAG_DIRTY, keywords);
b92813e2f96d4b28f989528ed5dd6115da7d9bdbTimo Sirainen }
b35f7104715edee0cfac6d46ab0b342033867eb7Timo Sirainen }
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen break;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen }
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen return 0;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen}
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainenint maildir_sync_last_commit(struct index_mailbox *ibox)
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen{
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen struct maildir_index_sync_context ctx;
fd3d711f219fd6813492acbe051e04327f0ca0f0Timo Sirainen uint32_t seq;
b35f7104715edee0cfac6d46ab0b342033867eb7Timo Sirainen uoff_t offset;
b35f7104715edee0cfac6d46ab0b342033867eb7Timo Sirainen int ret;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen if (ibox->commit_log_file_seq == 0)
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen return 0;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen memset(&ctx, 0, sizeof(ctx));
fd3d711f219fd6813492acbe051e04327f0ca0f0Timo Sirainen ctx.ibox = ibox;
fd3d711f219fd6813492acbe051e04327f0ca0f0Timo Sirainen
fd3d711f219fd6813492acbe051e04327f0ca0f0Timo Sirainen ibox->syncing_commit = TRUE;
65d6ca3fb5450b81df0190d9e9aa62c00fed5116Timo Sirainen ret = mail_index_sync_begin(ibox->index, &ctx.sync_ctx, &ctx.view,
fd3d711f219fd6813492acbe051e04327f0ca0f0Timo Sirainen ibox->commit_log_file_seq,
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen ibox->commit_log_file_offset, FALSE, FALSE);
757726d9acbd04cf0d0d4be8ce14e11525476a0bTimo Sirainen if (ret > 0) {
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen ctx.trans = mail_index_transaction_begin(ctx.view, FALSE);
7c95b03620a03a43dd72d39608cea5fc77393ad6Timo Sirainen
b35f7104715edee0cfac6d46ab0b342033867eb7Timo Sirainen while ((ret = mail_index_sync_next(ctx.sync_ctx,
fd3d711f219fd6813492acbe051e04327f0ca0f0Timo Sirainen &ctx.sync_rec)) > 0) {
fd3d711f219fd6813492acbe051e04327f0ca0f0Timo Sirainen if (maildir_sync_record(ibox, &ctx) < 0) {
fd3d711f219fd6813492acbe051e04327f0ca0f0Timo Sirainen ret = -1;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen break;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen }
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen }
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen if (mail_index_transaction_commit(ctx.trans, &seq, &offset) < 0)
b35f7104715edee0cfac6d46ab0b342033867eb7Timo Sirainen ret = -1;
b35f7104715edee0cfac6d46ab0b342033867eb7Timo Sirainen if (mail_index_sync_commit(ctx.sync_ctx) < 0)
d9de52132072d80b8c268094b879c0ef5a108db3Timo Sirainen ret = -1;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen }
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen ibox->syncing_commit = FALSE;
65d6ca3fb5450b81df0190d9e9aa62c00fed5116Timo Sirainen
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen if (ret == 0) {
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen ibox->commit_log_file_seq = 0;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen ibox->commit_log_file_offset = 0;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen } else {
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen mail_storage_set_index_error(ibox);
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen }
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen return ret;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen}
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainenstatic struct maildir_sync_context *
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainenmaildir_sync_context_new(struct index_mailbox *ibox)
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen{
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen struct maildir_sync_context *ctx;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen ctx = t_new(struct maildir_sync_context, 1);
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen ctx->ibox = ibox;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen ctx->new_dir = t_strconcat(ibox->path, "/new", NULL);
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen ctx->cur_dir = t_strconcat(ibox->path, "/cur", NULL);
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen return ctx;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen}
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainenstatic void maildir_sync_deinit(struct maildir_sync_context *ctx)
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen{
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen if (ctx->uidlist_sync_ctx != NULL)
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen (void)maildir_uidlist_sync_deinit(ctx->uidlist_sync_ctx);
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen}
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainenstatic int maildir_fix_duplicate(struct index_mailbox *ibox, const char *dir,
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen const char *old_fname)
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen{
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen const char *new_fname, *old_path, *new_path;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen int ret = 0;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen t_push();
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen old_path = t_strconcat(dir, "/", old_fname, NULL);
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen new_fname = maildir_generate_tmp_filename(&ioloop_timeval);
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen new_path = t_strconcat(ibox->path, "/new/", new_fname, NULL);
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen if (rename(old_path, new_path) == 0) {
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen i_warning("Fixed duplicate in %s: %s -> %s",
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen ibox->path, old_fname, new_fname);
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen } else if (errno != ENOENT) {
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen mail_storage_set_critical(ibox->box.storage,
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen "rename(%s, %s) failed: %m", old_path, new_path);
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen ret = -1;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen }
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen t_pop();
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen return ret;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen}
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainenstatic int maildir_scan_dir(struct maildir_sync_context *ctx, int new_dir)
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen{
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen struct mail_storage *storage = ctx->ibox->box.storage;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen const char *dir;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen DIR *dirp;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen string_t *src, *dest;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen struct dirent *dp;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen enum maildir_uidlist_rec_flag flags;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen int move_new, ret = 1;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen
641f0c0900ee6e7cf9667f4b40ed95cec7d0cdcaTimo Sirainen dir = new_dir ? ctx->new_dir : ctx->cur_dir;
641f0c0900ee6e7cf9667f4b40ed95cec7d0cdcaTimo Sirainen dirp = opendir(dir);
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen if (dirp == NULL) {
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen mail_storage_set_critical(storage,
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen "opendir(%s) failed: %m", dir);
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen return -1;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen }
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen t_push();
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen src = t_str_new(1024);
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen dest = t_str_new(1024);
d77c309fccbc6a7594f8cb08fb01009fa613c568Timo Sirainen
d77c309fccbc6a7594f8cb08fb01009fa613c568Timo Sirainen move_new = new_dir && !mailbox_is_readonly(&ctx->ibox->box) &&
d77c309fccbc6a7594f8cb08fb01009fa613c568Timo Sirainen !ctx->ibox->keep_recent;
d77c309fccbc6a7594f8cb08fb01009fa613c568Timo Sirainen while ((dp = readdir(dirp)) != NULL) {
c282435b57b6f9696fc12d99ea70468b7bdfe24cTimo Sirainen if (dp->d_name[0] == '.')
c282435b57b6f9696fc12d99ea70468b7bdfe24cTimo Sirainen continue;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen
7bd6001d84ecc1792ddfd54fe8efa63c509d90b1Timo Sirainen ret = maildir_uidlist_sync_next_pre(ctx->uidlist_sync_ctx,
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen dp->d_name);
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen if (ret == 0) {
8fcff4c5b52f24d9c681805fdf06b486f1d0fcbeTimo Sirainen /* new file and we couldn't lock uidlist, check this
8fcff4c5b52f24d9c681805fdf06b486f1d0fcbeTimo Sirainen later in next sync. */
d67f54632110cfb6aafe2d7cd1f99b031c0b208aTimo Sirainen if (new_dir)
d482b35af87f5fd872bad007da0475813a401a49Timo Sirainen ctx->ibox->last_new_mtime = 0;
d482b35af87f5fd872bad007da0475813a401a49Timo Sirainen else
d67f54632110cfb6aafe2d7cd1f99b031c0b208aTimo Sirainen ctx->ibox->dirty_cur_time = ioloop_time;
d67f54632110cfb6aafe2d7cd1f99b031c0b208aTimo Sirainen continue;
d67f54632110cfb6aafe2d7cd1f99b031c0b208aTimo Sirainen }
d67f54632110cfb6aafe2d7cd1f99b031c0b208aTimo Sirainen if (ret < 0)
8fcff4c5b52f24d9c681805fdf06b486f1d0fcbeTimo Sirainen break;
d67f54632110cfb6aafe2d7cd1f99b031c0b208aTimo Sirainen
8fcff4c5b52f24d9c681805fdf06b486f1d0fcbeTimo Sirainen flags = 0;
8fcff4c5b52f24d9c681805fdf06b486f1d0fcbeTimo Sirainen if (move_new) {
8fcff4c5b52f24d9c681805fdf06b486f1d0fcbeTimo Sirainen str_truncate(src, 0);
641f0c0900ee6e7cf9667f4b40ed95cec7d0cdcaTimo Sirainen str_truncate(dest, 0);
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen str_printfa(src, "%s/%s", ctx->new_dir, dp->d_name);
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen str_printfa(dest, "%s/%s", ctx->cur_dir, dp->d_name);
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen if (strchr(dp->d_name, ':') == NULL)
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen str_append(dest, ":2,");
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen if (rename(str_c(src), str_c(dest)) == 0) {
ead79af955bccc360de08c150918474c1809748eTimo Sirainen /* we moved it - it's \Recent for us */
ead79af955bccc360de08c150918474c1809748eTimo Sirainen ctx->ibox->dirty_cur_time = ioloop_time;
641f0c0900ee6e7cf9667f4b40ed95cec7d0cdcaTimo Sirainen flags |= MAILDIR_UIDLIST_REC_FLAG_MOVED |
d67f54632110cfb6aafe2d7cd1f99b031c0b208aTimo Sirainen MAILDIR_UIDLIST_REC_FLAG_RECENT;
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen } else if (ENOTFOUND(errno)) {
641f0c0900ee6e7cf9667f4b40ed95cec7d0cdcaTimo Sirainen /* someone else moved it already */
641f0c0900ee6e7cf9667f4b40ed95cec7d0cdcaTimo Sirainen flags |= MAILDIR_UIDLIST_REC_FLAG_MOVED;
641f0c0900ee6e7cf9667f4b40ed95cec7d0cdcaTimo Sirainen } else if (ENOSPACE(errno)) {
641f0c0900ee6e7cf9667f4b40ed95cec7d0cdcaTimo Sirainen /* not enough disk space, leave here */
641f0c0900ee6e7cf9667f4b40ed95cec7d0cdcaTimo Sirainen flags |= MAILDIR_UIDLIST_REC_FLAG_NEW_DIR |
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen MAILDIR_UIDLIST_REC_FLAG_RECENT;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen move_new = FALSE;
641f0c0900ee6e7cf9667f4b40ed95cec7d0cdcaTimo Sirainen } else {
641f0c0900ee6e7cf9667f4b40ed95cec7d0cdcaTimo Sirainen flags |= MAILDIR_UIDLIST_REC_FLAG_NEW_DIR |
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen MAILDIR_UIDLIST_REC_FLAG_RECENT;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen mail_storage_set_critical(storage,
641f0c0900ee6e7cf9667f4b40ed95cec7d0cdcaTimo Sirainen "rename(%s, %s) failed: %m",
641f0c0900ee6e7cf9667f4b40ed95cec7d0cdcaTimo Sirainen str_c(src), str_c(dest));
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen }
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen } else if (new_dir) {
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen flags |= MAILDIR_UIDLIST_REC_FLAG_NEW_DIR |
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen MAILDIR_UIDLIST_REC_FLAG_RECENT;
641f0c0900ee6e7cf9667f4b40ed95cec7d0cdcaTimo Sirainen }
641f0c0900ee6e7cf9667f4b40ed95cec7d0cdcaTimo Sirainen
641f0c0900ee6e7cf9667f4b40ed95cec7d0cdcaTimo Sirainen ret = maildir_uidlist_sync_next(ctx->uidlist_sync_ctx,
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen dp->d_name, flags);
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen if (ret <= 0) {
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen if (ret < 0)
641f0c0900ee6e7cf9667f4b40ed95cec7d0cdcaTimo Sirainen break;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen /* possibly duplicate - try fixing it */
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen if (maildir_fix_duplicate(ctx->ibox,
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen dir, dp->d_name) < 0) {
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen ret = -1;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen break;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen }
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen }
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen }
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen if (closedir(dirp) < 0) {
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen mail_storage_set_critical(storage,
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen "closedir(%s) failed: %m", dir);
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen }
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen t_pop();
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen return ret < 0 ? -1 : 0;
d77c309fccbc6a7594f8cb08fb01009fa613c568Timo Sirainen}
d77c309fccbc6a7594f8cb08fb01009fa613c568Timo Sirainen
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainenstatic int maildir_sync_quick_check(struct maildir_sync_context *ctx,
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen int *new_changed_r, int *cur_changed_r)
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen{
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen struct index_mailbox *ibox = ctx->ibox;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen struct stat st;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen time_t new_mtime, cur_mtime;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen *new_changed_r = *cur_changed_r = FALSE;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen if (stat(ctx->new_dir, &st) < 0) {
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen mail_storage_set_critical(ibox->box.storage,
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen "stat(%s) failed: %m", ctx->new_dir);
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen return -1;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen }
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen new_mtime = st.st_mtime;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen if (stat(ctx->cur_dir, &st) < 0) {
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen mail_storage_set_critical(ibox->box.storage,
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen "stat(%s) failed: %m", ctx->cur_dir);
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen return -1;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen }
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen cur_mtime = st.st_mtime;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen if (ibox->dirty_cur_time == 0) {
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen /* cur stamp is kept in index, we don't have to sync if
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen someone else has done it and updated the index. make sure
bde6382cf65fba6165dc3603f5419e194d87f404Timo Sirainen we have a fresh index with latest sync_stamp. */
bde6382cf65fba6165dc3603f5419e194d87f404Timo Sirainen struct mail_index_view *view;
bde6382cf65fba6165dc3603f5419e194d87f404Timo Sirainen const struct mail_index_header *hdr;
bde6382cf65fba6165dc3603f5419e194d87f404Timo Sirainen
bde6382cf65fba6165dc3603f5419e194d87f404Timo Sirainen if (mail_index_refresh(ibox->index) < 0) {
bde6382cf65fba6165dc3603f5419e194d87f404Timo Sirainen mail_storage_set_index_error(ibox);
bde6382cf65fba6165dc3603f5419e194d87f404Timo Sirainen return -1;
bde6382cf65fba6165dc3603f5419e194d87f404Timo Sirainen }
bde6382cf65fba6165dc3603f5419e194d87f404Timo Sirainen
bde6382cf65fba6165dc3603f5419e194d87f404Timo Sirainen view = mail_index_view_open(ibox->index);
bde6382cf65fba6165dc3603f5419e194d87f404Timo Sirainen if (mail_index_get_header(view, &hdr) < 0) {
bde6382cf65fba6165dc3603f5419e194d87f404Timo Sirainen mail_index_view_close(view);
bde6382cf65fba6165dc3603f5419e194d87f404Timo Sirainen mail_storage_set_index_error(ibox);
bde6382cf65fba6165dc3603f5419e194d87f404Timo Sirainen return -1;
bde6382cf65fba6165dc3603f5419e194d87f404Timo Sirainen }
bde6382cf65fba6165dc3603f5419e194d87f404Timo Sirainen ibox->last_cur_mtime = hdr->sync_stamp;
bde6382cf65fba6165dc3603f5419e194d87f404Timo Sirainen mail_index_view_close(view);
bde6382cf65fba6165dc3603f5419e194d87f404Timo Sirainen }
bde6382cf65fba6165dc3603f5419e194d87f404Timo Sirainen
bde6382cf65fba6165dc3603f5419e194d87f404Timo Sirainen if (new_mtime != ibox->last_new_mtime ||
b7b81543899e306c71e6152516d8698416162bcbTimo Sirainen new_mtime >= ibox->last_new_sync_time - MAILDIR_SYNC_SECS) {
b7b81543899e306c71e6152516d8698416162bcbTimo Sirainen *new_changed_r = TRUE;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen ibox->last_new_mtime = new_mtime;
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen ibox->last_new_sync_time = ioloop_time;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen }
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen
0c17af9d3f9323136a94e66605776ed4462a172dTimo Sirainen if (cur_mtime != ibox->last_cur_mtime ||
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen (ibox->dirty_cur_time != 0 &&
b7b81543899e306c71e6152516d8698416162bcbTimo Sirainen ioloop_time - ibox->dirty_cur_time > MAILDIR_SYNC_SECS)) {
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen /* cur/ changed, or delayed cur/ check */
033557e1c8ebec5ae31f2f24fab90226a1945168Timo Sirainen *cur_changed_r = TRUE;
033557e1c8ebec5ae31f2f24fab90226a1945168Timo Sirainen ibox->last_cur_mtime = cur_mtime;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen ibox->dirty_cur_time =
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen cur_mtime >= ioloop_time - MAILDIR_SYNC_SECS ?
d67f54632110cfb6aafe2d7cd1f99b031c0b208aTimo Sirainen cur_mtime : 0;
d67f54632110cfb6aafe2d7cd1f99b031c0b208aTimo Sirainen }
d67f54632110cfb6aafe2d7cd1f99b031c0b208aTimo Sirainen
d67f54632110cfb6aafe2d7cd1f99b031c0b208aTimo Sirainen return 0;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen}
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainenint maildir_sync_index(struct index_mailbox *ibox, int partial)
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen{
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen struct maildir_index_sync_context sync_ctx;
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen struct maildir_uidlist_iter_ctx *iter;
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen struct mail_index_transaction *trans;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen struct mail_index_view *view;
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen const struct mail_index_header *hdr;
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen const struct mail_index_record *rec;
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen uint32_t seq, uid;
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen enum maildir_uidlist_rec_flag uflags;
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen const char *filename;
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen enum mail_flags flags;
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen keywords_mask_t keywords;
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen uint32_t uid_validity, next_uid;
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen int ret;
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen memset(&sync_ctx, 0, sizeof(sync_ctx));
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen sync_ctx.ibox = ibox;
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen if (mail_index_sync_begin(ibox->index, &sync_ctx.sync_ctx, &view,
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen (uint32_t)-1, (uoff_t)-1,
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen FALSE, FALSE) <= 0) {
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen mail_storage_set_index_error(ibox);
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen return -1;
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen }
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen sync_ctx.view = view;
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen ret = mail_index_get_header(view, &hdr);
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen i_assert(ret == 0); /* view is locked, can't happen */
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen uid_validity = maildir_uidlist_get_uid_validity(ibox->uidlist);
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen if (uid_validity != hdr->uid_validity &&
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen uid_validity != 0 && hdr->uid_validity != 0) {
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen /* uidvalidity changed and mailbox isn't being initialized,
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen index must be rebuilt */
b7b81543899e306c71e6152516d8698416162bcbTimo Sirainen mail_storage_set_critical(ibox->box.storage,
b7b81543899e306c71e6152516d8698416162bcbTimo Sirainen "Maildir %s sync: UIDVALIDITY changed (%u -> %u)",
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen ibox->path, hdr->uid_validity, uid_validity);
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen mail_index_mark_corrupted(ibox->index);
d8b77aef97e89f1ccc5cbdaef77be9052279e35fTimo Sirainen (void)mail_index_sync_rollback(sync_ctx.sync_ctx);
e06c0b65c16ccce69bbee009ead14d7d3d17a256Timo Sirainen return -1;
31ddc75584c5cde53d2e78a737587f2e7fdcb0d2Timo Sirainen }
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen
de4288b7369945a31c4001add9445fd0195a358dTimo Sirainen trans = mail_index_transaction_begin(view, FALSE);
de4288b7369945a31c4001add9445fd0195a358dTimo Sirainen sync_ctx.trans = trans;
de4288b7369945a31c4001add9445fd0195a358dTimo Sirainen
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen seq = 0;
de4288b7369945a31c4001add9445fd0195a358dTimo Sirainen iter = maildir_uidlist_iter_init(ibox->uidlist);
de4288b7369945a31c4001add9445fd0195a358dTimo Sirainen while (maildir_uidlist_iter_next(iter, &uid, &uflags, &filename)) {
31ddc75584c5cde53d2e78a737587f2e7fdcb0d2Timo Sirainen maildir_filename_get_flags(filename, &flags, keywords);
e06c0b65c16ccce69bbee009ead14d7d3d17a256Timo Sirainen
92888ef30960c30ccc9e030fe7eab5d4d04a7d1cTimo Sirainen if ((uflags & MAILDIR_UIDLIST_REC_FLAG_RECENT) != 0 &&
92888ef30960c30ccc9e030fe7eab5d4d04a7d1cTimo Sirainen (uflags & MAILDIR_UIDLIST_REC_FLAG_NEW_DIR) != 0 &&
e06c0b65c16ccce69bbee009ead14d7d3d17a256Timo Sirainen (uflags & MAILDIR_UIDLIST_REC_FLAG_MOVED) == 0) {
e06c0b65c16ccce69bbee009ead14d7d3d17a256Timo Sirainen /* mail is recent for next session as well */
e06c0b65c16ccce69bbee009ead14d7d3d17a256Timo Sirainen flags |= MAIL_RECENT;
6969f13267ad8495a132ece39d34be36b9883f37Timo Sirainen }
6969f13267ad8495a132ece39d34be36b9883f37Timo Sirainen
e06c0b65c16ccce69bbee009ead14d7d3d17a256Timo Sirainen __again:
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen seq++;
e06c0b65c16ccce69bbee009ead14d7d3d17a256Timo Sirainen
e06c0b65c16ccce69bbee009ead14d7d3d17a256Timo Sirainen if (seq > hdr->messages_count) {
e06c0b65c16ccce69bbee009ead14d7d3d17a256Timo Sirainen if (uid < hdr->next_uid) {
7c95b03620a03a43dd72d39608cea5fc77393ad6Timo Sirainen /* most likely a race condition: we read the
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen maildir, then someone else expunged messages
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen and committed changes to index. so, this
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen message shouldn't actually exist. mark it
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen racy and check in next sync.
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen
d8b77aef97e89f1ccc5cbdaef77be9052279e35fTimo Sirainen the difference between this and the later
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen check is that this one happens when messages
d482b35af87f5fd872bad007da0475813a401a49Timo Sirainen are expunged from the end */
d482b35af87f5fd872bad007da0475813a401a49Timo Sirainen if ((uflags &
d482b35af87f5fd872bad007da0475813a401a49Timo Sirainen MAILDIR_UIDLIST_REC_FLAG_NONSYNCED) != 0) {
d482b35af87f5fd872bad007da0475813a401a49Timo Sirainen /* partial syncing */
d482b35af87f5fd872bad007da0475813a401a49Timo Sirainen continue;
d482b35af87f5fd872bad007da0475813a401a49Timo Sirainen }
d482b35af87f5fd872bad007da0475813a401a49Timo Sirainen if ((uflags &
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen MAILDIR_UIDLIST_REC_FLAG_RACING) != 0) {
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen mail_storage_set_critical(
b7b81543899e306c71e6152516d8698416162bcbTimo Sirainen ibox->box.storage,
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen "Maildir %s sync: "
d67f54632110cfb6aafe2d7cd1f99b031c0b208aTimo Sirainen "UID < next_uid "
316689d0ef55e2fa4e2fb4ac5b1ea35ce65688d3Timo Sirainen "(%u < %u, file = %s)",
316689d0ef55e2fa4e2fb4ac5b1ea35ce65688d3Timo Sirainen ibox->path, uid, hdr->next_uid,
316689d0ef55e2fa4e2fb4ac5b1ea35ce65688d3Timo Sirainen filename);
316689d0ef55e2fa4e2fb4ac5b1ea35ce65688d3Timo Sirainen mail_index_mark_corrupted(ibox->index);
316689d0ef55e2fa4e2fb4ac5b1ea35ce65688d3Timo Sirainen ret = -1;
316689d0ef55e2fa4e2fb4ac5b1ea35ce65688d3Timo Sirainen break;
316689d0ef55e2fa4e2fb4ac5b1ea35ce65688d3Timo Sirainen }
316689d0ef55e2fa4e2fb4ac5b1ea35ce65688d3Timo Sirainen ibox->dirty_cur_time = ioloop_time;
316689d0ef55e2fa4e2fb4ac5b1ea35ce65688d3Timo Sirainen maildir_uidlist_add_flags(ibox->uidlist,
a33fde36c57839342dafdf3c8d4b5c009e3ef4bcTimo Sirainen filename,
a33fde36c57839342dafdf3c8d4b5c009e3ef4bcTimo Sirainen MAILDIR_UIDLIST_REC_FLAG_RACING);
a33fde36c57839342dafdf3c8d4b5c009e3ef4bcTimo Sirainen
a33fde36c57839342dafdf3c8d4b5c009e3ef4bcTimo Sirainen seq--;
a33fde36c57839342dafdf3c8d4b5c009e3ef4bcTimo Sirainen continue;
316689d0ef55e2fa4e2fb4ac5b1ea35ce65688d3Timo Sirainen }
316689d0ef55e2fa4e2fb4ac5b1ea35ce65688d3Timo Sirainen
316689d0ef55e2fa4e2fb4ac5b1ea35ce65688d3Timo Sirainen mail_index_append(trans, uid, &seq);
316689d0ef55e2fa4e2fb4ac5b1ea35ce65688d3Timo Sirainen mail_index_update_flags(trans, seq, MODIFY_REPLACE,
6969f13267ad8495a132ece39d34be36b9883f37Timo Sirainen flags, keywords);
6969f13267ad8495a132ece39d34be36b9883f37Timo Sirainen continue;
316689d0ef55e2fa4e2fb4ac5b1ea35ce65688d3Timo Sirainen }
6969f13267ad8495a132ece39d34be36b9883f37Timo Sirainen
6969f13267ad8495a132ece39d34be36b9883f37Timo Sirainen if (mail_index_lookup(view, seq, &rec) < 0) {
316689d0ef55e2fa4e2fb4ac5b1ea35ce65688d3Timo Sirainen ret = -1;
316689d0ef55e2fa4e2fb4ac5b1ea35ce65688d3Timo Sirainen break;
316689d0ef55e2fa4e2fb4ac5b1ea35ce65688d3Timo Sirainen }
316689d0ef55e2fa4e2fb4ac5b1ea35ce65688d3Timo Sirainen
316689d0ef55e2fa4e2fb4ac5b1ea35ce65688d3Timo Sirainen if (rec->uid < uid) {
316689d0ef55e2fa4e2fb4ac5b1ea35ce65688d3Timo Sirainen /* expunged */
316689d0ef55e2fa4e2fb4ac5b1ea35ce65688d3Timo Sirainen mail_index_expunge(trans, seq);
316689d0ef55e2fa4e2fb4ac5b1ea35ce65688d3Timo Sirainen goto __again;
316689d0ef55e2fa4e2fb4ac5b1ea35ce65688d3Timo Sirainen }
316689d0ef55e2fa4e2fb4ac5b1ea35ce65688d3Timo Sirainen
316689d0ef55e2fa4e2fb4ac5b1ea35ce65688d3Timo Sirainen if (rec->uid > uid) {
d67f54632110cfb6aafe2d7cd1f99b031c0b208aTimo Sirainen /* most likely a race condition: we read the
d67f54632110cfb6aafe2d7cd1f99b031c0b208aTimo Sirainen maildir, then someone else expunged messages and
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen committed changes to index. so, this message
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen shouldn't actually exist. mark it racy and check
d8b77aef97e89f1ccc5cbdaef77be9052279e35fTimo Sirainen in next sync. */
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen if ((uflags &
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen MAILDIR_UIDLIST_REC_FLAG_NONSYNCED) != 0) {
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen /* partial syncing */
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen seq--;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen continue;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen }
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen if ((uflags & MAILDIR_UIDLIST_REC_FLAG_RACING) != 0) {
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen mail_storage_set_critical(ibox->box.storage,
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen "Maildir %s sync: "
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen "UID inserted in the middle of mailbox "
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen "(%u > %u, file = %s)",
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen ibox->path, rec->uid, uid, filename);
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen mail_index_mark_corrupted(ibox->index);
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen ret = -1;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen break;
a038139a470d2942759b9b86a9852aee7b460996Timo Sirainen }
a038139a470d2942759b9b86a9852aee7b460996Timo Sirainen
a038139a470d2942759b9b86a9852aee7b460996Timo Sirainen ibox->dirty_cur_time = ioloop_time;
316689d0ef55e2fa4e2fb4ac5b1ea35ce65688d3Timo Sirainen maildir_uidlist_add_flags(ibox->uidlist, filename,
316689d0ef55e2fa4e2fb4ac5b1ea35ce65688d3Timo Sirainen MAILDIR_UIDLIST_REC_FLAG_RACING);
a33fde36c57839342dafdf3c8d4b5c009e3ef4bcTimo Sirainen
a33fde36c57839342dafdf3c8d4b5c009e3ef4bcTimo Sirainen seq--;
a33fde36c57839342dafdf3c8d4b5c009e3ef4bcTimo Sirainen continue;
a33fde36c57839342dafdf3c8d4b5c009e3ef4bcTimo Sirainen }
a33fde36c57839342dafdf3c8d4b5c009e3ef4bcTimo Sirainen
a33fde36c57839342dafdf3c8d4b5c009e3ef4bcTimo Sirainen if ((rec->flags & MAIL_RECENT) != 0) {
316689d0ef55e2fa4e2fb4ac5b1ea35ce65688d3Timo Sirainen index_mailbox_set_recent(ibox, seq);
a038139a470d2942759b9b86a9852aee7b460996Timo Sirainen if (ibox->keep_recent) {
0e2686dfe29a18772fa4026bad53e2c7c560403fTimo Sirainen flags |= MAIL_RECENT;
0e2686dfe29a18772fa4026bad53e2c7c560403fTimo Sirainen } else {
a038139a470d2942759b9b86a9852aee7b460996Timo Sirainen mail_index_update_flags(trans, seq,
6969f13267ad8495a132ece39d34be36b9883f37Timo Sirainen MODIFY_REMOVE,
a038139a470d2942759b9b86a9852aee7b460996Timo Sirainen MAIL_RECENT, keywords);
a038139a470d2942759b9b86a9852aee7b460996Timo Sirainen }
a038139a470d2942759b9b86a9852aee7b460996Timo Sirainen }
bde6382cf65fba6165dc3603f5419e194d87f404Timo Sirainen
316689d0ef55e2fa4e2fb4ac5b1ea35ce65688d3Timo Sirainen if ((uflags & MAILDIR_UIDLIST_REC_FLAG_NONSYNCED) != 0) {
316689d0ef55e2fa4e2fb4ac5b1ea35ce65688d3Timo Sirainen /* partial syncing */
316689d0ef55e2fa4e2fb4ac5b1ea35ce65688d3Timo Sirainen continue;
316689d0ef55e2fa4e2fb4ac5b1ea35ce65688d3Timo Sirainen }
a038139a470d2942759b9b86a9852aee7b460996Timo Sirainen
a038139a470d2942759b9b86a9852aee7b460996Timo Sirainen if ((rec->flags & MAIL_INDEX_MAIL_FLAG_DIRTY) != 0) {
a038139a470d2942759b9b86a9852aee7b460996Timo Sirainen /* we haven't been able to update maildir with this
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen record's flag changes. don't sync them. */
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen continue;
0aa66a8e037c08604e71446d6ba52150a17dfc21Timo Sirainen }
0aa66a8e037c08604e71446d6ba52150a17dfc21Timo Sirainen
0aa66a8e037c08604e71446d6ba52150a17dfc21Timo Sirainen if (((uint8_t)flags & ~MAIL_RECENT) !=
0aa66a8e037c08604e71446d6ba52150a17dfc21Timo Sirainen (rec->flags & (MAIL_FLAGS_MASK^MAIL_RECENT)) ||
0aa66a8e037c08604e71446d6ba52150a17dfc21Timo Sirainen memcmp(keywords, rec->keywords,
0aa66a8e037c08604e71446d6ba52150a17dfc21Timo Sirainen INDEX_KEYWORDS_BYTE_COUNT) != 0) {
0aa66a8e037c08604e71446d6ba52150a17dfc21Timo Sirainen /* FIXME: this is wrong if there's pending changes in
0aa66a8e037c08604e71446d6ba52150a17dfc21Timo Sirainen transaction log already. it gets fixed in next sync
0aa66a8e037c08604e71446d6ba52150a17dfc21Timo Sirainen however.. */
0aa66a8e037c08604e71446d6ba52150a17dfc21Timo Sirainen mail_index_update_flags(trans, seq, MODIFY_REPLACE,
0aa66a8e037c08604e71446d6ba52150a17dfc21Timo Sirainen flags, keywords);
a33fde36c57839342dafdf3c8d4b5c009e3ef4bcTimo Sirainen } else if ((flags & MAIL_RECENT) == 0 &&
a33fde36c57839342dafdf3c8d4b5c009e3ef4bcTimo Sirainen (rec->flags & MAIL_RECENT) != 0) {
a33fde36c57839342dafdf3c8d4b5c009e3ef4bcTimo Sirainen /* just remove recent flag */
a33fde36c57839342dafdf3c8d4b5c009e3ef4bcTimo Sirainen memset(keywords, 0, sizeof(keywords));
a33fde36c57839342dafdf3c8d4b5c009e3ef4bcTimo Sirainen mail_index_update_flags(trans, seq, MODIFY_REMOVE,
fd3d711f219fd6813492acbe051e04327f0ca0f0Timo Sirainen MAIL_RECENT, keywords);
fd3d711f219fd6813492acbe051e04327f0ca0f0Timo Sirainen }
fd3d711f219fd6813492acbe051e04327f0ca0f0Timo Sirainen }
fd3d711f219fd6813492acbe051e04327f0ca0f0Timo Sirainen maildir_uidlist_iter_deinit(iter);
fd3d711f219fd6813492acbe051e04327f0ca0f0Timo Sirainen
fd3d711f219fd6813492acbe051e04327f0ca0f0Timo Sirainen if (!partial) {
d482b35af87f5fd872bad007da0475813a401a49Timo Sirainen /* expunge the rest */
d482b35af87f5fd872bad007da0475813a401a49Timo Sirainen for (seq++; seq <= hdr->messages_count; seq++)
d8b77aef97e89f1ccc5cbdaef77be9052279e35fTimo Sirainen mail_index_expunge(trans, seq);
d8b77aef97e89f1ccc5cbdaef77be9052279e35fTimo Sirainen }
d482b35af87f5fd872bad007da0475813a401a49Timo Sirainen
d482b35af87f5fd872bad007da0475813a401a49Timo Sirainen /* now, sync the index */
d482b35af87f5fd872bad007da0475813a401a49Timo Sirainen ibox->syncing_commit = TRUE;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen while ((ret = mail_index_sync_next(sync_ctx.sync_ctx,
d8b77aef97e89f1ccc5cbdaef77be9052279e35fTimo Sirainen &sync_ctx.sync_rec)) > 0) {
d482b35af87f5fd872bad007da0475813a401a49Timo Sirainen if (maildir_sync_record(ibox, &sync_ctx) < 0) {
d482b35af87f5fd872bad007da0475813a401a49Timo Sirainen ret = -1;
d482b35af87f5fd872bad007da0475813a401a49Timo Sirainen break;
d482b35af87f5fd872bad007da0475813a401a49Timo Sirainen }
d482b35af87f5fd872bad007da0475813a401a49Timo Sirainen }
d482b35af87f5fd872bad007da0475813a401a49Timo Sirainen ibox->syncing_commit = FALSE;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen if (ibox->dirty_cur_time == 0 &&
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen ibox->last_cur_mtime != (time_t)hdr->sync_stamp) {
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen uint32_t sync_stamp = ibox->last_cur_mtime;
e10c0b544b2516e92c5df9ef778e2a826eb02995Timo Sirainen
b7b81543899e306c71e6152516d8698416162bcbTimo Sirainen mail_index_update_header(trans,
b7b81543899e306c71e6152516d8698416162bcbTimo Sirainen offsetof(struct mail_index_header, sync_stamp),
b7b81543899e306c71e6152516d8698416162bcbTimo Sirainen &sync_stamp, sizeof(sync_stamp));
b7b81543899e306c71e6152516d8698416162bcbTimo Sirainen }
b7b81543899e306c71e6152516d8698416162bcbTimo Sirainen
b35f7104715edee0cfac6d46ab0b342033867eb7Timo Sirainen if (hdr->uid_validity == 0) {
714dfaac096e6190898aaa7cce315c2fa247f05aTimo Sirainen /* get the initial uidvalidity */
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen if (maildir_uidlist_update(ibox->uidlist) < 0)
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen ret = -1;
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen uid_validity = maildir_uidlist_get_uid_validity(ibox->uidlist);
b35f7104715edee0cfac6d46ab0b342033867eb7Timo Sirainen if (uid_validity == 0) {
b35f7104715edee0cfac6d46ab0b342033867eb7Timo Sirainen uid_validity = ioloop_time;
b35f7104715edee0cfac6d46ab0b342033867eb7Timo Sirainen maildir_uidlist_set_uid_validity(ibox->uidlist,
b35f7104715edee0cfac6d46ab0b342033867eb7Timo Sirainen uid_validity);
714dfaac096e6190898aaa7cce315c2fa247f05aTimo Sirainen }
b35f7104715edee0cfac6d46ab0b342033867eb7Timo Sirainen } else if (uid_validity == 0) {
bde6382cf65fba6165dc3603f5419e194d87f404Timo Sirainen maildir_uidlist_set_uid_validity(ibox->uidlist,
bde6382cf65fba6165dc3603f5419e194d87f404Timo Sirainen hdr->uid_validity);
e06c0b65c16ccce69bbee009ead14d7d3d17a256Timo Sirainen }
e06c0b65c16ccce69bbee009ead14d7d3d17a256Timo Sirainen
e06c0b65c16ccce69bbee009ead14d7d3d17a256Timo Sirainen if (uid_validity != hdr->uid_validity && uid_validity != 0) {
e06c0b65c16ccce69bbee009ead14d7d3d17a256Timo Sirainen mail_index_update_header(trans,
e06c0b65c16ccce69bbee009ead14d7d3d17a256Timo Sirainen offsetof(struct mail_index_header, uid_validity),
e06c0b65c16ccce69bbee009ead14d7d3d17a256Timo Sirainen &uid_validity, sizeof(uid_validity));
e06c0b65c16ccce69bbee009ead14d7d3d17a256Timo Sirainen }
92888ef30960c30ccc9e030fe7eab5d4d04a7d1cTimo Sirainen
92888ef30960c30ccc9e030fe7eab5d4d04a7d1cTimo Sirainen next_uid = maildir_uidlist_get_next_uid(ibox->uidlist);
92888ef30960c30ccc9e030fe7eab5d4d04a7d1cTimo Sirainen if (next_uid != 0 && hdr->next_uid != next_uid) {
92888ef30960c30ccc9e030fe7eab5d4d04a7d1cTimo Sirainen mail_index_update_header(trans,
92888ef30960c30ccc9e030fe7eab5d4d04a7d1cTimo Sirainen offsetof(struct mail_index_header, next_uid),
92888ef30960c30ccc9e030fe7eab5d4d04a7d1cTimo Sirainen &next_uid, sizeof(next_uid));
92888ef30960c30ccc9e030fe7eab5d4d04a7d1cTimo Sirainen }
92888ef30960c30ccc9e030fe7eab5d4d04a7d1cTimo Sirainen
92888ef30960c30ccc9e030fe7eab5d4d04a7d1cTimo Sirainen if (ret < 0) {
92888ef30960c30ccc9e030fe7eab5d4d04a7d1cTimo Sirainen mail_index_transaction_rollback(trans);
92888ef30960c30ccc9e030fe7eab5d4d04a7d1cTimo Sirainen mail_index_sync_rollback(sync_ctx.sync_ctx);
92888ef30960c30ccc9e030fe7eab5d4d04a7d1cTimo Sirainen } else {
92888ef30960c30ccc9e030fe7eab5d4d04a7d1cTimo Sirainen uint32_t seq;
92888ef30960c30ccc9e030fe7eab5d4d04a7d1cTimo Sirainen uoff_t offset;
92888ef30960c30ccc9e030fe7eab5d4d04a7d1cTimo Sirainen
92888ef30960c30ccc9e030fe7eab5d4d04a7d1cTimo Sirainen if (mail_index_transaction_commit(trans, &seq, &offset) < 0)
e06c0b65c16ccce69bbee009ead14d7d3d17a256Timo Sirainen ret = -1;
e06c0b65c16ccce69bbee009ead14d7d3d17a256Timo Sirainen else if (seq != 0) {
e06c0b65c16ccce69bbee009ead14d7d3d17a256Timo Sirainen ibox->commit_log_file_seq = seq;
e06c0b65c16ccce69bbee009ead14d7d3d17a256Timo Sirainen ibox->commit_log_file_offset = offset;
e06c0b65c16ccce69bbee009ead14d7d3d17a256Timo Sirainen }
e06c0b65c16ccce69bbee009ead14d7d3d17a256Timo Sirainen if (mail_index_sync_commit(sync_ctx.sync_ctx) < 0)
e06c0b65c16ccce69bbee009ead14d7d3d17a256Timo Sirainen ret = -1;
e06c0b65c16ccce69bbee009ead14d7d3d17a256Timo Sirainen }
e06c0b65c16ccce69bbee009ead14d7d3d17a256Timo Sirainen
e06c0b65c16ccce69bbee009ead14d7d3d17a256Timo Sirainen if (ret == 0) {
e06c0b65c16ccce69bbee009ead14d7d3d17a256Timo Sirainen ibox->commit_log_file_seq = 0;
e06c0b65c16ccce69bbee009ead14d7d3d17a256Timo Sirainen ibox->commit_log_file_offset = 0;
d9de52132072d80b8c268094b879c0ef5a108db3Timo Sirainen } else {
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen mail_storage_set_index_error(ibox);
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen }
d9de52132072d80b8c268094b879c0ef5a108db3Timo Sirainen
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen return ret;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen}
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainenstatic int maildir_sync_context(struct maildir_sync_context *ctx, int forced)
e06c0b65c16ccce69bbee009ead14d7d3d17a256Timo Sirainen{
8a3d609fdd84f5938c82e8e7eeb84a24ab41b317Timo Sirainen int ret, new_changed, cur_changed;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen if (!forced) {
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen if (maildir_sync_quick_check(ctx, &new_changed, &cur_changed) < 0)
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen return -1;
d9de52132072d80b8c268094b879c0ef5a108db3Timo Sirainen
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen if (!new_changed && !cur_changed)
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen return 0;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen } else {
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen new_changed = cur_changed = TRUE;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen }
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen /* we have to lock uidlist immediately, otherwise there's race
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen conditions with other processes who might write older maildir
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen file list into uidlist.
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen alternative would be to lock it when new files are found, but
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen the directory scans _must_ be restarted then.
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen
a38ef15060e45e5060bb24c403a580b9a57a818cTimo Sirainen if we got here through maildir_sync_last_commit(), we can't sync
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen index as it's already being synced. so, don't try locking uidlist
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen either, we only want to find new filename for some mail.
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen */
a38ef15060e45e5060bb24c403a580b9a57a818cTimo Sirainen if (!ctx->ibox->syncing_commit) {
a38ef15060e45e5060bb24c403a580b9a57a818cTimo Sirainen if ((ret = maildir_uidlist_try_lock(ctx->ibox->uidlist)) < 0)
a38ef15060e45e5060bb24c403a580b9a57a818cTimo Sirainen return ret;
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen if (ret == 0 && !forced) {
a38ef15060e45e5060bb24c403a580b9a57a818cTimo Sirainen /* we didn't get a lock, don't do syncing unless we
a38ef15060e45e5060bb24c403a580b9a57a818cTimo Sirainen really want to check for expunges or renames. new
a38ef15060e45e5060bb24c403a580b9a57a818cTimo Sirainen files won't be added. */
a38ef15060e45e5060bb24c403a580b9a57a818cTimo Sirainen return 0;
a38ef15060e45e5060bb24c403a580b9a57a818cTimo Sirainen }
641f0c0900ee6e7cf9667f4b40ed95cec7d0cdcaTimo Sirainen }
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen ctx->partial = !cur_changed;
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen ctx->uidlist_sync_ctx =
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen maildir_uidlist_sync_init(ctx->ibox->uidlist, ctx->partial);
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen if (maildir_scan_dir(ctx, TRUE) < 0)
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen return -1;
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen if (cur_changed) {
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen if (maildir_scan_dir(ctx, FALSE) < 0)
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen return -1;
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen }
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen /* finish uidlist syncing, but keep it still locked */
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen maildir_uidlist_sync_finish(ctx->uidlist_sync_ctx);
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen if (!ctx->ibox->syncing_commit) {
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen if (maildir_sync_index(ctx->ibox, ctx->partial) < 0)
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen return -1;
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen }
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen ret = maildir_uidlist_sync_deinit(ctx->uidlist_sync_ctx);
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen ctx->uidlist_sync_ctx = NULL;
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen return ret;
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen}
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainenint maildir_storage_sync_force(struct index_mailbox *ibox)
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen{
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen struct maildir_sync_context *ctx;
d77c309fccbc6a7594f8cb08fb01009fa613c568Timo Sirainen int ret;
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen ctx = maildir_sync_context_new(ibox);
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen ret = maildir_sync_context(ctx, TRUE);
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen maildir_sync_deinit(ctx);
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen return ret;
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen}
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainenstruct mailbox_sync_context *
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainenmaildir_storage_sync_init(struct mailbox *box, enum mailbox_sync_flags flags)
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen{
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen struct index_mailbox *ibox = (struct index_mailbox *)box;
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen struct maildir_sync_context *ctx;
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen int ret = 0;
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen if ((flags & MAILBOX_SYNC_FLAG_FAST) == 0 ||
65d6ca3fb5450b81df0190d9e9aa62c00fed5116Timo Sirainen ibox->sync_last_check + MAILBOX_FULL_SYNC_INTERVAL <= ioloop_time) {
65d6ca3fb5450b81df0190d9e9aa62c00fed5116Timo Sirainen ibox->sync_last_check = ioloop_time;
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen ctx = maildir_sync_context_new(ibox);
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen ret = maildir_sync_context(ctx, FALSE);
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen maildir_sync_deinit(ctx);
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen }
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen return index_mailbox_sync_init(box, flags, ret < 0);
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen}
d4dcb9c30dba354cff7af6d303ecef7698194c55Timo Sirainen