maildir-sync.c revision a038139a470d2942759b9b86a9852aee7b460996
5f5870385cff47efd2f58e7892f251cf13761528Timo Sirainen/* Copyright (C) 2004 Timo Sirainen */
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen Here's a description of how we handle Maildir synchronization and
8363f50d7b5d605912e55c34f7f28e9f4ce01341Timo Sirainen it's problems:
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen We want to be as efficient as we can. The most efficient way to
e8bdf1be00aec45d0c6dd72ad9c8be02a3dfc778Timo Sirainen check if changes have occured is to stat() the new/ and cur/
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen directories and uidlist file - if their mtimes haven't changed,
9334fbad0aabb2fed88f40b2205d0d6f80bdffa2Timo Sirainen there's no changes and we don't need to do anything.
de4e3a2e1e8f82b2d3226c090b71b518b43bf9cfTimo Sirainen Problem 1: Multiple changes can happen within a single second -
c36ec256c1bd1abe1c12e792cf64f0b7e3b3135aTimo Sirainen nothing guarantees that once we synced it, someone else didn't just
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen then make a modification. Such modifications wouldn't get noticed
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen until a new modification occured later.
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen Problem 2: Syncing cur/ directory is much more costly than syncing
f0a2d04321ba456e5c5ba821c0d1ed9e8e0e2e08Timo Sirainen new/. Moving mails from new/ to cur/ will always change mtime of
f0a2d04321ba456e5c5ba821c0d1ed9e8e0e2e08Timo Sirainen cur/ causing us to sync it as well.
f0a2d04321ba456e5c5ba821c0d1ed9e8e0e2e08Timo Sirainen Problem 3: We may not be able to move mail from new/ to cur/
f0a2d04321ba456e5c5ba821c0d1ed9e8e0e2e08Timo Sirainen because we're out of quota, or simply because we're accessing a
b5e6f6f27c1461f0f9f202615eeb738a645188c3Timo Sirainen read-only mailbox.
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen MAILDIR_SYNC_SECS
8363f50d7b5d605912e55c34f7f28e9f4ce01341Timo Sirainen -----------------
f0a2d04321ba456e5c5ba821c0d1ed9e8e0e2e08Timo Sirainen Several checks below use MAILDIR_SYNC_SECS, which should be maximum
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen clock drift between all computers accessing the maildir (eg. via
f0a2d04321ba456e5c5ba821c0d1ed9e8e0e2e08Timo Sirainen NFS), rounded up to next second. Our default is 1 second, since
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen everyone should be using NTP.
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen Note that setting it to 0 works only if there's only one computer
98922c5675bbbfadc84d58768bef867fe82256c2Timo Sirainen accessing the maildir. It's practically impossible to make two
98922c5675bbbfadc84d58768bef867fe82256c2Timo Sirainen clocks _exactly_ synchronized.
98922c5675bbbfadc84d58768bef867fe82256c2Timo Sirainen It might be possible to only use file server's clock by looking at
98922c5675bbbfadc84d58768bef867fe82256c2Timo Sirainen the atime field, but I don't know how well that would actually work.
98922c5675bbbfadc84d58768bef867fe82256c2Timo Sirainen cur directory
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen -------------
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen We have dirty_cur_time variable which is set to cur/ directory's
b5e6f6f27c1461f0f9f202615eeb738a645188c3Timo Sirainen mtime when it's >= time() - MAILDIR_SYNC_SECS and we _think_ we have
2cfe9983ce7a6280636ee12beccc2e865111967bTimo Sirainen synchronized the directory.
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen When dirty_cur_time is non-zero, we don't synchronize the cur/
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen directory until
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen a) cur/'s mtime changes
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen b) opening a mail fails with ENOENT
98922c5675bbbfadc84d58768bef867fe82256c2Timo Sirainen c) time() > dirty_cur_time + MAILDIR_SYNC_SECS
98922c5675bbbfadc84d58768bef867fe82256c2Timo Sirainen This allows us to modify the maildir multiple times without having
98922c5675bbbfadc84d58768bef867fe82256c2Timo Sirainen to sync it at every change. The sync will eventually be done to
98922c5675bbbfadc84d58768bef867fe82256c2Timo Sirainen make sure we didn't miss any external changes.
98922c5675bbbfadc84d58768bef867fe82256c2Timo Sirainen The dirty_cur_time is set when:
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen - we change message flags
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen - we expunge messages
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen - we move mail from new/ to cur/
b6a7e0a7899e7f5d60c23cdaa50e025e4c67d05fTimo Sirainen - we sync cur/ directory and it's mtime is >= time() - MAILDIR_SYNC_SECS
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen It's unset when we do the final syncing, ie. when mtime is
d42eb03b3a4e79a2da22a1be2de59b95660af2beTimo Sirainen older than time() - MAILDIR_SYNC_SECS.
f0a2d04321ba456e5c5ba821c0d1ed9e8e0e2e08Timo Sirainen new directory
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen -------------
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen If new/'s mtime is >= time() - MAILDIR_SYNC_SECS, always synchronize
f0a2d04321ba456e5c5ba821c0d1ed9e8e0e2e08Timo Sirainen it. dirty_cur_time-like feature might save us a few syncs, but
f0a2d04321ba456e5c5ba821c0d1ed9e8e0e2e08Timo Sirainen that might break a client which saves a mail in one connection and
f0a2d04321ba456e5c5ba821c0d1ed9e8e0e2e08Timo Sirainen tries to fetch it in another one. new/ directory is almost always
2d01cc1880cf2afd4fb1c8ad7fa6ce78e562e71eTimo Sirainen empty, so syncing it should be very fast anyway. Actually this can
fc40a9a002458e372ff4b9f6f4e15239520c0bcdTimo Sirainen still happen if we sync only new/ dir while another client is also
f0a2d04321ba456e5c5ba821c0d1ed9e8e0e2e08Timo Sirainen moving mails from it to cur/ - it takes us a while to see them.
b6a7e0a7899e7f5d60c23cdaa50e025e4c67d05fTimo Sirainen That's pretty unlikely to happen however, and only way to fix it
b6a7e0a7899e7f5d60c23cdaa50e025e4c67d05fTimo Sirainen would be to always synchronize cur/ after new/.
f0a2d04321ba456e5c5ba821c0d1ed9e8e0e2e08Timo Sirainen Normally we move all mails from new/ to cur/ whenever we sync it. If
b6a7e0a7899e7f5d60c23cdaa50e025e4c67d05fTimo Sirainen it's not possible for some reason, we mark the mail with "probably
b6a7e0a7899e7f5d60c23cdaa50e025e4c67d05fTimo Sirainen exists in new/ directory" flag.
f0a2d04321ba456e5c5ba821c0d1ed9e8e0e2e08Timo Sirainen If rename() still fails because of ENOSPC or EDQUOT, we still save
b6a7e0a7899e7f5d60c23cdaa50e025e4c67d05fTimo Sirainen the flag changes in index with dirty-flag on. When moving the mail
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen to cur/ directory, or when we notice it's already moved there, we
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen apply the flag changes to the filename, rename it and remove the
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen dirty flag. If there's dirty flags, this should be tried every time
f0a2d04321ba456e5c5ba821c0d1ed9e8e0e2e08Timo Sirainen after expunge or when closing the mailbox.
c60d1eda4df179d83d531647732d5e3e45064219Timo Sirainen This file contains UID <-> filename mappings. It's updated only when
c60d1eda4df179d83d531647732d5e3e45064219Timo Sirainen new mail arrives, so it may contain filenames that have already been
c60d1eda4df179d83d531647732d5e3e45064219Timo Sirainen deleted. Updating is done by getting uidlist.lock file, writing the
c60d1eda4df179d83d531647732d5e3e45064219Timo Sirainen whole uidlist into it and rename()ing it over the old uidlist. This
c60d1eda4df179d83d531647732d5e3e45064219Timo Sirainen means there's no need to lock the file for reading.
d42eb03b3a4e79a2da22a1be2de59b95660af2beTimo Sirainen Whenever uidlist is rewritten, it's mtime must be larger than the old
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen one's. Use utime() before rename() if needed. Note that inode checking
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen wouldn't have been sufficient as inode numbers can be reused.
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen This file is usually read the first time you need to know filename for
90adcaa0a00eba29b7fbd50ca66be11c8d086d6aTimo Sirainen given UID. After that it's not re-read unless new mails come that we
b0a901f1dbe9e05ac1c92a0974af6bce0274f31aTimo Sirainen don't know about.
b6a7e0a7899e7f5d60c23cdaa50e025e4c67d05fTimo Sirainen broken clients
ee1a3e217279dcd0f1cccbd3442e481b7dc90f05Timo Sirainen --------------
35283613d4c04ce18836e9fc431582c87b3710a0Timo Sirainen Originally the middle identifier in Maildir filename was specified
35283613d4c04ce18836e9fc431582c87b3710a0Timo Sirainen only as <process id>_<delivery counter>. That however created a
35283613d4c04ce18836e9fc431582c87b3710a0Timo Sirainen problem with randomized PIDs which made it possible that the same
35283613d4c04ce18836e9fc431582c87b3710a0Timo Sirainen PID was reused within one second.
bc6ef0b01b99c43ee46aa796420516e89a744c30Timo Sirainen So if within one second a mail was delivered, MUA moved it to cur/
a0aedab7cd06125e4d73638b1bd0c01c7caa2626Timo Sirainen and another mail was delivered by a new process using same PID as
992a13add4eea0810e4db0f042a595dddf85536aTimo Sirainen the first one, we likely ended up overwriting the first mail when
83bb013a99f0936995f9c7a1077822662d8fefdbTimo Sirainen the second mail was moved over it.
83bb013a99f0936995f9c7a1077822662d8fefdbTimo Sirainen Nowadays everyone should be giving a bit more specific identifier,
992a13add4eea0810e4db0f042a595dddf85536aTimo Sirainen for example include microseconds in it which Dovecot does.
b0a901f1dbe9e05ac1c92a0974af6bce0274f31aTimo Sirainen There's a simple way to prevent this from happening in some cases:
2d01cc1880cf2afd4fb1c8ad7fa6ce78e562e71eTimo Sirainen Don't move the mail from new/ to cur/ if it's mtime is >= time() -
01f4ee4a0243f3fe9af763e1a540cd5cff0d63f5Timo Sirainen MAILDIR_SYNC_SECS. The second delivery's link() call then fails
90adcaa0a00eba29b7fbd50ca66be11c8d086d6aTimo Sirainen because the file is already in new/, and it will then use a
35283613d4c04ce18836e9fc431582c87b3710a0Timo Sirainen different filename. There's a few problems with this however:
2d01cc1880cf2afd4fb1c8ad7fa6ce78e562e71eTimo Sirainen - it requires extra stat() call which is unneeded extra I/O
35283613d4c04ce18836e9fc431582c87b3710a0Timo Sirainen - another MUA might still move the mail to cur/
e9d29ae46d435aee85514decfe6ee27399ebf794Timo Sirainen - if first file's flags are modified by either Dovecot or another
e9d29ae46d435aee85514decfe6ee27399ebf794Timo Sirainen MUA, it's moved to cur/ (you _could_ just do the dirty-flagging
e9d29ae46d435aee85514decfe6ee27399ebf794Timo Sirainen but that'd be ugly)
e9d29ae46d435aee85514decfe6ee27399ebf794Timo Sirainen Because this is useful only for very few people and it requires
b6a7e0a7899e7f5d60c23cdaa50e025e4c67d05fTimo Sirainen extra I/O, I decided not to implement this. It should be however
2d01cc1880cf2afd4fb1c8ad7fa6ce78e562e71eTimo Sirainen quite easy to do since we need to be able to deal with files in new/
83bb013a99f0936995f9c7a1077822662d8fefdbTimo Sirainen It's also possible to never accidentally overwrite a mail by using
83bb013a99f0936995f9c7a1077822662d8fefdbTimo Sirainen link() + unlink() rather than rename(). This however isn't very
90adcaa0a00eba29b7fbd50ca66be11c8d086d6aTimo Sirainen good idea as it introduces potential race conditions when multiple
83bb013a99f0936995f9c7a1077822662d8fefdbTimo Sirainen clients are accessing the mailbox:
90adcaa0a00eba29b7fbd50ca66be11c8d086d6aTimo Sirainen Trying to move the same mail from new/ to cur/ at the same time:
90adcaa0a00eba29b7fbd50ca66be11c8d086d6aTimo Sirainen a) Client 1 uses slightly different filename than client 2,
83bb013a99f0936995f9c7a1077822662d8fefdbTimo Sirainen for example one sets read-flag on but the other doesn't.
90adcaa0a00eba29b7fbd50ca66be11c8d086d6aTimo Sirainen You have the same mail duplicated now.
90adcaa0a00eba29b7fbd50ca66be11c8d086d6aTimo Sirainen b) Client 3 sees the mail between Client 1's and 2's link() calls
90adcaa0a00eba29b7fbd50ca66be11c8d086d6aTimo Sirainen and changes it's flag. You have the same mail duplicated now.
ae1b268ffff743ad9927c304a1344c5cbd7f909dTimo Sirainen And it gets worse when they're unlink()ing in cur/ directory:
90adcaa0a00eba29b7fbd50ca66be11c8d086d6aTimo Sirainen c) Client 1 changes mails's flag and client 2 changes it back
90adcaa0a00eba29b7fbd50ca66be11c8d086d6aTimo Sirainen between 1's link() and unlink(). The mail is now expunged.
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen d) If you try to deal with the duplicates by unlink()ing another
ae1b268ffff743ad9927c304a1344c5cbd7f909dTimo Sirainen one of them, you might end up unlinking both of them.
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen So, what should we do then if we notice a duplicate? First of all,
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen it might not be a duplicate at all, readdir() might have just
83bb013a99f0936995f9c7a1077822662d8fefdbTimo Sirainen returned it twice because it was just renamed. What we should do is
90adcaa0a00eba29b7fbd50ca66be11c8d086d6aTimo Sirainen create a completely new base name for it and rename() it to that.
90adcaa0a00eba29b7fbd50ca66be11c8d086d6aTimo Sirainen If the call fails with ENOENT, it only means that it wasn't a
90adcaa0a00eba29b7fbd50ca66be11c8d086d6aTimo Sirainen duplicate after all.
7ede6554e451ec039a67beec7d6ee4aff61d386eTimo Sirainen struct maildir_uidlist_sync_ctx *uidlist_sync_ctx;
9334fbad0aabb2fed88f40b2205d0d6f80bdffa2Timo Sirainenstatic int maildir_expunge(struct index_mailbox *ibox, const char *path,
97eb53ade9057e6966dbb77289ad0204c7e1657bTimo Sirainenstatic int maildir_sync_flags(struct index_mailbox *ibox, const char *path,
c36ec256c1bd1abe1c12e792cf64f0b7e3b3135aTimo Sirainen struct maildir_index_sync_context *ctx = context;
c36ec256c1bd1abe1c12e792cf64f0b7e3b3135aTimo Sirainen (void)maildir_filename_get_flags(path, &flags, keywords);
c36ec256c1bd1abe1c12e792cf64f0b7e3b3135aTimo Sirainen mail_index_sync_flags_apply(&ctx->sync_rec, &flags8, keywords);
ecd69c4e8371853667e01b0c16d436ef7f7393e2Timo Sirainen newpath = maildir_filename_set_flags(path, flags8, keywords);
c36ec256c1bd1abe1c12e792cf64f0b7e3b3135aTimo Sirainen if (mail_index_sync_set_dirty(ctx->sync_ctx, ctx->seq) < 0)
c36ec256c1bd1abe1c12e792cf64f0b7e3b3135aTimo Sirainenstatic int maildir_sync_record(struct index_mailbox *ibox,
97eb53ade9057e6966dbb77289ad0204c7e1657bTimo Sirainen struct mail_index_sync_rec *sync_rec = &ctx->sync_rec;
236bedf76e31651ea9fca63fbdc25be673819526Timo Sirainen /* make it go through sequences to avoid looping through huge
d22301419109ed4a38351715e6760011421dadecTimo Sirainen holes in UID range */
c36ec256c1bd1abe1c12e792cf64f0b7e3b3135aTimo Sirainen if (mail_index_lookup_uid_range(view, sync_rec->uid1,
c36ec256c1bd1abe1c12e792cf64f0b7e3b3135aTimo Sirainen if (mail_index_lookup_uid(view, seq, &uid) < 0)
97eb53ade9057e6966dbb77289ad0204c7e1657bTimo Sirainen if (maildir_file_do(ibox, uid, maildir_expunge,
c36ec256c1bd1abe1c12e792cf64f0b7e3b3135aTimo Sirainen if (mail_index_lookup_uid_range(view, sync_rec->uid1,
b142deb9a831c89b1bb9129ada655f3e56b9d4ccTimo Sirainen for (ctx->seq = seq1; ctx->seq <= seq2; ctx->seq++) {
b142deb9a831c89b1bb9129ada655f3e56b9d4ccTimo Sirainen if (mail_index_lookup_uid(view, ctx->seq, &uid) < 0)
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen if (maildir_file_do(ibox, uid, maildir_sync_flags,
3eb63515855f386449c22233d1f1baf1ddfe8a2dTimo Sirainenint maildir_sync_last_commit(struct index_mailbox *ibox)
ae1b268ffff743ad9927c304a1344c5cbd7f909dTimo Sirainen ret = mail_index_sync_begin(ibox->index, &ctx.sync_ctx, &ctx.view,
d22301419109ed4a38351715e6760011421dadecTimo Sirainen while ((ret = mail_index_sync_next(ctx.sync_ctx,
97eb53ade9057e6966dbb77289ad0204c7e1657bTimo Sirainen if (mail_index_sync_end(ctx.sync_ctx, 0, 0) < 0)
cf49fc07f541c0f74578ac6c3b334ddade143aa1Timo Sirainenmaildir_sync_context_new(struct index_mailbox *ibox)
2ff23d6fb7e2ff85aa23b7f4769aeac1d0316a1bTimo Sirainen ctx->new_dir = t_strconcat(ibox->path, "/new", NULL);
cf49fc07f541c0f74578ac6c3b334ddade143aa1Timo Sirainen ctx->cur_dir = t_strconcat(ibox->path, "/cur", NULL);
c36ec256c1bd1abe1c12e792cf64f0b7e3b3135aTimo Sirainenstatic void maildir_sync_deinit(struct maildir_sync_context *ctx)
18065635d4e79dd96eb3b3215718abd12f6a6808Timo Sirainen (void)maildir_uidlist_sync_deinit(ctx->uidlist_sync_ctx);
97eb53ade9057e6966dbb77289ad0204c7e1657bTimo Sirainenstatic int maildir_fix_duplicate(struct index_mailbox *ibox, const char *dir,
ae1b268ffff743ad9927c304a1344c5cbd7f909dTimo Sirainen old_path = t_strconcat(dir, "/", old_fname, NULL);
ae1b268ffff743ad9927c304a1344c5cbd7f909dTimo Sirainen new_fname = maildir_generate_tmp_filename(&ioloop_timeval);
ae1b268ffff743ad9927c304a1344c5cbd7f909dTimo Sirainen new_path = t_strconcat(ibox->path, "/new/", new_fname, NULL);
ae1b268ffff743ad9927c304a1344c5cbd7f909dTimo Sirainen "rename(%s, %s) failed: %m", old_path, new_path);
a0aedab7cd06125e4d73638b1bd0c01c7caa2626Timo Sirainenstatic int maildir_scan_dir(struct maildir_sync_context *ctx, int new_dir)
ae1b268ffff743ad9927c304a1344c5cbd7f909dTimo Sirainen struct mail_storage *storage = ctx->ibox->box.storage;
a0aedab7cd06125e4d73638b1bd0c01c7caa2626Timo Sirainen const char *dir;
ae1b268ffff743ad9927c304a1344c5cbd7f909dTimo Sirainen move_new = new_dir && !mailbox_is_readonly(&ctx->ibox->box);
83bb013a99f0936995f9c7a1077822662d8fefdbTimo Sirainen ret = maildir_uidlist_sync_next_pre(ctx->uidlist_sync_ctx,
6d50c4d875bb05f9076e9e0ecbacb8beb2e9ae42Timo Sirainen str_printfa(src, "%s/%s", ctx->new_dir, dp->d_name);
6d50c4d875bb05f9076e9e0ecbacb8beb2e9ae42Timo Sirainen str_printfa(dest, "%s/%s", ctx->cur_dir, dp->d_name);
6d50c4d875bb05f9076e9e0ecbacb8beb2e9ae42Timo Sirainen /* we moved it - it's \Recent for us */
6d50c4d875bb05f9076e9e0ecbacb8beb2e9ae42Timo Sirainen /* someone else moved it already */
6d50c4d875bb05f9076e9e0ecbacb8beb2e9ae42Timo Sirainen /* not enough disk space, leave here */
3eb63515855f386449c22233d1f1baf1ddfe8a2dTimo Sirainen "rename(%s, %s) failed: %m",
97eb53ade9057e6966dbb77289ad0204c7e1657bTimo Sirainen } else if (new_dir) {
97eb53ade9057e6966dbb77289ad0204c7e1657bTimo Sirainen ret = maildir_uidlist_sync_next(ctx->uidlist_sync_ctx,
97eb53ade9057e6966dbb77289ad0204c7e1657bTimo Sirainen /* possibly duplicate - try fixing it */
e1eeb3be29afde2830c2b7ed594c6f1fef2f69dcTimo Sirainenstatic int maildir_sync_quick_check(struct maildir_sync_context *ctx,
ed34a210eff7707787ee154e5581528b8f4b2daaTimo Sirainen /* first sync in this session, get cur stamp from index */
ed34a210eff7707787ee154e5581528b8f4b2daaTimo Sirainen if (mail_index_get_header(ibox->view, &hdr) == 0)
e4e7475f646d66a257d682738fbff1f206ce4924Timo Sirainen new_mtime >= ibox->last_new_sync_time - MAILDIR_SYNC_SECS) {
e4e7475f646d66a257d682738fbff1f206ce4924Timo Sirainen ioloop_time - ibox->dirty_cur_time > MAILDIR_SYNC_SECS)) {
46c903eac3abcc7d32780da76ee6a79c26d185b6Timo Sirainen /* cur/ changed, or delayed cur/ check */
24cd47a2c8f7507e555459b7e841de771ba3c318Timo Sirainen cur_mtime >= ioloop_time - MAILDIR_SYNC_SECS ?
b142deb9a831c89b1bb9129ada655f3e56b9d4ccTimo Sirainenstatic int maildir_sync_index(struct maildir_sync_context *ctx)
89fb98e9eb7e95255a579c8e9d865383c2334a74Timo Sirainen if (mail_index_sync_begin(ibox->index, &sync_ctx.sync_ctx, &view,
e1eeb3be29afde2830c2b7ed594c6f1fef2f69dcTimo Sirainen i_assert(ret == 0); /* view is locked, can't happen */
5f44975ec6c5755dd74bcd4c47a123a7242ecab3Timo Sirainen trans = mail_index_transaction_begin(view, FALSE);
b19a1420da0618a10edf67c2cfd13c8c8633057aTimo Sirainen iter = maildir_uidlist_iter_init(ibox->uidlist);
b19a1420da0618a10edf67c2cfd13c8c8633057aTimo Sirainen while (maildir_uidlist_iter_next(iter, &uid, &uflags, &filename)) {
366d6311c9d5bac6613e3cd64619eb878adce9ecTimo Sirainen maildir_filename_get_flags(filename, &flags, keywords);
6df0ab0c1ab91f06b6418cb30eff44405a1b8f02Timo Sirainen if ((uflags & MAILDIR_UIDLIST_REC_FLAG_NONSYNCED) != 0) {
6df0ab0c1ab91f06b6418cb30eff44405a1b8f02Timo Sirainen /* partial syncing */
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen /* message not in index, but next_uid header
6ef7e31619edfaa17ed044b45861d106a86191efTimo Sirainen is updated? shouldn't really happen.. */
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainen "Maildir sync: UID < next_uid "
b6a7e0a7899e7f5d60c23cdaa50e025e4c67d05fTimo Sirainen "(%u < %u, file = %s)",
6564208826b0f46a00f010d1b5711d85944c3c88Timo Sirainen mail_index_update_flags(trans, seq, MODIFY_REPLACE,
225e82df5dd1e765f4e52b80c954558f00e5a7dfTimo Sirainen /* expunged */
225e82df5dd1e765f4e52b80c954558f00e5a7dfTimo Sirainen /* most likely a race condition: we read the
225e82df5dd1e765f4e52b80c954558f00e5a7dfTimo Sirainen maildir, then someone else expunged messages and
225e82df5dd1e765f4e52b80c954558f00e5a7dfTimo Sirainen committed changes to index. so, this message
900bb5e316d030cdebff7ee128ce65881dfb27f7Timo Sirainen shouldn't actually exist. check to be sure.
225e82df5dd1e765f4e52b80c954558f00e5a7dfTimo Sirainen FIXME: we could avoid this stat() and just mark
225e82df5dd1e765f4e52b80c954558f00e5a7dfTimo Sirainen this check in the uidlist and check it at next
8c9e48cd6de80da0fa32b9c0dee003472c9a7c0dTimo Sirainen const char *str;
de4e3a2e1e8f82b2d3226c090b71b518b43bf9cfTimo Sirainen "Maildir sync: UID inserted in the "
a10ed8c47534b4c6b6bf2711ccfe577e720a47b4Timo Sirainen "middle of mailbox "
a10ed8c47534b4c6b6bf2711ccfe577e720a47b4Timo Sirainen "(%u > %u, file = %s)",
b12b6da6f084212d42421a28ae329eae79751c42Timo Sirainen if ((rec->flags & MAIL_INDEX_MAIL_FLAG_DIRTY) != 0) {
b12b6da6f084212d42421a28ae329eae79751c42Timo Sirainen /* we haven't been able to update maildir with this
b12b6da6f084212d42421a28ae329eae79751c42Timo Sirainen record's flag changes. don't sync them. */
b12b6da6f084212d42421a28ae329eae79751c42Timo Sirainen maildir_filename_get_flags(filename, &flags, keywords);
3ec3632e12638396944e854a8f7a2bc6c67b981dTimo Sirainen if ((uint8_t)flags != (rec->flags & MAIL_FLAGS_MASK) ||
3ec3632e12638396944e854a8f7a2bc6c67b981dTimo Sirainen /* FIXME: this is wrong if there's syncs later.
3ec3632e12638396944e854a8f7a2bc6c67b981dTimo Sirainen it gets fixed in next sync however.. */
3ec3632e12638396944e854a8f7a2bc6c67b981dTimo Sirainen mail_index_update_flags(trans, seq, MODIFY_REPLACE,
b12b6da6f084212d42421a28ae329eae79751c42Timo Sirainen /* expunge the rest */
b12b6da6f084212d42421a28ae329eae79751c42Timo Sirainen for (seq++; seq <= hdr->messages_count; seq++)
b12b6da6f084212d42421a28ae329eae79751c42Timo Sirainen if (mail_index_transaction_commit(trans, &seq, &offset) < 0)
b12b6da6f084212d42421a28ae329eae79751c42Timo Sirainen else if (seq != 0) {
de4e3a2e1e8f82b2d3226c090b71b518b43bf9cfTimo Sirainen /* now, sync the index */
8c9e48cd6de80da0fa32b9c0dee003472c9a7c0dTimo Sirainen while ((ret = mail_index_sync_next(sync_ctx.sync_ctx,
8c9e48cd6de80da0fa32b9c0dee003472c9a7c0dTimo Sirainen if (maildir_sync_record(ibox, &sync_ctx) < 0) {
de4e3a2e1e8f82b2d3226c090b71b518b43bf9cfTimo Sirainen sync_stamp = ibox->dirty_cur_time != 0 ? 0 : ibox->last_cur_mtime;
de4e3a2e1e8f82b2d3226c090b71b518b43bf9cfTimo Sirainen if (mail_index_sync_end(sync_ctx.sync_ctx, sync_stamp, 0) < 0)
de4e3a2e1e8f82b2d3226c090b71b518b43bf9cfTimo Sirainenstatic int maildir_sync_context(struct maildir_sync_context *ctx)
de4e3a2e1e8f82b2d3226c090b71b518b43bf9cfTimo Sirainen if (maildir_sync_quick_check(ctx, &new_changed, &cur_changed) < 0)
8c9e48cd6de80da0fa32b9c0dee003472c9a7c0dTimo Sirainen maildir_uidlist_sync_init(ctx->ibox->uidlist, ctx->partial);
e15b305e90c9834734ccf35ed78f0ad29d570ee9Timo Sirainen /* finish uidlist syncing, but keep it still locked */
421d30619384e72a27e2a5d13ff6525aff4d17feTimo Sirainen if (maildir_uidlist_sync_finish(ctx->uidlist_sync_ctx)) {
a757f31393b9d6fc7760a9dec8363404ab3ae576Timo Sirainen ret = maildir_uidlist_sync_deinit(ctx->uidlist_sync_ctx);
90adcaa0a00eba29b7fbd50ca66be11c8d086d6aTimo Sirainenstatic int maildir_sync_context_readonly(struct maildir_sync_context *ctx)
87460b08cb97b31cde640d4975a6aa2c1d0e7226Timo Sirainen maildir_uidlist_sync_init(ctx->ibox->uidlist, FALSE);
ad48319996942463675b53877092ab7e13a7a75aTimo Sirainen ret = maildir_uidlist_sync_deinit(ctx->uidlist_sync_ctx);
0cb2e8eb55e70f8ebe1e8349bdf49e4cbe5d8834Timo Sirainenint maildir_storage_sync_readonly(struct index_mailbox *ibox)
int ret;
return ret;
int ret;
if (ret < 0)