mail-cache-decisions.c revision 1b97a59edb073e9a89ac43a21a9abe5d590d4a56
72ae534f5aef6d2e5d3f2f51299aede5abf9687eJakub Hrozek/* Copyright (C) 2004 Timo Sirainen */
72ae534f5aef6d2e5d3f2f51299aede5abf9687eJakub Hrozek Users can be divided to three groups:
72ae534f5aef6d2e5d3f2f51299aede5abf9687eJakub Hrozek 1. Most users will use only a single IMAP client which caches everything
72ae534f5aef6d2e5d3f2f51299aede5abf9687eJakub Hrozek locally. For these users it's quite pointless to do any kind of caching
72ae534f5aef6d2e5d3f2f51299aede5abf9687eJakub Hrozek as it only wastes disk space. That might also mean more disk I/O.
72ae534f5aef6d2e5d3f2f51299aede5abf9687eJakub Hrozek 2. Some users use multiple IMAP clients which cache everything locally.
72ae534f5aef6d2e5d3f2f51299aede5abf9687eJakub Hrozek These could benefit from caching until all clients have fetched the
72ae534f5aef6d2e5d3f2f51299aede5abf9687eJakub Hrozek data. After that it's useless.
72ae534f5aef6d2e5d3f2f51299aede5abf9687eJakub Hrozek 3. Some clients don't do permanent local caching at all. For example
72ae534f5aef6d2e5d3f2f51299aede5abf9687eJakub Hrozek Pine and webmails. These clients would benefit from caching everything.
72ae534f5aef6d2e5d3f2f51299aede5abf9687eJakub Hrozek Some locally caching clients might also access some data from server
72ae534f5aef6d2e5d3f2f51299aede5abf9687eJakub Hrozek again, such as when searching messages. They could benefit from caching
72ae534f5aef6d2e5d3f2f51299aede5abf9687eJakub Hrozek only these fields.
72ae534f5aef6d2e5d3f2f51299aede5abf9687eJakub Hrozek After thinking about these a while, I figured out that people who care
72ae534f5aef6d2e5d3f2f51299aede5abf9687eJakub Hrozek about performance most will be using Dovecot optimized LDA anyway
72ae534f5aef6d2e5d3f2f51299aede5abf9687eJakub Hrozek which updates the indexes/cache immediately. In that case even the first
72ae534f5aef6d2e5d3f2f51299aede5abf9687eJakub Hrozek user group would benefit from caching the same way as second group. LDA
72ae534f5aef6d2e5d3f2f51299aede5abf9687eJakub Hrozek reads the mail anyway, so it might as well extract some information
72ae534f5aef6d2e5d3f2f51299aede5abf9687eJakub Hrozek about it and store them into cache.
72ae534f5aef6d2e5d3f2f51299aede5abf9687eJakub Hrozek So, group 1. and 2. could be optimally implemented by keeping things
72ae534f5aef6d2e5d3f2f51299aede5abf9687eJakub Hrozek cached only for a while. I thought a week would be good. When cache file
72ae534f5aef6d2e5d3f2f51299aede5abf9687eJakub Hrozek is compressed, everything older than week will be dropped.
72ae534f5aef6d2e5d3f2f51299aede5abf9687eJakub Hrozek But how to figure out if user is in group 3? One quite easy rule would
63b8e826f62d2e8930c872de7d4cc8b5bc15d4a4Sumit Bose be to see if client is accessing messages older than a week. But with
63b8e826f62d2e8930c872de7d4cc8b5bc15d4a4Sumit Bose only that rule we might have already dropped useful cached data. It's
60e54ce78177b72ae82b8986e37dc4d09fce5681Lukas Slebodnik not very nice if we have to read and cache it twice.
60e54ce78177b72ae82b8986e37dc4d09fce5681Lukas Slebodnik Most locally caching clients always fetch new messages (all but body)
63b8e826f62d2e8930c872de7d4cc8b5bc15d4a4Sumit Bose when they see them. They fetch them in ascending order. Noncaching
63b8e826f62d2e8930c872de7d4cc8b5bc15d4a4Sumit Bose clients might fetch messages in pretty much any order, as they usually
72ae534f5aef6d2e5d3f2f51299aede5abf9687eJakub Hrozek don't fetch everything they can, only what's visible in screen. Some
72ae534f5aef6d2e5d3f2f51299aede5abf9687eJakub Hrozek will use server side sorting/threading which also makes messages to be
72ae534f5aef6d2e5d3f2f51299aede5abf9687eJakub Hrozek fetched in random order. Second rule would then be that if a session
72ae534f5aef6d2e5d3f2f51299aede5abf9687eJakub Hrozek doesn't fetch messages in ascending order, the fetched field type will
51b5e1475b3e0b7acac34ed382cfaca8411883a4Jakub Hrozek be permanently cached.
72ae534f5aef6d2e5d3f2f51299aede5abf9687eJakub Hrozek So, we have three caching decisions:
72ae534f5aef6d2e5d3f2f51299aede5abf9687eJakub Hrozek 1. Don't cache: Clients have never wanted the field
72ae534f5aef6d2e5d3f2f51299aede5abf9687eJakub Hrozek 2. Cache temporarily: Clients want this only once
72ae534f5aef6d2e5d3f2f51299aede5abf9687eJakub Hrozek 3. Cache permanently: Clients want this more than once
51b5e1475b3e0b7acac34ed382cfaca8411883a4Jakub Hrozek Different mailboxes have different decisions. Different fields have
51b5e1475b3e0b7acac34ed382cfaca8411883a4Jakub Hrozek different decisions.
51b5e1475b3e0b7acac34ed382cfaca8411883a4Jakub Hrozek There are some problems, such as if a client accesses message older than
30dd3f3e063dded0ec9f58bc2535a94727d8e96dJakub Hrozek a week, we can't know if user just started using a new client which is
30dd3f3e063dded0ec9f58bc2535a94727d8e96dJakub Hrozek just filling its local cache for the first time. Or it might be a
30dd3f3e063dded0ec9f58bc2535a94727d8e96dJakub Hrozek client user hasn't just used for over a week. In these cases we
295c8e301e31f9bf27d921f80c14dfa5864b2383Lukas Slebodnik shouldn't have marked the field to be permanently cached. User might
30dd3f3e063dded0ec9f58bc2535a94727d8e96dJakub Hrozek also switch clients from non-caching to caching.
30dd3f3e063dded0ec9f58bc2535a94727d8e96dJakub Hrozek So we should re-evaluate our caching decisions from time to time. This
63b8e826f62d2e8930c872de7d4cc8b5bc15d4a4Sumit Bose is done by checking the above rules constantly and marking when was the
63b8e826f62d2e8930c872de7d4cc8b5bc15d4a4Sumit Bose last time the decision was right. If decision hasn't matched for two
63b8e826f62d2e8930c872de7d4cc8b5bc15d4a4Sumit Bose months, it's changed. I picked two months because people go to at least
63b8e826f62d2e8930c872de7d4cc8b5bc15d4a4Sumit Bose one month vacations where they might still be reading mails, but with
63b8e826f62d2e8930c872de7d4cc8b5bc15d4a4Sumit Bose different clients.
63b8e826f62d2e8930c872de7d4cc8b5bc15d4a4Sumit Bosevoid mail_cache_decision_state_update(struct mail_cache_view *view,
63b8e826f62d2e8930c872de7d4cc8b5bc15d4a4Sumit Bose if (cache->fields[field].field.decision != MAIL_CACHE_DECISION_TEMP) {
63b8e826f62d2e8930c872de7d4cc8b5bc15d4a4Sumit Bose /* a) forced decision
63b8e826f62d2e8930c872de7d4cc8b5bc15d4a4Sumit Bose b) not cached, mail_cache_decision_add() will handle this
63b8e826f62d2e8930c872de7d4cc8b5bc15d4a4Sumit Bose c) permanently cached already, okay. */
63b8e826f62d2e8930c872de7d4cc8b5bc15d4a4Sumit Bose /* see if we want to change decision from TEMP to YES */
63b8e826f62d2e8930c872de7d4cc8b5bc15d4a4Sumit Bose if (mail_index_lookup_uid(view->view, seq, &uid) < 0)
63b8e826f62d2e8930c872de7d4cc8b5bc15d4a4Sumit Bose if (ioloop_time - cache->fields[field].last_used > 3600*24) {
63b8e826f62d2e8930c872de7d4cc8b5bc15d4a4Sumit Bose /* update last_used about once a day */
63b8e826f62d2e8930c872de7d4cc8b5bc15d4a4Sumit Bose cache->fields[field].last_used = (uint32_t)ioloop_time;
63b8e826f62d2e8930c872de7d4cc8b5bc15d4a4Sumit Bose /* a) nonordered access within this session. if client doesn't
63b8e826f62d2e8930c872de7d4cc8b5bc15d4a4Sumit Bose request messages in growing order, we assume it doesn't
63b8e826f62d2e8930c872de7d4cc8b5bc15d4a4Sumit Bose have a permanent local cache.
63b8e826f62d2e8930c872de7d4cc8b5bc15d4a4Sumit Bose b) accessing message older than one week. assume it's a
63b8e826f62d2e8930c872de7d4cc8b5bc15d4a4Sumit Bose client with no local cache. if it was just a new client
63b8e826f62d2e8930c872de7d4cc8b5bc15d4a4Sumit Bose generating the local cache for the first time, we'll
63b8e826f62d2e8930c872de7d4cc8b5bc15d4a4Sumit Bose drop back to TEMP within few months. */
63b8e826f62d2e8930c872de7d4cc8b5bc15d4a4Sumit Bose cache->fields[field].field.decision = MAIL_CACHE_DECISION_YES;
63b8e826f62d2e8930c872de7d4cc8b5bc15d4a4Sumit Bosevoid mail_cache_decision_add(struct mail_cache_view *view, uint32_t seq,
63b8e826f62d2e8930c872de7d4cc8b5bc15d4a4Sumit Bose unsigned int field)
63b8e826f62d2e8930c872de7d4cc8b5bc15d4a4Sumit Bose if (cache->fields[field].field.decision != MAIL_CACHE_DECISION_NO) {
63b8e826f62d2e8930c872de7d4cc8b5bc15d4a4Sumit Bose /* a) forced decision
63b8e826f62d2e8930c872de7d4cc8b5bc15d4a4Sumit Bose b) we're already caching it, so it just wasn't in cache */
63b8e826f62d2e8930c872de7d4cc8b5bc15d4a4Sumit Bose /* field used the first time */
63b8e826f62d2e8930c872de7d4cc8b5bc15d4a4Sumit Bose cache->fields[field].field.decision = MAIL_CACHE_DECISION_TEMP;