global: start relying on str_free(NULL) being a no-op Cleanup performed with the following semantic patch: @@ expression E; @@ - if (E != NULL) { - str_free(&E); - } + str_free(&E);

Updated copyright notices to include the year 2018.

fts-solr: Explicitly ask for XML responses With recent solr versions (7.0+) the default response writer changed from XML to JSON. This commit sets the "wt" query parameter for all requests, so that the right response type is written.

fts-solr: Don't double-escape headers.

fts-solr: Fixed HTTPS support by properly propagating SSL client settings from mail_user.

fts-solr: Escape {} chars when sending queries Fixes: java.lang.IllegalArgumentException: Invalid character found in the request target. The valid characters are defined in RFC 7230 and RFC 3986 Based on patch by Michael Welsh Duggan

Updated copyright notices to include the year 2017.

global: Replaced all instances of memset(p, 0, sizeof(*p)) with the new i_zero() macro. Used the following script: C_FILES=`git ls-files *.c` H_FILES=`git ls-files *.h` for F in "$C_FILES $H_FILES"; do echo "$F" perl -p -i -e 's/safe_memset\(&\(?([^,]*)\)?,\s*0,\s*sizeof\(\g1\)\)/i_zero_safe(&$1)/g' $F perl -p -i -e 's/safe_memset\(([^,]*),\s*0,\s*sizeof\(\*\g1\)\)/i_zero_safe($1)/g' $F perl -p -i -e 's/memset\(&\(?([^,]*)\)?,\s*0,\s*sizeof\(\g1\)\)/i_zero(&$1)/g' $F perl -p -i -e 's/memset\(([^,]*),\s*0,\s*sizeof\(\*\g1\)\)/i_zero($1)/g' $F done

global: Change string position/length from unsigned int to size_t Mainly to avoid truncating >4GB strings, which might potentially cause some security holes. Normally there are other limits, which prevent such excessive strings from being created in the first place. I'm sure this didn't find everything. Maybe everything could be found with compiler warnings. -Wconversion kind of does it, but it gives way too many unnecessary warnings. These were mainly found with: grep " = strlen" egrep "unsigned int.*(size|len)"

fts-backend-solr: Check that box is not NULL fts_backend_update_deinit will set box to NULL and call update_set_mailbox then.

Fixed searching multiple mailboxes When searching multiple mailboxes, a condition to search these mailboxes is added to the query string. However, this condition has to be separated from the preceding condition by a space (a '+' in this case, as it's encoded); otherwise, Solr considers it to be part of the previous condition, in which case it will fail to parse it properly. Therefore, this patch adds that one missing character so it works again.

global: unsigned int:1 -> bool:1 perl -i -pe 's/unsigned int ([^,:;]+):1;/bool $1:1;/' **/*.[ch]

lib-storage: add no_fts in struct mail_search_arg When flag is set, FTS will not be performed. fts plugin: handle no_fts flag No fts will be performed once flag is set. Original patch from Timo Sirainen <timo.sirainen@dovecot.fi>

fts-solr: Changed default operator to AND to fix multi-word queries. Previously for example searching from:foo@example.com was expanded to foo OR example OR com. Now it's foo AND example AND com, which makes a lot more sense.

global: freshen copyright git ls-files | xargs perl -p -i -e 's/(\d+)-201[0-5]/$1-2016/g;s/ (201[0-5]) Dovecot/ $1-2016 Dovecot/'

fts-solr: Fixed sending empty parameters. Solr probably doesn't do anything useful with them, but we shouldn't get 400 Bad Request errors.

fts-solr: Fixed escaping query parameters. Solr documentation says that "quoted string" would already work without escaping, but that doesn't seem to be true (we were also missing the \" escaping there). So we'll now escape all the special characters without quotes around it, which seems to work. Also added '/' to list of special characters, which is used by Solr 4.0.

fts-solr: Flush Solr indexing HTTP requests every 1000 mails. This avoids the HTTP request from becoming too huge.

fts-solr: Fix to previous memory leak fix - second request was failing. Duplicated data was sent to Solr because string wasn't truncated.

fts-solr: Fixed memory leak when indexing multiple mailboxes in one update context.

fts-solr: "highest UID for mailbox" lookup was actually returning "highest UID for user" If the default operator was OR instead of AND. This affected indexing mails in newly created mailboxes.

fts-solr: Avoid sending too large queries to Solr due to listing wanted mailboxes. This is mainly a problem if there are a lot of mailboxes and "All Mails" virtual mailbox. For now hardcoded to send max 10 mailboxes in the query, afterwards it just returns matches in all the mailboxes and we'll filter out the unwanted mailboxes.

various: use new uni_utf8_get_char*() interface No need for additional uni_utf8_char_bytes() calls if you can parse and know the size with one call. Signed-off-by: Phil Carmody <phil@dovecot.fi>

fts-solr: laxer check of uni_utf8_get_char_n() return value If uni_utf8_get_char*() were changed to return the number of bytes in the character on success, then all we care about is it being > 0 (i.e. not error, not truncated). Signed-off-by: Phil Carmody <phil@dovecot.fi>

fts-solr: Fixed memory leak at user deinit.

fts-solr: Crashfixes

fts-solr: Don't update "last indexed UID" if we couldn't actually add the mails to Solr.

fts-solr: fts_solr=use_libfts send data to Solr via space-separated tokens. In this case Solr should be configured to not do any kind of filtering and use only WhitespaceTokenizerFactory.

global: freshen copyright Robomatically: git ls-files | xargs perl -p -i -e 's/(\d+)-201[0-4]/$1-2015/g;s/ (201[0-4]) Dovecot/ $1-2015 Dovecot/' Happy 2015 everyone! Signed-off-by: Phil Carmody <phil@dovecot.fi>

doveadm fts rescan: For virtual namespaces just mark the last indexed UID to 0.

fts: Added fts_no_autofuzzy setting to require exact matches for found results. This is done by using the FTS search results as only filters on which the regular non-FTS search is done.

Updated copyright notices to include year 2014.

fts-solr: Fixed support for multiple Solr hosts when same process handles multiple users. E.g. indexer-worker was always using only the first user's Solr host.

fts-solr: Do only soft commits. Use a cronjob to do hard commits.

fts-solr: Don't crash if fts_solr setting is invalid.

fts-solr: Truncate header if it's over 1 MB. Previously we just warned about it.

fts-solr: Use built-in lib-http instead of libcurl. Removed libcurl dependency.

Oops :) Update copyrights to 2013 without breaking all .c files.

Updated copyright notices to include year 2013.