sssd_client: Do not use removed memory cache Resolves: https://fedorahosted.org/sssd/ticket/2726 Reviewed-by: Michal Židek <mzidek@redhat.com>

sss_client: Fix underflow of active_threads If the memory cache was not initialized and there was a failure in initialisation of memory cache context (e.g. memory cache file does not exist) then mc_context had to be destroyed to release resources. However the count of active threads in sss_cli_mc_ctx is already higher than zero because current thread is working wih the mc_context. But this counter was zero-ed with memset in sss_nss_mc_destroy_ctx due to issue with initialisation of memory cache. Then we have to decrease counter of active thread in function sss_nss_mc_get_ctx because initialisation of mc failed. And the result of this decrement is underflow of counter. Related to: https://fedorahosted.org/sssd/ticket/2726 Reviewed-by: Michal Židek <mzidek@redhat.com>

sss_client: Use unique lock for memory cache Previously the sma lock was used as for communication with responder. However it would cause a deadlock in case of re-checking memcache after acquiring the lock and before communication with responder.. Required by: https://fedorahosted.org/sssd/ticket/2581 Reviewed-by: Michal Židek <mzidek@redhat.com>

sss_client: Fix race condition in memory cache Thread safe initialisation was fixed in ticket #2380, but there is still race condition in reinitialisation. If caches is invalidated with command sss_cache -U (-G or -E) then client code will need to reinitialize fast memory cache. Let say we have two threads. The 1st thread find out that memory cache should be reinitialized; therefore the fast memory cached is unmapped and context destroyed. In the same time, 2nd thread tried to check header of memory cache whether it is initialized and valid. As a result of previously unmapped memory the 2nd thread access out of bound memory (SEGFAULT). The destroying of fast memory cache cannot be done any time. We need to be sure that there isn't any other thread which uses mmaped memory. The new counter of active threads was added for this purpose. The state of fast memory cache was converted from boolean to three value state (UNINITIALIZED, INITIALIZED, RECYCLED) UNINITIALIZED - the fast memory cache need to be initialized. - if there is a problem with initialisation the state will not change - after successful initialisation, the state will change to INITIALIZED INITIALIZED - if the cahe was invalidated or there is any other problem was detected in memory cache header the state will change to RECYCLED and memory cache IS NOT destroyed. RECYCLED - nothing will be done is there are any active threads which may use the data from mmaped memory - if there aren't active threads the fast memory cahe is destroyed and state is changed to UNINITIALIZED. https://fedorahosted.org/sssd/ticket/2445 Reviewed-by: Michal Židek <mzidek@redhat.com>

sss_client: Extract destroying of mmap cache to function Reviewed-by: Michal Židek <mzidek@redhat.com>

sss_client: Fix "struct sss_cli_mc_ctx" reinitialize-on-errors When we have difficulty setting up an sss_cli_mc_ctx structure, we try to clean things up so that we'll be ready to try again the next time we're called. Part of that is closing the descriptor of the file if we've opened it and using memset() to clear the structure. Now that sss_nss_mc_get_ctx() does its work in two phases, and each one may end up doing the cleanup, each needs to be careful to reset the descriptor field so that the new value provided by memset() (0) isn't mistakenly treated as a file which should be closed by the other. Resolves: https://fedorahosted.org/sssd/ticket/2409 Reviewed-by: Simo Sorce <simo@redhat.com>

sss_client: thread safe initialisation of sss_cli_mc_ctx In multi threaded application, it may happen that more threads will call function getpwuid(or similar) and sss client will not have initialized structure for fast memory cache. This structure is initialized just once. There isn't any problem with multi threaded application after successful initialisation. The race condition will happen if more threads try to initialise structure sss_cli_mc_ctx in function sss_nss_mc_get_ctx (ctx->initialized is false) It takes some time to initialise mmap cache: open file, get file size, mmap file, initialize structure sss_cli_mc_ctx. One of problems is that file with memory cache can be opened more times (file descriptor leak), but the race condition is with initialising structure sss_cli_mc_ctx. One tread will start to initialise this structure; another thread will think that structure is already initialised and will check consistency of this structure. It will fail because 1st thread did not finish initialisation. Therefore 2nd thread will return EINVAL and will do clean up in done section: munmap, close file and reset structure data. The 1st thread will finish an try to use memory cache, but structure was zero initialised by 2nd thread and it will cause dereference of NULL pointer in 1st thread (SIGSEGV) or dividing by zero in murmurhash function(SIGFPE) Function sss_nss_mc_get_ctx was split into two parts for simplification of locking and unlocking. The locking is used only in new static function sss_nss_mc_init_ctx. This function will not be called very often therefore the same mutex is used as in other nss functions. Resolves: https://fedorahosted.org/sssd/ticket/2380 Reviewed-by: Michal Židek <mzidek@redhat.com> Reviewed-by: Sumit Bose <sbose@redhat.com>

mmap_cache: Use two chains for hash collision. struct sss_mc_rec had two hash members (hash1 and hash2) but only one next member. This was a big problem in case of higher probability of hash collision. structure sss_mc_rec will have two next members (next1, next2) with this patch. next1 is related to hash1 and next2 is related to hash1. Iterating over chains is changed, because we need to choose right next pointer. Right next pointer will be chosen after comparing record hashes. This behaviour is wrapped in function sss_mc_next_slot_with_hash. Adding new record to chain is also changed. The situation is very similar to iterating. We need to choose right next pointer (next1 or next2). Right next pointer will be chosen after comparing record hashes. Adding reference to next slot is wrapped in function sss_mc_chain_slot_to_record_with_hash Size of structure sss_mc_rec was increased from 32 bytes to 40 bytes. Resolves: https://fedorahosted.org/sssd/ticket/2049

Rename _SSS_MC_SPECIAL If the environment variable _SSS_MC_SPECIAL is set to "NO", the mmap cache is skipped in the client code. The name is not very descriptive. This patch renames the variable to SSS_NSS_USE_MEMCACHE.

Fix sss_client breakage. Adding missing dependencies for linker. Missing dependency was introduced by commit 22d381367c27910fe82f476a76b9f4ede555e35a in changed file src/sss_client/nss_mc_common.c All function declaration for io.c was moved from util.h to separate file io.h, https://fedorahosted.org/sssd/ticket/1838

Reuse sss_open_cloexec at other places in code. Functions open_cloexec and openat_cloexec were renamed with prefix "sss_" and moved to separete file. Replacing duplicated code of function sss_open_cloexec everywhere in the source code. https://fedorahosted.org/sssd/ticket/1794

Potential resource leak in sss_nss_mc_get_record https://fedorahosted.org/sssd/ticket/1748

explicit null dereferenced in sss_nss_mc_get_record() https://fedorahosted.org/sssd/ticket/1724

Add a macro to copy with barriers We have 2 places where we memcpy memory and need barriers protection. Use a macro so we can consolidate code in one place. Second fix for: https://fedorahosted.org/sssd/ticket/1694

Add memory barrier to mmap cache client code loop Fixes https://fedorahosted.org/sssd/ticket/1694

Fix errors reported by rpmlint

Add close on exec support for old platforms Older platfroms like RHEL5 do not have support for O_CLOEXC and need an explicit fcntl after the fd is created. Add it conditionally so it can be clearly removed once we declared those platfroms obsolete and unsupported.

Do not leak file descriptors in client libs. We need to make sure the mc socket is not leaked otherwise child processes will pile up leaked file descriptors. Add O_CLOEXEC when opening the cache.

sss_client: Add common shared memory cache utils