sysdb: do not use objectClass for users and groups The majority of the object in the SSSD cache are users and groups. If there are many user and groups in the cache the index objects of the objectclass attributes 'user' and 'group' become large because the must hold references to all objects of those object classes. As a result the management of these index objects becomes costly because they must be parsed and split apart quite often. Additionally they are mostly useless because user and groups are lookup up by more specific attributes in general. Only when enumerating all user or groups this kind of index might be useful. There are two way of removing this kind of index from the user and group objects. Either by removing objectClass from the list of indexes and add a new attribute to all other type of object we want and index for. Or by replacing objectClass with a different attribute for the user and group objects. After some testing I think the latter one is the more reliable one and implemented it in this patch. Related to https://pagure.io/SSSD/sssd/issue/3503 Reviewed-by: Fabiano Fidêncio <fidencio@redhat.com> Reviewed-by: Jakub Hrozek <jhrozek@redhat.com>

LDAP_ID_CLEANUP: Use sysdb_search_*_by_timestamp() Use the appropriate methods for searching users and groups bv timestamp. Resolves: https://pagure.io/SSSD/sssd/issue/3369 Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> Reviewed-by: Jakub Hrozek <jhrozek@redhat.com>

LDAP: Sanitize group dn before using in filter Each string should be sanitized(rfc4515) before using ldbsearch. A group dn was not sanitized in the function cleanup_groups. Resolves: https://fedorahosted.org/sssd/ticket/2744 Reviewed-by: Pavel Březina <pbrezina@redhat.com>

Partially revert "LDAP: sanitize group name when used in filter" This reverts commit e2e334b2f51118cb14c7391c4e4e44ff247ef638. + temporary disable unit test Reviewed-by: Pavel Březina <pbrezina@redhat.com>

Revert "LDAP: end on ENOMEM" This reverts commit f31a57321fc0a2390bb0d6030053c49787e5e587. It blocked reverting commit e2e334b2f51118cb14c7391c4e4e44ff247ef638. Reviewed-by: Pavel Březina <pbrezina@redhat.com>

cleanup task: Expire all memberof targets when removing user Ticket: https://fedorahosted.org/sssd/ticket/2676 When user is removed from cache during cleanup task, mark all his memberof targets as expired to refresh member/ghost attributes on next request. Reviewed-by: Pavel Reichl <preichl@redhat.com>

LDAP: end on ENOMEM Reviewed-by: Pavel Březina <pbrezina@redhat.com>

LDAP: minor improvements in ldap id cleanup Reviewed-by: Pavel Březina <pbrezina@redhat.com>

LDAP: sanitize group name when used in filter cleanup_groups() uses DN of group in filter for ldbsearch. But the name might contain characters with special meaning for filtering like - "*()\/" Resolves: https://fedorahosted.org/sssd/ticket/2744 Reviewed-by: Pavel Březina <pbrezina@redhat.com>

SDAP: rename SDAP_CACHE_PURGE_TIMEOUT Enum member SDAP_CACHE_PURGE_TIMEOUT has counter-intuitive name as it's used to access 'ldap_purge_cache_timeout' option. SDAP_CACHE_PURGE_TIMEOUT is more fitting name. Reviewed-by: Petr Cech <pcech@redhat.com>

LDAP: Handle ENOENT better in the cleanup task The cleanup task handled both count=0 and ret=ENOENT separately which makes no sense, the count=0 handler was dead code previously. Set count=0 on ENOENT instead to just bubble through the DEBUG message gracefully as well. Reviewed-by: Pavel Reichl <preichl@redhat.com>

LDAP: Add better DEBUG messages to the cleanup task Some failures would shortcut to the done handler without telling us anything about why it failed. This commit decorates the cleanup task with more DEBUG statements. Reviewed-by: Pavel Reichl <preichl@redhat.com>

ptask: Add backoff feature to the ptask api. Reviewed-by: Jakub Hrozek <jhrozek@redhat.com> Reviewed-by: Simo Sorce <simo@redhat.com>

ptask: Allow adding random_offset to scheduled execution time Reviewed-by: Jakub Hrozek <jhrozek@redhat.com> Reviewed-by: Simo Sorce <simo@redhat.com>

Update DEBUG* invocations to use new levels Use a script to update DEBUG* macro invocations, which use literal numbers for levels, to use bitmask macros instead: grep -rl --include '*.[hc]' DEBUG . | while read f; do mv "$f"{,.orig} perl -e 'use strict; use File::Slurp; my @map=qw" SSSDBG_FATAL_FAILURE SSSDBG_CRIT_FAILURE SSSDBG_OP_FAILURE SSSDBG_MINOR_FAILURE SSSDBG_CONF_SETTINGS SSSDBG_FUNC_DATA SSSDBG_TRACE_FUNC SSSDBG_TRACE_LIBS SSSDBG_TRACE_INTERNAL SSSDBG_TRACE_ALL "; my $text=read_file(\*STDIN); my $repl; $text=~s/ ^ ( .* \b (DEBUG|DEBUG_PAM_DATA|DEBUG_GR_MEM) \s* \(\s* )( [0-9] )( \s*, ) ( \s* ) ( .* ) $ / $repl = $1.$map[$3].$4.$5.$6, length($repl) <= 80 ? $repl : $1.$map[$3].$4."\n".(" " x length($1)).$6 /xmge; print $text; ' < "$f.orig" > "$f" rm "$f.orig" done Reviewed-by: Jakub Hrozek <jhrozek@redhat.com> Reviewed-by: Stephen Gallagher <sgallagh@redhat.com> Reviewed-by: Simo Sorce <simo@redhat.com>

Make DEBUG macro invocations variadic Use a script to update DEBUG macro invocations to use it as a variadic macro, supplying format string and its arguments directly, instead of wrapping them in parens. This script was used to update the code: grep -rwl --include '*.[hc]' DEBUG . | while read f; do mv "$f"{,.orig} perl -e \ 'use strict; use File::Slurp; my $text=read_file(\*STDIN); $text=~s#(\bDEBUG\s*\([^(]+)\((.*?)\)\s*\)\s*;#$1$2);#gs; print $text;' < "$f.orig" > "$f" rm "$f.orig" done Reviewed-by: Jakub Hrozek <jhrozek@redhat.com> Reviewed-by: Stephen Gallagher <sgallagh@redhat.com> Reviewed-by: Simo Sorce <simo@redhat.com>

SYSDB: Drop the sysdb_ctx parameter - module sysdb_ops (part 2)

dp: convert cleanup task to be_ptask Resolves: https://fedorahosted.org/sssd/ticket/1968

Fix formating of variables with type: size_t

LDAP: Store cleanup timestamp after initial cleanup When the SSSD changes serves (and hence lastUSN) we perform a cleanup as well. However, after recent changes, we didn't set the cleanup timestamp correctly, which made the lastUSN logic fail.

LDAP: Make the cleanup task reusable for subdomains Instead of always performing the cleanup on the main domain, the task now accepts a sdap_domain structure to perform the cleanup on. This change will make the cleanup task reusable for subdomains.

LDAP: Make cleanup synchronous The LDAP cleanup request was asynchronous for no good reason, probably a leftover from the days of async sysdb. This patch makes it sychronous again, removing a lot of uneeded code.

Remove sysdb as a be context structure member The sysdb context is already available through the 'domain' structure.

Add domain argument to sysdb_delete_group() Also remove sysdb_delete_domgroup()

Add domain argument to sysdb_search_groups()

Add domain argument to sysdb_delete_user() Also remove sysdb_delete_domuser()

Add domain arg to sysdb_search_users()

Do not remove a group if it has members from subdomains Currently it is only checked if an expired group still has members of the local domain. If not, the group is delete from the cache. With this patch the whole cache, i.e. including subdomains, is searched for members.

Unify usage of sysdb transactions Removing bad examples of usage of sysdb_transaction_start/commit/end functions and making it more consistent (all files except of src/db/sysdb_*.c).

Ghost members - various small changes

Cleanup: Remove unused parameters

sysdb refactoring: memory context deleted This patch deletes memory context parameter in those places in sysdb where it is not necessary. The code using modified functions has been updated. Tests updated as well.

sysdb refactoring: deleted domain variables in sysdb API The patch also updates code using modified functions. Tests have also been adjusted.

Non-posix group processing - ldap provider and nss responder

Fix cleanup transaction Without setting in_transaction=true, if the sysdb operations threw an error, we wouldn't cancel the transaction.

Wrap cleanup task in a sysdb transaction

Don't clean up groups for which a user has it as primary GID We were cleaning up all groups that were expired and for which there existed no user with memberOf: <thegroup> as an attribute. This patch modifies the search to also check for cached users with this group's GID as their primary GID. Fixes https://fedorahosted.org/sssd/ticket/624

sysdb interface for adding fake users

Properly check that the timeout event was created for cleanup/enum We need to make sure that if we didn't create the timeout, that we cancel the request so there's no chance of ending up with two enumerations/cleanups running simultaneously. We'll attempt to reschedule later, if possible. https://fedorahosted.org/sssd/ticket/524

sysdb: convert sysdb_search_groups

sysdb: delete sysdb_delete_group

sysdb: convert sysdb_delete_user

sysdb: convert sysdb_search_users

Remove a check that was left behind When refactoring talloc_asprintf calls a check was left behind that cased the backend to go offline immediately.

Better cleanup task handling Implements a different mechanism for cleanup task. Instead of just deleting expired entries, this patch adds a new option account_cache_expiration for domains. If an entry is expired and the last login was more days in the past that account_cache_expiration, the entry is deleted. Groups are deleted if they are expired and and no user references them (no user has memberof: attribute pointing at that group). The parameter account_cache_expiration is not LDAP-specific, so that other future backends might use the same timeout setting. Fixes: #391

Do not check entries during cleanup task Do not attempt to validate expired entries in cache, just delete them. Also increase the cache timeouts. Fixes: #331

Do not schedule enumeration after a cleanup

Rename server/ directory to src/ Also update BUILD.txt