SDAP: rename SDAP_CACHE_PURGE_TIMEOUT Enum member SDAP_CACHE_PURGE_TIMEOUT has counter-intuitive name as it's used to access 'ldap_purge_cache_timeout' option. SDAP_CACHE_PURGE_TIMEOUT is more fitting name. Reviewed-by: Petr Cech <pcech@redhat.com>

LDAP: Remove dead assignment The variable ret was not read when function sysdb_has_enumerated returned ENOENT. Just boolean variable has_enumerated need to be changed. This dead assignment caused warning from clang static analyser Value stored to 'ret' is never read Reviewed-by: Pavel Reichl <preichl@redhat.com>

LDAP: disable the cleanup task by default Resolves: https://fedorahosted.org/sssd/ticket/2627 The cleanup task was designed to keep the cache size within certain limits. This is how it roughly works now: - find users who have never logged in by default. If account_cache_expiration is set, find users who loggged in later than account_cache_expiration - delete the matching set of users - find groups that have no members - delete the matching set of groups So unless account_cache_expiration is set to something sensible, only empty groups and expired users who never logged in are removed and that's quite a corner case. The above effectivelly walks the whole database, especially the groups step is quite slow with a huge database. The whole cleanup task also runs in a single sysdb transaction, which means all other transactions are blocked while the cleanup task crunches the database. Reviewed-by: Lukáš Slebodník <lslebodn@redhat.com>

enumeration: fix talloc context If for some reason ptask fails (e.g. timeout), req is talloc freed but because subreq is attached to ectx which is permanent it is finished anyway. Then a crash occures when we are trying to access callback data. The same happens in sdap_dom_enum_ex_send. Resolves: https://fedorahosted.org/sssd/ticket/2611 Reviewed-by: Pavel Reichl <preichl@redhat.com>

SYSDB: sysdb_get_bool() return ENOENT & unit tests sysdb_get_bool() return ENOENT if no result is found. Unit test for sysdb_get_bool() & sysdb_set_bool() was added. This patch also fixes ldap_setup_enumeration() to handle ENOENT returned by sysdb_has_enumerated(). Resolves: https://fedorahosted.org/sssd/ticket/1991 Reviewed-by: Lukáš Slebodník <lslebodn@redhat.com>

ptask: Add backoff feature to the ptask api. Reviewed-by: Jakub Hrozek <jhrozek@redhat.com> Reviewed-by: Simo Sorce <simo@redhat.com>

ptask: Allow adding random_offset to scheduled execution time Reviewed-by: Jakub Hrozek <jhrozek@redhat.com> Reviewed-by: Simo Sorce <simo@redhat.com>

Make DEBUG macro invocations variadic Use a script to update DEBUG macro invocations to use it as a variadic macro, supplying format string and its arguments directly, instead of wrapping them in parens. This script was used to update the code: grep -rwl --include '*.[hc]' DEBUG . | while read f; do mv "$f"{,.orig} perl -e \ 'use strict; use File::Slurp; my $text=read_file(\*STDIN); $text=~s#(\bDEBUG\s*\([^(]+)\((.*?)\)\s*\)\s*;#$1$2);#gs; print $text;' < "$f.orig" > "$f" rm "$f.orig" done Reviewed-by: Jakub Hrozek <jhrozek@redhat.com> Reviewed-by: Stephen Gallagher <sgallagh@redhat.com> Reviewed-by: Simo Sorce <simo@redhat.com>

LDAP: Pass a private context to enumeration ptask instead of hardcoded connection Previously, the sdap-domain enumeration request used a single connection context to download all the data. Now we'd like to use different connections to download different objects, so the ID context is passed in and the request itself decides which connection to use for the sdap-domain enumeration.

SYSDB: Drop redundant sysdb_ctx parameter from sysdb.c

AD: Download master domain info when enumerating https://fedorahosted.org/sssd/ticket/2068 With the current design, downloading master domain data was tied to subdomains refresh, triggered by responders. But because enumeration is a background task that can't be triggered on its own, we can't rely on responders to download the master domain data and we need to check the master domain on each enumeration request.

LDAP: sdap_id_setup_tasks accepts a custom enum request AD provider will override the default with its own.

LDAP: Convert enumeration to the ptask API https://fedorahosted.org/sssd/ticket/1942 Identity providers other than LDAP need to customize the enumeration in different ways while sharing the way the task is scheduled etc. The easiest way to accomplish it is to leverage the recently introduced ptask framework.

LDAP: Move the ldap enum request to its own reusable module The LDAP enumeration was too closely tied to the LDAP identity provider. Because some providers might need special handling such as refresh the master domain record before proceeding with the enumeration itself, this patch splits the request itself to a separate async request and lets the ldap_id_enum.c module only configure this new request. Also move the enum timestamp to sdap_domain to make the enum tracking per sdap domain. The cleanup timestamp will be moved in another patch.

LDAP: Remove unused constant The constant was not used since Euegene came up with his reconnection logic.

LDAP: Add enum_{users,groups}_recv to follow the tevent_req style The enum code was quite old and predated the tevent_req style. In particular, the enum code was checking tevent state direcly and not using _recv functions or the helper macros we added later. As a consequence, it was not easy to read. This patch adds the standard _recv functions to read the status of the enum requests.

Replace SDAP_ID_MAPPING checks with sdap_idmap_domain_has_algorithmic_mapping Currently the decision if external or algorithmic mapping should be used in the LDAP or AD provider was based on the value of the ldap_id_mapping config option. Since now all information about ID mapping is handled by libsss_idmap the check for this options can be replace with a call which checks the state via libss_idmap. https://fedorahosted.org/sssd/ticket/1961

LDAP: new SDAP domain structure Previously an sdap_id_ctx was always tied to one domain with a single set of search bases. But with the introduction of Global Catalog lookups, primary domain and subdomains might have different search bases. This patch introduces a new structure sdap_domain that contains an sssd domain or subdomain and a set of search bases. With this patch, there is only one sdap_domain that describes the primary domain.

LDAP: sdap_id_ctx might contain several connections With some LDAP server implementations, one server might provide different "views" of the identites on different ports. One example is the Active Directory Global catalog. The provider would contact different view depending on which operation it is performing and against which SSSD domain. At the same time, these views run on the same server, which means the same server options, enumeration, cleanup or Kerberos service should be used. So instead of using several different failover ports or several instances of sdap_id_ctx, this patch introduces a new "struct sdap_id_conn_ctx" that contains the connection cache to the particular view and an instance of "struct sdap_options" that contains the URI. No functional changes are present in this patch, currently all providers use a single connection. Multiple connections will be used later in the upcoming patches.

Remove sysdb as a be context structure member The sysdb context is already available through the 'domain' structure.

Add domain argument to sysdb_has/set_enumerated()

Clean up cache on server reinitialization https://fedorahosted.org/sssd/ticket/734 We successfully detect when the server is reinitialized by testing the new lastUSN value. The maximum USN values are set to zero, but the current cache content remains. This patch removes records that were deleted from the server. It uses the following approach: 1. remove entryUSN attribute from all entries 2. run enumeration 3. remove records that doesn't have entryUSN attribute updated We don't need to do this for sudo rules, they will be refreshed automatically during next smart/full refresh, or when an expired rule is deleted.

Add support for filtering atributes This patch adds support for filtering attributes when constructing attribute list from a map for LDAP query.

LDAP: Add attr_count return value to build_attrs_from_map() This is necessary because in several places in the code, we are appending to the attrs returned from this value, and if we relied on the map size macro, we would be appending after the NULL terminator if one or more attributes were defined as NULL.

LDAP: Add support for enumeration of ID-mapped users and groups

LDAP: Add enumeration support for services

LDAP: Add support for multiple search bases for group enumeration

LDAP: Add support for multiple search bases for user enumeration

LDAP: Convert ldap_*_search_filter Instead of making this a global option for all user lookups, make it only used if the search base is passed without an explicit filter.

LDAP: Support multiple group search bases (non-enumeration, RFC2307)

LDAP: Support multiple user search bases (non-enumeration)

sysdb refactoring: deleted domain variables in sysdb API The patch also updates code using modified functions. Tests have also been adjusted.

Fix returning groups when gidNumber attribute is not ordered https://fedorahosted.org/sssd/ticket/951

Fixed lastUSN checking improvements This patch fixes some issues with setting lastUSN attribute and it adds check against the highest user/group USN after enumeration to keep better track of the real highest USN. Optimal solution here would be to schedule a check of rootDSE entry right after the enumeration finishes, but for the moment this is good enough.

Add user and group search LDAP filter options https://fedorahosted.org/sssd/ticket/647

Require existence of username, uid and gid for user enumeration We will ignore users that do not have these three values.

Require existence of GID number and name in group searches https://fedorahosted.org/sssd/ticket/824

Add ldap_search_enumeration_timeout config option

Add timeout parameter to sdap_get_generic_send()

Start first enumeration immediately Previously, we would wait for ten seconds before starting an enumeration. However, this meant that on the first startup (before we had run our first enumeration) there was a ten-second window where clients would immediately get back a response with no entries instead of blocking until the enumeration completed. With this patch, SSSD will now run an enumeration immediately upon startup. Further startups will retain the ten-second delay so as not to slow down system bootups. https://fedorahosted.org/sssd/ticket/616

ldap: Use USN entries if available. Otherwise fallback to the default modifyTimestamp indicator

Dead assignments cleanup in providers code Dead assignments were deleted. Also prototype of function sdap_access_decide_offline() has been changed, since its return code was never used. Ticket: #586

Use new LDAP connection framework for LDAP user and group enumeration.

Properly check that the timeout event was created for cleanup/enum We need to make sure that if we didn't create the timeout, that we cancel the request so there's no chance of ending up with two enumerations/cleanups running simultaneously. We'll attempt to reschedule later, if possible. https://fedorahosted.org/sssd/ticket/524

Fix segfault in GSSAPI reconnect code Also clean up some duplicated code into a single common routine sdap_account_info_common_done()

Better handle sdap_handle memory from callers. Always just mark the sdap_handle as not connected and let later _send() functions to take care of freeing the handle before reconnecting. Introduce restart functions to avoid calling _send() functions in _done() functions error paths as this would have the same effect as directly freeing the sdap_handle and cause access to freed memory in sdap_handle_release() By freeing sdap_handle only in the connection _recv() function we guarantee it can never be done within sdap_handle_release() but only in a following event.

Rename server/ directory to src/ Also update BUILD.txt