Fix minor spelling mistakes in providers/* Reviewed-by: Lukáš Slebodník <lslebodn@redhat.com>

SDAP: Allow the mpg flag for the main domain This commit allows saving the users in the MPG domain in the SDAP layer. The commit contains the following changes: - abstracts the change where if the primary GID exists in the original object, it is saved instead as the SYSDB_PRIMARY_GROUP_GIDNUM attribute, which will allow the original primary GID to be exposed as a secondary group - if the primary GID does not exist, no SYSDB_PRIMARY_GROUP_GIDNUM is added. This will allow to handle LDAP objects that only contain the UID but no GID. Since this is a new use-case, a test is added later - a branch that handles the above is added to sdap_save_user() also for joined domains that set the MPG flag. Previously, only subdomains were handled. - to allow passing GID=0 to the sysdb layer, the range check is relaxed. Related: https://pagure.io/SSSD/sssd/issue/1872 Reviewed-by: Fabiano Fidêncio <fidencio@redhat.com> Reviewed-by: Pavel Březina <pbrezina@redhat.com>

UTIL: Remove ctype.h from util/util.h ctype.h is not used directly by util/util.h. The header file ctype.h must be included in 32 files and after removing it from util.h it had to be added only to 8 missing files Reviewed-by: Fabiano Fidêncio <fidencio@redhat.com>

LDAP: save non-POSIX users in application domains Related to: https://pagure.io/SSSD/sssd/issue/3310 If a user being saved by the LDAP provider does not have a UID or GID and the domain type is application, we save the user entry as non-POSIX. Reviewed-by: Sumit Bose <sbose@redhat.com>

sdap_get_users_send(): new argument mapped_attrs mapped_attrs can be a list of sysdb_attrs which are not available on the server side but should be store with the cached user entry. This is needed e.g. when the input to look up the user in LDAP is not an attribute which is stored in LDAP but some data where LDAP attributes are extracted from. The current use case is the certificate mapping library which can create LDAP search filters based on content of the certificate. To allow upcoming cache lookup to use the input directly it is stored in the user object in the cache. Related to https://pagure.io/SSSD/sssd/issue/3050 Reviewed-by: Jakub Hrozek <jhrozek@redhat.com> Reviewed-by: Lukáš Slebodník <lslebodn@redhat.com>

SDAP: Don't log an op failure when no users are found When no users are found during the search users process, just log (at TRACEL_ALL level) that zero users were retrieve and avoid logging (at OP_FAILURE level) that a failure has occurred, which may end up misleading admins, giving them the impression that something wrong has happened. Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> Resolves: https://fedorahosted.org/sssd/ticket/3089 Reviewed-by: Pavel Březina <pbrezina@redhat.com>

SDAP: add special handling for IPA Kerberos enterprise principal strings Unfortunately principal aliases with an alternative realm are stored in IPA as the string representation of an enterprise principal, i.e. name\@alt.realm@IPA.REALM. To allow searches with the plain alias 'name@alt.realm' the returned value is converted before it is saved to the cache. Reviewed-by: Jakub Hrozek <jhrozek@redhat.com>

LDAP: allow multiple user principals In general a user can have multiple principals and recent IPA version added support to defined multiple principals. With this patch SSSD does not only store the first but all principals read by LDAP from a server. Resolves https://fedorahosted.org/sssd/ticket/2958 Reviewed-by: Jakub Hrozek <jhrozek@redhat.com>

SDAP: Save user and group aliases qualified When saving users or groups, qualify their names. Otherwise (currently netgroups), store a plain username. Reviewed-by: Jakub Hrozek <jhrozek@redhat.com>

LDAP: Decorate the hot paths in the LDAP provider with systemtap probes During performance analysis, the LDAP provider and especially its nested group code proved to be the place where we spend the most time during account requests. Therefore, I decorated the LDAP provider with systemtap probes to be able to observe where the time is spent. The code allows passing of search properties (base, filter, ...) from marks to probes. Where applicable, the probes pass on these arguments to functions and build a human-readable string representation. Reviewed-by: Lukáš Slebodník <lslebodn@redhat.com>

SDAP: Add error code to debug message Reviewed-by: Michal Židek <mzidek@redhat.com>

SDAP: rename sdap_get_id_specific_filter More generic name is used now since it is not used only for id filters. Probably all references will be deleted when the code uses sdap_search_in_bases istead of custom search base iterators. Reviewed-by: Sumit Bose <sbose@redhat.com>

LDAP: remove unused param. in sdap_fallback_local_user Remove unused sdap_options parameter. Reviewed-by: Lukáš Slebodník <lslebodn@redhat.com>

LDAP: Filter out multiple entries when searching overlapping domains In case domain overlap, we might download multiple objects. To avoid saving them all, we attempt to filter out the objects from foreign domains. We can only do this optimization for non-wildcard lookups. Reviewed-by: Lukáš Slebodník <lslebodn@redhat.com>

LDAP: imposing sizelimit=1 for single-entry searches breaks overlapping domains https://fedorahosted.org/sssd/ticket/2723 In case there are overlapping sdap domains, a search for a single user might match and return multiple entries. For instance, with AD domains represented by search bases: DC=win,DC=trust,DC=test DC=child,DC=win,DC=trust,DC=test A search for user from win.trust.test would be based at: DC=win,DC=trust,DC=test but would match both search bases and return both users. Instead of performing complex filtering, just save both users. The responder would select the entry that matches the user's search. Reviewed-by: Lukáš Slebodník <lslebodn@redhat.com>

LDAP: Add the wildcard_limit option Related: https://fedorahosted.org/sssd/ticket/2553 Adds a new wildcard_limit option that is set by default to 1000 (one page). This option limits the number of entries that can by default be returned by a wildcard search. Reviewed-by: Pavel Březina <pbrezina@redhat.com>

LDAP: Add sdap_lookup_type enum Related: https://fedorahosted.org/sssd/ticket/2553 Change the boolan parameter of sdap_get_users_send and sdap_get_groups_send to a tri-state that controls whether we expect only a single entry (ie don't use the paging control), multiple entries with a search limit (wildcard request) or multiple entries with no limit (enumeration). Reviewed-by: Pavel Březina <pbrezina@redhat.com>

LDAP: Use sdap_get_and_parse_generic_/_recv Related: https://fedorahosted.org/sssd/ticket/2553 Using the new request sdap_get_and_parse_generic_send is a separate commit so that we can audit where the function is used during a code review. Reviewed-by: Pavel Březina <pbrezina@redhat.com>

SDAP: Remove unnecessary argument from sdap_save_user Reviewed-by: Pavel Březina <pbrezina@redhat.com>

SDAP: Set initgroups expire attribute at the end Initgrups consisted of two main steps: 1. store user to cache 2. store all user groups to cache. Previously the attribute SYSDB_INITGR_EXPIRE was set in the first step. So in case of epmty cache and parallel initgroups request in responders there was a small period when SYSDB_INITGR_EXPIRE was valid but groups were not cached. Therefore sometime responder could return zero supplementary groups. This patch moves the setting of initgroups expire attribute from 1st step to the end of 2nd step. In case of parallel initgroups requests in responder there are two other ways how we could get correct results even thought there was a bug. a) Time between two request was too small. User was not stored in cache yet and 2nd request waited for response from DP. b) Time between two request was big enough. All users groups were successfully stored in cache and 2nd request returned correct results. Resolves: https://fedorahosted.org/sssd/ticket/2634 Reviewed-by: Pavel Březina <pbrezina@redhat.com>

sdap: properly handle binary objectGuid attribute Although in the initial processing SSSD treats the binary value right at some point it mainly assumes that it is a string. Depending on the value this might end up with the correct binary value stored in the cache but in most cases there will be only a broken entry in the cache. This patch converts the binary value into a string representation which is described in [MS-DTYP] and stores the result in the cache. Resolves https://fedorahosted.org/sssd/ticket/2588 Reviewed-by: Jakub Hrozek <jhrozek@redhat.com>

LDAP: always store UUID if available Related to https://fedorahosted.org/sssd/ticket/2481 Reviewed-by: Jakub Hrozek <jhrozek@redhat.com>

UTIL: rename find_subdomain_by_sid The function was named "find_subdomain" yet it could find both main domain and subdomain. sed 's/find_subdomain_by_sid/find_domain_by_sid/' -i `find . -name "*.[ch]"` Reviewed-by: Jakub Hrozek <jhrozek@redhat.com>

LDAP: Make it possible to extend an attribute map https://fedorahosted.org/sssd/ticket/2073 This commit adds a new option ldap_user_extra_attrs that is unset by default. When set, the option contains a list of LDAP attributes the LDAP provider would download and store in addition to the usual set. The list can either contain LDAP attribute names only, or colon-separated tuples of LDAP attribute and SSSD cache attribute name. In case only LDAP attribute name is specified, the attribute is saved to the cache verbatim. Using a custom SSSD attribute name might be required by environments that configure several SSSD domains with different LDAP schemas. Reviewed-by: Simo Sorce <simo@redhat.com> Reviewed-by: Pavel Březina <pbrezina@redhat.com>

make make_realm_upper_case() static Reviewed-by: Jakub Hrozek <jhrozek@redhat.com>

Update DEBUG* invocations to use new levels Use a script to update DEBUG* macro invocations, which use literal numbers for levels, to use bitmask macros instead: grep -rl --include '*.[hc]' DEBUG . | while read f; do mv "$f"{,.orig} perl -e 'use strict; use File::Slurp; my @map=qw" SSSDBG_FATAL_FAILURE SSSDBG_CRIT_FAILURE SSSDBG_OP_FAILURE SSSDBG_MINOR_FAILURE SSSDBG_CONF_SETTINGS SSSDBG_FUNC_DATA SSSDBG_TRACE_FUNC SSSDBG_TRACE_LIBS SSSDBG_TRACE_INTERNAL SSSDBG_TRACE_ALL "; my $text=read_file(\*STDIN); my $repl; $text=~s/ ^ ( .* \b (DEBUG|DEBUG_PAM_DATA|DEBUG_GR_MEM) \s* \(\s* )( [0-9] )( \s*, ) ( \s* ) ( .* ) $ / $repl = $1.$map[$3].$4.$5.$6, length($repl) <= 80 ? $repl : $1.$map[$3].$4."\n".(" " x length($1)).$6 /xmge; print $text; ' < "$f.orig" > "$f" rm "$f.orig" done Reviewed-by: Jakub Hrozek <jhrozek@redhat.com> Reviewed-by: Stephen Gallagher <sgallagh@redhat.com> Reviewed-by: Simo Sorce <simo@redhat.com>

Make DEBUG macro invocations variadic Use a script to update DEBUG macro invocations to use it as a variadic macro, supplying format string and its arguments directly, instead of wrapping them in parens. This script was used to update the code: grep -rwl --include '*.[hc]' DEBUG . | while read f; do mv "$f"{,.orig} perl -e \ 'use strict; use File::Slurp; my $text=read_file(\*STDIN); $text=~s#(\bDEBUG\s*\([^(]+)\((.*?)\)\s*\)\s*;#$1$2);#gs; print $text;' < "$f.orig" > "$f" rm "$f.orig" done Reviewed-by: Jakub Hrozek <jhrozek@redhat.com> Reviewed-by: Stephen Gallagher <sgallagh@redhat.com> Reviewed-by: Simo Sorce <simo@redhat.com>

LDAP: Don't fail if subdomain cannot be found by sid Domain needn't contain sid if id_provider is ldap. With enabled id mapping, user couldn't be stored, because domain couldn't be found by sid. Resolves: https://fedorahosted.org/sssd/ticket/2172

LDAP: Fix a debug message

Remove unused parameter from sdap_save_user

LDAP: Split out a request to search for a user w/o saving Related: https://fedorahosted.org/sssd/ticket/2077 Certain situations require that a user entry is downloaded for further inpection, but not saved to the sysdb right away. This patch splits the previously monolithic request into one that just downloads the data and one that uses the new one to download and save the user.

SYSDB: Drop the sysdb_ctx parameter - module sysdb_ops (part 2)

sdap_save_user: try to determine domain by SID GC contains objects from both parent domain and subdomain. Lets say we have user with UID 5000 that belongs to a subdomain and overlapping search bases dc=ad,dc=pb and dc=sub,dc=ad,dc=pb. Now we call 'getent passwd 5000' and this request goes through data provider, searching in parent domain first. Even though this user does not belong to this domain it is found and stored as ad.pb user. With this patch we look at user's SID and put it into correct domain.

sdap_idmap_domain_has_algorithmic_mapping: add domain name argument When libss_idmap was only used to algorithmically map a SID to a POSIX ID a domain SID was strictly necessary and the only information needed to find a domain. With the introduction of external mappings there are cases where a domain SID is not available. Currently we relied on the fact that external mapping was always used as a default if not specific information about the domain was found. The lead to extra CPU cycles and potentially confusing debug messages. Adding the domain name as a search parameter will avoid this.

Fix formating of variables with type: size_t

sdap_save_user: save original primary GID of subdomain users If ID mapping is enabled we use magic private groups (MPG) for subdomains, i.e. the UID and the primary GID of the user will have the same numerical value. As a consequence the information about the original primary group might get lost because neither in AD domains nor on a typical UNIX system the user is an explicit member of it's primary group. With this patch the mapped GID or the original primary group is saved in the cached user object under a new attribute. Fixes https://fedorahosted.org/sssd/ticket/2027

LDAP: Use domain-specific name where appropriate The subdomain users user FQDN in their name attribute. However, handling of whether to use FQDN in the LDAP code was not really good. This patch introduces a utility function and converts code that was relying on user/group names matching to this utility function. This is a temporary fix until we can refactor the sysdb API in #2011.

Replace SDAP_ID_MAPPING checks with sdap_idmap_domain_has_algorithmic_mapping Currently the decision if external or algorithmic mapping should be used in the LDAP or AD provider was based on the value of the ldap_id_mapping config option. Since now all information about ID mapping is handled by libsss_idmap the check for this options can be replace with a call which checks the state via libss_idmap. https://fedorahosted.org/sssd/ticket/1961

LDAP: Do not store separate GID for subdomain users As the subdomains are MPG domains, we don't want to store a separate GID for the subdomain users, but rather just create a UPG.

Split generating primary GID for ID mapped users into a separate function Move the part of sdap_save_user into a separate function so that it can be special cased an only called for users in primary domains, not subdomain users.

LDAP: store FQDNs for trusted users and groups Because the NSS responder expects the name attribute to contain FQDN, we must save the name as FQDN in the LDAP provider if the domain we save to is a subdomain.

LDAP: always store SID if available Currently the string representation of a SID is only stored in the cache for debugging purpose if SID based ID-mapping is used. This patch unconditionally stores the SID if available to allow SID-to-name mappings from the cache.

ldap: Fallback option for rfc2307 schema Add option to fallback to fetch local users if rfc2307is being used. This is useful for cases where people added local users as LDAP members and rely on these group memberships to be maintained on the local host. Disabled by default as it violates identity domain separation. Ticket: https://fedorahosted.org/sssd/ticket/1020

Add domain argument to sysdb_store_user() Also remove sysdb_store_domuser()

AD: replace GID/UID, do not add another one The code would call sysdb_attrs_add_uint32 which added another UID or GID to the ID=0 we already downloaded from LDAP (0 is the default value) when ID-mapping an entry. This led to funky behaviour later on when we wanted to process the ID.

idmap: Silence DEBUG messages when dealing with built-in SIDs. When converting built-in SID to unix GID/UID a confusing debug message about the failed conversion was printed. This patch special cases these built-in objects. https://fedorahosted.org/sssd/ticket/1593

LDAP: fix uninitialized variable initialized variable, was causing build warning

LDAP: Only convert direct parents' ghost attribute to member https://fedorahosted.org/sssd/ticket/1612 This patch changes the handling of ghost attributes when saving the actual user entry. Instead of always linking all groups that contained the ghost attribute with the new user entry, the original member attributes are now saved in the group object and the user entry is only linked with its direct parents. As the member attribute is compared against the originalDN of the user, if either the originalDN or the originalMember attributes are missing, the user object is linked with all the groups as a fallback. The original member attributes are only saved if the LDAP schema supports nesting.

do not default fullname to gecos when schema = ad https://fedorahosted.org/sssd/ticket/1482 When we add fullname to user_attrs, then sysdb_add_basic_user() will set fullname to gecos when it initially creates the user object in the cache, but it will be overwritten in the same transaction when sysdb_store_user() adds all the user_attrs.

Unify usage of sysdb transactions Removing bad examples of usage of sysdb_transaction_start/commit/end functions and making it more consistent (all files except of src/db/sysdb_*.c).

Ghost members - removed sdap_check_aliases() This function is no longer necessary because we don't have fake user entries any more. The original purpose of this function was to check if there are fake user entries for particular user and, if yes, to update its membership.

LDAP: Add helper function to map IDs This function will also auto-create a new ID map if the domain has not been seen previously.

LDAP: Do not remove uidNumber and gidNumber attributes when saving id-mapped entries

LDAP: Add helper routine to convert LDAP blob to SID string

LDAP: Map the user's primaryGroupID

LDAP: Allow automatically-provisioning a domain and range If we get a user who is a member of a domain we haven't seen before, add a domain entry (auto-assigning its slice). Since we don't know the domain's real name, we'll just save the domain SID string as the name as well.

LDAP: Enable looking up ID-mapped users by name

Modifications to simplify list_missing_attrs

LDAP: Only use paging control on requests for multiple entries The paging control can cause issues on servers that put limits on how many paging controls can be active at one time (on some servers, it is limited to one per connection). We need to reduce our usage so that we only activate the paging control when making a request that may return an arbitrary number of results. https://fedorahosted.org/sssd/ticket/1202 phase one

Fix memory hierarchy when processing nested group memberships https://fedorahosted.org/sssd/ticket/1186

NSS: Add individual timeouts for entry types https://fedorahosted.org/sssd/ticket/1016

LDAP: Do not fail if RootDSE check cannot determine search bases https://fedorahosted.org/sssd/ticket/1152

Save original memberof, not memberof

Use the case sensitivity flag in the LDAP provider

Refactor saving sdap entities There was too much code duplication between sdap_save_{user,group,netgroup}. This patch removes the most egregious ones.

LDAP: Add support for multiple search bases for user enumeration

LDAP: Support multiple user search bases (non-enumeration)

SysDB commands that save lastUpdate allows this value to be passed in https://fedorahosted.org/sssd/ticket/836

Store name aliases for users, groups Also checks fake users for aliases when storing a real users so that getgrnam for a RFC2307 group that references a user by his secondary name followed by getpwnam for this user by his primary name works

sdap_async_accounts.c split The file has been split in three: sdap_async_users.c sdap_async_groups.c sdap_async_initgroups.c https://fedorahosted.org/sssd/ticket/864