nss: add a netgroup counter to struct nss_enum_index Netgroups are not looked up with the help of a single request but by calling setnetgrent(), getnetgrent() and endnetgrent() where getnetgrent() might be called multiple times depending on the number of netgroup elements. Since the caller does not provide a state the state has to be maintained by the SSSD nss responder. Besides the netgroup name this is mainly the number of elements already returned. This number is used to select the next element to return and currently it is assumed that there are not changes to the netgroup while the client is requesting the individual elements. But if e.g. the 3 nss calls are not used correctly or the netgroup is modified while the client is sending getnetgrent() calls the stored number might be out of range. To be on the safe side the stored number should be always compared with the current number of netgroup elements. Related to https://pagure.io/SSSD/sssd/issue/3679 Reviewed-by: Jakub Hrozek <jhrozek@redhat.com>

Fix minor spelling mistakes Reviewed-by: Lukáš Slebodník <lslebodn@redhat.com>

SYSDB: Use sysdb_domain_dn instead of raw ldb_dn_new_fmt Using ldb should be as much as an implementation detail as possible. Plus, it looks weird if one of the branch uses a sysdb function while another code branch uses a raw ldb call. Reviewed-by: Lukáš Slebodník <lslebodn@redhat.com>

SYSDB: Return EOK in case a non-fatal issue happened There may be the case where we aren't able to merge the timestamps from the fast ts db, which are treated as non-fatal issues. In case it happens, let's return EOK instead of propagating the non-fatal error. Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> Reviewed-by: Lukáš Slebodník <lslebodn@redhat.com> Reviewed-by: Jakub Hrozek <jhrozek@redhat.com>

sysdb: do not use objectClass for users and groups The majority of the object in the SSSD cache are users and groups. If there are many user and groups in the cache the index objects of the objectclass attributes 'user' and 'group' become large because the must hold references to all objects of those object classes. As a result the management of these index objects becomes costly because they must be parsed and split apart quite often. Additionally they are mostly useless because user and groups are lookup up by more specific attributes in general. Only when enumerating all user or groups this kind of index might be useful. There are two way of removing this kind of index from the user and group objects. Either by removing objectClass from the list of indexes and add a new attribute to all other type of object we want and index for. Or by replacing objectClass with a different attribute for the user and group objects. After some testing I think the latter one is the more reliable one and implemented it in this patch. Related to https://pagure.io/SSSD/sssd/issue/3503 Reviewed-by: Fabiano Fidêncio <fidencio@redhat.com> Reviewed-by: Jakub Hrozek <jhrozek@redhat.com>

SYSDB: Better debugging for email conflicts Add DEBUG message when conflicts in FQ names or emails are detected. Also improve man page to hint on how to work around issue with conflicting emails. Note: We store emails in two different attributes in sysdb: - SYSDB_USER_EMAIL - SYSDB_NAME_ALIAS - this one is lowercased and used in getpwnam searches. Resolves: https://fedorahosted.org/sssd/ticket/3293 Reviewed-by: Fabiano Fidêncio <fidencio@redhat.com>

SYSDB: Make the usage of the filter more generic for search_ts_matches() In order to make this function re-usable in different parts of our code, let's start passing an already built filter to it instead of having the specific code building the name filter there. Related: https://pagure.io/SSSD/sssd/issue/3369 Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> Reviewed-by: Jakub Hrozek <jhrozek@redhat.com>

SYSDB: Internally expose sysdb_search_ts_matches() This function will be used in the follow-up patches. As it's going to be "exposed", let's also rename it from search_ts_matches() to sysdb_search_ts_matches(). Related: https://pagure.io/SSSD/sssd/issue/3369 Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> Reviewed-by: Jakub Hrozek <jhrozek@redhat.com>

SYSDB: Return ERR_NO_TS when there's no timestamp cache present This change affects sysdb_search_ts_{users,groups} functions and is mainly needed in order to avoid breaking our current tests due to the changes planned for fixing https://pagure.io/SSSD/sssd/issue/3369. Related: https://pagure.io/SSSD/sssd/issue/3369 Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> Reviewed-by: Jakub Hrozek <jhrozek@redhat.com>

SYSDB: When searching for UPNs, search either the whole DB or only the given domain The search-by-UPN functions always searched for the whole domain. In some cases, the caller depends on the result coming from the domain specified by the 'domain' parameter. This is the case in the cache_req code at least. Even though it should be safe to just switch to always searching the whole domain, in order to allow us to examine the code carefully and test each codepath, let's introduce a boolean option to the search functions. Currently it defaults to false in all codepaths and as we test the individual ones, we can flip the option to true until we finally remove the option altogether. Reviewed-by: Sumit Bose <sbose@redhat.com>

sysdb: add parent_dom to sysdb_get_direct_parents() Currently sysdb_get_direct_parents() only return direct parents from the same domain as the child object. In setups with sub-domains this might not be sufficient. A new option parent_dom is added which allows to specify a domain the direct parents should be lookup up in. If it is NULL the whole cache is searched. Reviewed-by: Jakub Hrozek <jhrozek@redhat.com>

views: properly override group member names Resolves https://fedorahosted.org/sssd/ticket/2948 Reviewed-by: Jakub Hrozek <jhrozek@redhat.com>

SYSDB: Search functions don't need to construct per-domain names All user and group names are standardized to be fully qualified, so there's no need to provide per-domain names anymore. Reviewed-by: Jakub Hrozek <jhrozek@redhat.com>

sysdb: Use ldb_result as output in sysdb_search_ts_{users,groups} Passing address of unsigned to the output argument size_t causes access out of boundaries for type unsigned and and wrong data on big endian. It looks like functions sysdb_search_ts_{users,groups} need to store results in structure ldb_result anyway for further processing. Therefore it will be better to convert output arguments size_t* + ldb_message*** into structure ldb_result and avoid using additional helper variable with type size_t before each invocation of these functions. Reviewed-by: Sumit Bose <sbose@redhat.com>

SYSDB: If modifyTimestamp is the same, only update the TS cache Resolves: https://fedorahosted.org/sssd/ticket/2602 If the entry being saved contains the original modifyTimestamp attribute and the modifyTimestamp attribute is the same as the one we already saved to the timestamp cache, only the expire timestamps in the asynchronous timestamp cache will be bumped and the sysdb code will avoid writes to the main cache completely. If the modifyTimestamp is either missing or differs, we assume the entry had changed and do a full write to the main cache. Also amends the generic sysdb_set_attrs* and similar functions that their results is also reflected in the timestamps cache. Reviewed-by: Sumit Bose <sbose@redhat.com>

SYSDB: Search the timestamp caches in addition to the sysdb cache When a sysdb entry is searched, the sysdb cache is consulted first for users or groups. If an entry is found in the sysdb cache, the attributes from the timestamp cache are merged to return the full and up-to-date set of attributes. The merging is done with a single BASE search which is a direct lookup into the underlying key-value database, so it should be relatively fast. More complex merging is done only for enumeration by filter which is currently done only via the IFP back end and should be quite infrequent, so I hope we can justify a more complex merging there. Reviewed-by: Sumit Bose <sbose@redhat.com>

sysdb: add searches by certificate with overrides Reviewed-by: Jakub Hrozek <jhrozek@redhat.com>

cache_req: add support for UPN Reviewed-by: Sumit Bose <sbose@redhat.com>

views: allow ghost members for LOCAL view LOCAL view does not allow the case when both ghost member and user override is created so it is safe to allow ghost members for this view. Resolves: https://fedorahosted.org/sssd/ticket/2790 Reviewed-by: Sumit Bose <sbose@redhat.com>

Update few debug messages It reduces a noise caused by canonicalization of non-existing user. Resolves: https://fedorahosted.org/sssd/ticket/2678 Reviewed-by: Pavel Reichl <preichl@redhat.com>

SYSDB: Add functions to look up multiple entries including name and custom filter Related: https://fedorahosted.org/sssd/ticket/2553 Adds new sysdb function: - sysdb_enumpwent_filter - sysdb_enumpwent_filter_with_views - sysdb_enumgrent_filter - sysdb_enumgrent_filter_with_views These are similar to enumeration functions, but optionally allow to specify a filter to be applied on user/group names. Also an additional custom filter can be applied. Reviewed-by: Pavel Březina <pbrezina@redhat.com>

IPA: allow initgroups by UUID for FreeIPA users If a FreeIPA user is searched with the help of an override name the UUID from the override anchor is used to search the user. Currently the initgroups request only allows searches by SID or name. With this patch a UUID can be used as well. Related to https://fedorahosted.org/sssd/ticket/2642 Reviewed-by: Jakub Hrozek <jhrozek@redhat.com>

IPA: allow initgroups by SID for AD users If a user from a trusted AD domain is search with the help of an override name the SID from the override anchor is used to search the user in AD. Currently the initgroups request only allows searches by name. With this patch a SID can be used as well. Resolves https://fedorahosted.org/sssd/ticket/2632 Reviewed-by: Jakub Hrozek <jhrozek@redhat.com>

sysdb: use sysdb_user/group_dn Reviewed-by: Jakub Hrozek <jhrozek@redhat.com>

sysdb_add_overrides_to_object: add new parameter and multi-value support With the new parameter an attribute list other than the default one can be used. Override attributes with multiple values (e.g. SSH public keys) are now supported as well. Reviewed-by: Jakub Hrozek <jhrozek@redhat.com>

sysdb_get_user_attr_with_views: add mandatory override attributes This patch add another attribute with is needs for override processing to the attribute list of sysdb_get_user_attr_with_views(). With two attribute it does not seem useful to check for existence and add each of the attributes conditionally. With this patch they are added unconditionally if the domain has views. Additionally the attributes are not removed in the end because it is expected that they do not cause any harm. Reviewed-by: Jakub Hrozek <jhrozek@redhat.com>

Add sysdb_get_user_attr_with_views Reviewed-by: Sumit Bose <sbose@redhat.com>

sysdb: add sysdb_enumpw/grent_with_views() Reviewed-by: Pavel Březina <pbrezina@redhat.com>

sysdb: add sysdb_getgrnam_with_views and sysdb_getgrgid_with_views Reviewed-by: Pavel Březina <pbrezina@redhat.com>

sysdb: add sysdb_initgroups_with_views() Reviewed-by: Pavel Březina <pbrezina@redhat.com>

sysdb: add sysdb_getpwnam/uid_with_views() View-aware drop-in replacements for sysdb_getpwnam() and sysdb_getpwuid(). Reviewed-by: Pavel Březina <pbrezina@redhat.com>

sysdb_get_user_attr: use fqn for subdomain users Name of subdomain users is stored with domain part in the sysdb. We need to use fully qualified names for those user so we can look them up. Reviewed-by: Lukáš Slebodník <lslebodn@redhat.com>

SYSDB: move sysdb_get_real_name() from sysdb.c to sysdb_search.c The sysdb.c should be reserved for utility and setup functions. Search functions belong to sysdb_search.c Keeping functions in specialized modules helps to maintain nice dependencies and in overall makes unit testing easier. Moreover, the function was not unit tested, which needed fixing. Reviewed-by: Lukáš Slebodník <lslebodn@redhat.com>

SYSDB: SSS_LDB_SEARCH - macro around ldb_search This patch amends previous patch 5153e8b9793dea1e212ca08af0f77ea1d023cbb7. Macro SSS_LDB_SEARCH is used instead of using fuction sss_ldb_search as a wrapper around ldb_search which could lead to premature expansion of variadic parameters. Part of solution for: https://fedorahosted.org/sssd/ticket/1991 Reviewed-by: Lukáš Slebodník <lslebodn@redhat.com>

SYSDB: sysdb_getnetgr returns ENOENT Replace call of ldb_search by sss_ldb_search to make sure that ENOENT is returned if no results were found. Resolves: https://fedorahosted.org/sssd/ticket/1991 Reviewed-by: Lukáš Slebodník <lslebodn@redhat.com> Reviewed-by: Jakub Hrozek <jhrozek@redhat.com>

Update DEBUG* invocations to use new levels Use a script to update DEBUG* macro invocations, which use literal numbers for levels, to use bitmask macros instead: grep -rl --include '*.[hc]' DEBUG . | while read f; do mv "$f"{,.orig} perl -e 'use strict; use File::Slurp; my @map=qw" SSSDBG_FATAL_FAILURE SSSDBG_CRIT_FAILURE SSSDBG_OP_FAILURE SSSDBG_MINOR_FAILURE SSSDBG_CONF_SETTINGS SSSDBG_FUNC_DATA SSSDBG_TRACE_FUNC SSSDBG_TRACE_LIBS SSSDBG_TRACE_INTERNAL SSSDBG_TRACE_ALL "; my $text=read_file(\*STDIN); my $repl; $text=~s/ ^ ( .* \b (DEBUG|DEBUG_PAM_DATA|DEBUG_GR_MEM) \s* \(\s* )( [0-9] )( \s*, ) ( \s* ) ( .* ) $ / $repl = $1.$map[$3].$4.$5.$6, length($repl) <= 80 ? $repl : $1.$map[$3].$4."\n".(" " x length($1)).$6 /xmge; print $text; ' < "$f.orig" > "$f" rm "$f.orig" done Reviewed-by: Jakub Hrozek <jhrozek@redhat.com> Reviewed-by: Stephen Gallagher <sgallagh@redhat.com> Reviewed-by: Simo Sorce <simo@redhat.com>

Make DEBUG macro invocations variadic Use a script to update DEBUG macro invocations to use it as a variadic macro, supplying format string and its arguments directly, instead of wrapping them in parens. This script was used to update the code: grep -rwl --include '*.[hc]' DEBUG . | while read f; do mv "$f"{,.orig} perl -e \ 'use strict; use File::Slurp; my $text=read_file(\*STDIN); $text=~s#(\bDEBUG\s*\([^(]+)\((.*?)\)\s*\)\s*;#$1$2);#gs; print $text;' < "$f.orig" > "$f" rm "$f.orig" done Reviewed-by: Jakub Hrozek <jhrozek@redhat.com> Reviewed-by: Stephen Gallagher <sgallagh@redhat.com> Reviewed-by: Simo Sorce <simo@redhat.com>

Use lower-case name for case-insensitive searches The patch makes sure that a completely lower-cased version of a fully qualified name is used for case insensitive searches. Currently there are code paths where the domain name was used as configured and was not lower-cased. To make sure this patch does not break with old entries in the cache or case sensitive domains a third template was added to the related filters templates which is either filled with a completely lower-cased version or with the old version. The other two template values are unchanged.

SYSDB: simplification of condition in if statement else-if statement is checking for condition which is handled by previous if statement.

SYSDB: typos & debug macro constants

SYSDB: Skip malformed netgroup attribute. It was not easy find out why netgroup could not be covert into result entries. Problem was that nisNetgroupTriple contained unexpected string "(,user01)" This patch will ignore only malformed attribute and processing of netgroup will not fail. Resolves: https://fedorahosted.org/sssd/ticket/2137

SYSDB: Drop the sysdb_ctx parameter from the sysdb_search module

Fix formating of variables with type: size_t

New utility function sss_get_domain_name Instead of copying a block of code that checks whether domain is a subdomain and uses only name of FQDN as appropriate, wrap the logic into a function.

Add utility functions for formatting fully-qualified names Instead of using printf-like functions directly, provide two wrappers that would encapsulate formatting the fully-qualified names. No functional change is present in this patch.

Fix initialization of multiple variables

Introduce IS_SUBDOMAIN() macro Fixes https://fedorahosted.org/sssd/ticket/1766

Move mpg flag to the domain where it belongs A sysdb contains now multiple domains, but the mpg property is a property of a specific domain not of the underlying database.

Add domain argument to sysdb_get_user_attr()

Add domain argument to sysdb_initgroups()

Add domain option to sysdb_get/netgr/attrs() fns

Pass domain to sysdb_enum<pw/gr>ebt() functions

Pass domain to sysdb_get<pwu/grg><id() functions

Pass domain to sysdb_get<pw/gr>nam() functions Also allows us to remove sysdb_subdom_get<pw/gr>nam() wrappers and restore fqnames proper value in subdomains, by testing for a parent domain being present or not.

Refactor the way subdomain accounts are saved The original sysdb code had a strong assumption that only users from one domain are saved in the databse, with the subdomain feature, we have changed reality, but have not adjusted all the code arund the sysdb calls to not rely on the original assumption. One of the side effects of this incongrunece is that currently group memberships do not return fully qualified names for subdomain users as they should. In oreder to fix this and other potential issues surrounding the violation of the original assumption, we need to fully qualify subdomain user names. By savin them fully qualified we do not risk aliasing local users and have group memberhips or other name based matching code mistake a domain user with subdomain usr or vice versa.

Search netgroups by alias, too https://fedorahosted.org/sssd/ticket/1228

Fix FTBFS related to -Werror=format-security

Sanitize DN in sysdb_get_direct_parents

Fix typo in sysdb_get_direct_parents

Return users and groups based on alias https://fedorahosted.org/sssd/ticket/926

Add a sysdb_get_direct_parents function

Remaining memory context variables renamed memctx to mem_ctx tmpctx to tmp_ctx

sysdb refactoring: memory context deleted This patch deletes memory context parameter in those places in sysdb where it is not necessary. The code using modified functions has been updated. Tests updated as well.

sysdb refactoring: deleted domain variables in sysdb API The patch also updates code using modified functions. Tests have also been adjusted.

sysdb refactoring: renamed ctx variable to sysdb

Sanitize search filters for the sysdb

Also return member groups to the client

Netgroups sysdb API

Initgroups on a non-cached user should go to the data provider We were accidentally returning an error when sysdb_getpwnam() returned zero results internally in sysdb_initgroups(). The correct behavior here is to return EOK and a result object with zero entries.

sysdb: remove obsolete helpers from sysdb

sysdb: convert sysdb_initgroups

sysdb: convert sysdb_enumgrent

sysdb: convert sysdb_enumpwent

sysdb: convert sysdb_get_user_attr

sysdb: convert sysdb_getgrgid

sysdb: convert sysdb_getgrnam

sysdb: convert sysdb_getpwuid

sysdb: convert sysdb_getpwnam

Rename server/ directory to src/ Also update BUILD.txt