cache: Check for max_id/min_id in cache_req The cache_req code doesn't check the min_id/max_id boundaries for requests by ID. Extending the .lookup_fn function in each plugin that searches by ID for a check that returns non-zero if the entry is out of the range and 0 if not. Resolves: https://pagure.io/SSSD/sssd/issue/3569 Reviewed-by: Jakub Hrozek <jhrozek@redhat.com> Reviewed-by: Fabiano Fidêncio <fidencio@redhat.com>

CACHE_REQ: Use the domain-locator request to only search domains where the entry was found Uses the internal cache_req interface around the getAccountDomain to only search the domain returned by the cache_req_locate_domain_recv() request. If that request returns that no domain matched, all domains (belonging to the currently processed main domain) are skipped by setting the per-type negative cache. if a domain is reported as containing an object, all domains except that one are marked with the negative cache entries. Resolves: https://pagure.io/SSSD/sssd/issue/3468 Reviewed-by: Pavel Březina <pbrezina@redhat.com> Reviewed-by: Sumit Bose <sbose@redhat.com>

CACHE_REQ: Add cache_req_data_set_bypass_dp() Similar to cache_req_data_set_bypass_cache() cache_req_data_set_bypass_dp() can be used to control how the cache_req framework performs the lookup. If cache_req_data_set_bypass_dp() is used with 'true' only a cache lookup is performed and no request is send to the backend even if no entry was found. Related to https://pagure.io/SSSD/sssd/issue/2478 Reviewed-by: Jakub Hrozek <jhrozek@redhat.com>

CACHE_REQ: Copy the cr_domain list for each request Let's copy the cr_domain list for each request as this list may be free'd due to a refresh domains request. Resolves: https://pagure.io/SSSD/sssd/issue/3551 Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> Reviewed-by: Pavel Březina <pbrezina@redhat.com>

Print a warning when enumeration is requested but disabled Add an explanatory message to be logged once, at the start-up, mentioning that in case enumeration is not enabled, getent passwd won't return all users by design. The debug level chosen to show the message is SSS_LOG_NOTICE. Resolves: https://pagure.io/SSSD/sssd/issue/2301 Reviewed-by: Jakub Hrozek <jhrozek@redhat.com> Reviewed-by: Fabiano Fidêncio <fidencio@redhat.com> Reviewed-by: Lukáš Slebodník <lslebodn@redhat.com>

CACHE_REQ: Pull sessionRecording attrs from initgr After entires are retrieved by cache_req for user info requests (except initgr), overlay them with sessionRecording attribute retrieved from an initgr request made additionally for each entry. Do not do additional initgr requests with selective session recording enabled, if we don't have any group names to match against in session recording configuration. Only do user name matches instead. Reviewed-by: Pavel Březina <pbrezina@redhat.com>

CACHE_REQ: Propagate num_results to cache_req_state The num_results field in struct cache_req_state was only set in case of well-known objects, set it also for the regular results for uniformity, and for later use by session recording code. Reviewed-by: Pavel Březina <pbrezina@redhat.com>

CACHE_REQ: Ensure the domains are updated for "filter" related calls As contacting the infopipe responder on a "filter" related call may lead to the situation where the cr_domains' list is not populated yet (as the domains and subdomains lists from the data provider are not processed yet), let's explicitly call sss_dp_get_domains() for those cases and avoid returning a wrong result to the caller. This situation may happen only because the schedule_get_domains_task(), that's called when the infopipe responder is initialized, may take some time to run/finish. While I'm not exactly sure whether it's the best solution to avoid the "race", it seems to be sane enough to avoid the issues. Resolves: https://pagure.io/SSSD/sssd/issue/3387 Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> Reviewed-by: Sumit Bose <sbose@redhat.com> Reviewed-by: Pavel Březina <pbrezina@redhat.com>

CACHE_REQ: Allow configurationless shortname lookups Configurationless shortnames lookups must be allowed when a domains' resolution order is present and the (head) domain is not enforcing the usage of fully-qualified-names. With this patch SSSD does not require any kind of changes from client side for taking advantage of shortname lookups. Related: https://pagure.io/SSSD/sssd/issue/3001 Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> Reviewed-by: Pavel Březina <pbrezina@redhat.com>

CACHE_REQ: Domain type selection in cache_req Related to: https://pagure.io/SSSD/sssd/issue/3310 Adds a new enumeration cache_req_dom_type. It is a tri-state that allows the caller to select which domains can be contacted - either only POSIX, only application domains or any type. Not all plugins of cache_req have the new parameter added -- only those that are usable/useful in a non-POSIX environment. For example, it makes no sense to allow the selection for calls by ID because those are inherently POSIX-specific. Also, services or netgroups are supported only coming from POSIX domains. At the moment, the patch should not change any behaviour as all calls default to contacting POSIX domains only. Reviewed-by: Pavel Březina <pbrezina@redhat.com>

CACHE_REQ: Make use of domainResolutionOrder domainResolutionOrder has been introduced in the previous commits and allows the admin to set up a specific order which the domains will be resolved during a lookup and with this patch we can take advantage of this. In order to have it working a new structure has been added (struct domain_resolution_order) to the responder context and will be used by the cache_req to perform the lookups based on this list. As the ipaDomainResolutionOrder may be set globally on IPA or per View, SSSD does respect the following precedence order: View > Globally. The way the list is built is quite simple, basically having the domains present on ipaDomainResolutionOrder as the first domains (in that specific order) and then appending the remaining domains to this list. The final result is a completely flat list with all the domains respecting the specified order (it's important to remember that the domains not specified won't follow any specific order, they're just "random" based on the domains list present in the responder context. Related: https://pagure.io/SSSD/sssd/issue/3001 Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> Reviewed-by: Sumit Bose <sbose@redhat.com> Reviewed-by: Pavel Březina <pbrezina@redhat.com>

CACHE_REQ: Check the caches first This patch introduces a new configurable option to define whether the responder should query all domains' caches before querying the Data Providers. This new option is called cache_first and, by default, it's disabled, meaning that, for each provider, the responder may contact the cache and the data provider in the same iteration. Co-Author: Pavel Březina <pbrezina@redhat.com> Related: https://pagure.io/SSSD/sssd/issue/3001 Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> Reviewed-by: Pavel Březina <pbrezina@redhat.com>

CACHE_REQ: Move cache_req_next_domain() into a new tevent request This change prepares for the upcoming patch, which will have to support more than one iteration over all domains, leaving us with a better control of the whole logic around this iteration. Related: https://pagure.io/SSSD/sssd/issue/3001 Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> Reviewed-by: Pavel Březina <pbrezina@redhat.com>

CACHE_REQ: Move result manipulation into a separate module This patch is preparing the field for coming up patches where new lookup logic will be added. Taking this into consideration let's move the result manipulation code into a separate module and focus purely in the lookups logic in the main module. Related: https://pagure.io/sssd/sssd/issue/3001 Reviewed-by: Fabiano Fidêncio <fidencio@redhat.com>

CACHE_REQ: Make cache_req_{create_and_,}add_result() more generic This patch is preparing the field for coming up patches where this function will be re-used. In order to do so, let's remove the dependency for tevent states on both cache_req_create_and_add_result() and cache_req_add_result(). Related: https://pagure.io/sssd/sssd/issue/3001 Reviewed-by: Fabiano Fidêncio <fidencio@redhat.com>

cache_req: add host by name search Reviewed-by: Jakub Hrozek <jhrozek@redhat.com>

cache_req: add api to create ldb_result from message Some sysdb methods doesn't return ldb_result as output but return ldb_message instead. Changing sysdb to be consistent is too big so I added this helper function that will wrap resulting message into ldb_result. Reviewed-by: Jakub Hrozek <jhrozek@redhat.com>

cache_req: add ability to not use default domain suffix This will be used in the next plugin "host by name" where it is not desirable to use default domain suffix if set. Reviewed-by: Jakub Hrozek <jhrozek@redhat.com>

sss_parse_inp_send: provide default_domain as parameter It is not always desirable to consider default_domain from configuration but expect none instead. For example when we search host certificates. This is currently not used in this patch since host lookups parse name directly with sss_parse_name but it will be used in the next patch. Reviewed-by: Jakub Hrozek <jhrozek@redhat.com>

cache_req: do not set ncache if dp request fails We will only remember entry in the negative cache if the data provider requests succeeded because only then we can be sure that the entry does not exist. Resolves: https://fedorahosted.org/sssd/ticket/3151 Reviewed-by: Lukáš Slebodník <lslebodn@redhat.com>

cache_req: improve debugging The original debugging was unnecessarily large. I removed request name from all debug messages (keeping the request number). The name is now print only when a new request is created. I also added an explicit message to see what object are we currently looking up. For example: [cache_req_set_plugin] (0x2000): CR #4: Setting "Group by ID" plugin [cache_req_send] (0x0400): CR #4: New request 'Group by ID' [cache_req_select_domains] (0x0400): CR #4: Performing a multi-domain search [cache_req_set_domain] (0x0400): CR #4: Using domain [LDAP.PB] [cache_req_search_send] (0x0400): CR #4: Looking up GID:20001@LDAP.PB [cache_req_search_ncache] (0x0400): CR #4: Checking negative cache for [GID:20001@LDAP.PB] [cache_req_search_ncache] (0x0400): CR #4: [GID:20001@LDAP.PB] is not present in negative cache [cache_req_search_cache] (0x0400): CR #4: Looking up [GID:20001@LDAP.PB] in cache [cache_req_search_send] (0x0400): CR #4: Returning [GID:20001@LDAP.PB] from cache [cache_req_create_and_add_result] (0x0400): CR #4: Found 1 entries in domain LDAP.PB [cache_req_done] (0x0400): CR #4: Finished: Success Resolves: https://fedorahosted.org/sssd/ticket/3151 Reviewed-by: Lukáš Slebodník <lslebodn@redhat.com>

cache_req: make plug-ins definition const Resolves: https://fedorahosted.org/sssd/ticket/3151 Reviewed-by: Lukáš Slebodník <lslebodn@redhat.com>

cache_req: add object by id This request returns either user or group object. Resolves: https://fedorahosted.org/sssd/ticket/3151 Reviewed-by: Lukáš Slebodník <lslebodn@redhat.com>

cache_req: add object by name This request returns either user or group object. Resolves: https://fedorahosted.org/sssd/ticket/3151 Reviewed-by: Lukáš Slebodník <lslebodn@redhat.com>

cache_req: allow to return well known object as result When we are looking name by sid or sid by name we want to be also able to translate well known sid/name into its pair equivalent. This adds the ability to mock cache_req_result if a well known object is found. Resolves: https://fedorahosted.org/sssd/ticket/3151 Reviewed-by: Lukáš Slebodník <lslebodn@redhat.com>

cache_req: allow shallow copy of result For enumeration purpose, we want to limit the number of returned results This is a preparation for changes in NSS responder. Resolves: https://fedorahosted.org/sssd/ticket/3151 Reviewed-by: Lukáš Slebodník <lslebodn@redhat.com>

cache_req: add support for netgroups Resolves: https://fedorahosted.org/sssd/ticket/3151 Reviewed-by: Lukáš Slebodník <lslebodn@redhat.com>

cache_req: add support for services enumeration Resolves: https://fedorahosted.org/sssd/ticket/3151 Reviewed-by: Lukáš Slebodník <lslebodn@redhat.com>

cache_req: add support for service by port Resolves: https://fedorahosted.org/sssd/ticket/3151 Reviewed-by: Lukáš Slebodník <lslebodn@redhat.com>

cache_req: add support for service by name Resolves: https://fedorahosted.org/sssd/ticket/3151 Reviewed-by: Lukáš Slebodník <lslebodn@redhat.com>

cache_req: add group enumeration Resolves: https://fedorahosted.org/sssd/ticket/3151 Reviewed-by: Lukáš Slebodník <lslebodn@redhat.com>

cache_req: add user enumeration Resolves: https://fedorahosted.org/sssd/ticket/3151 Reviewed-by: Lukáš Slebodník <lslebodn@redhat.com>

cache_req: add ability to filter domains by enumeration Skip domains without enumeration if a plug-in requires it. This is preparation for enumeration support inside cache_req. Resolves: https://fedorahosted.org/sssd/ticket/3151 Reviewed-by: Lukáš Slebodník <lslebodn@redhat.com>

cache_req: add ability to gather result from all domains This is preparation for enumeration support inside cache_req. Resolves: https://fedorahosted.org/sssd/ticket/3151 Reviewed-by: Lukáš Slebodník <lslebodn@redhat.com>

cache_req: encapsulate output data into structure In enumeration calls we want to get objects from all domains, not only from the first matched domain. We move the cache search result into a structure that contains combination of domain and ldb_result. This is preparation for enumeration support inside cache_req. Resolves: https://fedorahosted.org/sssd/ticket/3151 Reviewed-by: Lukáš Slebodník <lslebodn@redhat.com>

cache_req: move from switch to plugins, add plugins This patch adds all existing functionality into plugins. Reviewed-by: Jakub Hrozek <jhrozek@redhat.com>

cache_req: move from switch to plugins; add logic cache_req grown quite big from the original code and it turned out that using switch statements to branch code for different cases makes the code quite hard to read and further extend and any modification to the logic itself is difficult. This patch changes the switch statements to plugins with small functions and separates logic into multiple modules. This gives us better control over the code and improves readability and maintainability while keeping code duplication to minimum. Reviewed-by: Jakub Hrozek <jhrozek@redhat.com> Reviewed-by: Petr Cech <pcech@redhat.com>