SYSDB: Use sysdb_domain_dn instead of raw ldb_dn_new_fmt Using ldb should be as much as an implementation detail as possible. Plus, it looks weird if one of the branch uses a sysdb function while another code branch uses a raw ldb call. Reviewed-by: Lukáš Slebodník <lslebodn@redhat.com>

sysdb: add sysdb_attrs_copy() Related to https://pagure.io/SSSD/sssd/issue/3050 Reviewed-by: Jakub Hrozek <jhrozek@redhat.com> Reviewed-by: Lukáš Slebodník <lslebodn@redhat.com>

SYSDB: Adding message about reason why cache changed Resolves: https://fedorahosted.org/sssd/ticket/3060 Reviewed-by: Lukáš Slebodník <lslebodn@redhat.com>

SYSDB: Add a utility function to return a list of qualified names Adds a utility function the LDAP provider can use. This is different from sss_create_internal_fqname_list in the sense that the LDAP provider passes in the attribute name that contains the name attribute value. Reviewed-by: Sumit Bose <sbose@redhat.com>

SYSDB: Check if group attributes differ before saving a group Adds a new function sysdb_entry_attrs_diff() used in group saving code. This function is used to check if the result of updating a group would result in actually changing the sysdb entry -- often, we would try to dump the same data to the cache during update. If that's the case, the update code now only updates the timestamp cache, avoiding costly writes. Reviewed-by: Sumit Bose <sbose@redhat.com>

SYSDB: If modifyTimestamp is the same, only update the TS cache Resolves: https://fedorahosted.org/sssd/ticket/2602 If the entry being saved contains the original modifyTimestamp attribute and the modifyTimestamp attribute is the same as the one we already saved to the timestamp cache, only the expire timestamps in the asynchronous timestamp cache will be bumped and the sysdb code will avoid writes to the main cache completely. If the modifyTimestamp is either missing or differs, we assume the entry had changed and do a full write to the main cache. Also amends the generic sysdb_set_attrs* and similar functions that their results is also reflected in the timestamps cache. Reviewed-by: Sumit Bose <sbose@redhat.com>

SYSDB: Search the timestamp caches in addition to the sysdb cache When a sysdb entry is searched, the sysdb cache is consulted first for users or groups. If an entry is found in the sysdb cache, the attributes from the timestamp cache are merged to return the full and up-to-date set of attributes. The merging is done with a single BASE search which is a direct lookup into the underlying key-value database, so it should be relatively fast. More complex merging is done only for enumeration by filter which is currently done only via the IFP back end and should be quite infrequent, so I hope we can justify a more complex merging there. Reviewed-by: Sumit Bose <sbose@redhat.com>

SYSDB: Move sysdb initialization into a new module sysdb_init.c The sysdb initialization was in the sysdb.c module. With adding initialization of the timestamp cache, this module would become too big with too many private functions meant only for initialization. This patch only moves code around, there are no functional changes to the initialization. Reviewed-by: Sumit Bose <sbose@redhat.com>

SYSDB: Add systemtap probes to track sysdb transactions Actually adds marks for sysdb transactions that receive the transaction nesting level as an argument. The nesting is passed on from probes to marks along with a human-friendly description. The transaction commit is decorated with two probes, before and after. This would allow the caller to distinguish between the time we spend in the transaction (which might be important, because if a transaction is active on an ldb context, even the readers are blocked before the transaction completes) and the time we spend commiting the transaction (which is important because that's when the disk writes occur) The probes would be installed into /usr/share/systemtap/tapset on RHEL and Fedora. This is in line with systemtap's paths which are described in detail in "man 7 stappaths". Reviewed-by: Lukáš Slebodník <lslebodn@redhat.com>

SYSDB: Track transaction nesting in sysdb_ctx Adds an integer that tracks how deeply nested we are in sysdb transactions. This will become useful later, because generally we are only interested in level-0 transactions when probing, so we'll want to pass the transaction nesting to the systemtap probes. Reviewed-by: Lukáš Slebodník <lslebodn@redhat.com>

sysdb: add sysdb_attrs_add_base64_blob() Reviewed-by: Jakub Hrozek <jhrozek@redhat.com>

sysdb: add sysdb_{add,replace,delete}_ulong() As the add_ulong() convenience can add, replace or remove a unsigned long according to the operation received as its argument, some confusion can easily happen due to its misleading name. In order to improve the explicitness of our code, let's introduce sysdb_add_ulong(), sysdb_replace_ulong() and sysdb_delete_ulong(). These new functions are basically wrappers of add_ulong() (now sysdb_ldb_msg_ulong_helper()), calling it using the proper flag according to each function. Any code previously using add_ulong() is now adapted to use these brand new functions. Related: https://fedorahosted.org/sssd/ticket/1656 Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org> Reviewed-by: Petr Cech <pcech@redhat.com>

sysdb: move add_ulong() convenience to sysdb.c Considering that sysdb.c is about utilities around our cache and that sysdb_ops.c is about operations on objects, seems that add_ulong() could fit better in sysdb.c. This move is a suggestion from Jakub Hrozek. Related: https://fedorahosted.org/sssd/ticket/1656 Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org> Reviewed-by: Petr Cech <pcech@redhat.com>

sysdb: add sysdb_{add,replace,delete}_string() As the add_string() convenience can add, replace or delete a string according to the operation received as its argument, some confusion can easily happen due to its misleading name. In order to improve the explicitness of our code, let's introduce sysdb_add_string(), sysdb_replace_string() and sysdb_delete_string(). These new functions are basically wrappers of add_string() (now sysdb_ldb_msg_string_helper()), calling it using the proper flag according to each function. Any code previously using add_string() is now adapted to use these brand new functions. Resolves: https://fedorahosted.org/sssd/ticket/1656 Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org> Reviewed-by: Petr Cech <pcech@redhat.com>

sysdb: move add_string() convenience to sysdb.c Considering that sysdb.c is about utilities around our cache and that sysdb_ops.c is about operations on objects, seems that add_string() could fit better in sysdb.c. This move is a suggestion from Jakub Hrozek. Related: https://fedorahosted.org/sssd/ticket/1656 Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org> Reviewed-by: Petr Cech <pcech@redhat.com>

SUDO: make sudo sysdb interface more reusable Reviewed-by: Sumit Bose <sbose@redhat.com>

IPA: fix override with the same name If the user name of a AD user is overridden with the name itself in an IPA override object SSSD adds this name twice to the alias list causing an ldb error when trying to write the user object to the cache. As a result the user is not available. This patch makes sure that there are no duplicated alias names. Resolves https://fedorahosted.org/sssd/ticket/2874 Reviewed-by: Jakub Hrozek <jhrozek@redhat.com>

SYSDB: Index the objectSIDString attribute Reviewed-by: Michal Židek <mzidek@redhat.com>

Log reason in debug message why ldb_modify failed Reviewed-by: Sumit Bose <sbose@redhat.com>

be_refresh: support groups Resolves: https://fedorahosted.org/sssd/ticket/2346 Reviewed-by: Jakub Hrozek <jhrozek@redhat.com>

be_refresh: support users Resolves: https://fedorahosted.org/sssd/ticket/2346 Reviewed-by: Jakub Hrozek <jhrozek@redhat.com>

SYSDB: sysdb_get_bool() return ENOENT & unit tests sysdb_get_bool() return ENOENT if no result is found. Unit test for sysdb_get_bool() & sysdb_set_bool() was added. This patch also fixes ldap_setup_enumeration() to handle ENOENT returned by sysdb_has_enumerated(). Resolves: https://fedorahosted.org/sssd/ticket/1991 Reviewed-by: Lukáš Slebodník <lslebodn@redhat.com>

SYSDB: Allow calling chown on the sysdb file from monitor Sysdb must be accessible for the nonroot sssd processes. Reviewed-by: Pavel Reichl <preichl@redhat.com> Reviewed-by: Simo Sorce <simo@redhat.com>

sysdb: add sysdb_attrs_add_val_safe() and sysdb_attrs_add_string_safe() sysdb_attrs_add_val_safe() works like sysdb_attrs_add_val() but checks if the attribute value to add already exists. In this case the value list is not changed. This is useful if values are added from different sources at different times to avoid LDB_ERR_ATTRIBUTE_OR_VALUE_EXISTS errors from ldb_modify() later on. sysdb_attrs_add_string_safe() does the same for string arguments Reviewed-by: Pavel Březina <pbrezina@redhat.com>

SYSDB: move sysdb_get_real_name() from sysdb.c to sysdb_search.c The sysdb.c should be reserved for utility and setup functions. Search functions belong to sysdb_search.c Keeping functions in specialized modules helps to maintain nice dependencies and in overall makes unit testing easier. Moreover, the function was not unit tested, which needed fixing. Reviewed-by: Lukáš Slebodník <lslebodn@redhat.com>

sysdb_get_real_name: allow UPN as input

SYSDB: Remove useless NULL test. There is a test for NULL after calling talloc_strndup and variable version was used a few times in strcmp before debug message. Reviewed-by: Pavel Reichl <preichl@redhat.com>

sysdb: make canonicalUserPrincipalName case-insensitive Reviewed-by: Jakub Hrozek <jhrozek@redhat.com>

SYSDB: utility call sysdb_attrs_add_lower_case_string Resolves: https://fedorahosted.org/sssd/ticket/2056 Reviewed-by: Sumit Bose <sbose@redhat.com>

Update DEBUG* invocations to use new levels Use a script to update DEBUG* macro invocations, which use literal numbers for levels, to use bitmask macros instead: grep -rl --include '*.[hc]' DEBUG . | while read f; do mv "$f"{,.orig} perl -e 'use strict; use File::Slurp; my @map=qw" SSSDBG_FATAL_FAILURE SSSDBG_CRIT_FAILURE SSSDBG_OP_FAILURE SSSDBG_MINOR_FAILURE SSSDBG_CONF_SETTINGS SSSDBG_FUNC_DATA SSSDBG_TRACE_FUNC SSSDBG_TRACE_LIBS SSSDBG_TRACE_INTERNAL SSSDBG_TRACE_ALL "; my $text=read_file(\*STDIN); my $repl; $text=~s/ ^ ( .* \b (DEBUG|DEBUG_PAM_DATA|DEBUG_GR_MEM) \s* \(\s* )( [0-9] )( \s*, ) ( \s* ) ( .* ) $ / $repl = $1.$map[$3].$4.$5.$6, length($repl) <= 80 ? $repl : $1.$map[$3].$4."\n".(" " x length($1)).$6 /xmge; print $text; ' < "$f.orig" > "$f" rm "$f.orig" done Reviewed-by: Jakub Hrozek <jhrozek@redhat.com> Reviewed-by: Stephen Gallagher <sgallagh@redhat.com> Reviewed-by: Simo Sorce <simo@redhat.com>

Make DEBUG macro invocations variadic Use a script to update DEBUG macro invocations to use it as a variadic macro, supplying format string and its arguments directly, instead of wrapping them in parens. This script was used to update the code: grep -rwl --include '*.[hc]' DEBUG . | while read f; do mv "$f"{,.orig} perl -e \ 'use strict; use File::Slurp; my $text=read_file(\*STDIN); $text=~s#(\bDEBUG\s*\([^(]+)\((.*?)\)\s*\)\s*;#$1$2);#gs; print $text;' < "$f.orig" > "$f" rm "$f.orig" done Reviewed-by: Jakub Hrozek <jhrozek@redhat.com> Reviewed-by: Stephen Gallagher <sgallagh@redhat.com> Reviewed-by: Simo Sorce <simo@redhat.com>

DB: Add sss_ldb_el_to_string_list

Add sysdb_attrs_get_int32_t

Add sysdb_attrs_add_lc_name_alias

Fix parameter name. We use '_' as prefix for output parameters. In function sysdb_get_rdn we wrongly used this prefix for input parameter, which caused some confusion when reading the code.

SYSDB: Drop redundant sysdb_ctx parameter from sysdb.c

SYSDB: Drop the sysdb_ctx parameter from the sysdb_search module

Remove the alt_db_path parameter of sysdb_init This parameter was never used. https://fedorahosted.org/sssd/ticket/1765

sysdb: try dealing with binary-content attributes https://fedorahosted.org/sssd/ticket/1818 I have here a LDAP user entry which has this attribute loginAllowedTimeMap:: AAAAAAAAAP///38AAP///38AAP///38AAP///38AAP///38AAAAAAAAA In the function sysdb_attrs_add_string(), called from sdap_attrs_add_ldap_attr(), strlen() is called on this blob, which is the wrong thing to do. The result of strlen is then used to populate the .v_length member of a struct ldb_val - and this will set it to zero in this case. (There is also the problem that there may not be a '\0' at all in the blob.) Subsequently, .v_length being 0 makes ldb_modify(), called from sysdb_set_entry_attr(), return LDB_ERR_INVALID_ATTRIBUTE_SYNTAX. End result is that users do not get stored in the sysdb, and programs like `id` or `getent ...` show incomplete information. The bug was encountered with sssd-1.8.5. sssd-1.5.11 seemed to behave fine, but that may not mean that is the absolute lower boundary of introduction of the problem.

Kill sysdb->domain Finally remove this upside-down dependency.

Move mpg flag to the domain where it belongs A sysdb contains now multiple domains, but the mpg property is a property of a specific domain not of the underlying database.

Pass the domain to upgrade functions

Add domain arguemnt to sysdb_get_real_name()

Add domain argument to sysdb_has/set_enumerated()

Pass domain to sysdb_get<pw/gr>nam() functions Also allows us to remove sysdb_subdom_get<pw/gr>nam() wrappers and restore fqnames proper value in subdomains, by testing for a parent domain being present or not.

Upgrade DB and move ranges into top level object

Make sysdb_custom_subtree_dn() require a domain.

Make sysdb_custom_dn() require a domain.

Make sysdb_domain_dn() require a domain.

Make sysdb_netgroup_base_dn() require a domain.

Make sysdb_netgroup_dn() require a domain explictly.

Make sysdb_group_dn() require a domain explictly.

Make sysdb_user_dn() require a domain explictly.

Remove the sysdb_ctx_get_domain() function. We are deprecating sysdb->domain so kill the function that gives access to this member as we should stop relying on it being available (or correct).

Refactor single domain initialization Bring it out of sysdb, which will slowly remove internal dependencies on domains and instead will always require them to be passed by callers.

Refactor sysdb initialization Change the way sysdbs are initialized. Make callers responsible for providing the list of domains. Remove the returned array of sysdb contexts, it was used only by sss_cache and not really necessary there either as that tool can easily iterate the domains. Make sysdb ctx children of their respective domains. Neither sysdb context nor domains are ever freed until a program is done so there shouldn't be any memory hierarchy issue. As plus we simplify the code by removing a destructor and a setter function.

Translate LDB_ERR_ATTRIBUTE_OR_VALUE_EXISTS to EEXIST Currently only the LDB error code indicating that an entry already exists is translated to EEXIST. To make debugging easier and return a better indication of the reason for an error in the logs this patch translates the LDB error code for an already existing attribute or value to EEXIST as well.

Null pointer dereferenced. https://fedorahosted.org/sssd/ticket/1674

Handle conversion to fully qualified usernames In subdomains we have to use fully qualified usernames. Unfortunately we have no other good option than simply removing caches for users of subdomains. This is because the memberof plugin does not support the rename operation.

Display more information on DB version mismatch https://fedorahosted.org/sssd/ticket/1589 Added check for determining, whether database version is higher or lower than expected. To distinguish it from other errors it uses following retun values (further used for appropriate error message): EMEDIUMTYPE for lower version than expected EUCLEAN for higher version than expected When SSSD or one of it's tools fails on DB version mismatch, new error message is showed suggesting how to proceed.

sysdb: add sysdb_base_dn() Add a help function which returns the ldb_dn object for the base dn of the cache.

SSH: Expire hosts in known_hosts

SYSDB: Remove unnecessary domain parameter from several sysdb calls The domain can be read from the sysdb object. Removing the domain string makes the API more self-contained.

AUTOFS: convert the existing autofs entries during a sysdb upgrade

LDB_ERR_INVALID_ATTRIBUTE_SYNTAX added to sysdb_error_to_errno.

SYSDB: Make sysdb_attrs_get_el_int() public Also rename it to sysdb_attrs_get_el_ext()

SYSDB: Use ldb_msg_add_string for simple string additions

SYSDB: Add log message for unexpected LDB errors

Add function sysdb_attrs_copy_values() This function copies all values from one sysdb_attrs structure to another

Ghost members - sysdb upgrade routine It is remotely possible to have sysdb in an inconsistent state that might need upgrade. Consider scenario when user asks for group information. Some fake users are added as a part of this operation. Before users can be fully resolved and stored properly, SSSD is shut down and upgrade is performed. In this case we need to go over all fake user records (uidNumber=0) and replace each of them with ghost record in all group objects that are stated in its memberof attribute.

SYSDB: check return value In addition to testing the number of elements, also check the return value of sysdb_attrs_get_el.

Sysdb routines for subdomains

Removed unused function sysdb_attrs_users_from_ldb_vals()

Save alias of the primary name, too

SYSDB: Save only lowercased aliases in case-insensitive domains https://fedorahosted.org/sssd/ticket/1253

Remove sysdb_get_ctx_from_list()

Keep sysdb context in domain info struct

SYSDB: index sudoUser Most of the the searches in the Sudo responder include the sudoUser attribute. Indexing it will make the responder faster.

SYSDB: Add sysdb_attrs_get_uint16_t

SYSDB: Add indexes for servicePort and serviceProtocol

SYSDB: Redundant check is redundant. Coverity 12480

sysdb_get_bool() and sysdb_get_bool() functions

SUDO Integration review issues

Export the function to convert ldb_result to sysdb_attrs It will be reused later in the sudo responder

Use the case sensitivity flag in the LDAP provider

sysdb_get_real_name helper function

SYSDB: add index for nameAlias

SYSDB: New source file for sysdb upgrade routines

Use explicit base 10 for converting strings to integers https://fedorahosted.org/sssd/ticket/1013

Add sysdb interface to get name aliases

Return the first value of name if the multivalued name attribute does not match RDN https://fedorahosted.org/sssd/ticket/926

sysdb refactoring: memory context deleted This patch deletes memory context parameter in those places in sysdb where it is not necessary. The code using modified functions has been updated. Tests updated as well.

sysdb refactoring: deleted domain variables in sysdb API The patch also updates code using modified functions. Tests have also been adjusted.

Added sysdb_ctx_get_domain function

sysdb refactoring: renamed ctx variable to sysdb

Fix potential double-free issue tmp_ctx is a child of ctx.

Cancel sysdb upgrade transaction if commit fails

Allow NULL memctx in sysdb_custom_subtree_dn ldb_dn_new_fmt() has a bug and cannot take a NULL memory context

Added sysdb_attrs_get_bool() function

Make sysdb_ctx_list public structure Also create a routine to initialize it

Added originalDN to attributes with case-insensitive search https://fedorahosted.org/sssd/ticket/808

Reopen the LDB after modifying it If we change any of the special entries such as indexes or plugins, we need to close and reopen the LDB to ensure that they take effect.

Run all appropriate upgrades Previously, if we were upgrading from version 0.4 or older, we would only run sysdb_upgrade_04() and exit, instead of also running sysdb_upgrade_05()

Don't leak memory if sysdb_domain_init() fails

sysdb: use header defined macros instead of explicit values

Check result of talloc_strdup() properly Coverity 10737

Don't crash on error if _name parameter unspecified Coverity 10738

Don't crash if we get a multivalued name without an origDN Coverity 10740 and 10739

Add sysdb_attrs_primary_name_list() routine This routine will replace the use of sysdb_attrs_to_list() for any case where we're trying to get the name of the entry. It's a necessary precaution in case the name is multi-valued.

Add sysdb_attrs_primary_name() This function will check a sysdb_attrs struct for the primary name of the entity it represents. If there are multiple entries, it will pick the one that matches the RDN. If none match, it will throw an error.

Create sysdb_get_rdn() function This function takes a DN formatted string and returns the RDN value from it.

Check LDB_MODULES_PATH for sysdb

Introduce sysdb_ldb_connect()

Make 'make check' look nice again With current libldb releases 'make check' will print a lot of 'unable to dlopen' messages although the test will succeed. This patch place the memberof module into a directory of its own to avoid these messages. Additionally this patch introduces TESTS_ENVIRONMENT which allows us to remove the SYSDB_TEST preprocessor definition.

Clear up -Wunused-but-set-variable warnings

Add additional indexing for sysdb Adds an index for dataExpireTimestamp This is used for determining which users need to be removed during the cleanup task. If enumeration is enabled (or huge numbers of users have been cached), the cleanup task runs very slowly due to the non-indexed search. Also adds an index for ONELEVEL lookups, to speed up situations where we would need to request all entries under a particular node in the LDB.

Add sysdb_has_enumerated and sysdb_set_enumerated helper functions Includes a unit test

Fix const cast issue with sysdb_attrs_users_from_str_list

Sanitize sysdb DN helpers

Add sysdb utility function for sanitizing DN

Always use uint32_t for UID/GID numbers

Add sysdb_attrs_get_ulong utility function

Add sysdb_netgroup_base_dn()

Netgroups sysdb API

Fix sysdb_attrs_to_list

Fix sysdb_group_dn_name

Dead assignments cleanup in various places in SSSD Three assignments deleted, two return code inspection added. Also found and fixed one critical bug caused by dead assignment. Ticket: #590

Add sysdb_group_dn_name utility function

Add sysdb_attrs_to_list() utility function

Add sysdb_attrs_get_string_array()

Add ldap_access_filter option This option (applicable to access_provider=ldap) allows the admin to set an additional LDAP search filter that must match in order for a user to be granted access to the system. Common examples for this would be limiting access to users by in a particular group, for example: ldap_access_filter = memberOf=cn=access_group,ou=Groups,dc=example,dc=com

sysydb: Finally stop using a common event context This commit completes the migration to a synchronous sysdb

sysdb: remove remaining traces of sysdb_handle

sysdb: remove async transactions not used anymore

sysdb: add synchronous transaction functions

sysdb: start conversion from async to sync allow nesting during transition

Add forgotten \n in DEBUG statements Logs from confdb with missing '\n' in the DEBUG statements annoyed me so I decided to fix them. I also made a quick grep through the code and found other places so I fixed them too.

Add --with-test-dir option to configure All 'make check' tests will chdir() into this directory before running the suite. This provides the option of having temporary files generated in a tmpfs or ramdisk

Rename server/ directory to src/ Also update BUILD.txt