sysdb: add userMappedCertificate to the index Related to https://pagure.io/SSSD/sssd/issue/3503 Reviewed-by: Fabiano Fidêncio <fidencio@redhat.com>

sysdb: remove IDXONE and objectClass from users and groups This patch does the needed sysdb update for the previous to patches. It removes the one-level search index IDXONE and replaces objectClass with objectCategory in the user and group objects. Related to https://pagure.io/SSSD/sssd/issue/3503 Reviewed-by: Fabiano Fidêncio <fidencio@redhat.com> Reviewed-by: Jakub Hrozek <jhrozek@redhat.com>

Fix minor spelling mistakes Merges: https://pagure.io/SSSD/sssd/pull-request/3556 Reviewed-by: Lukáš Slebodník <lslebodn@redhat.com>

sysdb: add missing indices Resolves https://pagure.io/SSSD/sssd/issue/3472 Reviewed-by: Jakub Hrozek <jhrozek@redhat.com>

SYSDB: Internally expose sysdb_search_ts_matches() This function will be used in the follow-up patches. As it's going to be "exposed", let's also rename it from search_ts_matches() to sysdb_search_ts_matches(). Related: https://pagure.io/SSSD/sssd/issue/3369 Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> Reviewed-by: Jakub Hrozek <jhrozek@redhat.com>

SUBDOMAINS: Allow use_fully_qualified_names for subdomains Allow option use_fully_qualified_names in subdomain section. This option was recently added to subdomain_inherit. Resolves: https://pagure.io/SSSD/sssd/issue/3337 Reviewed-by: Fabiano Fidêncio <fidencio@redhat.com>

UTIL: Store UPN suffixes when creating a new subdomain We used to store UPN suffixes pointer into the domain structure only if the domain changed, not when a new domain was created. As an effect, the enterprise principals flag was not enabled unless a domain changed, preventing logins with enterprise principals. Reviewed-by: Sumit Bose <sbose@redhat.com>

SYSDB: Upgrade sysdb to use qualified names for users and groups, sudo rules and override objects Runs a sysdb upgrade that changes objects that represent users, groups, sudo rules and overrides to the new schema, which uses the fully qualified names. Reviewed-by: Sumit Bose <sbose@redhat.com>

SYSDB: Allow passing a context to sysdb upgrade functions We decide on whether to upgrade or not based on a pointer value, not a boolean. This pointer points to a structure that the upgrade invoker (typically the monitor) can use to fill auxilary data the sysdb upgrade has no means of instantiating. Reviewed-by: Sumit Bose <sbose@redhat.com>

sysdb: Use ldb_result as output in sysdb_search_ts_{users,groups} Passing address of unsigned to the output argument size_t causes access out of boundaries for type unsigned and and wrong data on big endian. It looks like functions sysdb_search_ts_{users,groups} need to store results in structure ldb_result anyway for further processing. Therefore it will be better to convert output arguments size_t* + ldb_message*** into structure ldb_result and avoid using additional helper variable with type size_t before each invocation of these functions. Reviewed-by: Sumit Bose <sbose@redhat.com>

SYSDB: Check if group attributes differ before saving a group Adds a new function sysdb_entry_attrs_diff() used in group saving code. This function is used to check if the result of updating a group would result in actually changing the sysdb entry -- often, we would try to dump the same data to the cache during update. If that's the case, the update code now only updates the timestamp cache, avoiding costly writes. Reviewed-by: Sumit Bose <sbose@redhat.com>

SYSDB: If modifyTimestamp is the same, only update the TS cache Resolves: https://fedorahosted.org/sssd/ticket/2602 If the entry being saved contains the original modifyTimestamp attribute and the modifyTimestamp attribute is the same as the one we already saved to the timestamp cache, only the expire timestamps in the asynchronous timestamp cache will be bumped and the sysdb code will avoid writes to the main cache completely. If the modifyTimestamp is either missing or differs, we assume the entry had changed and do a full write to the main cache. Also amends the generic sysdb_set_attrs* and similar functions that their results is also reflected in the timestamps cache. Reviewed-by: Sumit Bose <sbose@redhat.com>

SYSDB: Search the timestamp caches in addition to the sysdb cache When a sysdb entry is searched, the sysdb cache is consulted first for users or groups. If an entry is found in the sysdb cache, the attributes from the timestamp cache are merged to return the full and up-to-date set of attributes. The merging is done with a single BASE search which is a direct lookup into the underlying key-value database, so it should be relatively fast. More complex merging is done only for enumeration by filter which is currently done only via the IFP back end and should be quite infrequent, so I hope we can justify a more complex merging there. Reviewed-by: Sumit Bose <sbose@redhat.com>

SYSDB: Open a timestamps cache for caching domains For all domain types, except the local domain, open a connection to a new ldb file located at /var/lib/sss/db names timestamps_$domain.ldb. Constructs the ldb file path manually in sysdb_check_upgrade_02() but that should be acceptable because nobody should be running such an old cache these days anyway. Reviewed-by: Sumit Bose <sbose@redhat.com>

SYSDB: Add a second, timestamp-only ldb cache Uses the generic functions in sysdb_init.c to open a new ldb database. The path to the database and the ldb context are stored in the sysdb_ctx. The database will be used to store ephemeral attributes such as timestamps. Because these attributes are not required for SSSD operation and the intent is for writes to this cache to be very fast, the database is opened with LDB_FLG_NOSYNC flag. At the same time, none of the attributes of the cache is required for sssd operation, so if we fail to open the database, we just start over. Adds a separate base LDIF with attributes that are supposed to be indexed in the timestamp database as well as a separate timestamp cache version. Reviewed-by: Sumit Bose <sbose@redhat.com>

SYSDB: Refactor database connection Changes the functions in sysdb_init.c so that the functions are usable to initialize a second cache to store timestamps. In particular, functions that operated on sysdb->ldb now operate on a generic ldb context so that a new ldb_ts context can be used later. Existing functions that initialize the sysdb cache call the generic functions with sysdb->ldb as a parameter. Splits out a function to initialize an empty ldb database with a generic LDIF. Splits out the sysdb upgrade function because the upgrade will only be used by the sysdb cache, the timestamp cache will start with a different version and might receive separate upgrade functions in the future. The ldb connection function accepts ldb flags parameter, currently unused. Reviewed-by: Sumit Bose <sbose@redhat.com>

SYSDB: Track transaction nesting in sysdb_ctx Adds an integer that tracks how deeply nested we are in sysdb transactions. This will become useful later, because generally we are only interested in level-0 transactions when probing, so we'll want to pass the transaction nesting to the systemtap probes. Reviewed-by: Lukáš Slebodník <lslebodn@redhat.com>

sysdb: add sysdb_{add,replace,delete}_ulong() As the add_ulong() convenience can add, replace or remove a unsigned long according to the operation received as its argument, some confusion can easily happen due to its misleading name. In order to improve the explicitness of our code, let's introduce sysdb_add_ulong(), sysdb_replace_ulong() and sysdb_delete_ulong(). These new functions are basically wrappers of add_ulong() (now sysdb_ldb_msg_ulong_helper()), calling it using the proper flag according to each function. Any code previously using add_ulong() is now adapted to use these brand new functions. Related: https://fedorahosted.org/sssd/ticket/1656 Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org> Reviewed-by: Petr Cech <pcech@redhat.com>

sysdb: add sysdb_{add,replace,delete}_string() As the add_string() convenience can add, replace or delete a string according to the operation received as its argument, some confusion can easily happen due to its misleading name. In order to improve the explicitness of our code, let's introduce sysdb_add_string(), sysdb_replace_string() and sysdb_delete_string(). These new functions are basically wrappers of add_string() (now sysdb_ldb_msg_string_helper()), calling it using the proper flag according to each function. Any code previously using add_string() is now adapted to use these brand new functions. Resolves: https://fedorahosted.org/sssd/ticket/1656 Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org> Reviewed-by: Petr Cech <pcech@redhat.com>

SYSDB: Index the objectSIDString attribute Reviewed-by: Michal Židek <mzidek@redhat.com>

UTIL/SYSDB: Move new_subdomain() to sysdb_subdomains.c and make it private In order to make updating the subdomain list a two-step process. Therefore we need to make sure that update_subdomains() is the only interface towards the SSSD that changes the subdomain list. Move the new_subdomain() function to sysdb_subdomains.c and only make it available through a private header so it's usable by unit tests. Reviewed-by: Sumit Bose <sbose@redhat.com>

sysdb: make canonicalUserPrincipalName case-insensitive Reviewed-by: Jakub Hrozek <jhrozek@redhat.com>

Kill sysdb->domain Finally remove this upside-down dependency.

Move mpg flag to the domain where it belongs A sysdb contains now multiple domains, but the mpg property is a property of a specific domain not of the underlying database.

Pass the domain to upgrade functions

Upgrade DB and move ranges into top level object

The Big sysdb/domain split-up! This commit is the first of a complex work of untangling domain and sysdb. It turns out the idea of keeping a reference to the domain within the sysdb was a poor one so we need to split the domain out and change all functions that needs one to get it explicitly from their callers.

Handle conversion to fully qualified usernames In subdomains we have to use fully qualified usernames. Unfortunately we have no other good option than simply removing caches for users of subdomains. This is because the memberof plugin does not support the rename operation.

SSH: Expire hosts in known_hosts

AUTOFS: convert the existing autofs entries during a sysdb upgrade

Ghost members - sysdb upgrade routine It is remotely possible to have sysdb in an inconsistent state that might need upgrade. Consider scenario when user asks for group information. Some fake users are added as a part of this operation. Before users can be fully resolved and stored properly, SSSD is shut down and upgrade is performed. In this case we need to go over all fake user records (uidNumber=0) and replace each of them with ghost record in all group objects that are stated in its memberof attribute.

SYSDB: index sudoUser Most of the the searches in the Sudo responder include the sudoUser attribute. Indexing it will make the responder faster.

SYSDB: Add indexes for servicePort and serviceProtocol

SYSDB: Move add_string and add_ulong to sysdb_private.h

Fixed a typo in sysdb_upgrade_07() declaration

SYSDB: add index for nameAlias

SYSDB: New source file for sysdb upgrade routines

Make sysdb_ctx_list public structure Also create a routine to initialize it

Added originalDN to attributes with case-insensitive search https://fedorahosted.org/sssd/ticket/808

Add additional indexing for sysdb Adds an index for dataExpireTimestamp This is used for determining which users need to be removed during the cleanup task. If enumeration is enabled (or huge numbers of users have been cached), the cleanup task runs very slowly due to the non-indexed search. Also adds an index for ONELEVEL lookups, to speed up situations where we would need to request all entries under a particular node in the LDB.

sysydb: Finally stop using a common event context This commit completes the migration to a synchronous sysdb

sysdb: remove remaining traces of sysdb_handle

Rename server/ directory to src/ Also update BUILD.txt