chap-synchronization.xml revision de5564d97c6a376f8309aa693f2fc79054715dc2
<?xml version="1.0" encoding="UTF-8"?>
<!--
! CCPL HEADER START
!
! This work is licensed under the Creative Commons
! Attribution-NonCommercial-NoDerivs 3.0 Unported License.
! To view a copy of this license, visit
! or send a letter to Creative Commons, 444 Castro Street,
! Suite 900, Mountain View, California, 94041, USA.
!
! You can also obtain a copy of the license at
! See the License for the specific language governing permissions
! and limitations under the License.
!
! If applicable, add the following below this CCPL HEADER, with the fields
! enclosed by brackets "[]" replaced with your own identifying information:
! Portions Copyright [yyyy] [name of copyright owner]
!
! CCPL HEADER END
!
! Copyright 2011-2012 ForgeRock AS
!
-->
<chapter xml:id='chap-synchronization'
xmlns='http://docbook.org/ns/docbook'
version='5.0' xml:lang='en'
xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'
xsi:schemaLocation='http://docbook.org/ns/docbook http://docbook.org/xml/5.0/xsd/docbook.xsd'
xmlns:xlink='http://www.w3.org/1999/xlink'
xmlns:xinclude='http://www.w3.org/2001/XInclude'>
<title>Configuring Synchronization</title>
<indexterm>
<primary>Synchronization</primary>
</indexterm>
<para>One of the core services of OpenIDM is synchronizing identity data from
different resources. This chapter explains what you must know to get started
configuring OpenIDM's flexible synchronization mechanism, and illustrates the
concepts with examples.</para>
<section xml:id="sync-types">
<title>Types of Synchronization</title>
<indexterm>
<primary>Synchronization</primary>
<secondary>Direct (push)</secondary>
</indexterm>
<para>Synchronization happens either when OpenIDM receives a change directly,
or when OpenIDM discovers a change on an external resource.</para>
<para>For direct changes to OpenIDM, OpenIDM immediately pushes updates to
all external resources configured to receive the updates. A direct change
can originate not only as a write request through the REST interface, but
also as an update resulting from reconciliation with another resource.</para>
<variablelist>
<para>OpenIDM discovers changes on external resources through reconciliation,
and through LiveSync.</para>
<varlistentry>
<term>Reconciliation</term>
<listitem>
<indexterm>
<primary>Reconciliation</primary>
</indexterm>
<para>In identity management, <firstterm>reconciliation</firstterm> is the
process of bidirectional synchronization of objects between different data
stores. Reconciliation applies mainly to user objects, though OpenIDM
can reconcile any objects, including groups and roles.</para>
<para>To perform reconciliation, OpenIDM analyzes both source and target
systems to uncover the differences that it must reconcile. Reconciliation
can therefore be a heavyweight process. When working with large data sets,
finding all changes can be more work than processing the changes.</para>
<para>Reconciliation is, however, thorough. It recognizes system error
conditions and catches changes that might be missed by the more
lightweight LiveSync mechanism. Reconciliation therefore serves as the
basis for compliance and reporting functionality.</para>
</listitem>
</varlistentry>
<varlistentry>
<term>LiveSync</term>
<listitem>
<indexterm>
<primary>LiveSync</primary>
</indexterm>
<para><firstterm>LiveSync</firstterm> performs the same job as
reconciliation. LiveSync relies on a change log on the external resource
to determine which objects have changed.</para>
<para>LiveSync is intended to react quickly to changes as they happen.
LiveSync is however a best effort mechanism that in some cases can miss
changes.</para>
<para>Furthermore, not all resources provide the change log mechanism
that LiveSync requires. The change long provides OpenIDM with a list of
objects changed since the last request such that OpenIDM does not need to
scan all objects for changes. OpenDJ provides an external change log.
Active Directory also provides a change log.</para>
</listitem>
</varlistentry>
</variablelist>
<para>In all cases, OpenIDM relies on mappings that you configure to
determine what to synchronize and how to carry out synchronization.
LiveSync relies on the set of mappings that you configure once per OpenIDM
server, whereas reconciliation also allows you to configure specific mappings
for a particular reconciliation job.</para>
<para>You must trigger OpenIDM to poll for changes on external resources,
usually by scheduling reconciliation or LiveSync as described in the chapter
on <link xlink:href="integrators-guide#chap-scheduler-conf"
Synchronization</citetitle></link>. Alternatively, you can start
reconciliation through the REST interface.</para>
<screen width="91"><?dbfo pgwide="1"?>$ curl
--header "X-OpenIDM-Username: openidm-admin"
--header "X-OpenIDM-Password: openidm-admin"
--request POST
"http://localhost:8080/openidm/sync?_action=recon&mapping=systemLdapAccounts_managedUser"</screen>
</section>
<section xml:id="sync-flexible-data">
<title>Flexible Data Model</title>
<indexterm>
<primary>Objects</primary>
<secondary>Managed objects</secondary>
</indexterm>
<para>Identity management software tends to favor either a meta-directory
data model, where all data are mirrored in a central repository, or a
virtual data model, where only a minimum set of attributes are stored
centrally, and most are loaded on demand from the external resources
on which they are stored. The meta-directory model offers fast access at
the risk of getting out-of-date data. The virtual model guarantees fresh
data, but pays for that guarantee in terms of performance.</para>
<para>OpenIDM leaves the data model choice with you. You determine the right
trade offs for a particular deployment. OpenIDM does not hard code any
particular schema or set of attributes stored in the repository. Instead,
you define how external system objects map onto managed objects, and OpenIDM
dynamically updates the repository to store the managed object attributes
that you configure.</para>
<para>You can, for example, choose to follow the data model defined in the
Simple Cloud Identity Management (<link xlink:show="new"
>SCIM</link>) specification. The following object represents a SCIM
user.</para>
<programlisting language="javascript">
{
"userName": "james1",
"familyName": "Berg",
"givenName": "James",
"email": [
"james1@example.com"
],
"description": "Created by OpenIDM REST.",
"password": "asdfkj23",
"displayName": "James Berg",
"phoneNumber": "12345",
"employeeNumber": "12345",
"userType": "Contractor",
"title": "Vice President",
"active": true
}</programlisting>
<note>
<para>Avoid using the dash character ( <literal>-</literal> ) in property
names, like <literal>last-name</literal>, as dashes in names make
JavaScript syntax more complex. If you cannot avoid the dash, then write
<literal>source['last-name']</literal> instead of
</note>
</section>
<section xml:id="basic-flow">
<title>Basic Data Flow Configuration</title>
<para>Data flow for synchronization involves three types of configuration
files, two of which you typically edit, and also a links table that OpenIDM
maintains in its repository, as well as scripts needed to check objects and
manipulate attributes. The two types of configuration files you edit are the
connector configuration files, with one file per external resource, and the
synchronization mappings file, with one file per OpenIDM instance.</para>
<variablelist>
<varlistentry>
<term>Connector configuration files</term>
<listitem>
<indexterm>
<primary>Synchronization</primary>
<secondary>Connectors</secondary>
</indexterm>
<para>Connector configuration files are described in the chapter on <link
xlink:href="integrators-guide#chap-resource-conf"
External Resources</citetitle></link>. Connector configuration files are
>resource-name</replaceable>.json</filename>,
where <replaceable>resource-name</replaceable> reflects the connector
technology and external resource, such as
<literal>openicf-xml</literal>.</para>
<para>An excerpt from an example connector configuration follows. The
example shows the name for the connector and two attributes of an account
object type. In the attribute mapping definitions, the attribute name is
mapped from the <literal>nativeName</literal>, the attribute name used on
the external resource, to the attribute name used in OpenIDM. Thus the
example shows that <literal>sn</literal> from LDAP is mapped to
<literal>lastName</literal> in OpenIDM. The <literal>homePhone</literal>
attribute can have multiple values.</para>
<programlisting language="javascript">
{
"name": "MyLDAP",
"objectTypes": {
"account": {
"lastName": {
"type": "string",
"required": true,
"nativeName": "sn",
"nativeType": "string"
},
"homePhone": {
"type": "array",
"items": {
"type": "string",
"nativeType": "string"
},
"nativeName": "homePhone",
"nativeType": "string"
}
}
}
}</programlisting>
<para>In order for OpenIDM to access external resource objects and
attributes, the object and its attributes must match the connector
configuration. Also, the connector file strictly maps external resource
objects to OpenIDM objects. To construct attributes and to manipulate
their values, you use the synchronization mappings file.</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Synchronization mappings file</term>
<listitem>
<indexterm>
<primary>Synchronization</primary>
<secondary>Mappings</secondary>
</indexterm>
<indexterm>
<primary>Mappings</primary>
</indexterm>
<para>The synchronization mappings file is
represent the core configuration for OpenIDM synchronization.</para>
Each mapping specifies how attributes from source objects correspond to
attributes on target objects. The source and target indicate the direction
for the data flow, so you must define a mapping for each data flow. For
example, if you want data flows from an LDAP server to the repository
and also from the repository to the LDAP server, you must define two
separate mappings.</para>
<para>You identify external resource sources and targets as
<literal>system/<replaceable>name</replaceable>/<replaceable
>object-type</replaceable></literal>, where
<replaceable>name</replaceable> is the name used in the connector
configuration file, and <replaceable>object-type</replaceable> is the
object defined in the connector configuration file list of object types.
For objects in OpenIDM's internal repository, you use
<literal>managed/<replaceable>object-type</replaceable></literal>, where
<replaceable>object-type</replaceable> is defined in
by convention is set to a string of the form
<literal><replaceable>source</replaceable>_<replaceable
>target</replaceable></literal>, as shown in the following example.</para>
<programlisting language="javascript" xml:id="basic-ldap-mapping">
{
"mappings": [
{
"name": "systemLdapAccounts_managedUser",
"properties": [
{
"target": "familyName",
"source": "lastName"
},
{
"target": "homePhone",
"source": "homePhone"
},
{
"target": "phoneExtension",
"default": "0047"
},
{
"target": "mail",
"comment": "Set mail if non-empty.",
"source": "email",
"condition": {
"type": "text/javascript",
"source": "(source.email != null)"
}
},
{
"target": "displayName",
"source": "";
"transform": {
"type": "text/javascript",
"source": "(source.lastName +', ' + source.firstName;)"
}
}
]
}
]
}</programlisting>
<para>In this example, the source is the external resource,
<literal>MyLDAP</literal>, and the target is OpenIDM's repository,
specifically the managed user objects. The <literal>properties</literal>
reflect OpenIDM attribute names. For example, the mapping has the
attribute <literal>lastName</literal> defined in the
<literal>MyLDAP</literal> connector configuration file mapped to
<literal>familyName</literal> in the OpenIDM managed user object. Notice
that the attribute names come from the connector configuration, rather
than the external resource itself.</para>
<indexterm>
<primary>Synchronization</primary>
<secondary>Creating attributes</secondary>
</indexterm>
<para>You can create attributes on the target as part of the
mapping. In the example, OpenIDM creates a
<literal>phoneExtension</literal> attribute with a default value of
<literal>0047</literal>.</para>
<indexterm>
<primary>Synchronization</primary>
<secondary>Conditions</secondary>
</indexterm>
<para>You can also set up conditions under which OpenIDM maps attributes
as shown for the email attribute in the example. By default, OpenIDM
synchronizes all attributes. In the example, the mail attribute is set
only if the script for the condition returns
<literal>true</literal>.</para>
<indexterm>
<primary>Synchronization</primary>
<secondary>Transforming attributes</secondary>
</indexterm>
<para>OpenIDM lets you transform attributes as well. In the example, the
<literal>displayName</literal> attribute is set using a combination of
the <literal>lastName</literal> and <literal>firstName</literal> attribute
values from the source. For transformations, the <literal>source</literal>
property is optional. However, the source object is only available
when you specify the <literal>source</literal> property. Therefore, in
<literal>displayName</literal>, the example specifies <literal>"source" :
""</literal>.</para>
<para>To add a flow from the repository to <literal>MyLDAP</literal>,
<literal>managedUser_systemLdapAccounts</literal>.</para>
<mediaobject xml:id="figure-service-tree"><?dbfo pgwide="1"?>
<alt>OpenIDM name space and object paths</alt>
<imageobject>
</imageobject>
<textobject>
<para>The image shows paths to objects in the OpenIDM name space.</para>
</textobject>
<caption>
<para>OpenIDM stores managed objects in the repository, and exposes
external resources are exposed under
</caption>
</mediaobject>
<variablelist>
<indexterm>
<primary>Synchronization</primary>
<secondary>Filtering</secondary>
</indexterm>
<para>By default, OpenIDM synchronizes all objects matching those defined
in the connector configuration for the resource. Many connectors let you
limit the scope of objects the connector accesses. For example, the LDAP
connector lets you specify base DNs and LDAP filters so you need not
access every entry in the directory. Yet, OpenIDM also lets you filter
what is considered a valid source or valid target for synchronization by
using JavaScript code. To apply these filters, use the
<literal>validSource</literal>, and <literal>validTarget</literal>
properties in your mapping.</para>
<varlistentry>
<term>validSource</term>
<listitem>
<para>A script that determines if a source object is valid to be
mapped. The script yields a boolean value: <literal>true</literal>
indicates the source object is valid; <literal>false</literal> can be
used to defer mapping until some condition is met. In the root scope,
the source object is provided in the <literal>"source"</literal>
property. If the script is not specified, then all source objects are
considered valid.</para>
<programlisting language="javascript">
{
"validTarget": {
"type": "text/javascript",
"source": "target.employeeType == 'internal'"
}
}</programlisting>
</listitem>
</varlistentry>
<varlistentry>
<term>validTarget</term>
<listitem>
<para>A script used during reconciliation's second phase, that determines if a target
object is valid to be mapped. The script yields a boolean value:
<literal>true</literal> indicates the target object is valid;
<literal>false</literal> indicates that the target object should not be
included in reconciliation. In the root scope, the source object is
provided in the <literal>"target"</literal> property. If the script is
not specified, then all target objects are considered valid for
mapping.</para>
<programlisting language="javascript">
{
"validSource": {
"type": "text/javascript",
"source": "source.ldapPassword != null"
}
}</programlisting>
</listitem>
</varlistentry>
</variablelist>
<para>During synchronization, your scripts always have access to a
<literal>source</literal> object and a <literal>target</literal> object.
Examples already shown in this section use <literal>source.<replaceable
>attributeName</replaceable></literal> to retrieve attributes from the
source objects. Your scripts can also write to target attributes using
<literal>target.<replaceable>attributeName</replaceable></literal>
syntax.</para>
<programlisting language="javascript">
{
"onUpdate": {
"type": "text/javascript",
}
}</programlisting>
<para>See the <link xlink:href="integrators-guide#appendix-scripting"
Reference</citetitle></link> appendix for more on scripting.</para>
</listitem>
</varlistentry>
</variablelist>
<section xml:id="sync-encrypted-values">
<title>Using Encrypted Values</title>
<indexterm>
<primary>Synchronization</primary>
<secondary>Encryption</secondary>
</indexterm>
<para>OpenIDM supports reversible encryption of attribute values for
managed objects. Attribute values to encrypt include passwords (if passwords
are not already encrypted on the external resource, which would usually exclude them from the synchronization process, see the chapter about <link
xlink:href="integrators-guide#chap-passwords"
xlink:role="http://docbook.org/xlink/role/olink">
<citetitle>Passwords</citetitle>
</link>), and also authentication
questions, credit card numbers, and social security numbers.</para>
<para>You configure encryption in the managed object configuration. The file
example shows a managed object configuration that encrypts and decrypts
<literal>securityAnswer</literal>, <literal>ssn</literal>, and
<literal>password</literal> attributes using the default symmetric
key, and additional scripts for extra passwords.</para>
<programlisting language="javascript">
{
"objects": [
{
"name": "user",
"properties": [
{
"name": "securityAnswer",
"encryption": {
"key": "openidm-sym-default"
}
},
{
"name": "ssn",
"encryption": {
"key": "openidm-sym-default"
}
},
{
"name": "password",
"encryption": {
"key": "openidm-sym-default"
}
}
],
"onStore": {
"type": "text/javascript",
"file": "script/encryptExtraPassword.js"
},
"onRetrieve": {
"type": "text/javascript",
"file": "script/decryptExtraPassword.js"
}
}
]
}</programlisting>
<para>Do not use the default symmetric key,
<literal>openidm-sym-default</literal>, in production. See the chapter on
<link xlink:href="integrators-guide#chap-security"
& Hardening OpenIDM</citetitle></link> for instructions on adding your
own symmetric key.</para>
</section>
<section xml:id="constructing-attributes">
<title>Constructing & Manipulating Attributes</title>
<indexterm>
<primary>Synchronization</primary>
<secondary>Creating attributes</secondary>
</indexterm>
<para>OpenIDM lets you construct and manipulate attributes using scripts
triggered when an object is created (onCreate), updated (onUpdate), or
deleted (onDelete), or when a link is created (onLink), or removed
(onUnlink).</para>
<para>The following example derives a DN for an LDAP entry when the entry
is created in the internal repository.</para>
<programlisting language="javascript">
{
"onCreate": {
"type": "text/javascript",
"source":
"target.dn = 'uid=' + source.uid + ',ou=people,dc=example,dc=com'"
}
}</programlisting>
</section>
<section xml:id="reusing-links">
<title>Reusing Links</title>
<indexterm>
<primary>Synchronization</primary>
<secondary>Reusing links</secondary>
</indexterm>
<para>When two mappings exist to sync the same objects bidirectionally,
you can use the <literal>links</literal> property in one mapping to have
OpenIDM use the same internally managed link for both mappings. Otherwise,
if no <literal>links</literal> property is specified, OpenIDM maintains a
link for each mapping.</para>
<para>The following excerpt shows two mappings, one from MyLDAP accounts
to managed users, and another from managed users to MyLDAP accounts. In
the second mapping, the <literal>link</literal> property tells OpenIDM
to reuse the links created in the first mapping, rather than create new
links.</para>
<programlisting language="javascript">
{
"mappings": [
{
"name": "systemMyLDAPAccounts_managedUser",
},
{
"name": "managedUser_systemMyLDAPAccounts",
"links": "systemMyLDAPAccounts_managedUser"
}
]
}</programlisting>
</section>
</section>
<section xml:id="handling-sync">
<title>Synchronization Situations & Actions</title>
<indexterm>
<primary>Synchronization</primary>
<secondary>Situations</secondary>
</indexterm>
<indexterm>
<primary>Synchronization</primary>
<secondary>Actions</secondary>
</indexterm>
<para>During synchronization, OpenIDM categorizes objects by situation.
Situations are characterized by whether an object exists on a source or
target system, whether OpenIDM has registered a link between the source
object and the target object, and whether the object is considered valid.
OpenIDM takes action depending on the situation.</para>
<para>You can define actions for particular situations in the
<literal>policies</literal> section of a synchronization mapping, as
shown in the following excerpt.</para>
<programlisting language="javascript">
{
"policies": [
{
"situation": "CONFIRMED",
"action": "UPDATE"
},
{
"situation": "FOUND",
"action": "IGNORE"
},
{
"situation": "ABSENT",
"action": "CREATE"
},
{
"situation": "AMBIGUOUS",
"action": "IGNORE"
},
{
"situation": "MISSING",
"action": "IGNORE"
},
{
"situation": "UNQUALIFIED",
"action": "IGNORE"
},
{
"situation": "UNASSIGNED",
"action": "IGNORE"
}
]
}</programlisting>
<para>If you do not define a policy for a particular situation, OpenIDM
takes the default action for the situation.</para>
<para>The situations and their corresponding actions are described in the
sections below.</para>
<section xml:id="sync-situations">
<title>Synchronization Situations</title>
<para>OpenIDM performs synchronization action in two phases. First, OpenIDM
performs the so called source reconciliation, where OpenIDM accounts for source objects
and associated links based on the mapping configured. Second, OpenIDM runs
the target reconciliation, where OpenIDM iterates over the target objects not
processed in the first phase.</para>
<orderedlist>
<para>During reconciliation OpenIDM builds three lists, assigning values
to the objects to reconcile.</para>
<listitem>
<para>All valid objects from the source</para>
<para>OpenIDM assigns valid source objects <literal>qualifies=1</literal>.
Invalid objects, including those not found in the source system, and those
filtered out by the script specified in the <literal>validSource</literal>
property get <literal>qualifies=0</literal>.</para>
</listitem>
<listitem>
<para>All records from the appropriate link table</para>
<para>Objects with corresponding links in the link table of the repository
get <literal>link=1</literal>. Objects without corresponding links get
<literal>link=0</literal>.</para>
</listitem>
<listitem>
<para>All valid objects on the target system</para>
<para>Object found in the target system get <literal>target=1</literal>.
Objects not found in the target system get
<literal>target=0</literal>.</para>
</listitem>
</orderedlist>
<variablelist>
<para>Based on the values assigned to objects during source reconciliation,
OpenIDM assigns situations, listed here with their default actions.</para>
<varlistentry>
<term>"CONFIRMED" (qualifies=1, link=1, target=1)</term>
<listitem>
<para>The mapping qualifies for a target object, and a link to an existing target
object was found. Detected during change events and reconciliation. Default action: "UPDATE".</para>
</listitem>
</varlistentry>
<varlistentry>
<term>"FOUND" (qualifies=1, link=0, target=1)</term>
<listitem>
<para>The mapping qualifies for a target object, there is no link to a
target object, and there is a single target object, correlated by the logic found in the correlationQuery.
Detected during change events and reconciliation. Default action:
"UPDATE".</para>
</listitem>
</varlistentry>
<varlistentry>
<term>"ABSENT" (qualifies=1, link=0, target=0)</term>
<listitem>
<para>The mapping qualifies for a target object, there is no link to a
target object, and there is no correlated target object found.
Detected during change events and reconciliation. Default action:
"CREATE".</para>
</listitem>
</varlistentry>
<varlistentry>
<term>"AMBIGUOUS" (qualifies=1, link=0, target>1)</term>
<listitem>
<para>The mapping qualifies for a target object, there is no link to a
target object, but there is more than one correlated target object. Detected during source object changes and reconciliation. Default
action: "EXCEPTION".</para>
</listitem>
</varlistentry>
<varlistentry>
<term>"MISSING" (qualifies=1, link=1, target=0)</term>
<listitem>
<para>The mapping is qualified for a target object, and there is a
qualified link to a target object, but the target object is missing.
Only detected during reconciliation and source object changes in synchronous mappings.
Default action: "EXCEPTION".</para>
</listitem>
</varlistentry>
<varlistentry>
<term>"UNQUALIFIED" (qualifies=0, link=0 or 1, target=1 or >1)</term>
<listitem>
<para>The mapping is not qualified for a source object. There is one or more targets found through the correlation logic. Detected during change events and
reconciliation. Default action: "DELETE".</para>
</listitem>
</varlistentry>
<varlistentry>
<term>"SOURCE_IGNORED" (qualifies=0, link=0, target=0)</term>
<listitem>
<para>The source object is unqualified (by validSource), no link, no target is found. Detected during source object changes and reconciliation. Default action: "IGNORE".</para>
</listitem>
</varlistentry>
</variablelist>
<para>Based on the values assigned to objects during target reconciliation,
OpenIDM assigns situations, listed here with their default actions.</para>
<variablelist>
<varlistentry>
<term>"TARGET_IGNORED" (qualifies=0)</term>
<listitem>
<para>During target reconciliation the target becomes unqualified by the "validTagrt" script. Only detected during reconciliation. Default action: "IGNORE"</para>
</listitem>
</varlistentry>
<varlistentry>
<term>"UNASSIGNED" (qualifies=1, link=0)</term>
<listitem>
<para>A target object exists for which there is no link. Only detected
during reconciliation. Default action: "EXCEPTION".</para>
</listitem>
</varlistentry>
<varlistentry>
<term>"CONFIRMED" (qualifies=1, link=1, source=1)</term>
<listitem>
<para>The mapping qualifies for a target object, and a link to a
source object exists. Detected only during reconciliation. Default
action: "UPDATE".</para>
</listitem>
</varlistentry>
<varlistentry>
<term>"UNQUALIFIED" (qualifies=0, link=1, source=1, but source does not qualify)</term>
<listitem>
<para>The mapping is not qualified (by validTarget) for a target object, and there is a
link from an existing source object where the source exists. Detected during change events and
reconciliation. Default action: "DELETE".</para>
</listitem>
</varlistentry>
<varlistentry>
<term>SOURCE_MISSING (qualifies=1, link=1, source=0)</term>
<listitem>
<para>The target qualifies and a link is found. But the source object is missing. Only detected during reconciliation. Default action: "EXCEPTION".</para>
</listitem>
</varlistentry>
</variablelist>
<para>The following sections reiterate in detail how OpenIDM assigns
situations during each of the two synchronization phases.</para>
</section>
<section xml:id="source-reconciliation">
<title>Source Reconciliation</title>
<para>OpenIDM starts reconciliation and LiveSync by reading a list of
objects from the resource. For reconciliation, the list includes all
objects available through the connector. For LiveSync, the list contains
only changed objects. The connector can filter objects out of the list, too.
You can filter objects out of the list by using the
<literal>validSource</literal> property.</para>
<para>OpenIDM then iterates over the list, checking each entry against the
<literal>validSource</literal> filter, classifying objects according to
their situations as described in <xref linkend="sync-situations"/>. OpenIDM
uses the list of links for the current mapping to classify objects. Finally,
OpenIDM executes the action configured for the situation.</para>
<para>The following table shows how OpenIDM assigns the appropriate
situation during source reconciliation, depending on whether a valid
source exists (Source Qualifies), whether a link with the appropriate type
exists in the repository (Link Exists), and how many target objects are
found either based on links or correlation results.</para>
<table pgwide="1" rules="none">
<title>Resolving Source Reconciliation Situations</title>
<tgroup cols="8">
<colspec colnum="1" colname="c1" colwidth="1*" />
<colspec colnum="2" colname="c2" colwidth="1*" />
<colspec colnum="3" colname="c3" colwidth="1*" />
<colspec colnum="4" colname="c4" colwidth="1*" />
<colspec colnum="5" colname="c5" colwidth="1*" />
<colspec colnum="6" colname="c6" colwidth="1*" />
<colspec colnum="7" colname="c7" colwidth="1*" />
<colspec colnum="8" colname="c8" colwidth="2*" />
<thead>
<row>
<entry namest="c1" nameend="c2" align="left">Source Qualifies?</entry>
<entry namest="c3" nameend="c4" align="left">Link Exists?</entry>
<entry namest="c5" nameend="c7" align="left">Target Objects
Found<footnote><para>If no link exists for the source object, then
OpenIDM executes a correlation query.</para></footnote></entry>
<entry morerows="1" valign="top" align="left">Situation</entry>
</row>
<row>
<entry>Yes</entry>
<entry>No</entry>
<entry>Yes</entry>
<entry>No</entry>
<entry>0</entry>
<entry>1</entry>
<entry>> 1</entry>
</row>
</thead>
<tbody>
<row>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry>X</entry>
<entry>X</entry>
<entry> </entry>
<entry> </entry>
<entry>SOURCE_IGNORED</entry>
</row>
<row>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry>UNQUALIFIED</entry>
</row>
<row>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry> </entry>
<entry>X</entry>
<entry>UNQUALIFIED</entry>
</row>
<row>
<entry> </entry>
<entry>X</entry>
<entry>X</entry>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry> </entry>
<entry>UNQUALIFIED</entry>
</row>
<row>
<entry> </entry>
<entry>X</entry>
<entry>X</entry>
<entry> </entry>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry>UNQUALIFIED</entry>
</row>
<row>
<entry> </entry>
<entry>X</entry>
<entry>X</entry>
<entry> </entry>
<entry> </entry>
<entry> </entry>
<entry>X</entry>
<entry>UNQUALIFIED</entry>
</row>
<row>
<entry>X</entry>
<entry> </entry>
<entry> </entry>
<entry>X</entry>
<entry>X</entry>
<entry> </entry>
<entry> </entry>
<entry>ABSENT</entry>
</row>
<row>
<entry>X</entry>
<entry> </entry>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry>FOUND</entry>
</row>
<row>
<entry>X</entry>
<entry> </entry>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry> </entry>
<entry>X</entry>
<entry>AMBIGUOUS</entry>
</row>
<row>
<entry>X</entry>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry> </entry>
<entry>MISSING</entry>
</row>
<row>
<entry>X</entry>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry>CONFIRMED</entry>
</row>
</tbody>
</tgroup>
</table>
</section>
<section xml:id="target-reconciliation">
<title>Target Reconciliation</title>
<para>During source reconciliation, OpenIDM cannot detect situations where
no source object exists, such as the UNASSIGNED situation. When no source
object exists, OpenIDM detects the situation during the second
reconciliation phase, target reconciliation. During target reconciliation,
OpenIDM iterates over all target objects that do not have a representation
on the source, checking each object against the
<literal>validTarget</literal> filter, determining the appropriate situation,
and executing the action configured for the situation.</para>
<para>The following table shows how OpenIDM assigns the appropriate
situation during target reconciliation, depending on whether a valid target
exists (Target Qualifies), whether a link with an appropriate type exists
in the repository (Link Exists), whether a source object exists
(Source Exists), and whether the source object qualifies (Source Qualifies).
Not all situations assigned during source reconciliation are assigned
during target reconciliation.</para>
<table pgwide="1" rules="none">
<title>Resolving Target Reconciliation Situations</title>
<tgroup cols="9">
<colspec colnum="1" colname="c1" colwidth="1*" />
<colspec colnum="2" colname="c2" colwidth="1*" />
<colspec colnum="3" colname="c3" colwidth="1*" />
<colspec colnum="4" colname="c4" colwidth="1*" />
<colspec colnum="5" colname="c5" colwidth="1*" />
<colspec colnum="6" colname="c6" colwidth="1*" />
<colspec colnum="7" colname="c7" colwidth="1*" />
<colspec colnum="8" colname="c8" colwidth="1*" />
<colspec colnum="9" colname="c9" colwidth="2*" />
<thead>
<row>
<entry namest="c1" nameend="c2" align="left">Target Qualifies?</entry>
<entry namest="c3" nameend="c4" align="left">Link Exists?</entry>
<entry namest="c5" nameend="c6" align="left">Source Exists?</entry>
<entry namest="c7" nameend="c8" align="left">Source Qualifies?</entry>
<entry morerows="1" valign="top" align="left">Situation</entry>
</row>
<row>
<entry>Yes</entry>
<entry>No</entry>
<entry>Yes</entry>
<entry>No</entry>
<entry>Yes</entry>
<entry>No</entry>
<entry>Yes</entry>
<entry>No</entry>
</row>
</thead>
<tbody>
<row>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry> </entry>
<entry> </entry>
<entry> </entry>
<entry> </entry>
<entry> </entry>
<entry>TARGET_IGNORED</entry>
</row>
<row>
<entry>X</entry>
<entry> </entry>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry> </entry>
<entry>UNASSIGNED</entry>
</row>
<row>
<entry>X</entry>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry>CONFIRMED</entry>
</row>
<row>
<entry>X</entry>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry> </entry>
<entry>X</entry>
<entry>UNQUALIFIED</entry>
</row>
<row>
<entry>X</entry>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry> </entry>
<entry>SOURCE_MISSING</entry>
</row>
</tbody>
</tgroup>
</table>
</section>
<section xml:id="sync-actions">
<title>Synchronization Actions</title>
<variablelist>
<para>Once OpenIDM has assigned a situation to an object, OpenIDM takes
the actions configured in the mapping. If no action is configured, then
OpenIDM takes the default action for the situation. OpenIDM supports the
following actions.</para>
<varlistentry>
<term>"CREATE"</term>
<listitem>
<para>Create and link a target object.</para>
</listitem>
</varlistentry>
<varlistentry>
<term>"UPDATE"</term>
<listitem>
<para>Link and update a target object.</para>
</listitem>
</varlistentry>
<varlistentry>
<term>"DELETE"</term>
<listitem>
<para>Delete and unlink the target object.</para>
</listitem>
</varlistentry>
<varlistentry>
<term>"LINK"</term>
<listitem>
<para>Link the correlated target object.</para>
</listitem>
</varlistentry>
<varlistentry>
<term>"UNLINK"</term>
<listitem>
<para>Unlink the linked target object.</para>
</listitem>
</varlistentry>
<varlistentry>
<term>"EXCEPTION"</term>
<listitem>
<para>Flag the link situation as an exception.</para>
</listitem>
</varlistentry>
<varlistentry>
<term>"IGNORE"</term>
<listitem>
<para>Do not change the link or target object state.</para>
</listitem>
</varlistentry>
</variablelist>
</section>
</section>
<section xml:id="correlation">
<title>Correlation Queries</title>
<indexterm>
<primary>Synchronization</primary>
<secondary>Correlation queries</secondary>
</indexterm>
<indexterm>
<primary>Correlation queries</primary>
</indexterm>
<para>Every time OpenIDM creates an object through synchronization, it
creates a link between the source and target objects. OpenIDM then uses the
link to determine the object's situation during later synchronization
operations.</para>
<para>Initial, bulk synchronization operations can involve correlating
many objects that exist both on source and target systems. In this case,
OpenIDM uses correlation queries to find target objects that already exist,
and that correspond to source objects. For the target objects that match
a correlation query, OpenIDM needs only to create a link, rather than a
new target object.</para>
<para>Correlation queries run against target resources. The query syntax
therefore depends on the target system, and is either specific to the data
store underlying the OpenIDM repository, or to OpenICF query
capabilities.</para>
<section xml:id="correlation-query-managed-object">
<title>Managed Object as Correlation Query Target</title>
<para>Queries on managed objects in the repository must be defined in the
configuration file for the repository, which is either
<para>The following example shows a correlation query defined in
<programlisting language="javascript">
"for-userName" : "SELECT * FROM ${_resource} WHERE userName = '${uid}'"</programlisting>
<para>The following correlation query example shows the JavaScript to call
the query defined for OrientDB. The <literal>_query-id</literal> property
value matches the name of the query specified in
replaces <literal>${uid}</literal> in the query. OpenIDM replaces
<literal>${_resource}</literal> in the query with the name of table that
holds managed objects.</para>
<programlisting language="javascript">
{
"correlationQuery": {
"type": "text/javascript",
"source":
"var query = {'_query-id' : 'for-userName', 'uid' : source.name}; query;"
}
}</programlisting>
<para>The query can return zero or more objects, so the situation OpenIDM
assigns to the source object depends on the number of target objects
returned.</para>
<para>With a JDBC-based repository, the query defined in
to how the tables are indexed. The correlation query you define in
</section>
<section xml:id="correlation-query-system-object">
<title>System Object as Correlation Query Target</title>
<para>Correlation queries on system objects access the connector. The
connector then executes the query on the external resource.</para>
<para>Your correlation query JavaScript must return a map holding a generic
query with the following elements.</para>
<itemizedlist>
<listitem>
<para>A condition such as "Equals"</para>
</listitem>
<listitem>
<para>The naming attribute to compare on the system object. In the example
that follows, the naming attribute is <literal>uid</literal>.</para>
</listitem>
<listitem>
<para>The value from the source object to use in the search filter. You
set this as the value of the <literal>value</literal> property, which
takes an array. In the example that follows, the value to use in the
</listitem>
</itemizedlist>
<programlisting language="javascript">
varmap={"query": {"Equals": {"field": "uid", "values": [ source.userName ]}}};
map;</programlisting>
</section>
</section>
<section xml:id="advanced-dataflow">
<title>Advanced Data Flow Configuration</title>
<indexterm>
<primary>Mappings</primary>
<secondary>Hooks for scripting</secondary>
</indexterm>
<para><xref linkend="basic-flow"/> shows how to trigger scripts when objects
are created and updated. Other situations require you to trigger scripts
in response to other synchronization actions. For example, you might not
want OpenIDM to delete a managed user directly when an external account is
deleted, but instead unlink the objects and deactivate the user in another
resource. (Alternatively, you might delete the object in OpenIDM but
nevertheless execute a script.) The following example shows a more advanced
mapping configuration.</para>
<programlisting linenumbering="numbered" language="javascript">
{
"mappings": [
{
"name": "systemLdapAccount_managedUser",
"validSource": {
"type": "text/javascript",
"file": "script/isValid.js"
},
"correlationQuery": {
"type": "text/javascript",
"file": "script/ldapCorrelationQuery.js"
},
"properties": [
{
"source": "uid",
"transform": {
"type": "text/javascript",
"source": "source.toLowerCase()"
},
"target": "userName"
},
{
"source": "",
"transform": {
"type": "text/javascript",
"source": "if (source.myGivenName)
{source.myGivenName;} else {source.givenName;}"
},
"target": "givenName"
},
{
"source": "",
"transform": {
"type": "text/javascript",
"source": "if (source.mySn)
{source.mySn;} else {source.sn;}"
},
"target": "familyName"
},
{
"source": "cn",
"target": "fullname"
},
{
"comment": "Multi-valued in LDAP, single-valued in AD.
Retrieve first non-empty value.",
"source": "title",
"transform": {
"type": "text/javascript",
"file": "script/getFirstNonEmpty.js"
},
"target": "title"
},
{
"condition": {
"type": "text/javascript",
"source": "var clearObj = openidm.decrypt(object);
((clearObj.password != null) &&
},
"transform": {
"type": "text/javascript",
"source": "source.password"
},
"target": "__PASSWORD__"
}
],
"onCreate": {
"type": "text/javascript",
"source": "target.ldapPassword = null;
target.adPassword = null;
target.password = null;
target.ldapStatus = 'New Account'"
},
"onUpdate": {
"type": "text/javascript",
"source": "target.ldapStatus = 'OLD'"
},
"onUnlink": {
"type": "text/javascript",
"file": "script/triggerAdDisable.js"
},
"policies": [
{
"situation": "CONFIRMED",
"action": "UPDATE"
},
{
"situation": "FOUND",
"action": "UPDATE"
},
{
"situation": "ABSENT",
"action": "CREATE"
},
{
"situation": "AMBIGUOUS",
"action": "EXCEPTION"
},
{
"situation": "MISSING",
"action": "EXCEPTION"
},
{
"situation": "UNQUALIFIED",
"action": "UNLINK"
},
{
"situation": "UNASSIGNED",
"action": "EXCEPTION"
}
]
}
]
}</programlisting>
<variablelist>
<para>Here is the complete list of properties you can use as hooks in
mapping configurations to call scripts.
</para>
<varlistentry>
<term>Triggered by Situation</term>
<listitem>
<para>onCreate, onUpdate, onDelete, onLink, onUnlink</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Object Filter</term>
<listitem>
<para>vaildSource, validTarget</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Correlating Objects</term>
<listitem>
<para>correlationQuery</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Triggered on Reconciliation</term>
<listitem>
<para>result</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Scripts Inside Properties</term>
<listitem>
<para>condition, transform</para>
</listitem>
</varlistentry>
</variablelist>
<para>Your scripts can get data from any connected system at any time by
<literal>id</literal> is the identifier of the object to read.</para>
<para>The following example reads a managed user object from the
repository.</para>
<programlisting language="javascript">
<para>The following example reads an account from an external LDAP
resource.</para>
<programlisting language="javascript">
</section>
<section xml:id="alternative-mapping">
<title>Alternative Mappings</title>
<indexterm>
<primary>Mappings</primary>
<secondary>Scheduled reconciliation</secondary>
</indexterm>
<para>Mappings for synchronization are usually stored in
and for pushing changes made to managed objects to external resources.
You can, however, provide alternative mappings for scheduled reconciliation
by adding the mapping to the scheduler configuration instead of referencing
<programlisting language="javascript">
{
"enabled": true,
"type": "cron",
"schedule": "0 08 16 * * ?",
"invokeService": "sync",
"invokeContext": {
"action": "reconcile",
<emphasis role="bold">"mapping": {
"name": "CSV_XML",
"properties": [
{
"source": "firstname",
"target": "firstname"
},
...
],
"policies": [...]
}</emphasis>
}
}</programlisting>
</section>
</chapter>