chap-synchronization.xml revision 7b85693a3ced68e4f3697280b999a5a710b7a17d
<?xml version="1.0" encoding="UTF-8"?>
<!--
! CCPL HEADER START
!
! This work is licensed under the Creative Commons
! Attribution-NonCommercial-NoDerivs 3.0 Unported License.
! To view a copy of this license, visit
! or send a letter to Creative Commons, 444 Castro Street,
! Suite 900, Mountain View, California, 94041, USA.
!
! You can also obtain a copy of the license at
! See the License for the specific language governing permissions
! and limitations under the License.
!
! If applicable, add the following below this CCPL HEADER, with the fields
! enclosed by brackets "[]" replaced with your own identifying information:
! Portions Copyright [yyyy] [name of copyright owner]
!
! CCPL HEADER END
!
! Copyright 2011-2014 ForgeRock AS
!
-->
<chapter xml:id='chap-synchronization'
xmlns='http://docbook.org/ns/docbook'
version='5.0' xml:lang='en'
xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'
xsi:schemaLocation='http://docbook.org/ns/docbook
xmlns:xlink='http://www.w3.org/1999/xlink'>
<title>Configuring Synchronization</title>
<indexterm>
<primary>Synchronization</primary>
</indexterm>
<para>
One of the core services of OpenIDM is synchronizing identity data from
different resources. This chapter explains what you must know to get started
configuring OpenIDM's flexible synchronization mechanism, and illustrates the
concepts with examples.
</para>
<section xml:id="sync-types">
<title>Types of Synchronization</title>
<indexterm>
<primary>Synchronization</primary>
<secondary>Direct (push)</secondary>
</indexterm>
<para>
Synchronization happens either when OpenIDM receives a change directly, or
when OpenIDM discovers a change on an external resource.
</para>
<para>
For direct changes to OpenIDM, OpenIDM immediately pushes updates to all
external resources configured to receive the updates. A direct change can
originate not only as a write request through the REST interface, but also as
an update resulting from reconciliation with another resource.
</para>
<variablelist>
<para>
OpenIDM discovers and synchronizes changes from external resources
through reconciliation and LiveSync.
</para>
<para>
In contrast, OpenIDM synchronizes changes from internal resources to
external targets using automatic sync.
</para>
<varlistentry>
<term>Reconciliation</term>
<listitem>
<indexterm>
<primary>Reconciliation</primary>
</indexterm>
<para>
In identity management, <firstterm>reconciliation</firstterm> is the
process of bidirectional synchronization of objects between different data
stores. Reconciliation applies mainly to user objects, although OpenIDM
can reconcile any objects, including groups and roles.
</para>
<para>
To perform reconciliation, OpenIDM analyzes both source and target systems
to uncover the differences that it must reconcile. Reconciliation can
therefore be a heavyweight process. When working with large data sets,
finding all changes can be more work than processing the changes.
</para>
<para>
Reconciliation is, however, thorough. It recognizes system error
conditions and catches changes that might be missed by the more
lightweight LiveSync mechanism. Reconciliation therefore serves as the
basis for compliance and reporting functionality.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>LiveSync</term>
<listitem>
<indexterm>
<primary>LiveSync</primary>
</indexterm>
<para>
<firstterm>LiveSync</firstterm> captures the changes that occur on a
remote system, then pushes those changes to OpenIDM. OpenIDM uses the
defined mappings to replay the changes where they are required - either in
the OpenIDM repository, or on another remote system, or both. Unlike
reconciliation, LiveSync uses a polling system, and is intended to react
quickly to changes as they happen.
</para>
<para>
To perform this polling, LiveSync relies on a change detection mechanism
on the external resource to determine which objects have changed. The
change detection mechanism is specific to the external resource, and can
be a time stamp, a sequence number, a change vector or other any method of
recording changes that have occurred on the system. For example, OpenDJ
implements a change log that provides OpenIDM with a list of objects that
have changed since the last request. Active Directory implements a change
sequence number, and certain databases might have a
<literal>lastChange</literal> attribute.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Automatic sync</term>
<listitem>
<para>
<firstterm>Automatic sync</firstterm> automatically pushes changes made in
the OpenIDM internal repository to external systems.
</para>
<para>
The section that follows describes additional features that
can be included with automatic sync.
</para>
</listitem>
</varlistentry>
</variablelist>
<para>
To determine what to synchronize, and how to carry out synchronization,
OpenIDM relies on mappings configured in the
rely on the mappings configured once per OpenIDM server.
</para>
<para>
For reconciliation or LiveSync, you can schedule changes as described in
<link xlink:href="integrators-guide#chap-scheduler-conf"
Tasks and Events</citetitle></link>.
</para>
<section xml:id="sync-types-automatic">
<title>How Automatic Sync works with <literal>onSync</literal></title>
<para>
When automatic sync is used to push a large number of changes to external
databases, that process takes time. Problems such as lost connections
could happen, resulting in partial synchronization.
</para>
<para>
For example, if a Human Resources manager adds a group of new employees in
one database, a partial synchronization may mean that some of those
employees do not have access to their email or other systems.
</para>
<para>
Automatic sync assumes that the external data sources include all data
currently in the internal OpenIDM repository. If that does not reflect
your configuration, you should start either with an initial reconciliation,
or with LiveSync as described in the Installation Guide,
xlink:href="install-guide#more-sample6"><citetitle>Sample 6 - LiveSync
with an AD Server</citetitle></link>.
</para>
<para>
An example of automatic sync is depicted in the OpenIDM Installation Guide,
xlink:href="install-guide#more-sample5b"><citetitle>Sample 5b - Compensated
Synchronization with <literal>onSync</literal></citetitle></link>. That
sample demonstrates how OpenIDM compensates for the failure of an external
resource.
</para>
<para>
OpenIDM includes an optional <literal>onSync</literal> hook, which you
A sample excerpt is shown here:
</para>
<programlisting language="javascript">
...
"onDelete" : {
"type" : "text/javascript",
"file" : "ui/onDelete-user-cleanup.js"
},
"onSync" : {
"type" : "text/javascript",
"file" : "compensate.js"
},
"properties" : [
...
</programlisting>
<para>
Note the reference from <literal>onSync</literal> to
</para>
<para>
When a change to a user is made, either in the UI or through a REST call,
a change is made to the managed object for that user.
</para>
<para>
The automatic sync feature of OpenIDM attempts to synchronize the change
(and any other pending changes) to the external data store(s). The
automatic sync process is configured in the associated
</para>
<para>
start with source and target mappings similar to:
</para>
<programlisting language="javascript">
...
"mappings" : [
{
"name" : "managedUser_systemAdAccounts",
...
</programlisting>
<para>
Based on this excerpt, automatic synchronization takes information from the
internal OpenIDM managed user data store, and synchronizes that information
to the AD data store. Once complete, it proceeds down the file to the
next <literal>source</literal> and <literal>target</literal> to be
synchronized.
</para>
<para>
partial synchronization. If synchronization proceeds to completion, OpenIDM
exits from the script, based on some of the first active entries in the file:
</para>
<programlisting language="javascript">
if (syncResults.success) {
logger.debug("sync was a success; no compensation necessary");
return;
}</programlisting>
<para>
If synchronization does not complete, OpenIDM proceeds through the
messages related to the sync action (<literal>notifyCreate, notifyUpdate,
notifyDelete</literal>, along with the error that caused the sync failure,
which should be available in the standard OpenIDM log file.
</para>
<para>
connections to an external resource can lead to data stores that are
out of sync, such as the example cited earlier where some new employees
do not have access to their corporate email accounts.
</para>
<para>
result in each data store using the information it had before
automatic synchronization started. OpenIDM stores that information,
temporarily, in properties such as <literal>oldObject</literal> and
<literal>oldTarget</literal>.
</para>
<para>
In this particular example, human resource managers should see that
new employees are not shown in their database. Then, the administrators
of the OpenIDM system can check log files for errors, address them, and
then restart automatic synchronization with a new REST call.
</para>
</section>
</section>
<section xml:id="sync-flexible-data">
<title>Flexible Data Model</title>
<indexterm>
<primary>Objects</primary>
<secondary>Managed objects</secondary>
</indexterm>
<para>
Identity management software tends to favor either a meta-directory data
model, where all data are mirrored in a central repository, or a virtual data
model, where only a minimum set of attributes are stored centrally, and most
are loaded on demand from the external resources in which they are stored.
The meta-directory model offers fast access at the risk of getting outdated
data. The virtual model guarantees fresh data, but pays for that guarantee in
terms of performance.
</para>
<para>
OpenIDM leaves the data model choice up to you. You determine the right trade
offs for a particular deployment. OpenIDM does not hard code any particular
schema or set of attributes stored in the repository. Instead, you define how
external system objects map onto managed objects, and OpenIDM dynamically
updates the repository to store the managed object attributes that you
configure.
</para>
<para>
You can, for example, choose to follow the data model defined in the Simple
Cloud Identity Management (<link xlink:show="new"
>SCIM</link>) specification. The following object represents a SCIM user.
</para>
<programlisting language="javascript">
{
"userName": "james1",
"familyName": "Berg",
"givenName": "James",
"email": [
"james1@example.com"
],
"description": "Created by OpenIDM REST.",
"password": "asdfkj23",
"displayName": "James Berg",
"phoneNumber": "12345",
"employeeNumber": "12345",
"userType": "Contractor",
"title": "Vice President",
"active": true
}</programlisting>
<note>
<para>
Avoid using the dash character ( <literal>-</literal> ) in property names,
like <literal>last-name</literal>, as dashes in names make JavaScript syntax
more complex. If you cannot avoid the dash, then write
<literal>source['last-name']</literal> instead of
</para>
</note>
</section>
<section xml:id="basic-flow">
<title>Basic Data Flow Configuration</title>
<para>Data flow for synchronization involves the following elements:
</para>
<itemizedlist>
<listitem><para>Connector configuration files
resource.</para></listitem>
<listitem><para>Synchronization mappings file
</para></listitem>
<listitem><para>A links table that OpenIDM maintains in its repository.</para></listitem>
<listitem><para>The scripts required to check objects and manipulate attributes.</para></listitem>
</itemizedlist>
<section xml:id="connector-config-files">
<title>Connector Configuration Files</title>
<indexterm>
<primary>Synchronization</primary>
<secondary>Connectors</secondary>
</indexterm>
<para>Connector configuration files map external resource objects to OpenIDM
objects, and are described in detail in the chapter on <link
xlink:href="integrators-guide#chap-resource-conf"
External Resources</citetitle></link>. Connector configuration files are
>resource-name</replaceable>.json</filename>,
where <replaceable>resource-name</replaceable> reflects the connector
technology and external resource, such as
<literal>openicf-xml</literal>.</para>
<para>An excerpt from an example connector configuration follows. The
example shows the name for the connector and two attributes of an account
object type. In the attribute mapping definitions, the attribute name is
mapped from the <literal>nativeName</literal>, the attribute name used on
the external resource, to the attribute name used in OpenIDM. Thus the
example shows that the <literal>sn</literal> attribute in LDAP is mapped to
<literal>lastName</literal> in OpenIDM. The <literal>homePhone</literal>
attribute can have multiple values.</para>
<programlisting language="javascript">
{
"name": "MyLDAP",
"objectTypes": {
"account": {
"lastName": {
"type": "string",
"required": true,
"nativeName": "sn",
"nativeType": "string"
},
"homePhone": {
"type": "array",
"items": {
"type": "string",
"nativeType": "string"
},
"nativeName": "homePhone",
"nativeType": "string"
}
}
}
}</programlisting>
<para>In order for OpenIDM to access external resource objects and
attributes, the object and its attributes must match the connector
configuration. Note that the connector file only maps external resource
objects to OpenIDM objects. To construct attributes and to manipulate
their values, you use the synchronization mappings file.</para>
</section>
<section xml:id="synchronization-mappings-file">
<title>Synchronization Mappings File</title>
<indexterm>
<primary>Synchronization</primary>
<secondary>Mappings</secondary>
</indexterm>
<indexterm>
<primary>Mappings</primary>
</indexterm>
<para>
represents the core configuration for OpenIDM synchronization.
</para>
Each mapping specifies how attributes from source objects correspond to
attributes on target objects. The source and target indicate the direction
for the data flow, so you must define a separate mapping for each data
flow. For example, if you want data flows from an LDAP server to the
repository and also from the repository to the LDAP server, you must
define two separate mappings.</para>
<para>You identify external resource sources and targets as
<literal>system/<replaceable>name</replaceable>/<replaceable
>object-type</replaceable></literal>, where
<replaceable>name</replaceable> is the name used in the connector
configuration file, and <replaceable>object-type</replaceable> is the
object defined in the connector configuration file list of object types.
For objects in OpenIDM's internal repository, you use
<literal>managed/<replaceable>object-type</replaceable></literal>, where
<replaceable>object-type</replaceable> is defined in
by convention is set to a string of the form
<literal><replaceable>source</replaceable>_<replaceable
>target</replaceable></literal>, as shown in the following example.</para>
<programlisting language="javascript" xml:id="basic-ldap-mapping">
{
"mappings": [
{
"name": "systemLdapAccounts_managedUser",
"properties": [
{
"target": "sn",
"source": "lastName"
},
{
"target": "telephoneNumber",
"source": "homePhone"
},
{
"target": "phoneExtension",
"default": "0047"
},
{
"target": "mail",
"comment": "Set mail if non-empty.",
"source": "email",
"condition": {
"type": "text/javascript",
"source": "(object.email != null)"
}
},
{
"target": "displayName",
"source": "",
"transform": {
"type": "text/javascript",
"source": "source.lastName +', ' + source.firstName;"
}
}
]
}
]
}</programlisting>
<para>In this example, the source is the external resource,
<literal>MyLDAP</literal>, and the target is OpenIDM's repository,
specifically the managed user objects. The <literal>properties</literal>
reflect OpenIDM attribute names. For example, the mapping has the
attribute <literal>lastName</literal> defined in the
<literal>MyLDAP</literal> connector configuration file mapped to
<literal>sn</literal> in the OpenIDM managed user object. Notice
that the attribute names come from the connector configuration, rather
than the external resource itself.</para>
<indexterm>
<primary>Synchronization</primary>
<secondary>Creating attributes</secondary>
</indexterm>
<para>
You can create attributes on the target as part of the mapping. In the
preceding example, a <literal>phoneExtension</literal> attribute with a
default value of <literal>0047</literal> is created on the target.
</para>
<para>
You can also use the <literal>"default"</literal> property to specify a
value that should be assigned to the target property. When determining
the value of the target property, any associated conditions are evaluated
first, followed by the transform script, if present. The default value is
applied (for update and create actions) if the <literal>"source"</literal>
property and the <literal>"transform"</literal> script yield a null value.
The default value overrides the target value, if one exists.
</para>
<indexterm>
<primary>Synchronization</primary>
<secondary>Conditions</secondary>
</indexterm>
<para>You can also set up conditions under which OpenIDM maps attributes
as shown for the email attribute in the example. By default, OpenIDM
synchronizes all attributes. In the example, the mail attribute is set
only if the script for the condition returns <literal>true</literal>.
</para>
<indexterm>
<primary>Synchronization</primary>
<secondary>Transforming attributes</secondary>
</indexterm>
<para>OpenIDM also enables you to transform attributes. In the example,
the value of the <literal>displayName</literal> attribute is set using a
combination of the <literal>lastName</literal> and
<literal>firstName</literal> attribute values from the source. For
transformations, the <literal>source</literal> property is optional.
However, the source object is only available when you specify the
<literal>source</literal> property. Therefore, in order to use
to calculate the <literal>displayName</literal>, the example specifies
<literal>"source" : ""</literal>.</para>
<para>To add a flow from the repository to <literal>MyLDAP</literal>,
<literal>managedUser_systemLdapAccounts</literal>.</para>
<para>The following image shows the paths to objects in the OpenIDM
namespace.</para>
<mediaobject xml:id="figure-service-tree"><?dbfo pgwide="1"?>
<alt>OpenIDM namespace and object paths</alt>
<imageobject>
</imageobject>
<textobject>
<para>The image shows paths to objects in the OpenIDM namespace.</para>
</textobject>
<caption>
<para>OpenIDM stores managed objects in the repository, and exposes
external resources are exposed under
</caption>
</mediaobject>
<variablelist>
<indexterm>
<primary>Synchronization</primary>
<secondary>Filtering</secondary>
</indexterm>
<para>
By default, OpenIDM synchronizes all objects that match those defined in
the connector configuration for the resource. Many connectors allow you
to limit the scope of objects that the connector accesses. For example,
the LDAP connector allows you to specify base DNs and LDAP filters so
that you do not need to access every entry in the directory. OpenIDM also
allows you to filter what is considered a valid source or valid target
for synchronization by using scripts. To apply these filters, use the
<literal>validSource</literal>, and <literal>validTarget</literal>
properties in your mapping.
</para>
<varlistentry>
<term>validSource</term>
<listitem>
<para>A script that determines if a source object is valid to be
mapped. The script yields a boolean value: <literal>true</literal>
indicates that the source object is valid; <literal>false</literal>
can be used to defer mapping until some condition is met. In the root
scope, the source object is provided in the <literal>"source"</literal>
property. If the script is not specified, then all source objects are
considered valid.</para>
<programlisting language="javascript">
{
"validSource": {
"type": "text/javascript",
"source": "source.ldapPassword != null"
}
}
</programlisting>
</listitem>
</varlistentry>
<varlistentry>
<term>validTarget</term>
<listitem>
<para>A script, used during reconciliation's second phase, that
determines if a target object is valid to be mapped. The script yields
a boolean value: <literal>true</literal> indicates that the target
object is valid; <literal>false</literal> indicates that the target
object should not be included in reconciliation. In the root scope, the
source object is provided in the <literal>"target"</literal> property.
If the script is not specified, then all target objects are considered
valid for mapping.</para>
<programlisting language="javascript">
{
"validTarget": {
"type": "text/javascript",
"source": "target.employeeType == 'internal'"
}
}
</programlisting>
</listitem>
</varlistentry>
</variablelist>
<para>During synchronization, your scripts always have access to a
<literal>source</literal> object and a <literal>target</literal> object.
Examples already shown in this section use <literal>source.<replaceable
>attributeName</replaceable></literal> to retrieve attributes from the
source objects. Your scripts can also write to target attributes using
<literal>target.<replaceable>attributeName</replaceable></literal>
syntax.</para>
<programlisting language="javascript">
{
"onUpdate": {
"type": "text/javascript",
}
}</programlisting>
<para>See the <link xlink:href="integrators-guide#appendix-scripting"
Reference</citetitle></link> appendix for more on scripting.</para>
<para>By default, all mappings participate in automatic synchronization
operations. You can prevent a specific mapping from participating in
automatic synchronization by setting the <literal>"enableSync"</literal>
property of that mapping to false. In the following example, automatic
synchronization is disabled. This means that changes to objects in the internal
repository are not automatically propagated to the LDAP directory. To
propagate changes to the LDAP directory, reconciliation must be launched
manually.</para>
<programlisting language="javascript">
{
"mappings" : [
{
"name" : "managedUser_systemLdapAccounts",
"enableSync" : false,
....
}
</programlisting>
<para>
Note that if <literal>enableSync</literal> is set to <literal>false</literal>
for a system to managed user mapping (for example
<literal>"systemLdapAccounts_managedUser"</literal>), LiveSync is disabled
for that mapping.
</para>
<para>
If a source resource is empty, the default behavior is for a reconciliation
operation to exit, without failure, and to log a warning, similar to the
following:
</para>
<screen>
2014-03-20 10:41:18:918 WARN Cannot perform reconciliation with an empty source
object set, unless explicitly configured to allow it.
</screen>
<para>
The reconciliation summary is also logged in the reconciliation audit log.
</para>
<para>
This behavior prevents reconciliation operations from accidentally deleting
everything in a target resource. For example, in the event that a source
system is unavailable but erroneously reporting its status as "up", the
absence of source objects should not result in objects being removed on the
target resource.
</para>
<para>
There might be situations in which you do want reconciliations of an empty
source resource to proceed. In this case, you can override the default
behavior by setting the <literal>"allowEmptySourceSet"</literal> property to
<literal>true</literal> in the mapping. For example:
</para>
<programlisting>
{
"mappings" : [
{
"name" : "systemXmlfileAccounts_managedUser",
"allowEmptySourceSet" : true,
...
</programlisting>
<para>
Reconciliation of an empty source effectively wipes out the target.
</para>
<para>
You can update mappings in the synchronization configuration file
do not update a mapping that is currently in use by a reconciliation
process.
</para>
</section>
<section xml:id="sync-encrypted-values">
<title>Using Encrypted Values</title>
<indexterm>
<primary>Synchronization</primary>
<secondary>Encryption</secondary>
</indexterm>
<para>OpenIDM supports reversible encryption of attribute values for
managed objects. Attribute values to encrypt include passwords,
authentication questions, credit card numbers, and social security numbers.
If passwords are already encrypted on the external resource, they are
generally excluded from the synchronization process. For more information,
see <link xlink:href="integrators-guide#chap-passwords"
xlink:role="http://docbook.org/xlink/role/olink">
<citetitle>Managing Passwords</citetitle></link>.</para>
<para>You configure encryption in the managed object configuration (in the
of that file shows a managed object configuration that encrypts and decrypts
<literal>securityAnswer</literal>, <literal>ssn</literal>, and
<literal>password</literal> attributes using the default symmetric
key, and additional scripts for extra passwords.</para>
<programlisting language="javascript">
{
"objects": [
{
"name": "user",
...
"properties": [
{
"name": "securityAnswer",
"encryption": {
"key": "openidm-sym-default"
}
},
{
"name": "ssn",
"encryption": {
"key": "openidm-sym-default"
}
},
{
"name": "password",
"encryption": {
"key": "openidm-sym-default"
}
}
],
...
}
]
}</programlisting>
<para>Do not use the default symmetric key,
<literal>openidm-sym-default</literal>, in production. See the chapter on
<link xlink:href="integrators-guide#chap-security"
and Hardening OpenIDM</citetitle></link> for instructions on adding your
own symmetric key.</para>
</section>
<section xml:id="restricting-http-access">
<title>Restricting HTTP Access to Sensitive Data</title>
<para>You can protect specific sensitive data stored in the repository by
marking the corresponding properties as "private". Private data, whether it
is encrypted or not, is not accessible over the REST interface. Properties
that are marked as private are removed from an object when that object is
retrieved over REST.</para>
<para>To mark a property as private, set its <literal>"scope"</literal> to
file.</para>
shows how HTTP access is prevented on the <literal>password</literal> and
<literal>securityAnswer</literal> properties.</para>
<programlisting language="javascript">
"properties" : [
{
"name" : "securityAnswer",
"encryption" : {
"key" : "openidm-sym-default"
},
"scope" : "private"
},
{
"name" : "password",
"encryption" : {
"key" : "openidm-sym-default"
},
"scope" : "private"
</programlisting>
<para>A potential caveat with using private properties is that such
properties are <emphasis>removed</emphasis> if an object is updated by
using an HTTP <literal>PUT</literal> request. A <literal>PUT</literal>
request replaces the entire object in the repository. Because properties
that are marked as private are ignored in HTTP requests, these properties
are effectively removed from the object when the update is done. To work
around this limitation, do not use <literal>PUT</literal> requests if you
have configured private properties. Instead, use a <literal>PATCH</literal>
request to update only those properties that need to be changed.
</para>
<para>For example, to update the <literal>givenName</literal> of user jdoe,
you could run the following command:</para>
<screen>
$ <userinput>curl \
--cacert self-signed.crt \
--header "X-OpenIDM-Username: openidm-admin" \
--header "X-OpenIDM-Password: openidm-admin" \
--header "Content-Type: application/json" \
--request POST \
--data '[
{
"operation":"replace",
"field":"/givenName",
"value":"Jon"
}
]' \
"https://localhost:8443/openidm/managed/user?_action=patch&_queryId=for-userName&uid=jdoe"</userinput>
</screen>
<note>
<para>The filtering of private data applies only to direct HTTP read
and query calls on managed objects. No automatic filtering is done for
internal callers, and the data that these callers choose to expose.</para>
</note>
</section>
<section xml:id="constructing-attributes">
<title>Constructing and Manipulating Attributes</title>
<indexterm>
<primary>Synchronization</primary>
<secondary>Creating attributes</secondary>
</indexterm>
<para>
OpenIDM enables you to construct and manipulate attributes using scripts
that are triggered when an object is created (onCreate), updated (onUpdate),
retrieved (onRetrieve), or deleted (onDelete). Additional scripts are
available when a managed object requires validation (onValidate), and when
an object is about to be stored in the repository (onStore). Similar scripts
are available for when a link is created (onLink) or removed (onUnlink).
</para>
<para>The following example derives a DN for an LDAP entry when the entry
is created in the internal repository.</para>
<programlisting language="javascript">
{
"onCreate": {
"type": "text/javascript",
"source":
"target.dn = 'uid=' + source.uid + ',ou=people,dc=example,dc=com'"
}
}</programlisting>
<para>
In addition, OpenIDM supports the use of post-action scripts, including
after the creation of an object is complete (postCreate), after the update
of an object is complete (postUpdate), and after the deletion of an object
(postDelete).
</para>
</section>
<section xml:id="reusing-links">
<title>Reusing Links</title>
<indexterm>
<primary>Synchronization</primary>
<secondary>Reusing links</secondary>
</indexterm>
<para>When two mappings exist to synchronize the same objects bidirectionally,
you can use the <literal>links</literal> property in one mapping to have
OpenIDM use the same internally managed link for both mappings. Otherwise,
if no <literal>links</literal> property is specified, OpenIDM maintains a
link for each mapping.</para>
<para>The following excerpt shows two mappings, one from MyLDAP accounts
to managed users, and another from managed users to MyLDAP accounts. In
the second mapping, the <literal>link</literal> property tells OpenIDM
to reuse the links created in the first mapping, rather than create new
links.</para>
<programlisting language="javascript">
{
"mappings": [
{
"name": "systemMyLDAPAccounts_managedUser",
},
{
"name": "managedUser_systemMyLDAPAccounts",
"links": "systemMyLDAPAccounts_managedUser"
}
]
}</programlisting>
</section>
</section>
<section xml:id="recon-over-rest">
<title>Managing Reconciliation Over REST</title>
<para>
You can trigger, cancel, and monitor reconciliation operations over REST,
using the REST endpoint
</para>
<section xml:id="triggering-recons">
<title>Triggering a Reconciliation Run</title>
<para>
The following example triggers a reconciliation operation based on the
<literal>systemLdapAccounts_managedUser</literal> mapping. The mapping is
</para>
<screen>$ curl \
--cacert self-signed.crt \
--header "X-OpenIDM-Username: openidm-admin" \
--header "X-OpenIDM-Password: openidm-admin" \
--header "Content-Type: application/json" \
--request POST \
"https://localhost:8443/openidm/recon?_action=recon&mapping=systemLdapAccounts_managedUser"</screen>
<para>
By default, an assigned reconciliation run ID is returned immediately when
the reconciliation operation is initiated. Clients can make subsequent calls
to the reconciliation service, using this reconciliation run ID to query its
state and to call operations on it.
</para>
<para>
For example, the reconciliation run initiated previously would return
something similar to the following:
</para>
<screen>{"_id":"0890ad62-4738-4a3f-8b8e-f3c83bbf212e"}</screen>
<para>
To have the entire reconciliation run complete before the reconciliation
run ID is returned, set the <literal>waitForCompletion</literal> property to
<literal>true</literal> when the reconciliation is initiated. For example:
</para>
<screen>$ curl \
--cacert self-signed.crt \
--header "X-OpenIDM-Username: openidm-admin" \
--header "X-OpenIDM-Password: openidm-admin" \
--header "Content-Type: application/json" \
--request POST \
</screen>
</section>
<section xml:id="recon-details">
<title>Obtaining the Details of a Reconciliation Run</title>
<para>
You can display the details of a particular reconciliation run over REST by
specifying the reconciliation run ID in the URL. For example, the following
call shows the details of the reconciliation run initiated in the previous
section:
</para>
<screen>$ <userinput>curl \
--cacert self-signed.crt \
--header "X-OpenIDM-Username: openidm-admin" \
--header "X-OpenIDM-Password: openidm-admin" \
--request GET \
<computeroutput>{
"ended": "2014-03-06T07:00:32.094Z",
"_id": "7a07c100-4f11-4d7e-bf8e-fa4594f99d58",
"mapping": "systemLdapAccounts_managedUser",
"state": "SUCCESS",
"stage": "COMPLETED_SUCCESS",
"stageDescription": "reconciliation completed.",
"progress": {
"links": {
"created": 0,
"existing": {
"total": "1",
"processed": 1
}
},
"target": {
"created": 0,
"existing": {
"total": "3",
"processed": 3
}
},
"source": {
"existing": {
"total": "1",
"processed": 1
}
}
},
"situationSummary": {
"UNASSIGNED": 2,
"TARGET_IGNORED": 0,
"SOURCE_IGNORED": 0,
"MISSING": 0,
"FOUND": 0,
"AMBIGUOUS": 0,
"UNQUALIFIED": 0,
"CONFIRMED": 1,
"SOURCE_MISSING": 0,
"ABSENT": 0
},
"started": "2014-03-06T07:00:31.907Z"</computeroutput>
}</screen>
</section>
<section xml:id="canceling-recons">
<title>Canceling a Reconciliation Run</title>
<para>
You can cancel a reconciliation run by sending a REST call with the
<literal>cancel</literal> action, specifying the reconciliation run ID. For
example, the following call cancels the reconciliation run initiated in the
previous section:
</para>
<screen><?dbfo pgwide="1"?>$ curl \
--cacert self-signed.crt \
--header "X-OpenIDM-Username: openidm-admin" \
--header "X-OpenIDM-Password: openidm-admin" \
--header "Content-Type: application/json" \
--request POST \
<para>
The output for a reconciliation cancellation request is similar to the
following:
</para>
<screen>{
"status":"SUCCESS",
"action":"cancel",
"_id":"0890ad62-4738-4a3f-8b8e-f3c83bbf212e"
}</screen>
<para>
If you specified that the call should wait for completion before the ID is
returned, you can obtain the reconciliation run ID from the list of active
reconciliations, as described in the following section.
</para>
</section>
<section xml:id="listing-recons">
<title>Listing Reconciliation Runs</title>
<para>
You can display a list of reconciliation processes that have completed, and
those that are in progress, by running a RESTful GET on
example displays all reconciliation runs.
</para>
<screen>$ curl \
--cacert self-signed.crt \
--header "X-OpenIDM-Username: openidm-admin" \
--header "X-OpenIDM-Password: openidm-admin" \
--request GET \
</screen>
<para>
The output of such a request is similar to the following, with one item for
each reconciliation run.
</para>
<screen>{
"reconciliations": [
{
"ended": "2014-03-06T06:14:11.845Z",
"_id": "4286510e-986a-4521-bfa4-8cd1e039a7f5",
"mapping": "systemLdapAccounts_managedUser",
"state": "SUCCESS",
"stage": "COMPLETED_SUCCESS",
"stageDescription": "reconciliation completed.",
"progress": {
"links": {
"created": 1,
"existing": {
"total": "0",
"processed": 0
}
},
"target": {
"created": 1,
"existing": {
"total": "2",
"processed": 2
}
},
"source": {
"existing": {
"total": "1",
"processed": 1
}
}
},
"situationSummary": {
"UNASSIGNED": 2,
"TARGET_IGNORED": 0,
"SOURCE_IGNORED": 0,
"MISSING": 0,
"FOUND": 0,
"AMBIGUOUS": 0,
"UNQUALIFIED": 0,
"CONFIRMED": 0,
"SOURCE_MISSING": 0,
"ABSENT": 1
},
"started": "2014-03-06T06:14:04.722Z"
},
]
}</screen>
<variablelist>
<para>
Each reconciliation run has the following properties:
</para>
<varlistentry>
<term><literal>_id</literal></term>
<listitem>
<para>
The ID of the reconciliation run.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><literal>mapping</literal></term>
<listitem>
<para>
file.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><literal>state</literal></term>
<listitem>
<para>
The high level state of the reconciliation run. Values can be as follows:
</para>
<itemizedlist>
<listitem>
<para>
<literal>ACTIVE</literal>
</para>
<para>
The reconciliation run is in progress.
</para>
</listitem>
<listitem>
<para>
<literal>CANCELED</literal>
</para>
<para>
The reconciliation run was successfully canceled.
</para>
</listitem>
<listitem>
<para>
<literal>FAILED</literal>
</para>
<para>
The reconciliation run was terminated because of failure.
</para>
</listitem>
<listitem>
<para>
<literal>SUCCESS</literal>
</para>
<para>
The reconciliation run completed successfully.
</para>
</listitem>
</itemizedlist>
</listitem>
</varlistentry>
<varlistentry>
<term><literal>stage</literal></term>
<listitem>
<para>
The current stage of the reconciliation run's progress. Values can be as
follows:
</para>
<itemizedlist>
<listitem>
<para>
<literal>ACTIVE_INITIALIZED</literal>
</para>
<para>
The initial stage, when a reconciliation run is first created.
</para>
</listitem>
<listitem>
<para>
<literal>ACTIVE_QUERY_ENTRIES</literal>
</para>
<para>
Querying the source, target and possibly link sets to reconcile.
</para>
</listitem>
<listitem>
<para>
<literal>ACTIVE_RECONCILING_SOURCE</literal>
</para>
<para>
Reconciling the set of IDs retrieved from the mapping source.
</para>
</listitem>
<listitem>
<para>
<literal>ACTIVE_RECONCILING_TARGET</literal>
</para>
<para>
Reconciling any remaining entries from the set of IDs retrieved from
the mapping target, that were not matched or processed during the
source phase.
</para>
</listitem>
<listitem>
<para>
<literal>ACTIVE_LINK_CLEANUP</literal>
</para>
<para>
Checking whether any links are now unused and should be cleaned up.
</para>
</listitem>
<listitem>
<para>
<literal>ACTIVE_PROCESSING_RESULTS</literal>
</para>
<para>
Post-processing of reconciliation results.
</para>
</listitem>
<listitem>
<para>
<literal>ACTIVE_CANCELING</literal>
</para>
<para>
Attempting to abort a reconciliation run in progress.
</para>
</listitem>
<listitem>
<para>
<literal>COMPLETED_SUCCESS</literal>
</para>
<para>
Successfully completed processing the reconciliation run.
</para>
</listitem>
<listitem>
<para>
<literal>COMPLETED_CANCELED</literal>
</para>
<para>
Completed processing because the reconciliation run was aborted.
</para>
</listitem>
<listitem>
<para>
<literal>COMPLETED_FAILED</literal>
</para>
<para>
Completed processing because of a failure.
</para>
</listitem>
</itemizedlist>
</listitem>
</varlistentry>
<varlistentry>
<term><literal>stageDescription</literal></term>
<listitem>
<para>
A description of the stages described previously.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><literal>progress</literal></term>
<listitem>
<para>
The progress object has the following structure (annotated here with
comments):
</para>
<programlisting language="javascript" width="91">
"progress":{
"source":{ // Progress on set of existing entries in the mapping source
"existing":{
"processed":1001,
"total":"1001" // Total number of entries in source set, if known, “?” otherwise
}
},
"target":{ // Progress on set of existing entries in the mapping target
"existing":{
"processed":1001,
"total":"1001" // Total number of entries in target set, if known, “?” otherwise
},
"created":0 // New entries that were created
},
"links":{ // Progress on set of existing links between source and target
"existing":{
"processed":1001,
"total":"1001" // Total number of existing links, if known, “?” otherwise
},
"created":0 // Denotes new links that were created
}
},
</programlisting>
</listitem>
</varlistentry>
</variablelist>
</section>
<section xml:id="livesync-over-rest">
<title>Triggering LiveSync Over REST</title>
<para>
The ability to trigger LiveSync operations over REST, or by using the
resource API, enables you to use an external scheduler to trigger a LiveSync
operation, rather than using the OpenIDM scheduling mechanism.
</para>
<itemizedlist>
<para>
There are two ways in which to trigger a LiveSync operation over REST.
</para>
<listitem>
<para>
Use the <literal>_action=liveSync</literal> parameter directly on the
resource. This is the recommended method. The following example calls a
LiveSync operation on the user accounts in an external LDAP system.
</para>
<screen>$ curl \
--cacert self-signed.crt \
--header "X-OpenIDM-Username: openidm-admin" \
--header "X-OpenIDM-Password: openidm-admin" \
--header "Content-Type: application/json" \
--request POST \
</listitem>
<listitem>
<para>
Target the <literal>system</literal> endpoint and supply a
<literal>source</literal> parameter to identify the object that should be
synchronized. This method matches the scheduler configuration and can
therefore be used to test schedules before they are implemented.
</para>
<para>
The following example calls the same LiveSync operation as the previous
example.
</para>
<screen>$ curl \
--cacert self-signed.crt \
--header "X-OpenIDM-Username: openidm-admin" \
--header "X-OpenIDM-Password: openidm-admin" \
--header "Content-Type: application/json" \
--request POST \
</listitem>
</itemizedlist>
<para>
A successful LiveSync operation returns the following response:
</para>
<screen>{
"_rev": "4",
"_id": "SYSTEMLDAPACCOUNT",
"connectorData": {
"nativeType": "integer",
"syncToken": 1
}
}</screen>
<para>
Do not run two identical LiveSync operations simultaneously - rather, ensure
that the first operation has completed before a second similar operation is
launched.
</para>
<para>
To troubleshoot a LiveSync operation that has not succeeded, you can include
an optional parameter (<literal>detailedFailure</literal>) to return
additional information. For example:
</para>
<screen>$ curl \
--cacert self-signed.crt \
--header "X-OpenIDM-Username: openidm-admin" \
--header "X-OpenIDM-Password: openidm-admin" \
--header "Content-Type: application/json" \
--request POST \
"https://localhost:8443/openidm/system/ldap/account?_action=liveSync&detailedFailure=true"</screen>
<note>
<para>
The first time that a LiveSync operation is called, no synchronization
token exists in the database to establish which changes have already been
processed. The default LiveSync behavior is to locate the last existing
entry in the change log, and to store that entry in the database as the
current starting position from which changes should be applied. This
behavior prevents LiveSync from processing changes that might already have
been processed during an initial data load. Subsequent LiveSync operations
will pick up and process any new changes.
</para>
<para>
Typically, in setting up LiveSync on a new system, you would load the data
initially (by using reconciliation, for example) and then enable LiveSync,
starting from that base point.
</para>
</note>
</section>
</section>
<section xml:id="recon-by-query">
<title>Restricting Reconciliation by Using Queries</title>
<para>
Every reconciliation operation performs a query on the source, and on the
target resource, to determine which records should be reconciled. The default
source and target queries are <literal>query-all-ids</literal>, which means
that all records in both the source and the target are considered candidates
for that reconciliation operation.
</para>
<para>
You can restrict reconciliation to specific entries by defining explicit
source or target queries in the mapping configuration.
</para>
<para>
For example, to restrict reconciliation to only those records whose
<literal>employeeType</literal> on the source resource is
<literal>Permanent</literal>, you might specify a source query as follows:
</para>
<programlisting language="javascript">"mappings" : [
{
"name" : "managedUser_systemLdapAccounts",
"sourceQuery" : : {
"queryFilter" : "employeeType eq \"Permanent\""
},
...</programlisting>
<para>
The format of the query can be any query type that is supported by the
resource, and can include additional parameters, if applicable. OpenIDM
${docTargetVersion} supports the following query types.
</para>
<itemizedlist>
<para>
For queries on managed objects:
</para>
<listitem>
<para>
<literal>_queryId</literal> for arbitrary predefined, parameterized
queries
</para>
</listitem>
<listitem>
<para>
<literal>_queryExpression</literal> for client-supplied queries, in native
query format
</para>
</listitem>
</itemizedlist>
<itemizedlist>
<para>
For queries on system objects:
</para>
<listitem>
<para>
<literal>_queryId=query-all-ids</literal> (the only supported predefined
query)
</para>
</listitem>
<listitem>
<para>
<literal>_queryFilter</literal> for arbitrary filters, in common filter
notation
</para>
</listitem>
</itemizedlist>
<para>
The source and target queries send the query to the resource that is defined
for that source or target, by default. You can override the resource to which
the query is sent by specifying a <literal>resourceName</literal> in the
query. For example, to query a specific endpoint instead of the source
resource, you might modify the preceding source query as follows:
</para>
<programlisting language="javascript">"mappings" : [
{
"name" : "managedUser_systemLdapAccounts",
"sourceQuery" : {
"resourceName" : "endpoint/scriptedQuery"
"queryFilter" : "employeeType eq \"Permanent\""
},
...</programlisting>
<para>
To override a source or target query that is defined in the mapping, you can
specify the query when you call the reconciliation operation. For example, if
you wanted to reconcile all employee entries, and not just the permanent
employees, you would run the reconciliation operation as follows:
</para>
<screen>$ curl \
--cacert self-signed.crt \
--header "X-OpenIDM-Username: openidm-admin" \
--header "X-OpenIDM-Password: openidm-admin" \
--header "Content-Type: application/json" \
--request POST \
--data '{"sourceQuery": {"_queryId" : "query-all-ids"}}' \
"https://localhost:8443/openidm/recon?_action=recon&mapping=managedUser_systemLdapAccounts"</screen>
<para>
By default, a reconciliation operation runs both the source and target phase,
that is both the source and the target resources are queried. To specify that
either the source or target resource is not queried, set
<literal>runSourcePhase</literal> or <literal>runTargetPhase</literal> to
<literal>false</literal> in the mapping configuration
target resource from being queried during the reconciliation operation
configured in the previous example, you would amend the mapping configuration
as follows:
</para>
<programlisting>{
"mappings" : [
{
"name" : "systemLdapAccounts_managedUser",
"sourceQuery" : {
"queryFilter" : "employeeType eq \"Permanent\""
},
"runTargetPhase" : false,
... </programlisting>
</section>
<section xml:id="recon-by-id">
<title>Restricting Reconciliation to a Specific ID</title>
<para>
In the same way that you can restrict reconciliation operations to specific
records by using queries, you can specify an ID to restrict a reconciliation
operation to a particular record.
</para>
<para>
To restrict reconciliation to a specific ID, use the
<literal>reconById</literal> action, instead of the <literal>recon</literal>
action when you call the reconciliation operation. Specify the ID with the
<literal>ids</literal> parameter. Currently reconciling more than one ID with
the <literal>reconById</literal> action is not supported.
</para>
<para>
The following example is based on the data from Sample 2b, which maps an
LDAP server with the OpenIDM repository. The example reconciles only the user
<literal>bjensen</literal>, using the
<literal>managedUser_systemLdapAccounts</literal> mapping to update the user
account in LDAP with the data from the OpenIDM repository. The
<literal>_id</literal> for <literal>bjensen</literal> in this example is
<literal>b3c2f414-e7b3-46aa-8ce6-f4ab1e89288c</literal>.
The example assumes that automatic synchronization has been disabled and that
a reconciliation operation is required to copy changes made in the repository
to the LDAP system.
</para>
<screen width="102">$ curl \
--cacert self-signed.crt \
--header "X-OpenIDM-Username: openidm-admin" \
--header "X-OpenIDM-Password: openidm-admin" \
--header "Content-Type: application/json" \
--request POST \
<para>
A reconciliation by ID takes the default reconciliation options that are
specified in the mapping, so the source and target queries, and source and
target phases described in the previous section apply equally to
reconciliation by ID.
</para>
</section>
<section xml:id="querying-recon-logs">
<title>Querying the Reconciliation Audit Log</title>
<para>
Reconciliation operations are logged in the file
repository. You can read and query the reconciliation audit logs over the
REST interface, as outlined in the following examples.
</para>
<para>
based on the <literal>entryType</literal> of the entry. Fields that are not
required for the specific entry type are stripped away from the response.
For example, a <literal>summary</literal> entry would not need to include a
null <literal>targetObjectId</literal> field, as this would not add
information to a summary. You can specify that this auto-formatting be
disabled and return the full entry for all entry types. To disable entry
formatting, include <literal>formatted=false</literal> as a query parameter
in the request.
</para>
<para>
To return all reconciliation operations logged in the audit log, run a
</para>
<screen>$ <userinput>curl \
--cacert self-signed.crt \
--header "X-OpenIDM-Username: openidm-admin" \
--header "X-OpenIDM-Password: openidm-admin" \
--request GET \
<para>
The following code sample shows an extract of the audit log after the first
reconciliation operation in Sample 1.
</para>
<programlisting language="javascript">{
"entries": [
{
"mapping": "systemXmlfileAccounts_managedUser",
"situation": "ABSENT",
"reconciling": "source",
"ambiguousTargetObjectIds": "",
"action": "CREATE",
"actionId": "e5f3190d-41d9-4bea-907d-b287a9436a45",
"exception": "",
"_id": "fe250514-d3e1-477a-bb90-88bd4525d70b",
"entryType": "entry",
"timestamp": "2014-09-08T08:57:47.575Z",
"reconId": "e5f3190d-41d9-4bea-907d-b287a9436a45",
"rootActionId": "e5f3190d-41d9-4bea-907d-b287a9436a45",
"status": "SUCCESS",
"message": null,
"messageDetail": null
},
{
"mapping": "systemXmlfileAccounts_managedUser",
"exception": "",
"_id": "10e4195b-7b38-4b99-9916-a6d2de137c11",
"entryType": "start",
"timestamp": "2014-09-08T08:57:47.218Z",
"reconId": "e5f3190d-41d9-4bea-907d-b287a9436a45",
"rootActionId": "e5f3190d-41d9-4bea-907d-b287a9436a45",
"status": "SUCCESS",
"message": "Reconciliation initiated by openidm-admin",
"messageDetail": null
},
{
"mapping": "systemXmlfileAccounts_managedUser",
"exception": "",
"_id": "d8634325-78f6-4504-b9f4-ba7b9103e391",
"entryType": "summary",
"timestamp": "2014-09-08T08:57:47.607Z",
"reconId": "e5f3190d-41d9-4bea-907d-b287a9436a45",
"rootActionId": "e5f3190d-41d9-4bea-907d-b287a9436a45",
"status": "SUCCESS",
"message": "SOURCE_IGNORED: 0 MISSING: 0 FOUND: 0 AMBIGUOUS: 0 UNQUALIFIED: 0 CONFIRMED: 0 SOURCE_MISSING: 0 ABSENT: 2 TARGET_IGNORED: 0 UNASSIGNED: 0 ",
"messageDetail": {
"stage": "COMPLETED_SUCCESS",
"stageDescription": "reconciliation completed.",
"progress": {
"links": {
"created": 2,
"existing": {
"processed": 0,
"total": "0"
}
},
"source": {
"existing": {
"processed": 2,
"total": "2"
}
},
"target": {
"created": 2,
"existing": {
"processed": 0,
"total": "0"
}
}
},
"duration": 388,
"situationSummary": {
"SOURCE_MISSING": 0,
"FOUND": 0,
"SOURCE_IGNORED": 0,
"UNQUALIFIED": 0,
"UNASSIGNED": 0,
"TARGET_IGNORED": 0,
"CONFIRMED": 0,
"AMBIGUOUS": 0,
"ABSENT": 2,
"MISSING": 0
},
"statusSummary": {
"FAILURE": 0,
"SUCCESS": 2
},
"state": "SUCCESS",
"mapping": "systemXmlfileAccounts_managedUser",
"started": "2014-09-08T08:57:47.218Z",
"ended": "2014-09-08T08:57:47.606Z"
}
},
{
"mapping": "systemXmlfileAccounts_managedUser",
"situation": "ABSENT",
"reconciling": "source",
"ambiguousTargetObjectIds": "",
"action": "CREATE",
"actionId": "e5f3190d-41d9-4bea-907d-b287a9436a45",
"exception": "",
"_id": "939fd113-1158-4f5c-a7f7-6c4b005dce2f",
"entryType": "entry",
"timestamp": "2014-09-08T08:57:47.579Z",
"reconId": "e5f3190d-41d9-4bea-907d-b287a9436a45",
"rootActionId": "e5f3190d-41d9-4bea-907d-b287a9436a45",
"status": "SUCCESS",
"message": null,
"messageDetail": null
}
]
} </programlisting>
<para>
Most of the fields in this audit log are self-explanatory. Each distinct
reconciliation operation is identified by its <literal>reconId</literal>.
Each entry in the log is identified by a unique <literal>_id</literal>. The
first log entry indicates the status for the complete reconciliation
operation. Successive entries indicate the status for each record affected
by the reconciliation.
</para>
<para>
To obtain information on a specific audit log entry, include its entry
<literal>_id</literal> in the URL. For example:
</para>
<screen>$ curl \
--cacert self-signed.crt \
--header "X-OpenIDM-Username: openidm-admin" \
--header "X-OpenIDM-Password: openidm-admin" \
--request GET \
<para>
The following sample output shows the results of a read operation on a
specific reconciliation audit entry.
</para>
<programlisting language="javascript">{
"mapping": "systemXmlfileAccounts_managedUser",
"situation": "ABSENT",
"reconciling": "source",
"ambiguousTargetObjectIds": "",
"action": "CREATE",
"actionId": "e5f3190d-41d9-4bea-907d-b287a9436a45",
"exception": "",
"_id": "fe250514-d3e1-477a-bb90-88bd4525d70b",
"entryType": "entry",
"timestamp": "2014-09-08T08:57:47.575Z",
"reconId": "e5f3190d-41d9-4bea-907d-b287a9436a45",
"rootActionId": "e5f3190d-41d9-4bea-907d-b287a9436a45",
"status": "SUCCESS",
"message": null,
"messageDetail": null
} </programlisting>
<para>
To query the audit log for a particular reconciliation operation, use the
<literal>audit-by-recon-id</literal> keyword, specifying the reconciliation
ID, as follows:
</para>
<screen><userinput>$ curl \
--cacert self-signed.crt \
--header "X-OpenIDM-Username: openidm-admin" \
--header "X-OpenIDM-Password: openidm-admin" \
--request GET \
"https://localhost:8443/openidm/audit/recon?_queryId=audit-by-recon-id&reconId=<reconID>"</userinput></screen>
<para>
Output similar to the following is returned, for the specified
reconciliation operation:
</para>
<programlisting language="javascript">{
"remainingPagedResults": -1,
"pagedResultsCookie": null,
"resultCount": 4,
"result": [
{
"mapping": "systemXmlfileAccounts_managedUser",
"exception": "",
"_id": "d8634325-78f6-4504-b9f4-ba7b9103e391",
"entryType": "summary",
"timestamp": "2014-09-08T08:57:47.607Z",
"reconId": "e5f3190d-41d9-4bea-907d-b287a9436a45",
"rootActionId": "e5f3190d-41d9-4bea-907d-b287a9436a45",
"status": "SUCCESS",
"message": "SOURCE_IGNORED: 0 MISSING: 0 FOUND: 0 AMBIGUOUS: 0 UNQUALIFIED: 0
CONFIRMED: 0 SOURCE_MISSING: 0 ABSENT: 2 TARGET_IGNORED: 0 UNASSIGNED: 0 ",
"messageDetail": {
"stage": "COMPLETED_SUCCESS",
"stageDescription": "reconciliation completed.",
"progress": {
"links": {
"created": 2,
"existing": {
"processed": 0,
"total": "0"
}
},
"source": {
"existing": {
"processed": 2,
"total": "2"
}
},
"target": {
"created": 2,
"existing": {
"processed": 0,
"total": "0"
}
}
},
"duration": 388,
"situationSummary": {
"SOURCE_MISSING": 0,
"FOUND": 0,
"SOURCE_IGNORED": 0,
"UNQUALIFIED": 0,
"UNASSIGNED": 0,
"TARGET_IGNORED": 0,
"CONFIRMED": 0,
"AMBIGUOUS": 0,
"ABSENT": 2,
"MISSING": 0
},
"statusSummary": {
"FAILURE": 0,
"SUCCESS": 2
},
"state": "SUCCESS",
"mapping": "systemXmlfileAccounts_managedUser",
"started": "2014-09-08T08:57:47.218Z",
"ended": "2014-09-08T08:57:47.606Z"
}
},
{
"mapping": "systemXmlfileAccounts_managedUser",
"exception": "",
"_id": "10e4195b-7b38-4b99-9916-a6d2de137c11",
"entryType": "start",
"timestamp": "2014-09-08T08:57:47.218Z",
"reconId": "e5f3190d-41d9-4bea-907d-b287a9436a45",
"rootActionId": "e5f3190d-41d9-4bea-907d-b287a9436a45",
"status": "SUCCESS",
"message": "Reconciliation initiated by openidm-admin",
"messageDetail": null
},
{
"mapping": "systemXmlfileAccounts_managedUser",
"situation": "ABSENT",
"reconciling": "source",
"ambiguousTargetObjectIds": "",
"action": "CREATE",
"actionId": "e5f3190d-41d9-4bea-907d-b287a9436a45",
"exception": "",
"_id": "fe250514-d3e1-477a-bb90-88bd4525d70b",
"entryType": "entry",
"timestamp": "2014-09-08T08:57:47.575Z",
"reconId": "e5f3190d-41d9-4bea-907d-b287a9436a45",
"rootActionId": "e5f3190d-41d9-4bea-907d-b287a9436a45",
"status": "SUCCESS",
"message": null,
"messageDetail": null
},
{
"mapping": "systemXmlfileAccounts_managedUser",
"situation": "ABSENT",
"reconciling": "source",
"ambiguousTargetObjectIds": "",
"action": "CREATE",
"actionId": "e5f3190d-41d9-4bea-907d-b287a9436a45",
"exception": "",
"_id": "939fd113-1158-4f5c-a7f7-6c4b005dce2f",
"entryType": "entry",
"timestamp": "2014-09-08T08:57:47.579Z",
"reconId": "e5f3190d-41d9-4bea-907d-b287a9436a45",
"rootActionId": "e5f3190d-41d9-4bea-907d-b287a9436a45",
"status": "SUCCESS",
"message": null,
"messageDetail": null
}
]
}</programlisting>
<para>
To query the audit log for a specific reconciliation situation, use the
<literal>audit-by-recon-id-situation</literal> keyword, specifying the
reconciliation ID and the situation that you want to query. For example,
the following query returns all ABSENT records found during the specified
reconciliation operation:
</para>
<screen>$ <userinput>curl \
--cacert self-signed.crt \
--header "X-OpenIDM-Username: openidm-admin" \
--header "X-OpenIDM-Password: openidm-admin" \
--request GET \
</screen>
<para>
Output similar to the following is returned, with one entry for each record
that matches the situation queried:
</para>
<programlisting language="javascript">{
"remainingPagedResults": -1,
"pagedResultsCookie": null,
"resultCount": 2,
"result": [
{
"mapping": "systemXmlfileAccounts_managedUser",
"situation": "ABSENT",
"reconciling": "source",
"ambiguousTargetObjectIds": "",
"action": "CREATE",
"actionId": "e5f3190d-41d9-4bea-907d-b287a9436a45",
"exception": "",
"_id": "fe250514-d3e1-477a-bb90-88bd4525d70b",
"entryType": "entry",
"timestamp": "2014-09-08T08:57:47.575Z",
"reconId": "e5f3190d-41d9-4bea-907d-b287a9436a45",
"rootActionId": "e5f3190d-41d9-4bea-907d-b287a9436a45",
"status": "SUCCESS",
"message": null,
"messageDetail": null
},
{
"mapping": "systemXmlfileAccounts_managedUser",
"situation": "ABSENT",
"reconciling": "source",
"ambiguousTargetObjectIds": "",
"action": "CREATE",
"actionId": "e5f3190d-41d9-4bea-907d-b287a9436a45",
"exception": "",
"_id": "939fd113-1158-4f5c-a7f7-6c4b005dce2f",
"entryType": "entry",
"timestamp": "2014-09-08T08:57:47.579Z",
"reconId": "e5f3190d-41d9-4bea-907d-b287a9436a45",
"rootActionId": "e5f3190d-41d9-4bea-907d-b287a9436a45",
"status": "SUCCESS",
"message": null,
"messageDetail": null
}
]
}</programlisting>
<para>
The activity logs track all operations on internal (managed) and external
(system) objects. Entries in the activity log contain identifiers for the
reconciliation or synchronization action that triggered the activity, and
for the original caller and the relationships between related actions.
</para>
<para>
You can access the activity logs over REST with the following call:
</para>
<screen>$ curl \
--cacert self-signed.crt \
--header "X-OpenIDM-Username: openidm-admin" \
--header "X-OpenIDM-Password: openidm-admin" \
--request GET \
<para>
The following extract of the activity log shows the last entry in the log,
which was a password change for user bjensen.
</para>
<programlisting language="javascript">{
"entries": [
...
},
"before": null,
"requester": "openidm-admin",
"parentActionId": "c2c102bc-7b32-4020-b5aa-9a7d63652cb6",
"_id": "bbaff1e0-923b-48f0-b053-b1614cbb3647",
"activityId": "c2c102bc-7b32-4020-b5aa-9a7d63652cb6",
"timestamp": "2014-03-13T16:20:54.811Z",
"action": "CREATE",
"message": "create",
"rev": "1",
"rootActionId": "c2c102bc-7b32-4020-b5aa-9a7d63652cb6"
},
{
"passwordChanged": true,
"changedFields": [
"/password"
],
"status": "SUCCESS",
"after": {
"securityAnswer": {
"$crypto": {
"value": {
"key": "openidm-sym-default",
"iv": "8CvlA6rWN03MAhLSKJmbvw==",
"data": "oJBTrrX+wFAygFZkLuGPrhB/jAIICcdIBuCX1eEbpS0="
},
"type": "x-simple-encryption"
}
},
...
</programlisting>
<para>
To return activity information for a specific action, include the
<literal>_id</literal> of the action in the endpoint, for example:
</para>
<screen><?dbfo pgwide="1"?>$ curl \
--cacert self-signed.crt \
--header "X-OpenIDM-Username: openidm-admin" \
--header "X-OpenIDM-Password: openidm-admin" \
--request GET \
<para>
Results similar to the following are returned:
</para>
<programlisting language="javascript">{
"passwordChanged": true,
"changedFields": [
"/password"
],
"status": "SUCCESS",
"after": {
"securityAnswer": {
"$crypto": {
"value": {
"key": "openidm-sym-default",
"iv": "HpsyTtTXc2pfNrXlYbro7Q==",
"data": "0M6O7geNjalJ7e0EGSG9B90eaeF8zJuogdL74hcAIRg="
},
"type": "x-simple-encryption"
}
},
"userName": "bjensen@example.com",
"stateProvince": "",
"postalAddress": "",
"effectiveAssignments": {},
"roles": "openidm-authorized",
"telephoneNumber": "1234567",
"accountStatus": "active",
"password": {
"$crypto": {
"value": {
"key": "openidm-sym-default",
"iv": "dkRjURz761HaObBuLl+EkA==",
"data": "9chNPUlXotHy195ERj6vlg=="
},
"type": "x-simple-encryption"
}
},
"effectiveRoles": [
"openidm-authorized"
],
"givenName": "Barbara",
"lastPasswordAttempt": "Thu Mar 13 2014 07:23:12 GMT-0800 (GMT-08:00)",
"address2": "",
"passwordAttempts": "0",
"sn": "Jensen",
"mail": "bjensen@example.com",
"securityQuestion": "1",
"city": "",
"country": "",
"_rev": "7",
"lastPasswordSet": "",
"postalCode": "",
"_id": "bjensen",
"description": "Created By XML1"
},
"before": {
"securityAnswer": "Some security answer",
"userName": "bjensen@example.com",
"stateProvince": "",
"postalAddress": "",
"roles": "openidm-authorized",
"telephoneNumber": "1234567",
"password": {
"$crypto": {
"value": {
"key": "openidm-sym-default",
"iv": "bqhRyLW1lI+KZROcpgyukg==",
"data": "qO8A76GqNqftVVwOlasyPw=="
},
"type": "x-simple-encryption",
"securityQuestion": "1",
"givenName": "Barbara",
"address2": "",
"lastPasswordAttempt": "Thu Mar 13 2014 07:23:12 GMT-0800 (GMT-08:00)",
"passwordAttempts": "0",
"sn": "Jensen",
"mail": "bjensen@example.com",
"country": "",
"city": "",
"_rev": "7",
"lastPasswordSet": "",
"postalCode": "",
"_id": "bjensen",
"description": "Created By XML1",
"accountStatus": "active"
},
"requester": "openidm-admin",
"parentActionId": "71ddeed8-9006-4578-b869-13e15a3ce6b5",
"_id": "ee88adb8-3329-4f81-a8f2-d9c8e0fbf72b",
"activityId": "71ddeed8-9006-4578-b869-13e15a3ce6b5",
"timestamp": "2014-03-13T16:21:27.086Z",
"action": "UPDATE",
"message": "update",
"rev": "7",
"rootActionId": "71ddeed8-9006-4578-b869-13e15a3ce6b5"
} </programlisting>
<para>
Each action in the activity log has a <literal>rootActionId</literal> and a
<literal>parentActionId</literal>. The <literal>rootActionId</literal> is
the ID that was assigned to the incoming or initiating request. The
<literal>parentActionId</literal> is the ID that is associated with the
overall action. So, for example, if an HTTP request invokes a script that
changes a user's password, the HTTP request is assigned the
<literal>rootActionId</literal> and the action taken by the script is
assigned the <literal>parentActionId</literal>. You can query the activity
log for the details of a specific action by including the
<literal>parentActionId</literal> in the query. For example:
</para>
<screen><userinput>$ curl \
--cacert self-signed.crt \
--header "X-OpenIDM-Username: openidm-admin" \
--header "X-OpenIDM-Password: openidm-admin" \
--request GET \
"https://localhost:8443/openidm/audit/activity?_queryId=audit-by-activity-parent-action&parentActionId=0aaba292-1dd3-4e98-a0e2-04bec9ae5209"</userinput></screen>
<para>
The following sample output shows the result of a query that requests
details of the password change for bjensen.
</para>
<programlisting language="javascript">{
"remainingPagedResults": -1,
"pagedResultsCookie": null,
"resultCount": 2,
"result": [
{
"rootActionId": "71ddeed8-9006-4578-b869-13e15a3ce6b5",
"changedFields": [
"/password"
],
"action": "UPDATE",
"before": {
"securityAnswer": "Some security answer",
"userName": "bjensen@example.com",
"stateProvince": "",
"postalAddress": "",
"roles": "openidm-authorized",
"telephoneNumber": "1234567",
"password": "CAngetin1",
"securityQuestion": "1",
"givenName": "Barbara",
"address2": "",
"lastPasswordAttempt": "Thu Mar 13 2014 07:23:12 GMT-0800 (GMT-08:00)",
"passwordAttempts": "0",
"sn": "Jensen",
"mail": "bjensen@example.com",
"country": "",
"city": "",
"_rev": "7",
"lastPasswordSet": "",
"postalCode": "",
"_id": "bjensen",
"description": "Created By XML1",
"accountStatus": "active"
},
"status": "SUCCESS",
"_rev": "1",
"_id": "ee88adb8-3329-4f81-a8f2-d9c8e0fbf72b",
"parentActionId": "71ddeed8-9006-4578-b869-13e15a3ce6b5",
"timestamp": "2014-03-13T16:21:27.086Z",
"message": "update",
"activityId": "71ddeed8-9006-4578-b869-13e15a3ce6b5",
"after": {
"securityAnswer": {
"$crypto": {
"value": {
"key": "openidm-sym-default",
"iv": "HpsyTtTXc2pfNrXlYbro7Q==",
"data": "0M6O7geNjalJ7e0EGSG9B90eaeF8zJuogdL74hcAIRg="
},
"type": "x-simple-encryption"
}
},
"userName": "bjensen@example.com",
"stateProvince": "",
"postalAddress": "",
"effectiveAssignments": {},
"roles": "openidm-authorized",
"telephoneNumber": "1234567",
"accountStatus": "active",
"password": {
"$crypto": {
"value": {
"key": "openidm-sym-default",
"iv": "dkRjURz761HaObBuLl+EkA==",
"data": "9chNPUlXotHy195ERj6vlg=="
},
"type": "x-simple-encryption"
}
},
"effectiveRoles": [
"openidm-authorized"
],
"givenName": "Barbara",
"lastPasswordAttempt": "Thu Mar 13 2014 07:23:12 GMT-0800 (GMT-08:00)",
"address2": "",
"passwordAttempts": "0",
"sn": "Jensen",
"mail": "bjensen@example.com",
"securityQuestion": "1",
"city": "",
"country": "",
"_rev": "7",
"lastPasswordSet": "",
"postalCode": "",
"_id": "bjensen",
"description": "Created By XML1"
},
"rev": "7",
"requester": "openidm-admin",
"passwordChanged": true
}
]
}</programlisting>
<note>
<para>
For audit logs in the repository, you can define custom queries using the
parameterized query mechanism. For more information, see the section on
<link xlink:href="integrators-guide#parameterized-queries"
Queries</citetitle></link>.
</para>
</note>
<para>
For more information about the entries in these logs, see the chapter that
covers <link xlink:href="integrators-guide#chap-auditing"
Logs</citetitle></link>.
</para>
</section>
<section xml:id="livesync-retry-strategy">
<title>Configuring the LiveSync Retry Policy</title>
<para>
OpenIDM enables you to specify what should happen if a LiveSync operation
reports a failure for an operation. By configuring the LiveSync retry
policy, you can specify how many times a failed modification should be
reattempted and what should happen in the event that the modification is
unsuccessful after the specified number of attempts. If no retry policy is
configured, OpenIDM reattempts the change an infinite number of times, until
the change is successful. This behavior can increase data consistency in the
case of transient failures (for example, when the connection to the database
is temporarily lost). However, in situations where the cause of the failure
is permanent (for example, if the change does not meet certain policy
requirements) the change will never succeed, regardless of the number of
attempts. In this case, the infinite retry behavior can effectively block
subsequent LiveSync operations from starting.
</para>
<para>
Generally, a scheduled reconciliation operation will eventually force
consistency. However, to prevent repeated retries that block the LiveSync
mechanism, you should restrict the number of times OpenIDM reattempts the
same modification. You can then specify what OpenIDM does with failed
LiveSync changes. The failed modification can be stored in a "dead letter
queue", discarded, or reapplied. Alternatively, an administrator can be
notified of the failure by email or by some other means. This behavior can
be scripted. The default configuration, in the samples provided with
OpenIDM, is to retry a failed modification five times, and then to log and
ignore the failure.
</para>
<para>
The LiveSync retry policy is configured in the connector configuration file
configuration files have a retry policy defined as follows:
</para>
<programlisting language="javascript">
"syncFailureHandler" : {
"maxRetries" : 5,
"postRetryAction" : "logged-ignore"
},
</programlisting>
<para>
The <literal>maxRetries</literal> field specifies the number of attempts
that OpenIDM should make to process the failed modification. The value of
this property must be a positive integer, or <literal>-1</literal>. A value
of zero indicates that failed modifications should not be reattempted. In
this case, the post retry action is executed immediately when a LiveSync
operation fails. A value of <literal>-1</literal> (or omitting the
<literal>maxRetries</literal> property, or the entire
<literal>syncFailureHandler</literal> from the configuration) indicates that
failed modifications should be retried an infinite number of times. In this
case, no post retry action is executed.
</para>
<para>
The default retry policy relies on the scheduler, or whatever invokes the
LiveSync operation. Therefore, if retries are enabled and a LiveSync
modification fails, OpenIDM will retry the modification the next time that
LiveSync is invoked.
</para>
<para>
The <literal>postRetryAction</literal> field indicates what action OpenIDM
should take in the event that the maximum number of retries has been reached
(or if <literal>maxRetries</literal> has been set to zero). The post retry
action can be one of the following:
</para>
<itemizedlist>
<listitem>
<para>
<literal>logged-ignore</literal> indicates that OpenIDM should ignore the
failed modification, and log its occurrence.
</para>
</listitem>
<listitem>
<para>
<literal>dead-letter-queue</literal> indicates that OpenIDM should save
the details of the failed modification in a table in the repository
(accessible over REST at
<literal>repo/synchronisation/deadLetterQueue/<replaceable>provisioner-name</replaceable></literal>).
</para>
</listitem>
<listitem>
<para>
<literal>script</literal> specifies a custom script that should be
executed when the maximum number of retries has been reached. For
information about using custom scripts in the configuration, see the
<link xlink:href="integrators-guide#appendix-scripting"
Reference</citetitle></link>.
</para>
<para>
In addition to the regular objects described in the Scripting Reference,
the following objects are available in the script scope:
</para>
<itemizedlist>
<listitem>
<para>
<literal>syncFailure</literal>
</para>
<para>
Provides details about the failed record. The structure of the
<literal>syncFailure</literal> object is as follows:
</para>
<programlisting language="javascript">
"syncFailure" :
{
"token" : <replaceable>the ID of the token</replaceable>,
"systemIdentifier" : <replaceable>a string identifier that matches the "name" property in
provisioner.openicf.json</replaceable>,
"objectType" : <replaceable>the object type being synced, one of the keys in the
"objectTypes" property in provisioner.openicf.json</replaceable>,
"uid" : <replaceable>the UID of the object (for example uid=joe,ou=People,dc=example,dc=com)</replaceable>,
"failedRecord", <replaceable>the record that failed to synchronize</replaceable>
},
</programlisting>
<para>
To access these fields, include
<literal>syncFailure.<replaceable>fieldname</replaceable></literal> in
your script.
</para>
</listitem>
<listitem>
<para>
<literal>failureCause</literal>
</para>
<para>
Provides the exception that caused the original LiveSync failure.
</para>
</listitem>
<listitem>
<para>
<literal>failureHandlers</literal>
</para>
<para>
OpenIDM currently provides two synchronization failure handlers "out of
the box". <literal>loggedIgnore</literal> indicates that the failure
should be logged, after which no further action should be taken.
<literal>deadLetterQueue</literal> indicates that the failed record
should be written to a specific table in the repository, where further
action can be taken. To invoke one of the internal failure handlers from
your script, use a call similar to the following (shown here for
JavaScript):
</para>
</listitem>
</itemizedlist>
<para>
Two sample scripts are provided in
logs failures, and one that sends them to the dead letter queue in the
repository.</para>
</listitem>
</itemizedlist>
<para>
The following sample provisioner configuration file extract shows a
LiveSync retry policy that specifies a maximum of four retries before the
failed modification is sent to the dead letter queue.
</para>
<programlisting language="javascript">
...
},<emphasis role="heavy">
"syncFailureHandler" : {
"maxRetries" : 4,
"postRetryAction" : dead-letter-queue
},</emphasis>
"poolConfigOption" : {
...
</programlisting>
<para>
In the case of a failed modification, a message similar to the following is
output to the log file:
</para>
<screen>INFO: sync retries = 1/4, retrying</screen>
<para>
OpenIDM reattempts the modification, the specified number of times. If the
modification is still unsuccessful, a message similar to the following is
logged:
</para>
<screen><?dbfo pgwide="1"?>INFO: sync retries = 4/4, retries exhausted
Jul 19, 2013 11:59:30 AM
INFO: uid=jdoe,ou=people,dc=example,dc=com saved to dead letter queue
</screen>
<para>
The log message indicates the entry for which the modification failed
(<literal>uid=jdoe</literal>, in this example).
</para>
<para>You can view the failed modification in the dead letter queue, over
the REST interface, as follows:
</para>
<screen><?dbfo pgwide="1"?>$ <userinput>curl \
--cacert self-signed.crt \
--header "X-OpenIDM-Username: openidm-admin" \
--header "X-OpenIDM-Password: openidm-admin" \
--request GET \
"https://localhost:8443/openidm/repo/synchronisation/deadLetterQueue/ldap?_queryId=query-all-ids"</userinput>
<computeroutput>{
"query-time-ms": 2,
"result":
[
{
"_id": "4",
"_rev": "0"
}
],
"conversion-time-ms": 0
}</computeroutput></screen>
<para>
To view the details of a specific failed modification, include its ID in
the URL:
</para>
<screen>$ <userinput>curl \
--cacert self-signed.crt \
--header "X-OpenIDM-Username: openidm-admin" \
--header "X-OpenIDM-Password: openidm-admin" \
--request GET \
<computeroutput>{
"objectType": "account",
"systemIdentifier": "ldap",
ReferenceError: \"bad\" is not defined.
"token": 4,
"failedRecord": "<replaceable>complete record, in xml format</replaceable>"
"uid": "uid=jdoe,ou=people,dc=example,dc=com",
"_rev": "0",
"_id": "4"
} </computeroutput></screen>
</section>
<section xml:id="handling-sync">
<title>Synchronization Situations and Actions</title>
<indexterm>
<primary>Synchronization</primary>
<secondary>Situations</secondary>
</indexterm>
<indexterm>
<primary>Synchronization</primary>
<secondary>Actions</secondary>
</indexterm>
<para>
During synchronization, OpenIDM categorizes objects according to their
<firstterm>situation</firstterm>. Situations are characterized by whether an
object exists on a source or target system, whether OpenIDM has registered a
link between the source object and the target object, and whether the object
is considered <firstterm>valid</firstterm>, as assessed by the
<literal>validSource</literal> and <literal>validTarget</literal> scripts.
OpenIDM then takes a specific action, depending on the situation.
</para>
<para>
You can define actions for particular situations in the
<literal>policies</literal> section of a synchronization mapping, as shown in
the following excerpt.
</para>
<programlisting language="javascript">
{
"policies": [
{
"situation": "CONFIRMED",
"action": "UPDATE"
},
{
"situation": "FOUND",
"action": "UPDATE"
},
{
"situation": "ABSENT",
"action": "CREATE"
},
{
"situation": "AMBIGUOUS",
"action": "EXCEPTION"
},
{
"situation": "MISSING",
"action": "EXCEPTION"
},
{
"situation": "UNQUALIFIED",
"action": "DELETE"
},
{
"situation": "UNASSIGNED",
"action": "EXCEPTION"
}
]
}</programlisting>
<para>
If you do not define a policy for a particular situation, OpenIDM takes the
<firstterm>default action</firstterm> for the situation.
</para>
<para>
The following sections describe the possible situations and their default
corresponding actions.
</para>
<section xml:id="sync-situations">
<title>Synchronization Situations</title>
<orderedlist>
<para>
OpenIDM performs a reconciliation operation in two phases:
</para>
<listitem>
<para>
<firstterm>Source reconciliation</firstterm>, where OpenIDM accounts for
source objects and associated links, based on the configured mapping.
</para>
</listitem>
<listitem>
<para>
<firstterm>Target reconciliation</firstterm>, where OpenIDM iterates over
the target objects that were not processed in the first phase.
</para>
</listitem>
</orderedlist>
<orderedlist>
<para>
During the source reconciliation phase, OpenIDM builds three lists,
assigning values to the objects to reconcile.
</para>
<listitem>
<para>
All valid objects from the source
</para>
<para>
OpenIDM assigns valid source objects <literal>qualifies=1</literal>.
Invalid objects, including those that were not found in the source system,
and those that were filtered out by the script specified in the
<literal>validSource</literal> property, are assigned
<literal>qualifies=0</literal>.
</para>
</listitem>
<listitem>
<para>
All records from the appropriate links table
</para>
<para>
Objects that have a corresponding link in the links table of the
repository are assigned <literal>link=1</literal>. Objects that do not
have a corresponding link are assigned <literal>link=0</literal>.
</para>
</listitem>
<listitem>
<para>
All valid objects on the target system
</para>
<para>
Objects that are found in the target system are assigned
<literal>target=1</literal>. Objects that are not found in the target
system are assigned <literal>target=0</literal>.
</para>
</listitem>
</orderedlist>
<variablelist>
<para>
Based on the values assigned to objects during source reconciliation,
OpenIDM assigns situations, listed here with default and appropriate
alternative actions.
</para>
<varlistentry>
<term>"CONFIRMED" (qualifies=1, link=1, target=1)</term>
<listitem>
<para>
The source object qualifies for a target object, and a link to an
existing target object was found. This situation is detected during
change events and during reconciliation.
</para>
<para>
Default action: <literal>UPDATE</literal> the target object.
</para>
<para>
Other valid actions: <literal>IGNORE, REPORT, NOREPORT, ASYNC</literal>
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>"FOUND" (qualifies=1, link=0, target=1)</term>
<listitem>
<para>
The source object qualifies for a target object and there is no link to
an existing target object. There is a single target object, that
correlates with this source object, according to the logic in the
correlation query. This situation is detected during change events and
reconciliation.
</para>
<para>
Default action: <literal>UPDATE</literal> the target object.
</para>
<para>
Other valid actions: <literal>EXCEPTION, IGNORE, REPORT,
NOREPORT, ASYNC</literal>
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>"FOUND_ALREADY_LINKED" (qualifies=1, link=1, target=1)</term>
<listitem>
<para>
The source object qualifies for a target object and there is no link from
that source object to an existing target object. There is a single target
object, that correlates with this source object, according to the logic
in the correlation query, but that target object is already linked to a
different source object. This situation is detected during change events
and reconciliation.
</para>
<para>
Default action: log an <literal>EXCEPTION</literal>.
</para>
<para>
Other valid actions: <literal>IGNORE, REPORT, NOREPORT, ASYNC</literal>
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>"ABSENT" (qualifies=1, link=0, target=0)</term>
<listitem>
<para>
The source object qualifies for a target object, there is no link to an
existing target object, and there is no correlated target object found.
This situation is detected during change events and reconciliation.
</para>
<para>
Default action: <literal>CREATE</literal> a target object.
</para>
<para>
Other valid actions: <literal>EXCEPTION, IGNORE, REPORT, NOREPORT, ASYNC
</literal>
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>"AMBIGUOUS" (qualifies=1, link=0, target>1)</term>
<listitem>
<para>
The source object qualifies for a target object, there is no link to an
existing target object, but there is more than one correlated target
object (that is, more than one possible match on the target system). This
situation is detected during source object changes and reconciliation.
</para>
<para>
Default action: log an <literal>EXCEPTION</literal>.
</para>
<para>
Other valid actions: <literal>IGNORE, REPORT, NOREPORT, ASYNC</literal>
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>"MISSING" (qualifies=1, link=1, target=0)</term>
<listitem>
<para>
The source object qualifies for a target object, and there is a link to a
target object, but the target object is missing. This situation is
detected during reconciliation operations and during source object changes.
</para>
<para>
Default action: log an <literal>EXCEPTION</literal>.
</para>
<para>
Other valid actions: <literal>CREATE, UNLINK, IGNORE, REPORT, NOREPORT.
ASYNC</literal>
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>"UNQUALIFIED" (qualifies=0, link=0 or 1, target=1 or >1)</term>
<listitem>
<para>
The source object is unqualified (by the "validSource" script). One or
more target objects are found through the correlation logic. This
situation is detected during change events and reconciliation.
</para>
<para>
Default action: <literal>DELETE</literal> the target object or objects.
</para>
<para>
Other valid actions: <literal>EXCEPTION, IGNORE, REPORT, NOREPORT,
ASYNC</literal>
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>"TARGET_IGNORED" (qualifies=0, link=0 or 1, target=1)</term>
<listitem>
<para>
The source object is unqualified (by the "validSource" script). One or
more target objects are found through the correlation logic. This
situation is detected only during source object changes.
</para>
<para>
It is different from "UNQUALIFIED", based on the status of the link and
target. If there is a link, the target is not valid. If there is no link
and exactly one target, that target is not valid.
</para>
<para>
Default action: <literal>IGNORE</literal> the target object until the
next full reconciliation operation.
</para>
<para>
Other valid actions: <literal>DELETE, UNLINK, EXCEPTION, REPORT,
NOREPORT, ASYNC</literal>
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>"SOURCE_IGNORED" (qualifies=0, link=0, target=0)</term>
<listitem>
<para>
The source object is unqualified (by the "validSource" script), no link
is found, and no correlated target exists. This situation is detected
during source object changes and reconciliation.
</para>
<para>
Default action: <literal>IGNORE</literal> the source object.
</para>
<para>
Other valid actions: <literal>EXCEPTION, REPORT, NOREPORT, ASYNC</literal>
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>"LINK_ONLY" (qualifies=n/a, link=1, target=0)</term>
<listitem>
<para>
The source may or may not be qualified, a link is found, but no target
object is found. This situation is detected only during source object
changes.
</para>
<para>
Default action: Log an <literal>EXCEPTION</literal>.
</para>
<para>
Other valid actions: <literal>UNLINK, IGNORE, REPORT, NOREPORT,
ASYNC</literal>
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>"ALL_GONE" (qualifies=n/a, link=0, cannot-correlate)</term>
<listitem>
<para>
The source object has been removed. No link is found. Correlation is not
possible, for one of the following reasons:
</para>
<itemizedlist>
<listitem>
<para>
No previous source value can be found
</para>
</listitem>
<listitem>
<para>
There is no correlation query
</para>
</listitem>
<listitem>
<para>
A previous value was found, and a correlation query exists, but no
corresponding target was found
</para>
</listitem>
</itemizedlist>
<para>This situation is detected only during source object changes.
</para>
<para>
Default action: "IGNORE" the source object.
</para>
<para>
Other valid actions: <literal>EXCEPTION, REPORT, NOREPORT,
ASYNC</literal>
</para>
</listitem>
</varlistentry>
</variablelist>
<orderedlist>
<para>
During the target reconciliation phase, OpenIDM assigns the following
values as it iterates through the target objects that were not accounted
for during the source reconciliation.
</para>
<listitem>
<para>
Valid objects from the target
</para>
<para>
OpenIDM assigns valid target objects <literal>qualifies=1</literal>.
Invalid objects, including those that are filtered out by the script
specified in the <literal>validTarget</literal> property, are assigned
<literal>qualifies=0</literal>.
</para>
</listitem>
<listitem>
<para>
All records from the appropriate links table
</para>
<para>
Objects that have a corresponding link in the links table of the
repository are assigned <literal>link=1</literal>. Objects that do not
have a corresponding link are assigned <literal>link=0</literal>.
</para>
</listitem>
<listitem>
<para>
All valid objects on the source system
</para>
<para>
Objects that are found in the source system are assigned
<literal>source=1</literal>. Objects that are not found in the source
system are assigned <literal>source=0</literal>.
</para>
</listitem>
</orderedlist>
<para>
Based on the values that are assigned to objects during the target
reconciliation phase, OpenIDM assigns situations, listed here with their
default actions.
</para>
<variablelist>
<varlistentry>
<term>"TARGET_IGNORED" (qualifies=0)</term>
<listitem>
<para>
During target reconciliation, the target becomes unqualified by the
"validTarget" script. This situation is detected only during
reconciliation operations.
</para>
<para>
Default action: <literal>IGNORE</literal> the target object.
</para>
<para>
Other valid actions: <literal>DELETE, UNLINK, REPORT, NOREPORT,
ASYNC</literal>
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>"UNASSIGNED" (qualifies=1, link=0)</term>
<listitem>
<para>
A valid target object exists, for which there is no link. This situation
is detected only during reconciliation operations.
</para>
<para>
Default action: log an <literal>EXCEPTION</literal>.
</para>
<para>
Other valid actions: <literal>IGNORE, REPORT, NOREPORT, ASYNC</literal>
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>"CONFIRMED" (qualifies=1, link=1, source=1)</term>
<listitem>
<para>
The target object qualifies, and a link to a source object exists. This
situation is detected only during reconciliation operations.
</para>
<para>
Default action: <literal>UPDATE</literal> the target object.
</para>
<para>
Other valid actions: <literal>IGNORE, REPORT, NOREPORT</literal>
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>"UNQUALIFIED" (qualifies=0, link=1, source=1, but source does not
qualify)</term>
<listitem>
<para>
The target object is unqualified, (by the "validTarget" script), but
there is a link to an existing source object, which is also unqualified.
This situation is detected during change events and reconciliation.
</para>
<para>
Default action: <literal>DELETE</literal> the target object.
</para>
<para>
Other valid actions: <literal>UNLINK, EXCEPTION, IGNORE, REPORT,
NOREPORT, ASYNC</literal>
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>SOURCE_MISSING (qualifies=1, link=1, source=0)</term>
<listitem>
<para>
The target object qualifies and a link is found. But the source object is
missing. This situation is detected during change events and
reconciliation.
</para>
<para>
Default action: log an "EXCEPTION".
</para>
<para>
Other valid actions: <literal>DELETE, UNLINK, IGNORE, REPORT,
NOREPORT, ASYNC</literal>
</para>
</listitem>
</varlistentry>
</variablelist>
<para>
The following sections reiterate in detail how OpenIDM assigns situations
during each of the two synchronization phases.
</para>
</section>
<section xml:id="source-reconciliation">
<title>Source Reconciliation</title>
<para>
OpenIDM starts reconciliation and LiveSync by reading a list of objects from
the resource. For reconciliation, the list includes all objects that are
available through the connector. For LiveSync, the list contains only
changed objects. The connector can filter objects out of the list, too, by
using the script specified in the <literal>validSource</literal> property.
</para>
<para>
OpenIDM then iterates over the list, checking each entry against the
<literal>validSource</literal> filter, and classifying objects according to
their situations as described in <xref linkend="sync-situations"/>. OpenIDM
uses the list of links for the current mapping to classify objects. Finally,
OpenIDM executes the action that is configured for each situation.
</para>
<para>
The following table shows how OpenIDM assigns the appropriate situation
during source reconciliation, depending on whether a valid source exists
(Source Qualifies), whether a link exists in the repository (Link Exists),
and how many target objects are found, based either on links or on the
results of the correlation query.
</para>
<table pgwide="1" rules="none">
<title>Resolving Source Reconciliation Situations</title>
<tgroup cols="8">
<colspec colnum="1" colname="c1" colwidth="1*" />
<colspec colnum="2" colname="c2" colwidth="1*" />
<colspec colnum="3" colname="c3" colwidth="1*" />
<colspec colnum="4" colname="c4" colwidth="1*" />
<colspec colnum="5" colname="c5" colwidth="1*" />
<colspec colnum="6" colname="c6" colwidth="1*" />
<colspec colnum="7" colname="c7" colwidth="1*" />
<colspec colnum="8" colname="c8" colwidth="2*" />
<thead>
<row>
<entry namest="c1" nameend="c2" align="left">Source Qualifies?</entry>
<entry namest="c3" nameend="c4" align="left">Link Exists?</entry>
<entry namest="c5" nameend="c7" align="left">Target Objects
Found<footnote><para>If no link exists for the source object, then
OpenIDM executes a correlation query. If no previous object is available,
OpenIDM cannot correlate.</para></footnote></entry>
<entry morerows="1" valign="top" align="left">Situation</entry>
</row>
<row>
<entry>Yes</entry>
<entry>No</entry>
<entry>Yes</entry>
<entry>No</entry>
<entry>0</entry>
<entry>1</entry>
<entry>> 1</entry>
</row>
</thead>
<tbody>
<row>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry>SOURCE_MISSING</entry>
</row>
<row>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry> </entry>
<entry>X</entry>
<entry>UNQUALIFIED</entry>
</row>
<row>
<entry> </entry>
<entry>X</entry>
<entry>X</entry>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry> </entry>
<entry>UNQUALIFIED</entry>
</row>
<row>
<entry> </entry>
<entry>X</entry>
<entry>X</entry>
<entry> </entry>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry>TARGET_IGNORED</entry>
</row>
<row>
<entry> </entry>
<entry>X</entry>
<entry>X</entry>
<entry> </entry>
<entry> </entry>
<entry> </entry>
<entry>X</entry>
<entry>UNQUALIFIED</entry>
</row>
<row>
<entry>X</entry>
<entry> </entry>
<entry> </entry>
<entry>X</entry>
<entry>X</entry>
<entry> </entry>
<entry> </entry>
<entry>ABSENT</entry>
</row>
<row>
<entry>X</entry>
<entry> </entry>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry>FOUND</entry>
</row>
<row>
<entry>X</entry>
<entry> </entry>
<entry> </entry>
<entry>X<footnote><para>A link does exist from the target object but it
not for this specific source object.</para></footnote></entry>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry>FOUND_ALREADY_LINKED</entry>
</row>
<row>
<entry>X</entry>
<entry> </entry>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry> </entry>
<entry>X</entry>
<entry>AMBIGUOUS</entry>
</row>
<row>
<entry>X</entry>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry> </entry>
<entry>MISSING</entry>
</row>
<row>
<entry>X</entry>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry>CONFIRMED</entry>
</row>
</tbody>
</tgroup>
</table>
</section>
<section xml:id="target-reconciliation">
<title>Target Reconciliation</title>
<para>
During source reconciliation, OpenIDM cannot detect situations where no
source object exists, such as the UNASSIGNED situation. When no source
object exists, OpenIDM detects the situation during the second
reconciliation phase, target reconciliation. During target reconciliation,
OpenIDM iterates over all target objects that do not have a representation
on the source, checking each object against the
<literal>validTarget</literal> filter, determining the appropriate situation,
and executing the action configured for the situation.
</para>
<para>
The following table shows how OpenIDM assigns the appropriate situation
during target reconciliation, depending on whether a valid target exists
(Target Qualifies), whether a link with an appropriate type exists in the
repository (Link Exists), whether a source object exists (Source Exists),
and whether the source object qualifies (Source Qualifies). Not all
situations assigned during source reconciliation are assigned during target
reconciliation.
</para>
<table pgwide="1" rules="none">
<title>Resolving Target Reconciliation Situations</title>
<tgroup cols="9">
<colspec colnum="1" colname="c1" colwidth="1*" />
<colspec colnum="2" colname="c2" colwidth="1*" />
<colspec colnum="3" colname="c3" colwidth="1*" />
<colspec colnum="4" colname="c4" colwidth="1*" />
<colspec colnum="5" colname="c5" colwidth="1*" />
<colspec colnum="6" colname="c6" colwidth="1*" />
<colspec colnum="7" colname="c7" colwidth="1*" />
<colspec colnum="8" colname="c8" colwidth="1*" />
<colspec colnum="9" colname="c9" colwidth="2*" />
<thead>
<row>
<entry namest="c1" nameend="c2" align="left">Target Qualifies?</entry>
<entry namest="c3" nameend="c4" align="left">Link Exists?</entry>
<entry namest="c5" nameend="c6" align="left">Source Exists?</entry>
<entry namest="c7" nameend="c8" align="left">Source Qualifies?</entry>
<entry morerows="1" valign="top" align="left">Situation</entry>
</row>
<row>
<entry>Yes</entry>
<entry>No</entry>
<entry>Yes</entry>
<entry>No</entry>
<entry>Yes</entry>
<entry>No</entry>
<entry>Yes</entry>
<entry>No</entry>
</row>
</thead>
<tbody>
<row>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry> </entry>
<entry> </entry>
<entry> </entry>
<entry> </entry>
<entry> </entry>
<entry>TARGET_IGNORED</entry>
</row>
<row>
<entry>X</entry>
<entry> </entry>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry> </entry>
<entry>UNASSIGNED</entry>
</row>
<row>
<entry>X</entry>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry>CONFIRMED</entry>
</row>
<row>
<entry>X</entry>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry> </entry>
<entry>X</entry>
<entry>UNQUALIFIED</entry>
</row>
<row>
<entry>X</entry>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry> </entry>
<entry>SOURCE_MISSING</entry>
</row>
</tbody>
</tgroup>
</table>
</section>
<section xml:id="autosync-and-livesync">
<title>Situations Specific to Automatic Synchronization and LiveSync</title>
<para>
Certain situations occur only during automatic synchronization (when OpenIDM
pushes changes made in the repository out to external systems) and LiveSync
(when OpenIDM polls external system change logs for changes and updates the
repository).
</para>
<para>
The following table shows the situations that pertain only to automatic sync
and LiveSync, when records are <emphasis>deleted</emphasis> from the source
or target resource.
</para>
<table pgwide="1" rules="none">
<title>Resolving Automatic Sync and LiveSync Delete Situations</title>
<tgroup cols="8">
<colspec colnum="1" colname="c1" colwidth="1*" />
<colspec colnum="2" colname="c2" colwidth="1*" />
<colspec colnum="3" colname="c3" colwidth="1*" />
<colspec colnum="4" colname="c4" colwidth="1*" />
<colspec colnum="5" colname="c5" colwidth="1*" />
<colspec colnum="6" colname="c6" colwidth="1*" />
<colspec colnum="7" colname="c7" colwidth="1*" />
<colspec colnum="8" colname="c8" colwidth="2*" />
<thead>
<row>
<entry namest="c1" nameend="c2" align="left">Source Qualifies?</entry>
<entry namest="c3" nameend="c4" align="left">Link Exists?</entry>
<entry namest="c5" nameend="c7" align="left">Target Objects Found
<footnote>
<para>
If no link exists for the source object, then OpenIDM executes a
correlation query. If no previous object is available, OpenIDM cannot
correlate.
</para>
</footnote>
</entry>
<entry morerows="1" valign="top" align="left">Situation</entry>
</row>
<row>
<entry>Yes</entry>
<entry>No</entry>
<entry>Yes</entry>
<entry>No</entry>
<entry>0</entry>
<entry>1</entry>
<entry>> 1</entry>
</row>
</thead>
<tbody>
<row>
<entry>N/A</entry>
<entry>N/A</entry>
<entry>X</entry>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry> </entry>
<entry>LINK_ONLY</entry>
</row>
<row>
<entry>N/A</entry>
<entry>N/A</entry>
<entry> </entry>
<entry>X</entry>
<entry>X</entry>
<entry> </entry>
<entry> </entry>
<entry>ALL_GONE</entry>
</row>
<row>
<entry>X</entry>
<entry> </entry>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry> </entry>
<entry>X</entry>
<entry>AMBIGUOUS</entry>
</row>
<row>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry>X</entry>
<entry> </entry>
<entry> </entry>
<entry>X</entry>
<entry>UNQUALIFIED</entry>
</row>
</tbody>
</tgroup>
</table>
</section>
<section xml:id="sync-actions">
<title>Synchronization Actions</title>
<variablelist>
<para>Once OpenIDM has assigned a situation to an object, OpenIDM takes
the actions configured in the mapping. If no action is configured, then
OpenIDM takes the default action for the situation. OpenIDM supports the
following actions.</para>
<varlistentry>
<term>"CREATE"</term>
<listitem>
<para>Create and link a target object.</para>
</listitem>
</varlistentry>
<varlistentry>
<term>"UPDATE"</term>
<listitem>
<para>Link and update a target object.</para>
</listitem>
</varlistentry>
<varlistentry>
<term>"DELETE"</term>
<listitem>
<para>Delete and unlink the target object.</para>
</listitem>
</varlistentry>
<varlistentry>
<term>"LINK"</term>
<listitem>
<para>Link the correlated target object.</para>
</listitem>
</varlistentry>
<varlistentry>
<term>"UNLINK"</term>
<listitem>
<para>Unlink the linked target object.</para>
</listitem>
</varlistentry>
<varlistentry>
<term>"EXCEPTION"</term>
<listitem>
<para>Flag the link situation as an exception.</para>
<para>You should not use this action for LiveSync mappings.</para>
</listitem>
</varlistentry>
<varlistentry>
<term>"IGNORE"</term>
<listitem>
<para>Do not change the link or target object state.</para>
</listitem>
</varlistentry>
<varlistentry>
<term>"REPORT"</term>
<listitem>
<para>
Do not perform any action but report on what would happen if the default
action were performed.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>"NOREPORT"</term>
<listitem>
<para>
Do not perform any action or generate any report.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>"ASYNC"</term>
<listitem>
<para>
An asynchronous process has been started so do not perform any action or
generate any report. <!--TODO Clarify this -->
</para>
</listitem>
</varlistentry>
</variablelist>
</section>
<section xml:id="script-actions">
<title>Providing a Script as an Action</title>
<para>
In addition to the static synchronization actions described in the previous
section, you can provide a script that is run in specific synchronization
file specifies that when a synchronization operation assesses an entry as
<literal>ABSENT</literal>, the workflow named
<literal>managedUserApproval</literal> is invoked. The parameters for the
workflow are passed in as properties of the <literal>action</literal>
parameter.
</para>
<programlisting language="javascript">
{
"situation" : "ABSENT",
"action" : {
"workflowName" : "managedUserApproval",
"type" : "text/javascript",
"file" : "workflow/triggerWorkflowFromSync.js"
}
}
</programlisting>
<para>
The variables available to these scripts are described in
<link xlink:href="integrators-guide#appendix-scripting"
Available in Scripts</citetitle></link> in the <citetitle>Scripting
Appendix</citetitle>.
</para>
</section>
</section>
<section xml:id="asynchronous-reconciliation">
<title>Asynchronous Reconciliation</title>
<para>Reconciliation can work in tandem with workflows to provide
additional business logic to the reconciliation process. You can define
scripts to determine the action that should be taken for a particular
reconciliation situation. A reconciliation process can launch a workflow
after it has assessed a situation, and then perform the reconciliation, or
some other action.</para>
<para>For example, you might want a reconciliation process to assess new user
accounts that need to be created on a target resource. However, new user
account creation might require some kind of approval from a manager before
the accounts are actually created. The initial reconciliation process can
assess the accounts that need to be created, launch a workflow to request
management approval for those accounts, and then relaunch the reconciliation
process to create the accounts, once the management approval has been
received.</para>
<para>In this scenario, the defined script returns <literal>IGNORE</literal>
for new accounts and the reconciliation engine does not continue processing
the given object. The script then initiates an asynchronous process which
calls back and completes the reconciliation process at a later stage.</para>
<para>A sample configuration for this scenario is available in
xlink:href="install-guide#more-sample9"
Asynchronous Reconciliation Using Workflows</citetitle></link> in the
<citetitle>Installation Guide</citetitle>.</para>
<orderedlist>
<para>Configuring asynchronous reconciliation involves the following
steps:</para>
<listitem>
<para>Create the workflow definition file (<literal>.xml or .bar</literal> file)
For more information about creating workflows, see <link
<citetitle>Integrating Business Processes and Workflows</citetitle></link>.
</para>
</listitem>
<listitem>
situation or situations that should call the workflow. Reference the
workflow name in the configuration for that situation.</para>
calls the <literal>managedUserApproval</literal> workflow if the
situation is assessed as <literal>ABSENT</literal>:</para>
<programlisting language="javascript">
{
"situation" : "ABSENT",
"action" : {
"workflowName" : "managedUserApproval",
"type" : "text/javascript",
"file" : "workflow/triggerWorkflowFromSync.js"
}
}, </programlisting>
</listitem>
<listitem>
<para>
In the sample configuration, the workflow calls a second, explicit
reconciliation process as a final step. This reconciliation process is
called on the <literal>sync</literal> context path, with the
<literal>performAction</literal> action
</para>
<para>
You can also use this kind of explicit reconciliation to perform a
specific action on a source or target record, regardless of the assessed
situation.
</para>
<para>
You can call such an operation over the REST interface, specifying the
action can be any one of the supported reconciliation actions, that is,
<literal>CREATE, UPDATE, DELETE, LINK, UNLINK, EXCEPTION, REPORT,
NOREPORT, ASYNC, IGNORE</literal>. In addition, if you specify a
<literal>reconId</literal>, the action that is taken is logged in the
for that reconciliation run.
</para>
<para>
The following sample command calls the DELETE action on user
<literal>bjensen</literal>, whose <literal>_id</literal> in the LDAP
directory is <literal>uid=bjensen,ou=People,dc=example,dc=com</literal>.
The user is deleted in the target resource, in this case, the
OpenIDM repository.
</para>
<para>
Note that the <literal>_id</literal> must be URL-encoded in the REST
call.
</para>
<screen><userinput>$ curl \
--cacert self-signed.crt \
--header "X-OpenIDM-Username: openidm-admin" \
--header "X-OpenIDM-Password: openidm-admin" \
--header "Content-Type: application/json" \
--request POST \
systemLdapAccounts_ManagedUser&action=DELETE"</userinput>
<computeroutput>{}</computeroutput></screen>
</listitem>
</orderedlist>
</section>
<section xml:id="case-sensitivity">
<title>Configuring Case Sensitivity for Data Stores</title>
<para>By default, OpenIDM is case-sensitive, which means that case is taken
into account when comparing IDs during reconciliation. For data stores that
are case-insensitive, such as OpenDJ, IDs and links that are created by a
reconciliation process may be stored with a different case to the way in
which they are stored in the OpenIDM repository. Such a situation can cause
problems during a reconciliation operation, as the links for these IDs may
not match.</para>
<para>For such data stores, you can configure OpenIDM to ignore case during
reconciliation operations. With case sensitivity turned off in OpenIDM, for
those specific mappings, comparisons are done without regard to case.</para>
<para>To specify that data stores are not case-sensitive, set the
<literal>"sourceIdsCaseSensitive"</literal> or <literal>"targetIdsCaseSensitive"</literal>
property to <literal>false</literal> in the mapping for those links. For
example, if the LDAP data store is case-insensitive, set the mapping from
the LDAP store to the managed user repository as follows:</para>
<programlisting language="javascript">
"mappings" : [
{
"name" : "systemLdapAccounts_managedUser",
"sourceIdsCaseSensitive" : false,
"properties" : [
...
</programlisting>
<para>If a mapping inherits links by using the <literal>"links"</literal>
property, it is not necessary to set case sensitivity, because the mapping
uses the setting of the referred links.</para>
<para>
Note that configuring OpenIDM to be case-insensitive when comparing links
does not make the OpenICF provisioner case-insensitive when it requests
data. For example, if a user entry is stored with the ID
<literal>testuser</literal> and you make a request for
provisioners will filter out the match because of the difference in case,
and will indicate that the record is not found. To prevent the provisioner
from performing this secondary filtering, set the
<literal>enableFilteredResultsHandler</literal> property to
<literal>false</literal> in the provisioner configuration. For example:
</para>
<screen>"resultsHandlerConfig" :
{
"enableFilteredResultsHandler":false,
}, </screen>
<caution>
<para>Do not disable the filtered results handler for the CSV file
connector. The CSV file connector does not perform filtering so if you
disable the filtered results handler for this connector, the full CSV
file will be returned for every request.</para>
</caution>
</section>
<section xml:id="reconciliation-optimization">
<title>Reconciliation Optimization</title>
<para>By default, reconciliation is configured to function in an optimized
way. Some of these optimizations might, however, be unsuitable for your
environment. The following sections describe the optimizations and how they
can be configured.</para>
<section xml:id="correlate-target-set">
<title>Correlating Empty Target Sets</title>
<para>To optimize a reconciliation operation, the reconciliation process does
not attempt to correlate source objects to target objects if the set of
target objects is empty when the correlation is started. This considerably
speeds up the process the first time the reconciliation is run. You can change
this behavior for a specific mapping by adding the
<literal>correlateEmptyTargetSet</literal> property to the mapping definition
and setting it to <literal>true</literal>. For example:</para>
<programlisting language="javascript">
{
"mappings": [
{
"name" : "systemMyLDAPAccounts_managedUser",
"correlateEmptyTargetSet" : true
},
]
}</programlisting>
<para>Be aware that this setting will have a performance impact on the
reconciliation process.</para>
</section>
<section xml:id="prefetching-links">
<title>Prefetching Links</title>
<para>All links are queried at the start of a correlation and the results of
that query are used. You can disable the prefetching of links, so that the
correlation process looks up each link in the database as it processes each
source or target object. You can disable the prefetching of links by adding
the <literal>prefetchLinks</literal> property to the mapping, and setting it
to <literal>false</literal>, for example:</para>
<programlisting language="javascript">
{
"mappings": [
{
"name": "systemMyLDAPAccounts_managedUser",
"prefetchLinks" : false
}
]
}</programlisting>
<para>Be aware that this setting will have a performance impact on the
reconciliation process.</para>
</section>
<section xml:id="parallel-recon-tasks">
<title>Parallel Reconciliation Threads</title>
<para>By default, reconciliation is executed in a multi-threaded manner,
that is, numerous threads are dedicated to the same reconciliation run.
Multithreading generally improves reconciliation run performance. The
default number of threads for a single reconciliation run is ten (plus the
main reconciliation thread). Under normal circumstances, you should not
need to change this number, however the default might not be appropriate
in the following situations:</para>
<itemizedlist>
<listitem>
<para>The hardware has many cores and supports more concurrent threads.
As a rule of thumb for performance tuning, start with setting the
thread number to two times the number of cores.</para>
</listitem>
<listitem>
<para>The source or target is an external system with high latency or
slow response times. Threads may then spend considerable time waiting
for a response from the external system. Increasing the available
threads enables the system to prepare or continue with additional
objects.</para>
</listitem>
</itemizedlist>
<para>To change the number of threads, set the <literal>taskThreads</literal>
<programlisting language="javascript">
"mappings" : [
{
"name" : "systemXmlfileAccounts_managedUser",
"taskThreads" : 20
...
}
]
}</programlisting>
<para>A value of <literal>0</literal> specifies that reconciliation is run
on the main reconciliation thread, that is, in a serial manner.</para>
</section>
</section>
<section xml:id="correlation">
<title>Correlation Queries</title>
<indexterm>
<primary>Synchronization</primary>
<secondary>Correlation queries</secondary>
</indexterm>
<indexterm>
<primary>Correlation queries</primary>
</indexterm>
<para>Every time OpenIDM creates an object through synchronization, it
creates a link between the source and target objects. OpenIDM then uses the
link to determine the object's situation during later synchronization
operations.</para>
<para>Initial, bulk synchronization operations can involve correlating
many objects that exist both on source and target systems. In this case,
OpenIDM uses correlation queries to find target objects that already exist,
and that correspond to source objects. For the target objects that match
a correlation query, OpenIDM needs only to create a link, rather than a
new target object.</para>
<para>Correlation queries run against target resources. The query syntax
therefore depends on the target system, and is either specific to the data
store underlying the OpenIDM repository, or to OpenICF query capabilities.
</para>
<section xml:id="correlation-query-managed-object">
<title>Managed Objects as Correlation Query Targets</title>
<para>Queries on managed objects in the repository must be defined in the
configuration file for the repository, which is either
<para>The following example shows a correlation query defined in
<programlisting language="javascript">
"for-userName" : "SELECT * FROM ${unquoted:_resource} WHERE userName = ${uid}"
</programlisting>
<para>By default, a <literal>${value}</literal> token replacement is
assumed to be a quoted string. If the value is not a quoted string, use the
<literal>unquoted:</literal> prefix, as shown above.</para>
<para>The following correlation query example shows the JavaScript to call
the query defined for OrientDB. The <literal>_queryId</literal> property
value matches the name of the query specified in
replaces <literal>${uid}</literal> in the query. OpenIDM replaces
<literal>${unquoted:_resource}</literal> in the query with the name of the
table that holds managed objects.</para>
<programlisting language="javascript">
{
"correlationQuery": {
"type": "text/javascript",
"source":
"var query = {'_queryId' : 'for-userName', 'uid' : source.name}; query;"
}
}</programlisting>
<para>The query can return zero or more objects, so the situation OpenIDM
assigns to the source object depends on the number of target objects
returned.</para>
<para>With a JDBC-based repository, the query defined in
to how the tables are indexed. The correlation query you define in
</section>
<section xml:id="correlation-query-system-object">
<title>System Objects as Correlation Query Targets</title>
<para>
Correlation queries on system objects access the connector. The connector
then executes the query on the external resource.
</para>
<para>
Your correlation query JavaScript must return a map that holds a generic
query with the following elements:
</para>
<itemizedlist>
<listitem>
<para>
A condition, such as "Equals"
</para>
</listitem>
<listitem>
<para>
The naming attribute to compare on the system object. In the example that
follows, the naming attribute is <literal>uid</literal>.
</para>
</listitem>
<listitem>
<para>
The value from the source object to use in the search filter. You set this
as the value of the <literal>value</literal> property, which takes an
array. In the example that follows, the value to use in the search filter
</para>
</listitem>
</itemizedlist>
<programlisting language="javascript">
var map = {"_queryFilter": 'uid eq "' + source.userName + '"'};
map;
</programlisting>
</section>
</section>
<section xml:id="advanced-dataflow">
<title>Advanced Data Flow Configuration</title>
<indexterm>
<primary>Mappings</primary>
<secondary>Hooks for scripting</secondary>
</indexterm>
<para><xref linkend="basic-flow"/> shows how to trigger scripts when objects
are created and updated. Other situations require you to trigger scripts
in response to other synchronization actions. For example, you might not
want OpenIDM to delete a managed user directly when an external account is
deleted, but instead unlink the objects and deactivate the user in another
resource. (Alternatively, you might delete the object in OpenIDM but
nevertheless execute a script.) The following example shows a more advanced
mapping configuration.</para>
<programlisting linenumbering="numbered" language="javascript">{
"mappings": [
{
"name": "systemLdapAccount_managedUser",
"validSource": {
"type": "text/javascript",
"file": "script/isValid.js"
},
"correlationQuery" : {
"type" : "text/javascript",
"source" : "var map = {'_queryFilter': 'uid eq \"' +
source.userName + '\"'}; map;"
},
"properties": [
{
"source": "uid",
"transform": {
"type": "text/javascript",
"source": "source.toLowerCase()"
},
"target": "userName"
},
{
"source": "",
"transform": {
"type": "text/javascript",
"source": "if (source.myGivenName)
{source.myGivenName;} else {source.givenName;}"
},
"target": "givenName"
},
{
"source": "",
"transform": {
"type": "text/javascript",
"source": "if (source.mySn)
{source.mySn;} else {source.sn;}"
},
"target": "familyName"
},
{
"source": "cn",
"target": "fullname"
},
{
"comment": "Multi-valued in LDAP, single-valued in AD.
Retrieve first non-empty value.",
"source": "title",
"transform": {
"type": "text/javascript",
"file": "script/getFirstNonEmpty.js"
},
"target": "title"
},
{
"condition": {
"type": "text/javascript",
"source": "var clearObj = openidm.decrypt(object);
((clearObj.password != null) &&
},
"transform": {
"type": "text/javascript",
"source": "source.password"
},
"target": "__PASSWORD__"
}
],
"onCreate": {
"type": "text/javascript",
"source": "target.ldapPassword = null;
target.adPassword = null;
target.password = null;
target.ldapStatus = 'New Account'"
},
"onUpdate": {
"type": "text/javascript",
"source": "target.ldapStatus = 'OLD'"
},
"onUnlink": {
"type": "text/javascript",
"file": "script/triggerAdDisable.js"
},
"policies": [
{
"situation": "CONFIRMED",
"action": "UPDATE"
},
{
"situation": "FOUND",
"action": "UPDATE"
},
{
"situation": "ABSENT",
"action": "CREATE"
},
{
"situation": "AMBIGUOUS",
"action": "EXCEPTION"
},
{
"situation": "MISSING",
"action": "EXCEPTION"
},
{
"situation": "UNQUALIFIED",
"action": "UNLINK"
},
{
"situation": "UNASSIGNED",
"action": "EXCEPTION"
}
]
}
]
}</programlisting>
<variablelist>
<para>The following list shows all the properties that you can use as hooks
in mapping configurations to call scripts.</para>
<varlistentry>
<term>Triggered by Situation</term>
<!-- All related to the "Supported script hooks" section of
<listitem>
<para>onCreate, onRead, onUpdate, onDelete, onValidate, onRetrieve,
onStore, onLink, onUnlink, postCreate, postUpdate, postDelete</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Object Filter</term>
<listitem>
<para>vaildSource, validTarget</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Correlating Objects</term>
<listitem>
<para>correlationQuery</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Triggered on Reconciliation</term>
<listitem>
<para>result</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Scripts Inside Properties</term>
<listitem>
<para>condition, transform</para>
</listitem>
</varlistentry>
</variablelist>
<para>Your scripts can get data from any connected system at any time by
<literal>id</literal> is the identifier of the object to read.</para>
<para>The following example reads a managed user object from the
repository.</para>
<programlisting language="javascript">
<para>The following example reads an account from an external LDAP
resource.</para>
<programlisting language="javascript">
externalAccount = openidm.read("system/ldap/account/uid=ddoe,ou=People,dc=example,dc=com");</programlisting>
<para>Note that the query targets a DN rather than a UID, as it did in the
previous example. The attribute that is used for the <literal>_id</literal>
is defined in the connector configuration file and, in this example, is set
to <literal>"uidAttribute" : "dn"</literal>. Although it is possible to use
a DN (or any unique attribute) for the <literal>_id</literal>, as a best
practice, you should use an attribute that is both unique and immutable.
</para>
</section>
<section xml:id="scheduling-synchronization">
<title>Scheduling Synchronization</title>
<indexterm>
<primary>Scheduler</primary>
</indexterm>
<indexterm>
<primary>Reconciliation</primary>
<secondary>Scheduling</secondary>
</indexterm>
<indexterm>
<primary>Synchronization</primary>
<secondary>Scheduling</secondary>
</indexterm>
<indexterm>
<primary>LiveSync</primary>
<secondary>Scheduling</secondary>
</indexterm>
<para>You can schedule synchronization operations, such as LiveSync and
reconciliation, using <command>cron</command>-like syntax.</para>
<para>This section describes scheduling for reconciliation and LiveSync,
however, you can also use OpenIDM's scheduler service to schedule any other
event by supplying a link to a script file, in which that event is defined.
For information about scheduling other events, and for a deeper understanding
of the OpenIDM scheduler service, see
<link xlink:href="integrators-guide#chap-scheduler-conf"
and Events</citetitle></link>.</para>
<section xml:id="configuring-sync-schedule">
<title>Configuring Scheduled Synchronization</title>
<para>
Each scheduled reconciliation and LiveSync task requires a schedule
files are named
.json</filename>, where <replaceable>schedule-name</replaceable> is a
logical name for the scheduled synchronization operation, such as
<literal>reconcile_systemXmlAccounts_managedUser</literal>.
</para>
<para>
Schedule configuration files have the following format:
</para>
<programlisting language="javascript">
{
"enabled" : true,
"persisted" : false,
"type" : "cron",
"startTime" : "<replaceable>(optional) time</replaceable>",
"endTime" : "<replaceable>(optional) time</replaceable>",
"schedule" : "<replaceable>cron expression</replaceable>",
"misfirePolicy" : "<replaceable>optional, string</replaceable>",
"timeZone" : "<replaceable>(optional) time zone</replaceable>",
"invokeService" : "<replaceable>service identifier</replaceable>",
"invokeContext" : "<replaceable>service specific context info</replaceable>"
}
</programlisting>
<para>
For an explanation of each of these properties, see
<link xlink:href="integrators-guide#chap-scheduler-conf"
and Events</citetitle></link>.
</para>
<para>
To schedule a reconciliation or LiveSync task, set the
<literal>invokeService</literal> property to either
<literal>"sync"</literal> (for reconciliation) or
<literal>"provisioner"</literal> for LiveSync.
</para>
<para>
The value of the <literal>invokeContext</literal> property depends on the
type of scheduled event. For reconciliation, the properties are set as
follows:
</para>
<programlisting language="javascript">
{
"invokeService": "sync",
"invokeContext": {
"action": "reconcile",
"mapping": "systemLdapAccount_managedUser"
}
}
</programlisting>
<para>
The <literal>"mapping"</literal> is either referenced by its name in the
the <literal>"mapping"</literal> property, as shown in the example in
<link xlink:href="integrators-guide#alternative-mapping"
Mappings</citetitle></link>.
</para>
<para>
For LiveSync, the properties are set as follows:
</para>
<programlisting language="javascript">
{
"invokeService": "provisioner",
"invokeContext": {
"action": "liveSync",
}
}
</programlisting>
<para>
The <literal>"source"</literal> property follows OpenIDM's convention for a
pointer to an external resource object and takes the form
</para>
</section>
<section xml:id="alternative-mapping">
<title>Alternative Mappings</title>
<indexterm>
<primary>Mappings</primary>
<secondary>Scheduled reconciliation</secondary>
</indexterm>
<para>
Mappings for synchronization are usually stored in
and for pushing changes made to managed objects to external resources. You
can, however, provide alternative mappings for scheduled reconciliation by
adding the mapping to the schedule configuration instead of referencing a
</para>
<programlisting language="javascript">
{
"enabled": true,
"type": "cron",
"schedule": "0 08 16 * * ?",
"invokeService": "sync",
"invokeContext": {
"action": "reconcile",
<emphasis role="bold">"mapping": {
"name": "CSV_XML",
"properties": [
{
"source": "firstname",
"target": "firstname"
},
...
],
"policies": [...]
}</emphasis>
}
}
</programlisting>
</section>
</section>
</chapter>