2N/A<?xml version="1.0" encoding="UTF-8"?>

<chapter xml:id='chap-synchronization'

xmlns='http://docbook.org/ns/docbook'

version='5.0' xml:lang='en'

xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'

xsi:schemaLocation='http://docbook.org/ns/docbook http://docbook.org/xml/5.0/xsd/docbook.xsd'

xmlns:xlink='http://www.w3.org/1999/xlink'

xmlns:xinclude='http://www.w3.org/2001/XInclude'>

<title>Configuring Synchronization</title>

<indexterm>

<primary>Synchronization</primary>

</indexterm>

<para>One of the core services of OpenIDM is synchronizing identity data from

different resources. This chapter explains what you must know to get started

configuring OpenIDM's flexible synchronization mechanism, and illustrates the

concepts with examples.</para>

<section xml:id="sync-types">

<title>Types of Synchronization</title>

<indexterm>

<primary>Synchronization</primary>

<secondary>Direct (push)</secondary>

</indexterm>

<para>Synchronization happens either when OpenIDM receives a change directly,

or when OpenIDM discovers a change on an external resource.</para>

<para>For direct changes to OpenIDM, OpenIDM immediately pushes updates to

all external resources configured to receive the updates. A direct change

can originate not only as a write request through the REST interface, but

also as an update resulting from reconciliation with another resource.</para>

<variablelist>

<para>OpenIDM discovers changes on external resources through reconciliation,

and through LiveSync.</para>

<varlistentry>

<term>Reconciliation</term>

<listitem>

<indexterm>

<primary>Reconciliation</primary>

</indexterm>

<para>In identity management, <firstterm>reconciliation</firstterm> is the

process of bidirectional synchronization of objects between different data

stores. Reconciliation applies mainly to user objects, although OpenIDM

can reconcile any objects, including groups and roles.</para>

<para>To perform reconciliation, OpenIDM analyzes both source and target

systems to uncover the differences that it must reconcile. Reconciliation

can therefore be a heavyweight process. When working with large data sets,

finding all changes can be more work than processing the changes.</para>

<para>Reconciliation is, however, thorough. It recognizes system error

conditions and catches changes that might be missed by the more

lightweight LiveSync mechanism. Reconciliation therefore serves as the

basis for compliance and reporting functionality.</para>

</listitem>

</varlistentry>

<varlistentry>

<term>LiveSync</term>

<listitem>

<indexterm>

<primary>LiveSync</primary>

</indexterm>

<para><firstterm>LiveSync</firstterm> performs the same job as

reconciliation. LiveSync relies on a change log on the external resource

to determine which objects have changed.</para>

<para>LiveSync is intended to react quickly to changes as they happen.

LiveSync is however a best effort mechanism that, in some cases, can miss

changes.</para>

<para>Furthermore, not all resources provide the change log mechanism

that LiveSync requires. The change log provides OpenIDM with a list of

objects that have changed since the last request, so that OpenIDM does not

need to scan all objects for changes. OpenDJ and Active Directory both

provide the change log required for LiveSync.</para>

</listitem>

</varlistentry>

</variablelist>

<para>To determine what to synchronize, and how to carry out synchronization,

OpenIDM relies on mappings that you configure. LiveSync relies on the set of

mappings that you configure once per OpenIDM server. Reconciliation allows

you to configure specific mappings for a particular reconciliation job.</para>

<para>You must trigger OpenIDM to poll for changes on external resources,

usually by scheduling reconciliation or LiveSync, as described in

<link xlink:href="integrators-guide#chap-scheduler-conf"

xlink:role="http://docbook.org/xlink/role/olink"><citetitle>Scheduling

Tasks and Events</citetitle></link>. Alternatively, you can manage

reconciliations over the REST interface, as described in the following

section.</para>

</section>

<section xml:id="recon-over-rest">

<title>Managing Reconciliation Over REST</title>

<para>You can trigger, cancel, and monitor reconciliation operations over

REST, using the REST endpoint

<literal>http://localhost:8080/openidm/recon</literal>.</para>

<section xml:id="triggering-recons">

<title>Triggering a Reconciliation Run</title>

<para>The following example triggers a reconciliation operation based on

the <literal>systemLdapAccounts_managedUser</literal> mapping. The mapping

is defined in the file <filename>conf/sync.json</filename>.</para>

<screen width="91"><?dbfo pgwide="1"?>$ curl

--header "X-OpenIDM-Username: openidm-admin"

--header "X-OpenIDM-Password: openidm-admin"

--request POST

"http://localhost:8080/openidm/recon?_action=recon&amp;mapping=systemLdapAccounts_managedUser"

</screen>

<para>By default, an assigned reconciliation run ID is returned immediately

when the reconciliation operation is initiated. Clients can make subsequent

calls to the reconciliation service, using this reconciliation run ID to

query its state and to call operations on it.</para>

<para>For example, the reconciliation run initiated previously would return

something similar to the following:</para>

<screen>{"_id":"0890ad62-4738-4a3f-8b8e-f3c83bbf212e"}</screen>

<para>To have the entire reconciliation run complete before the

reconciliation run ID is returned, set the

<literal>waitForCompletion</literal> property to <literal>true</literal>

when the reconciliation is initiated. For example:</para>

<screen width="91"><?dbfo pgwide="1"?>$ curl

--header "X-OpenIDM-Username: openidm-admin"

--header "X-OpenIDM-Password: openidm-admin"

--request POST

"http://localhost:8080/openidm/recon?_action=recon&amp;

mapping=systemLdapAccounts_managedUser&amp;waitForCompletion=true"

</screen>

</section>

<section xml:id="canceling-recons">

<title>Canceling a Reconciliation Run</title>

<para>You can cancel a reconciliation run by sending a REST call with

the <literal>cancel</literal> action, specifying the reconciliation run

ID. For example, the following call cancels the reconciliation run

initiated in the previous section:</para>

<screen width="91"><?dbfo pgwide="1"?>$curl

--header "X-OpenIDM-Username: openidm-admin"

--header "X-OpenIDM-Password: openidm-admin"

--request POST

"http://localhost:8080/openidm/recon/0890ad62-4738-4a3f-8b8e-f3c83bbf212e?_action=cancel"

</screen>

<para>The output for a reconciliation cancelation request is similar to the

following, but on a single line:</para>

<screen width="91">

{"_id":"0890ad62-4738-4a3f-8b8e-f3c83bbf212e",

"action":"cancel",

"status":"SUCCESS"}

</screen>

<para>If you specified that the call should wait for completion before

the ID is returned, you can obtain the reconciliation run ID from the list

of active reconciliations, as described in the following section.</para>

</section>

<section xml:id="listing-recons">

<title>Listing Reconciliation Runs</title>

<para>You can display a list of reconciliation processes that have

completed, and those that are in progress, by running a RESTful GET on

<literal>"http://localhost:8080/openidm/recon"</literal>. The following

example displays all reconciliation runs.</para>

<screen>$curl

--header "X-OpenIDM-Username: openidm-admin"

--header "X-OpenIDM-Password: openidm-admin"

--request GET

"http://localhost:8080/openidm/recon"

</screen>

<para>The output of such a request is similar to the following, with one

item for each reconciliation run. The output appears on a single line, but

has been indented here, for legibility.</para>

<screen width="91">{"reconciliations":[

{"_id":"d3040cc9-ec2e-41b8-86c4-72393087a626",

"mapping":"systemLdapAccounts_managedUser",

"state":"SUCCESS",

"stage":"COMPLETED_SUCCESS",

"stageDescription":"reconciliation completed.",

"progress":{

"progress":{

"source":{

"existing":{

"processed":1001,

"total":"1001"

}

},

"target":{

"existing":{

"processed":1001,

"total":"1001"

},

"created":0

},

"links":{

"existing":{

"processed":1001,

"total":"1001"

},

"created":0

}

}

},

"started":"2012-10-10T13:02:36.624+00:00",

"ended":"2012-10-10T13:02:54.960+00:00"},

</screen>

<variablelist>

<para>Each reconciliation run has the following properties:</para>

<varlistentry>

<term><literal>_id</literal></term>

<listitem>

<para>The ID of the reconciliation run.</para>

</listitem>

</varlistentry>

<varlistentry>

<term><literal>mapping</literal></term>

<listitem>

<para>The name of the mapping, defined in the <filename>conf/sync.json

</filename> file.</para>

</listitem>

</varlistentry>

<varlistentry>

<term><literal>state</literal></term>

<listitem>

<para>The high level state of the reconciliation run. Values can be as

follows:</para>

<itemizedlist>

<listitem>

<para><literal>ACTIVE</literal></para>

<para>The reconciliation run is in progress.</para>

</listitem>

<listitem>

<para><literal>CANCELED</literal></para>

<para>The reconciliation run was successfully canceled.</para>

</listitem>

<listitem>

<para><literal>FAILED</literal></para>

<para>The reconciliation run was terminated because of failure.</para>

</listitem>

<listitem>

<para><literal>SUCCESS</literal></para>

<para>The reconciliation run completed successfully.</para>

</listitem>

</itemizedlist>

</listitem>

</varlistentry>

<varlistentry>

<term><literal>stage</literal></term>

<listitem>

<para>The current stage of the reconciliation run's progress. Values can

be as follows:</para>

<itemizedlist>

<listitem>

<para><literal>ACTIVE_INITIALIZED</literal></para>

<para>The initial stage, when a reconciliation run is first created.

</para>

</listitem>

<listitem>

<para><literal>ACTIVE_QUERY_ENTRIES</literal></para>

<para>Querying the source, target and possibly link sets to reconcile.

</para>

</listitem>

<listitem>

<para><literal>ACTIVE_RECONCILING_SOURCE</literal></para>

<para>Reconciling the set of IDs retrieved from the mapping source.

</para>

</listitem>

<listitem>

<para><literal>ACTIVE_RECONCILING_TARGET</literal></para>

<para>Reconciling any remaining entries from the set of IDs retrieved

from the mapping target, that were not matched or processed during

the source phase.</para>

</listitem>

<listitem>

<para><literal>ACTIVE_LINK_CLEANUP</literal></para>

<para>Checking whether any links are now unused and should be cleaned

up.</para>

</listitem>

<listitem>

<para><literal>ACTIVE_PROCESSING_RESULTS</literal></para>

<para>Post-processing of reconciliation results.</para>

</listitem>

<listitem>

<para><literal>ACTIVE_CANCELING</literal></para>

<para>Attempting to abort a reconciliation run in progress.</para>

</listitem>

<listitem>

<para><literal>COMPLETED_SUCCESS</literal></para>

<para>Successfully completed processing the reconciliation run.</para>

</listitem>

<listitem>

<para><literal>COMPLETED_CANCELED</literal></para>

<para>Completed processing because the reconciliation run was aborted.

</para>

</listitem>

<listitem>

<para><literal>COMPLETED_FAILED</literal></para>

<para>Completed processing because of a failure.</para>

</listitem>

</itemizedlist>

</listitem>

</varlistentry>

<varlistentry>

<term><literal>stageDescription</literal></term>

<listitem>

<para>A description of the stages described previously.</para>

</listitem>

</varlistentry>

<varlistentry>

<term><literal>progress</literal></term>

<listitem>

<para>The progress object has the following structure (anotated here

with comments):</para>

<screen width="96">

"progress":{

"progress":{

"source":{ // Progress on the set of existing entries in the mapping source

"existing":{

"processed":1001,

"total":"1001" // Total number of entries in source set, if known, “?” otherwise

}

},

"target":{ // Progress on the set of existing entries in the mapping target

"existing":{

"processed":1001,

"total":"1001" // Total number of entries in target set, if known, “?” otherwise

},

"created":0 // New entries that were created

},

"links":{ // Progress on the set of existing links between source and target

"existing":{

"processed":1001,

"total":"1001" // Total number of existing links, if known, “?” otherwise

},

"created":0 // Denotes new links that were created

}

}

},

</screen>

</listitem>

</varlistentry>

</variablelist>

</section>

</section>

<section xml:id="sync-flexible-data">

<title>Flexible Data Model</title>

<indexterm>

<primary>Objects</primary>

<secondary>Managed objects</secondary>

</indexterm>

<para>Identity management software tends to favor either a meta-directory

data model, where all data are mirrored in a central repository, or a

virtual data model, where only a minimum set of attributes are stored

centrally, and most are loaded on demand from the external resources

in which they are stored. The meta-directory model offers fast access at

the risk of getting out-of-date data. The virtual model guarantees fresh

data, but pays for that guarantee in terms of performance.</para>

<para>OpenIDM leaves the data model choice up to you. You determine the right

trade offs for a particular deployment. OpenIDM does not hard code any

particular schema or set of attributes stored in the repository. Instead,

you define how external system objects map onto managed objects, and OpenIDM

dynamically updates the repository to store the managed object attributes

that you configure.</para>

<para>You can, for example, choose to follow the data model defined in the

Simple Cloud Identity Management (<link xlink:show="new"

xlink:href="http://www.simplecloud.info/specs/draft-scim-core-schema-00.html"

>SCIM</link>) specification. The following object represents a SCIM

user.</para>

<programlisting language="javascript">

{

"userName": "james1",

"familyName": "Berg",

"givenName": "James",

"email": [

"james1@example.com"

],

"description": "Created by OpenIDM REST.",

"password": "asdfkj23",

"displayName": "James Berg",

"phoneNumber": "12345",

"employeeNumber": "12345",

"userType": "Contractor",

"title": "Vice President",

"active": true

}</programlisting>

<note>

<para>Avoid using the dash character ( <literal>-</literal> ) in property

names, like <literal>last-name</literal>, as dashes in names make

JavaScript syntax more complex. If you cannot avoid the dash, then write

<literal>source['last-name']</literal> instead of

<literal>source.last-name</literal> in java script.</para>

</note>

</section>

<section xml:id="basic-flow">

<title>Basic Data Flow Configuration</title>

<para>Data flow for synchronization involves the following elements:

</para>

<itemizedlist>

<listitem><para>Three types of configuration files.</para></listitem>

<listitem><para>A links table that OpenIDM maintains in its repository.</para></listitem>

<listitem><para>The scripts required to check objects and manipulate attributes.</para></listitem>

</itemizedlist>

<para>Typically, you only edit two of the three types of configuration files

- connector configuration files, with one file per external resource, and the

synchronization mappings file (<filename>conf/sync.json</filename>), with one

file per OpenIDM instance.</para>

<variablelist>

<varlistentry>

<term>Connector configuration files</term>

<listitem>

<indexterm>

<primary>Synchronization</primary>

<secondary>Connectors</secondary>

</indexterm>

<para>Connector configuration files map external resource objects to OpenIDM

objects, and are described in detail in the chapter on <link

xlink:href="integrators-guide#chap-resource-conf"

xlink:role="http://docbook.org/xlink/role/olink"><citetitle>Connecting to

External Resources</citetitle></link>. Connector configuration files are

named <filename>openidm/conf/provisioner.<replaceable

>resource-name</replaceable>.json</filename>,

where <replaceable>resource-name</replaceable> reflects the connector

technology and external resource, such as

<literal>openicf-xml</literal>.</para>

<para>An excerpt from an example connector configuration follows. The

example shows the name for the connector and two attributes of an account

object type. In the attribute mapping definitions, the attribute name is

mapped from the <literal>nativeName</literal>, the attribute name used on

the external resource, to the attribute name used in OpenIDM. Thus the

example shows that the <literal>sn</literal> attribute in LDAP is mapped to

<literal>lastName</literal> in OpenIDM. The <literal>homePhone</literal>

attribute can have multiple values.</para>

<programlisting language="javascript">

{

"name": "MyLDAP",

"objectTypes": {

"account": {

"lastName": {

"type": "string",

"required": true,

"nativeName": "sn",

"nativeType": "string"

},

"homePhone": {

"type": "array",

"items": {

"type": "string",

"nativeType": "string"

},

"nativeName": "homePhone",

"nativeType": "string"

}

}

}

}</programlisting>

<para>In order for OpenIDM to access external resource objects and

attributes, the object and its attributes must match the connector

configuration. Note that the connector file only maps external resource

objects to OpenIDM objects. To construct attributes and to manipulate

their values, you use the synchronization mappings file.</para>

</listitem>

</varlistentry>

<varlistentry>

<term>Synchronization mappings file</term>

<listitem>

<indexterm>

<primary>Synchronization</primary>

<secondary>Mappings</secondary>

</indexterm>

<indexterm>

<primary>Mappings</primary>

</indexterm>

<para>The synchronization mappings file is

<filename>openidm/conf/sync.json</filename>. Synchronization mappings

represent the core configuration for OpenIDM synchronization.</para>

<para>The <filename>sync.json</filename> file describes a set of mappings.

Each mapping specifies how attributes from source objects correspond to

attributes on target objects. The source and target indicate the direction

for the data flow, so you must define a separate mapping for each data

flow. For example, if you want data flows from an LDAP server to the

repository and also from the repository to the LDAP server, you must

define two separate mappings.</para>

<para>You identify external resource sources and targets as

<literal>system/<replaceable>name</replaceable>/<replaceable

>object-type</replaceable></literal>, where

<replaceable>name</replaceable> is the name used in the connector

configuration file, and <replaceable>object-type</replaceable> is the

object defined in the connector configuration file list of object types.

For objects in OpenIDM's internal repository, you use

<literal>managed/<replaceable>object-type</replaceable></literal>, where

<replaceable>object-type</replaceable> is defined in

<filename>openidm/conf/managed.json</filename>. The name for the mapping

by convention is set to a string of the form

<literal><replaceable>source</replaceable>_<replaceable

>target</replaceable></literal>, as shown in the following example.</para>

<programlisting language="javascript" xml:id="basic-ldap-mapping">

{

"mappings": [

{

"name": "systemLdapAccounts_managedUser",

"source": "system/MyLDAP/account",

"target": "managed/user",

"properties": [

{

"target": "familyName",

"source": "lastName"

},

{

"target": "homePhone",

"source": "homePhone"

},

{

"target": "phoneExtension",

"default": "0047"

},

{

"target": "mail",

"comment": "Set mail if non-empty.",

"source": "email",

"condition": {

"type": "text/javascript",

"source": "(source.email != null)"

}

},

{

"target": "displayName",

"source": "";

"transform": {

"type": "text/javascript",

"source": "(source.lastName +', ' + source.firstName;)"

}

}

]

}

]

}</programlisting>

<para>In this example, the source is the external resource,

<literal>MyLDAP</literal>, and the target is OpenIDM's repository,

specifically the managed user objects. The <literal>properties</literal>

reflect OpenIDM attribute names. For example, the mapping has the

attribute <literal>lastName</literal> defined in the

<literal>MyLDAP</literal> connector configuration file mapped to

<literal>familyName</literal> in the OpenIDM managed user object. Notice

that the attribute names come from the connector configuration, rather

than the external resource itself.</para>

<indexterm>

<primary>Synchronization</primary>

<secondary>Creating attributes</secondary>

</indexterm>

<para>You can create attributes on the target as part of the

mapping. In the example, OpenIDM creates a

<literal>phoneExtension</literal> attribute with a default value of

<literal>0047</literal>.</para>

<indexterm>

<primary>Synchronization</primary>

<secondary>Conditions</secondary>

</indexterm>

<para>You can also set up conditions under which OpenIDM maps attributes

as shown for the email attribute in the example. By default, OpenIDM

synchronizes all attributes. In the example, the mail attribute is set

only if the script for the condition returns

<literal>true</literal>.</para>

<indexterm>

<primary>Synchronization</primary>

<secondary>Transforming attributes</secondary>

</indexterm>

<para>OpenIDM also enables you to transform attributes. In the example,

the value of the <literal>displayName</literal> attribute is set using a

combination of the <literal>lastName</literal> and

<literal>firstName</literal> attribute values from the source. For

transformations, the <literal>source</literal> property is optional.

However, the source object is only available when you specify the

<literal>source</literal> property. Therefore, in order to use

<literal>source.lastName</literal> and <literal>source.firstName</literal>

to calculate the <literal>displayName</literal>, the example specifies

<literal>"source" : ""</literal>.</para>

<para>To add a flow from the repository to <literal>MyLDAP</literal>,

you would define a mapping with source <literal>managed/user</literal>

and target <literal>system/MyLDAP/account</literal>, named for example

<literal>managedUser_systemLdapAccounts</literal>.</para>

<para>The following image shows the paths to objects in the OpenIDM

namespace.</para>

<mediaobject xml:id="figure-service-tree"><?dbfo pgwide="1"?>

<alt>OpenIDM namespace and object paths</alt>

<imageobject>

<imagedata fileref="images/ServiceTree.png" format="PNG" />

</imageobject>

<textobject>

<para>The image shows paths to objects in the OpenIDM namespace.</para>

</textobject>

<caption>

<para>OpenIDM stores managed objects in the repository, and exposes

them under <literal>/openidm/managed</literal>. System objects on

external resources are exposed under

<literal>/openidm/system</literal>.</para>

</caption>

</mediaobject>

<variablelist>

<indexterm>

<primary>Synchronization</primary>

<secondary>Filtering</secondary>

</indexterm>

<para>By default, OpenIDM synchronizes all objects that match those

defined in the connector configuration for the resource. Many connectors

allow you to limit the scope of objects that the connector accesses. For

example, the LDAP connector allows you to specify base DNs and LDAP

filters so that you do not need to access every entry in the directory.

OpenIDM also allows you to filter what is considered a valid source or

valid target for synchronization by using JavaScript code. To apply these

filters, use the <literal>validSource</literal>, and

<literal>validTarget</literal> properties in your mapping.</para>

<varlistentry>

<term>validSource</term>

<listitem>

<para>A script that determines if a source object is valid to be

mapped. The script yields a boolean value: <literal>true</literal>

indicates that the source object is valid; <literal>false</literal>

can be used to defer mapping until some condition is met. In the root

scope, the source object is provided in the <literal>"source"</literal>

property. If the script is not specified, then all source objects are

considered valid.</para>

<programlisting language="javascript">

{

"validSource": {

"type": "text/javascript",

"source": "source.ldapPassword != null"

}

}

</programlisting>

</listitem>

</varlistentry>

<varlistentry>

<term>validTarget</term>

<listitem>

<para>A script, used during reconciliation's second phase, that

determines if a target object is valid to be mapped. The script yields

a boolean value: <literal>true</literal> indicates that the target

object is valid; <literal>false</literal> indicates that the target

object should not be included in reconciliation. In the root scope, the

source object is provided in the <literal>"target"</literal> property.

If the script is not specified, then all target objects are considered

valid for mapping.</para>

<programlisting language="javascript">

{

"validTarget": {

"type": "text/javascript",

"source": "target.employeeType == 'internal'"

}

}

</programlisting>

</listitem>

</varlistentry>

</variablelist>

<para>During synchronization, your scripts always have access to a

<literal>source</literal> object and a <literal>target</literal> object.

Examples already shown in this section use <literal>source.<replaceable

>attributeName</replaceable></literal> to retrieve attributes from the

source objects. Your scripts can also write to target attributes using

<literal>target.<replaceable>attributeName</replaceable></literal>

syntax.</para>

<programlisting language="javascript">

{

"onUpdate": {

"type": "text/javascript",

"source": "if ((source.email != null) {target.mail = source.email;}"

}

}</programlisting>

<para>See the <link xlink:href="integrators-guide#appendix-scripting"

xlink:role="http://docbook.org/xlink/role/olink"><citetitle>Scripting

Reference</citetitle></link> appendix for more on scripting.</para>

</listitem>

</varlistentry>

</variablelist>

<section xml:id="sync-encrypted-values">

<title>Using Encrypted Values</title>

<indexterm>

<primary>Synchronization</primary>

<secondary>Encryption</secondary>

</indexterm>

<para>OpenIDM supports reversible encryption of attribute values for

managed objects. Attribute values to encrypt include passwords (if passwords

are not already encrypted on the external resource, which would usually

exclude them from the synchronization process, see the chapter about <link

xlink:href="integrators-guide#chap-passwords"

xlink:role="http://docbook.org/xlink/role/olink">

<citetitle>Passwords</citetitle></link>), and also authentication questions,

credit card numbers, and social security numbers.</para>

<para>You configure encryption in the managed object configuration. The file

to edit is <filename>openidm/conf/managed.json</filename>. The following

example shows a managed object configuration that encrypts and decrypts

<literal>securityAnswer</literal>, <literal>ssn</literal>, and

<literal>password</literal> attributes using the default symmetric

key, and additional scripts for extra passwords.</para>

<programlisting language="javascript">

{

"objects": [

{

"name": "user",

"properties": [

{

"name": "securityAnswer",

"encryption": {

"key": "openidm-sym-default"

}

},

{

"name": "ssn",

"encryption": {

"key": "openidm-sym-default"

}

},

{

"name": "password",

"encryption": {

"key": "openidm-sym-default"

}

}

],

"onStore": {

"type": "text/javascript",

"file": "script/encryptExtraPassword.js"

},

"onRetrieve": {

"type": "text/javascript",

"file": "script/decryptExtraPassword.js"

}

}

]

}</programlisting>

<para>Do not use the default symmetric key,

<literal>openidm-sym-default</literal>, in production. See the chapter on

<link xlink:href="integrators-guide#chap-security"

xlink:role="http://docbook.org/xlink/role/olink"><citetitle>Securing

&amp; Hardening OpenIDM</citetitle></link> for instructions on adding your

own symmetric key.</para>

</section>

<section xml:id="restricting-http-access">

<title>Restricting HTTP Access to Sensitive Data</title>

<para>You can protect specific sensitive data stored in the repository by

marking the corresponding properties as "private". Private data, whether it

is encrypted or not, is not accessible over the REST interface. Properties

that are marked as private are removed from an object when that object is

retrieved over REST.</para>

<para>To mark a property as private, set its <literal>"scope"</literal> to

<literal>"private"</literal> in the <filename>conf/managed.json</filename>

file.</para>

<para>The following extract of the <filename>managed.json</filename> file

shows how HTTP access is prevented on the <literal>password</literal> and

<literal>securityAnswer</literal> properties.</para>

<programlisting language="javascript">

"properties" : [

{

"name" : "securityAnswer",

"encryption" : {

"key" : "openidm-sym-default"

},

"scope" : "private"

},

{

"name" : "password",

"encryption" : {

"key" : "openidm-sym-default"

},

"scope" : "private"

</programlisting>

<para>A potential caveat with using private properties is that such

properties are <emphasis>removed</emphasis> if an object is updated by

using an HTTP <literal>PUT</literal> request. A <literal>PUT</literal>

request replaces the entire object in the repository. Because properties

that are marked as private are ignored in HTTP requests, these properties

are effectively removed from the object when the update is done. To work

around this limitation, do not use <literal>PUT</literal> requests if you

have configured private properties. Instead, use a <literal>PATCH</literal>

request to update only those properties that need to be changed.

</para>

<para>For example, to update the familyName of user joe, replace only the

familyName and not the entire user object, as follows:</para>

<screen width="100">$ curl

--header "X-OpenIDM-Username: openidm-admin"

--header "X-OpenIDM-Password: openidm-admin"

--header "Content-Type: application/json"

--request POST

--data '{

"replace":"familyName","value": "Brown"

}'

"http://localhost:8080/openidm/managed/user?_action=patch&amp;_query-id=for-userName&amp;uid=joe"

</screen>

<note>

<para>The filtering of private data applies only to direct HTTP read

and query calls on managed objects. No automatic filtering is done for

internal callers, and the data that these callers choose to expose.</para>

</note>

</section>

<section xml:id="constructing-attributes">

<title>Constructing &amp; Manipulating Attributes</title>

<indexterm>

<primary>Synchronization</primary>

<secondary>Creating attributes</secondary>

</indexterm>

<para>OpenIDM lets you construct and manipulate attributes using scripts

triggered when an object is created (onCreate), updated (onUpdate), or

deleted (onDelete), or when a link is created (onLink), or removed

(onUnlink).</para>

<para>The following example derives a DN for an LDAP entry when the entry

is created in the internal repository.</para>

<programlisting language="javascript">

{

"onCreate": {

"type": "text/javascript",

"source":

"target.dn = 'uid=' + source.uid + ',ou=people,dc=example,dc=com'"

}

}</programlisting>

</section>

<section xml:id="reusing-links">

<title>Reusing Links</title>

<indexterm>

<primary>Synchronization</primary>

<secondary>Reusing links</secondary>

</indexterm>

<para>When two mappings exist to sync the same objects bidirectionally,

you can use the <literal>links</literal> property in one mapping to have

OpenIDM use the same internally managed link for both mappings. Otherwise,

if no <literal>links</literal> property is specified, OpenIDM maintains a

link for each mapping.</para>

<para>The following excerpt shows two mappings, one from MyLDAP accounts

to managed users, and another from managed users to MyLDAP accounts. In

the second mapping, the <literal>link</literal> property tells OpenIDM

to reuse the links created in the first mapping, rather than create new

links.</para>

<programlisting language="javascript">

{

"mappings": [

{

"name": "systemMyLDAPAccounts_managedUser",

"source": "system/MyLDAP/account",

"target": "managed/user"

},

{

"name": "managedUser_systemMyLDAPAccounts",

"source": "managed/user",

"target": "system/MyLDAP/account",

"links": "systemMyLDAPAccounts_managedUser"

}

]

}</programlisting>

</section>

</section>

<section xml:id="handling-sync">

<title>Synchronization Situations &amp; Actions</title>

<indexterm>

<primary>Synchronization</primary>

<secondary>Situations</secondary>

</indexterm>

<indexterm>

<primary>Synchronization</primary>

<secondary>Actions</secondary>

</indexterm>

<para>During synchronization, OpenIDM categorizes objects by situation.

Situations are characterized by whether an object exists on a source or

target system, whether OpenIDM has registered a link between the source

object and the target object, and whether the object is considered valid.

OpenIDM takes action depending on the situation.</para>

<para>You can define actions for particular situations in the

<literal>policies</literal> section of a synchronization mapping, as

shown in the following excerpt.</para>

<programlisting language="javascript">

{

"policies": [

{

"situation": "CONFIRMED",

"action": "UPDATE"

},

{

"situation": "FOUND",

"action": "IGNORE"

},

{

"situation": "ABSENT",

"action": "CREATE"

},

{

"situation": "AMBIGUOUS",

"action": "IGNORE"

},

{

"situation": "MISSING",

"action": "IGNORE"

},

{

"situation": "UNQUALIFIED",

"action": "IGNORE"

},

{

"situation": "UNASSIGNED",

"action": "IGNORE"

}

]

}</programlisting>

<para>If you do not define a policy for a particular situation, OpenIDM

takes the default action for the situation.</para>

<para>The situations and their corresponding actions are described in the

sections below.</para>

<section xml:id="sync-situations">

<title>Synchronization Situations</title>

<para>OpenIDM performs synchronization action in two phases. First, OpenIDM

performs the source reconciliation, where OpenIDM accounts for source

objects and associated links based on the mapping configured. Second, OpenIDM

runs the target reconciliation, where OpenIDM iterates over the target objects

not processed in the first phase.</para>

<orderedlist>

<para>During reconciliation OpenIDM builds three lists, assigning values

to the objects to reconcile.</para>

<listitem>

<para>All valid objects from the source</para>

<para>OpenIDM assigns valid source objects <literal>qualifies=1</literal>.

Invalid objects, including those not found in the source system, and those

filtered out by the script specified in the <literal>validSource</literal>

property are assigned <literal>qualifies=0</literal>.</para>

</listitem>

<listitem>

<para>All records from the appropriate link table</para>

<para>Objects with corresponding links in the link table of the repository

get <literal>link=1</literal>. Objects without corresponding links get

<literal>link=0</literal>.</para>

</listitem>

<listitem>

<para>All valid objects on the target system</para>

<para>Object found in the target system get <literal>target=1</literal>.

Objects not found in the target system get

<literal>target=0</literal>.</para>

</listitem>

</orderedlist>

<variablelist>

<para>Based on the values assigned to objects during source reconciliation,

OpenIDM assigns situations, listed here with their default actions.</para>

<varlistentry>

<term>"CONFIRMED" (qualifies=1, link=1, target=1)</term>

<listitem>

<para>The mapping qualifies for a target object, and a link to an

existing target object was found. Detected during change events and

reconciliation. Default action: "UPDATE".</para>

</listitem>

</varlistentry>

<varlistentry>

<term>"FOUND" (qualifies=1, link=0, target=1)</term>

<listitem>

<para>The mapping qualifies for a target object, there is no link to a

target object, and there is a single target object, correlated by the

logic found in the correlationQuery. Detected during change events and

reconciliation. Default action: "UPDATE".</para>

</listitem>

</varlistentry>

<varlistentry>

<term>"ABSENT" (qualifies=1, link=0, target=0)</term>

<listitem>

<para>The mapping qualifies for a target object, there is no link to a

target object, and there is no correlated target object found.

Detected during change events and reconciliation. Default action:

"CREATE".</para>

</listitem>

</varlistentry>

<varlistentry>

<term>"AMBIGUOUS" (qualifies=1, link=0, target&gt;1)</term>

<listitem>

<para>The mapping qualifies for a target object, there is no link to a

target object, but there is more than one correlated target object.

Detected during source object changes and reconciliation. Default

action: "EXCEPTION".</para>

</listitem>

</varlistentry>

<varlistentry>

<term>"MISSING" (qualifies=1, link=1, target=0)</term>

<listitem>

<para>The mapping is qualified for a target object, and there is a

qualified link to a target object, but the target object is missing.

Only detected during reconciliation and source object changes in

synchronous mappings. Default action: "EXCEPTION".</para>

</listitem>

</varlistentry>

<varlistentry>

<term>"UNQUALIFIED" (qualifies=0, link=0 or 1, target=1 or &gt;1)</term>

<listitem>

<para>The mapping is not qualified for a source object. One or more

targets are found through the correlation logic. Detected during change

events and reconciliation. Default action: "DELETE".</para>

</listitem>

</varlistentry>

<varlistentry>

<term>"SOURCE_IGNORED" (qualifies=0, link=0, target=0)</term>

<listitem>

<para>The source object is unqualified (by validSource), no link, no

target is found. Detected during source object changes and

reconciliation. Default action: "IGNORE".</para>

</listitem>

</varlistentry>

</variablelist>

<para>Based on the values assigned to objects during target reconciliation,

OpenIDM assigns situations, listed here with their default actions.</para>

<variablelist>

<varlistentry>

<term>"TARGET_IGNORED" (qualifies=0)</term>

<listitem>

<para>During target reconciliation the target becomes unqualified by the

"validTarget" script. Only detected during reconciliation. Default

action: "IGNORE"</para>

</listitem>

</varlistentry>

<varlistentry>

<term>"UNASSIGNED" (qualifies=1, link=0)</term>

<listitem>

<para>A target object exists for which there is no link. Only detected

during reconciliation. Default action: "EXCEPTION".</para>

</listitem>

</varlistentry>

<varlistentry>

<term>"CONFIRMED" (qualifies=1, link=1, source=1)</term>

<listitem>

<para>The mapping qualifies for a target object, and a link to a

source object exists. Detected only during reconciliation. Default

action: "UPDATE".</para>

</listitem>

</varlistentry>

<varlistentry>

<term>"UNQUALIFIED" (qualifies=0, link=1, source=1, but source does not

qualify)</term>

<listitem>

<para>The mapping is not qualified (by validTarget) for a target object,

and there is a link from an existing source object where the source

exists. Detected during change events and reconciliation. Default

action: "DELETE".</para>

</listitem>

</varlistentry>

<varlistentry>

<term>SOURCE_MISSING (qualifies=1, link=1, source=0)</term>

<listitem>

<para>The target qualifies and a link is found. But the source object is

missing. Only detected during reconciliation. Default action:

"EXCEPTION".</para>

</listitem>

</varlistentry>

</variablelist>

<para>The following sections reiterate in detail how OpenIDM assigns

situations during each of the two synchronization phases.</para>

</section>

<section xml:id="source-reconciliation">

<title>Source Reconciliation</title>

<para>OpenIDM starts reconciliation and LiveSync by reading a list of

objects from the resource. For reconciliation, the list includes all

objects available through the connector. For LiveSync, the list contains

only changed objects. The connector can filter objects out of the list, too.

You can filter objects out of the list by using the

<literal>validSource</literal> property.</para>

<para>OpenIDM then iterates over the list, checking each entry against the

<literal>validSource</literal> filter, classifying objects according to

their situations as described in <xref linkend="sync-situations"/>. OpenIDM

uses the list of links for the current mapping to classify objects. Finally,

OpenIDM executes the action configured for the situation.</para>

<para>The following table shows how OpenIDM assigns the appropriate

situation during source reconciliation, depending on whether a valid

source exists (Source Qualifies), whether a link with the appropriate type

exists in the repository (Link Exists), and how many target objects are

found, either based on links or correlation results.</para>

<table pgwide="1" rules="none">

<title>Resolving Source Reconciliation Situations</title>

<tgroup cols="8">

<colspec colnum="1" colname="c1" colwidth="1*" />

<colspec colnum="2" colname="c2" colwidth="1*" />

<colspec colnum="3" colname="c3" colwidth="1*" />

<colspec colnum="4" colname="c4" colwidth="1*" />

<colspec colnum="5" colname="c5" colwidth="1*" />

<colspec colnum="6" colname="c6" colwidth="1*" />

<colspec colnum="7" colname="c7" colwidth="1*" />

<colspec colnum="8" colname="c8" colwidth="2*" />

<thead>

<row>

<entry namest="c1" nameend="c2" align="left">Source Qualifies?</entry>

<entry namest="c3" nameend="c4" align="left">Link Exists?</entry>

<entry namest="c5" nameend="c7" align="left">Target Objects

Found<footnote><para>If no link exists for the source object, then

OpenIDM executes a correlation query.</para></footnote></entry>

<entry morerows="1" valign="top" align="left">Situation</entry>

</row>

<row>

<entry>Yes</entry>

<entry>No</entry>

<entry>Yes</entry>

<entry>No</entry>

<entry>0</entry>

<entry>1</entry>

<entry>&gt; 1</entry>

</row>

</thead>

<tbody>

<row>

<entry>&#160;</entry>

<entry>X</entry>

<entry>&#160;</entry>

<entry>X</entry>

<entry>X</entry>

<entry>&#160;</entry>

<entry>&#160;</entry>

<entry>SOURCE_IGNORED</entry>

</row>

<row>

<entry>&#160;</entry>

<entry>X</entry>

<entry>&#160;</entry>

<entry>X</entry>

<entry>&#160;</entry>

<entry>X</entry>

<entry>&#160;</entry>

<entry>UNQUALIFIED</entry>

</row>

<row>

<entry>&#160;</entry>

<entry>X</entry>

<entry>&#160;</entry>

<entry>X</entry>

<entry>&#160;</entry>

<entry>&#160;</entry>

<entry>X</entry>

<entry>UNQUALIFIED</entry>

</row>

<row>

<entry>&#160;</entry>

<entry>X</entry>

<entry>X</entry>

<entry>&#160;</entry>

<entry>X</entry>

<entry>&#160;</entry>

<entry>&#160;</entry>

<entry>UNQUALIFIED</entry>

</row>

<row>

<entry>&#160;</entry>

<entry>X</entry>

<entry>X</entry>

<entry>&#160;</entry>

<entry>&#160;</entry>

<entry>X</entry>

<entry>&#160;</entry>

<entry>UNQUALIFIED</entry>

</row>

<row>

<entry>&#160;</entry>

<entry>X</entry>

<entry>X</entry>

<entry>&#160;</entry>

<entry>&#160;</entry>

<entry>&#160;</entry>

<entry>X</entry>

<entry>UNQUALIFIED</entry>

</row>

<row>

<entry>X</entry>

<entry>&#160;</entry>

<entry>&#160;</entry>

<entry>X</entry>

<entry>X</entry>

<entry>&#160;</entry>

<entry>&#160;</entry>

<entry>ABSENT</entry>

</row>

<row>

<entry>X</entry>

<entry>&#160;</entry>

<entry>&#160;</entry>

<entry>X</entry>

<entry>&#160;</entry>

<entry>X</entry>

<entry>&#160;</entry>

<entry>FOUND</entry>

</row>

<row>

<entry>X</entry>

<entry>&#160;</entry>

<entry>&#160;</entry>

<entry>X</entry>

<entry>&#160;</entry>

<entry>&#160;</entry>

<entry>X</entry>

<entry>AMBIGUOUS</entry>

</row>

<row>

<entry>X</entry>

<entry>&#160;</entry>

<entry>X</entry>

<entry>&#160;</entry>

<entry>X</entry>

<entry>&#160;</entry>

<entry>&#160;</entry>

<entry>MISSING</entry>

</row>

<row>

<entry>X</entry>

<entry>&#160;</entry>

<entry>X</entry>

<entry>&#160;</entry>

<entry>&#160;</entry>

<entry>X</entry>

<entry>&#160;</entry>

<entry>CONFIRMED</entry>

</row>

</tbody>

</tgroup>

</table>

</section>

<section xml:id="target-reconciliation">

<title>Target Reconciliation</title>

<para>During source reconciliation, OpenIDM cannot detect situations where

no source object exists, such as the UNASSIGNED situation. When no source

object exists, OpenIDM detects the situation during the second

reconciliation phase, target reconciliation. During target reconciliation,

OpenIDM iterates over all target objects that do not have a representation

on the source, checking each object against the

<literal>validTarget</literal> filter, determining the appropriate situation,

and executing the action configured for the situation.</para>

<para>The following table shows how OpenIDM assigns the appropriate

situation during target reconciliation, depending on whether a valid target

exists (Target Qualifies), whether a link with an appropriate type exists

in the repository (Link Exists), whether a source object exists

(Source Exists), and whether the source object qualifies (Source Qualifies).

Not all situations assigned during source reconciliation are assigned

during target reconciliation.</para>

<table pgwide="1" rules="none">

<title>Resolving Target Reconciliation Situations</title>

<tgroup cols="9">

<colspec colnum="1" colname="c1" colwidth="1*" />

<colspec colnum="2" colname="c2" colwidth="1*" />

<colspec colnum="3" colname="c3" colwidth="1*" />

<colspec colnum="4" colname="c4" colwidth="1*" />

<colspec colnum="5" colname="c5" colwidth="1*" />

<colspec colnum="6" colname="c6" colwidth="1*" />

<colspec colnum="7" colname="c7" colwidth="1*" />

<colspec colnum="8" colname="c8" colwidth="1*" />

<colspec colnum="9" colname="c9" colwidth="2*" />

<thead>

<row>

<entry namest="c1" nameend="c2" align="left">Target Qualifies?</entry>

<entry namest="c3" nameend="c4" align="left">Link Exists?</entry>

<entry namest="c5" nameend="c6" align="left">Source Exists?</entry>

<entry namest="c7" nameend="c8" align="left">Source Qualifies?</entry>

<entry morerows="1" valign="top" align="left">Situation</entry>

</row>

<row>

<entry>Yes</entry>

<entry>No</entry>

<entry>Yes</entry>

<entry>No</entry>

<entry>Yes</entry>

<entry>No</entry>

<entry>Yes</entry>

<entry>No</entry>

</row>

</thead>

<tbody>

<row>

<entry>&#160;</entry>

<entry>X</entry>

<entry>&#160;</entry>

<entry>&#160;</entry>

<entry>&#160;</entry>

<entry>&#160;</entry>

<entry>&#160;</entry>

<entry>&#160;</entry>

<entry>TARGET_IGNORED</entry>

</row>

<row>

<entry>X</entry>

<entry>&#160;</entry>

<entry>&#160;</entry>

<entry>X</entry>

<entry>&#160;</entry>

<entry>X</entry>

<entry>&#160;</entry>

<entry>&#160;</entry>

<entry>UNASSIGNED</entry>

</row>

<row>

<entry>X</entry>

<entry>&#160;</entry>

<entry>X</entry>

<entry>&#160;</entry>

<entry>X</entry>

<entry>&#160;</entry>

<entry>X</entry>

<entry>&#160;</entry>

<entry>CONFIRMED</entry>

</row>

<row>

<entry>X</entry>

<entry>&#160;</entry>

<entry>X</entry>

<entry>&#160;</entry>

<entry>X</entry>

<entry>&#160;</entry>

<entry>&#160;</entry>

<entry>X</entry>

<entry>UNQUALIFIED</entry>

</row>

<row>

<entry>X</entry>

<entry>&#160;</entry>

<entry>X</entry>

<entry>&#160;</entry>

<entry>&#160;</entry>

<entry>X</entry>

<entry>&#160;</entry>

<entry>&#160;</entry>

<entry>SOURCE_MISSING</entry>

</row>

</tbody>

</tgroup>

</table>

</section>

<section xml:id="sync-actions">

<title>Synchronization Actions</title>

<variablelist>

<para>Once OpenIDM has assigned a situation to an object, OpenIDM takes

the actions configured in the mapping. If no action is configured, then

OpenIDM takes the default action for the situation. OpenIDM supports the

following actions.</para>

<varlistentry>

<term>"CREATE"</term>

<listitem>

<para>Create and link a target object.</para>

</listitem>

</varlistentry>

<varlistentry>

<term>"UPDATE"</term>

<listitem>

<para>Link and update a target object.</para>

</listitem>

</varlistentry>

<varlistentry>

<term>"DELETE"</term>

<listitem>

<para>Delete and unlink the target object.</para>

</listitem>

</varlistentry>

<varlistentry>

<term>"LINK"</term>

<listitem>

<para>Link the correlated target object.</para>

</listitem>

</varlistentry>

<varlistentry>

<term>"UNLINK"</term>

<listitem>

<para>Unlink the linked target object.</para>

</listitem>

</varlistentry>

<varlistentry>

<term>"EXCEPTION"</term>

<listitem>

<para>Flag the link situation as an exception.</para>

</listitem>

</varlistentry>

<varlistentry>

<term>"IGNORE"</term>

<listitem>

<para>Do not change the link or target object state.</para>

</listitem>

</varlistentry>

</variablelist>

</section>

</section>

<section xml:id="reconciliation-optimization">

<title>Reconciliation Optimization</title>

<para>By default, reconciliation is configured to function in an optimized

way. Some of these optimizations might, however, be unsuitable for your

environment. The following sections describe the optimizations and how they

can be configured.</para>

<section xml:id="correlate-target-set">

<title>Correlating Empty Target Sets</title>

<para>To optimize a reconciliation operation, the reconciliation process does

not attempt to correlate source objects to target objects if the set of

target objects is empty when the correlation is started. This considerably

speeds up the process the first time the reconciliation is run. You can change

this behavior for a specific mapping by adding the

<literal>correlateEmptyTargetSet</literal> property to the mapping definition

and setting it to <literal>true</literal>. For example:</para>