<?xml version="1.0" encoding="UTF-8"?>
<!--
! CCPL HEADER START
!
! This work is licensed under the Creative Commons
! Attribution-NonCommercial-NoDerivs 3.0 Unported License.
! To view a copy of this license, visit
! or send a letter to Creative Commons, 444 Castro Street,
! Suite 900, Mountain View, California, 94041, USA.
!
! You can also obtain a copy of the license at
! See the License for the specific language governing permissions
! and limitations under the License.
!
! If applicable, add the following below this CCPL HEADER, with the fields
! enclosed by brackets "[]" replaced with your own identifying information:
! Portions Copyright [yyyy] [name of copyright owner]
!
! CCPL HEADER END
!
! Copyright 2011-2015 ForgeRock AS.
!
-->
<chapter xml:id='chap-indexing'
xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'
xsi:schemaLocation='http://docbook.org/ns/docbook
xmlns:xlink='http://www.w3.org/1999/xlink'>
<title>Indexing Attribute Values</title>
<indexterm>
<primary>Indexes</primary>
</indexterm>
<para>OpenDJ provides several indexing schemes to speed up searches.</para>
<para>When a client requests a directory search operation, the client sends
the server a filter expression such as
<literal>(&(uid=*jensen*)(l=Stavanger))</literal>. The server then uses
applicable indexes to find entries with attribute values likely to match
the search. If no indexes are applicable, then the server potentially has
to go through all entries to look for candidate matches.</para>
<para>Looking through all entries is resource-intensive for large directories.
For this reason, the <literal>unindexed-search</literal> privilege, allowing
users to request searches for which no applicable index exists, is reserved
for the directory root user by default.</para>
<para>Rather than granting the <literal>unindexed-search</literal> privilege
to more users and client applications, you configure indexes to correspond
to the searches that clients need to perform. See
<xref linkend="debug-search-indexes" /> for details.</para>
<para>This chapter first describes index types, and demonstrates how to
index attribute values. This chapter also lists the default indexing
configuration for OpenDJ directory server.</para>
<section xml:id="indexes-overview">
<title>Index Types & What Each Does</title>
<para>OpenDJ provides several different index types, each corresponding
to a different type of search.</para>
<section xml:id="indexes-approximate">
<title>Approximate Index</title>
<indexterm>
<primary>Indexes</primary>
<secondary>Approximate</secondary>
</indexterm>
<para>An approximate index is used to match values that "sound like" those
provided in the filter. An approximate index on <literal>cn</literal>
allows clients to find people even when they misspell names as in the
following example.</para>
<screen>
$ <userinput>ldapsearch --port 1389 --baseDN dc=example,dc=com "(cn~=Babs Jansen)" cn</userinput>
<computeroutput>dn: uid=bjensen,ou=People,dc=example,dc=com
cn: Barbara Jensen
cn: Babs Jensen</computeroutput>
</screen>
</section>
<section xml:id="indexes-equality">
<title>Equality Index</title>
<indexterm>
<primary>Indexes</primary>
<secondary>Equality</secondary>
</indexterm>
<para>An equality index is used to match values that correspond exactly
(though generally without case sensitivity) to the value provided in
the search filter. An equality index requires clients to match values
without wildcards or misspellings.</para>
<screen>
$ <userinput>ldapsearch --port 1389 --baseDN dc=example,dc=com "(uid=bjensen)" mail</userinput>
<computeroutput>dn: uid=bjensen,ou=People,dc=example,dc=com
mail: bjensen@example.com</computeroutput>
</screen>
</section>
<section xml:id="indexes-ordering">
<title>Ordering Index</title>
<indexterm>
<primary>Indexes</primary>
<secondary>Ordering</secondary>
</indexterm>
<para>An ordering index is used to match values for a filter that
specifies a range. The <literal>ds-sync-hist</literal> has an ordering
index by default because searches on that attribute often seek entries
with changes more recent than the last time a search was performed.</para>
<para>The following example shows a search that specifies ranges.</para>
<screen>
$ <userinput>ldapsearch --port 1389 --baseDN dc=example,dc=com \
"(&(uidNumber>=1120)(roomNumber>=4500))" uid</userinput>
<computeroutput>dn: uid=charvey,ou=People,dc=example,dc=com
uid: charvey
dn: uid=eward,ou=People,dc=example,dc=com
uid: eward
dn: uid=mvaughan,ou=People,dc=example,dc=com
uid: mvaughan
dn: uid=pchassin,ou=People,dc=example,dc=com
uid: pchassin</computeroutput>
</screen>
</section>
<section xml:id="indexes-presence">
<title>Presence Index</title>
<indexterm>
<primary>Indexes</primary>
<secondary>Presence</secondary>
</indexterm>
<para>A presence index is used to match the fact that an attribute is
present on the entry, regardless of the value. The <literal>aci</literal>
attribute is indexed for presence by default to allow quick retrieval
of entries with ACIs.</para>
<screen>
$ <userinput>ldapsearch --port 1389 --baseDN dc=example,dc=com "(aci=*)" -</userinput>
<computeroutput>dn: dc=example,dc=com
dn: ou=People,dc=example,dc=com</computeroutput>
</screen>
</section>
<section xml:id="indexes-substring">
<title>Substring Index</title>
<indexterm>
<primary>Indexes</primary>
<secondary>Substring</secondary>
</indexterm>
<para>A substring index is used to match values specified with wildcards
in the filter. Substring indexes can be expensive to maintain, especially
for large attribute values.</para>
<screen>
$ <userinput>ldapsearch --port 1389 --baseDN dc=example,dc=com "(cn=Barb*)" cn</userinput>
<computeroutput>dn: uid=bfrancis,ou=People,dc=example,dc=com
cn: Barbara Francis
dn: uid=bhal2,ou=People,dc=example,dc=com
cn: Barbara Hall
dn: uid=bjablons,ou=People,dc=example,dc=com
cn: Barbara Jablonski
dn: uid=bjensen,ou=People,dc=example,dc=com
cn: Barbara Jensen
cn: Babs Jensen
dn: uid=bmaddox,ou=People,dc=example,dc=com
cn: Barbara Maddox</computeroutput>
</screen>
</section>
<section xml:id="indexes-vlv">
<title>Virtual List View (Browsing) Index</title>
<indexterm>
<primary>Indexes</primary>
<secondary>Virtual list view (browsing)</secondary>
</indexterm>
<para>A VLV or browsing index are designed to help the server respond to
client applications that need virtual list view results, for example to
browse through a long list in a GUI. They also help the server respond
to clients that request server-side sorting of the search results.</para>
<para>VLV indexes correspond to particular searches. Configure your
VLV indexes using the Control Panel, and copy the command-line
equivalent from the Details pane for the operation, if necessary.</para>
</section>
<section xml:id="indexes-extensible">
<title>Extensible Matching Rule Index</title>
<indexterm>
<primary>Indexes</primary>
<secondary>Extensible matching rule</secondary>
</indexterm>
<para>In some cases you need an index for a matching rule other than those
described above. For example, OpenDJ supports generalized time based
matching so applications can search for all times later than, or earlier
than a specified time.</para>
</section>
</section>
<section xml:id="debug-search-indexes">
<title>Determining What Needs Indexing</title>
<indexterm>
<primary>Indexes</primary>
<secondary>Debugging searches</secondary>
</indexterm>
<para>OpenDJ search performance depends on indexes. As mentioned above,
unindexed searches are so resource intensive that by default OpenDJ refuses
to perform unindexed searches. This is because, in order to find candidate
matches for an unindexed search, OpenDJ has to scan the entire directory
database. Most searches should therefore use indexes.</para>
<para>A simple way of checking the indexes that match a search is to request
the <literal>debugsearchindex</literal> attribute in your results.</para>
<screen>
$ <userinput>ldapsearch \
--port 1389 \
--baseDN dc=example,dc=com \
--bindDN "cn=Directory Manager" \
--bindPassword password \
"(uid=user.1000)" \
debugsearchindex</userinput>
<computeroutput>dn: cn=debugsearch
debugsearchindex: filter=(uid=user.1000)[INDEX:uid.equality][COUNT:1] final=[COU
NT:1]</computeroutput>
</screen>
<para>When you request the <literal>debugsearchindex</literal> attribute,
instead of performing the search, OpenDJ returns debug information indicating
how it would process the search operation. In the example above you notice
OpenDJ hits the equality index for <literal>uid</literal> right away.</para>
<para>A less exact search requires more work from OpenDJ. In the following
example OpenDJ would have to return over 10,000 entries.</para>
<screen>
$ <userinput>ldapsearch \
--port 1389 \
--baseDN dc=example,dc=com \
--bindDN "cn=Directory Manager" \
--bindPassword password \
"(uid=*)" \
debugsearchindex</userinput>
<computeroutput>dn: cn=debugsearch
debugsearchindex: filter=(uid=*)[NOT-INDEXED] scope=wholeSubtree[LIMIT-EXCEEDED:
10002] final=[NOT-INDEXED]</computeroutput>
</screen>
<para>By default OpenDJ rejects unindexed searches when the number of
candidate entries goes beyond the search or look-though limit.</para>
<screen>
$ <userinput>ldapsearch --port 1389 --baseDN dc=example,dc=com "(uid=*)"</userinput>
<computeroutput>SEARCH operation failed
Result Code: 50 (Insufficient Access Rights)
Additional Information: You do not have sufficient privileges to perform
an unindexed search</computeroutput>
</screen>
<para>When an unindexed search is performed, it shows up in the access
log with the <literal>unindexed</literal> label.</para>
<programlisting language="none">
...SEARCH RES ... result=50 message="You do not have sufficient privileges
to perform an unindexed search" nentries=0 unindexed etime=1
</programlisting>
<para>If directory users tell you their client applications are getting this
error, then you can work with them either to help them make their search
filter specific enough to use existing indexes, or to index attributes they
need indexed in order to perform their searches. For example, if a
directory client application is having trouble performing a search with
a filters such as <literal>(objectClass=person)</literal>, you can suggest
that they adjust the search to be more specific, such as
<literal>(&(mail=username@maildomain.net)(objectClass=person))</literal>,
so that the server can use an index, in this case equality for mail, to
limit the number of candidate entries to check for matches.</para>
<para>You can view and edit what is indexed through OpenDJ Control Panel,
Indexes > Manage Indexes. Alternatively you can manage indexes using the
command-line tools demonstrated in <xref linkend="configure-indexes" />.
If an index already exists, but you suspect it is not working properly, see
<xref linkend="verify-index" />, too.</para>
<para>If you do need to allow some applications to perform unindexed searches,
because they need to retrieve very large numbers of entries for example, then
you can assign them the <literal>unindexed-search</literal> privilege. See
<link xlink:show="new" xlink:href="admin-guide#configure-privileges"
Privileges</citetitle></link> for details. A successful unindexed search also
shows up in the access log with the label <literal>unindexed</literal>,
usually with a large etime as well.</para>
<programlisting language="none">
...SEARCH RES conn=11 op=1 msgID=2 result=0 nentries=10000 unindexed etime=1129
</programlisting>
<para>There is a trade off between the cost of maintaining an index and the
value the index has in speeding up searches. Although monitoring index use
is not something to leave active in production due to the additional cost and
memory needed to maintain the statistics, in a test environment you can
activate index analysis using the <command>dsconfig set-backend-prop</command>
command.</para>
<screen>
$ <userinput>dsconfig \
set-backend-prop \
--port 4444 \
--hostname opendj.example.com \
--bindDN "cn=Directory Manager" \
--bindPassword password \
--backend-name userRoot \
--set index-filter-analyzer-enabled:true \
--no-prompt \
--trustAll</userinput>
</screen>
<para>The command causes OpenDJ to analyze filters used and keep the results
in memory, so that you can read them through the <literal>cn=monitor</literal>
interface.</para>
<screen>
$ <userinput>ldapsearch \
--port 1389 \
--baseDN "cn=userRoot Database Environment,cn=monitor" \
--bindDN "cn=Directory Manager" \
--bindPassword password \
"(objectclass=*)" \
filter-use</userinput>
<computeroutput>dn: cn=userRoot Database Environment,cn=monitor
filter-use: (mail=aa*@maildomain.net) hits:1 maxmatches:0 message:
filter-use: (objectClass=*) hits:1 maxmatches:-1 message:presence index type is
disabled for the objectClass attribute
filter-use: (uid=user.1000) hits:2 maxmatches:1 message:
filter-use: (uid=user.1001) hits:1 maxmatches:1 message:
filter-use: (cn=aa*) hits:1 maxmatches:10 message:
filter-use: (cn=b*) hits:1 maxmatches:834 message:</computeroutput>
</screen>
<para>The <literal>filter-use</literal> values consist of the filter, followed
by <literal>hits</literal> being the number of times the filter was used,
followed by <literal>maxmatches</literal> being the number of matches found
for the filter, followed by a message.</para>
<para>You can turn off index analysis with the <command>dsconfig
set-backend-prop</command> command as well.</para>
<screen>
$ <userinput>dsconfig \
set-backend-prop \
--port 4444 \
--hostname opendj.example.com \
--bindDN "cn=Directory Manager" \
--bindPassword password \
--backend-name userRoot \
--set index-filter-analyzer-enabled:false \
--no-prompt \
--trustAll</userinput>
</screen>
</section>
<section xml:id="configure-indexes">
<title>Configuring & Rebuilding Indexes</title>
<indexterm>
<primary>Indexes</primary>
<secondary>Configuring</secondary>
</indexterm>
<para>
You modify index configurations by using the
<link
xlink:show="new"
xlink:href="reference#dsconfig-1"
xlink:role="http://docbook.org/xlink/role/olink"
><command>dsconfig</command></link> command.
The configuration changes then take effect
after you rebuild the index according to the new configuration, using the
<link
xlink:show="new"
xlink:href="reference#rebuild-index-1"
xlink:role="http://docbook.org/xlink/role/olink"
><command>rebuild-index</command></link> command.
The <command>dsconfig --help-database</command> command lists subcommands
for creating, reading, updating, and deleting index configuration.
</para>
<tip>
<para>Indexes are per directory backend rather than per suffix. To maintain
separate indexes for different suffixes on the same directory server, put
the suffixes in different backends.</para>
</tip>
<section xml:id="configure-standard-index">
<title>Configuring a Standard Index</title>
<para>You can configure standard indexes from the Control Panel, and also
on the command line using the <command>dsconfig</command> command. After
you finish configuring the index, you must rebuild the index for the changes
to take effect.</para>
<example xml:id="create-index-example">
<title>Create a New Index</title>
<para>The following example creates a new substring index for
<literal>description</literal>.</para>
<screen>
$ <userinput>dsconfig \
create-local-db-index \
--port 4444 \
--hostname opendj.example.com \
--bindDN "cn=Directory Manager" \
--bindPassword password \
--backend-name userRoot \
--index-name description \
--set index-type:substring \
--trustAll \
--no-prompt</userinput>
</screen>
</example>
<example xml:id="approx-index-example">
<title>Configure an Approximate Index</title>
<indexterm>
<primary>Indexes</primary>
<secondary>Approximate</secondary>
</indexterm>
<para>The following example configures an approximate index for
<literal>cn</literal> (common name).</para>
<screen>
$ <userinput>dsconfig \
set-local-db-index-prop \
--port 4444 \
--hostname opendj.example.com \
--bindDN "cn=Directory Manager" \
--bindPassword password \
--backend-name userRoot \
--index-name cn \
--set index-type:approximate \
--trustAll \
--no-prompt</userinput>
</screen>
</example>
<example xml:id="extensible-match-index-example">
<title>Configure an Extensible Match Index</title>
<indexterm>
<primary>Indexes</primary>
<secondary>Extensible matching rule</secondary>
</indexterm>
<para>The OpenDJ Control Panel New Index window does not help you set up
extensible matching rule indexes. Use the <command>dsconfig</command>
command instead.</para>
<para>The following example configures an extensible matching rule
index for "later than" and "earlier than" generalized time matching on
a <literal>lastLoginTime</literal> attribute.</para>
<screen>
$ <userinput>dsconfig \
create-local-db-index \
--port 4444 \
--hostname opendj.example.com \
--bindDN "cn=Directory Manager" \
--bindPassword password \
--backend-name userRoot \
--set index-type:extensible \
--set index-extensible-matching-rule:1.3.6.1.4.1.26027.1.4.5 \
--set index-extensible-matching-rule:1.3.6.1.4.1.26027.1.4.6 \
--index-name lastLoginTime \
--trustAll \
--no-prompt</userinput>
</screen>
</example>
</section>
<section xml:id="configure-vlv">
<title>Configuring a Virtual List View Index</title>
<indexterm>
<primary>Indexes</primary>
<secondary>Virtual list view (browsing)</secondary>
</indexterm>
<para>In the OpenDJ Control Panel, select Manage Indexes >
New VLV Index, and then set up your VLV index using the New VLV
Index window.</para>
<mediaobject xml:id="figure-create-vlv-index">
<alt>New VLV Index window</alt>
<imageobject>
</imageobject>
<textobject>
<para>
The New VLV Index window helps you to configure a browsing index.
</para>
</textobject>
</mediaobject>
<para>After you finish configuring your index and click OK, the Control
Panel prompts you to make the additional changes necessary to complete the
VLV index configuration, and then to build the index.</para>
<para>You can also create the equivalent index configuration using the
<command>dsconfig</command> command.</para>
<screen>
$ <userinput>dsconfig \
create-local-db-vlv-index \
--port 4444 \
--hostname opendj.example.com \
--bindDn "cn=Directory Manager" \
--bindPassword password \
--backend-name userRoot \
--index-name people-by-last-name \
--set base-dn:ou=People,dc=example,dc=com \
--set filter:"(|(givenName=*)(sn=*))" \
--set scope:single-level \
--set sort-order:"+sn +givenName" \
--trustAll \
--no-prompt</userinput>
</screen>
<note>
<para>When referring to a virtual list view (VLV) index after creation, you
must add <literal>vlv.</literal> as a prefix. In other words, if you named
the VLV index <literal>people-by-last-name</literal>, you refer to it as
changing index properties such as the index entry limit, or verifying
indexes.</para>
</note>
</section>
<section xml:id="rebuild-index">
<title>Rebuilding Indexes</title>
<indexterm>
<primary>Indexes</primary>
<secondary>Rebuilding</secondary>
</indexterm>
<para>After you change an index configuration, or when you find that
an index is corrupt, you can rebuild the index. When you rebuild indexes,
you specify the base DN of the data to index, and either the list of indexes
to rebuild or <option>--rebuildAll</option>. You can rebuild indexes while
the server is offline, or while the server is online. If you rebuild the
index while the server is online, then you must schedule the rebuild process
as a task.</para>
<example xml:id="rebuild-index-example">
<title>Rebuild Index</title>
<para>The following example rebuilds the <literal>cn</literal> index
immediately with the server online.</para>
<screen>
$ <userinput>rebuild-index \
--port 4444 \
--hostname opendj.example.com \
--bindDN "cn=Directory Manager" \
--bindPassword password \
--baseDN dc=example,dc=com \
--index cn \
--start 0</userinput>
<computeroutput>Rebuild Index task 20110607171639867 scheduled to start Jun 7, 2011 5:16:39 PM</computeroutput>
</screen>
</example>
<example xml:id="rebuild-degraded-indexes-example">
<title>Rebuild Degraded Indexes</title>
<para>The following example rebuilds degraded indexes immediately with
the server online.</para>
<screen>
$ <userinput>rebuild-index \
--port 4444 \
--hostname opendj.example.com \
--bindDN "cn=Directory Manager" \
--bindPassword password \
--baseDN dc=example,dc=com \
--rebuildDegraded</userinput>
<computeroutput>...
message="Due to changes in the configuration, index
dc_example_dc_com_description is currently operating in a degraded state
and must be rebuilt before it can be used"
message="Rebuild of all degraded indexes started with 160 total entries
to process"
...
message="Rebuild complete. Processed 160 entries in 0 seconds (average
...
Rebuild Index task 20120131164324838 has been successfully completed</computeroutput>
</screen>
</example>
<example xml:id="clear-degraded-indexes-example">
<title>Clear New, Unused, "Degraded" Indexes</title>
<para>When you add a new attribute as described in <link
xlink:href="admin-guide#update-schema"
Directory Schema</citetitle></link>, and then create indexes for the new
attribute, the new indexes appear as degraded, even though the attribute
has not yet been used, and so indexes are sure to be empty, rather than
degraded.</para>
<para>In this special case, you can safely use the
<command>rebuild-index</command> command
<option>--clearDegradedState</option> option to avoid having to scan
the entire directory backend to rebuild the new, unused index. This
is shown in the following example, where an index has just been created
for <literal>newUnusedAttribute</literal>.</para>
<para>
Start by testing the index status by using the
<link
xlink:show="new"
xlink:href="reference#dbtest-1"
xlink:role="http://docbook.org/xlink/role/olink"
><command>dbtest</command></link> command.
The final column show in the output is the Index Valid column,
<literal>false</literal> before the rebuild, <literal>true</literal> after.
</para>
<note>
<para>
The <command>dbtest list-index-status</command> command
can take a long time to complete, as it reads all indexes for all backends.
</para>
</note>
<screen>
$ <userinput>dbtest \
list-index-status \
--backendID userRoot \
--baseDN dc=example,dc=com \
| grep newUnusedAttribute</userinput>
newUnusedAttribute.presence Index ...newUnusedAttribute.presence false...
$ <userinput>rebuild-index \
--port 4444 \
--hostname opendj.example.com \
--bindDN "cn=Directory Manager" \
--bindPassword password \
--baseDN dc=example,dc=com \
--clearDegradedState \
--index newUnusedAttribute \
--start 0</userinput>
<computeroutput>Rebuild Index task 20130211175925012 scheduled to start Feb 11, 2013 5:59:25
PM CET</computeroutput>
$ <userinput>dbtest \
list-index-status \
--backendID userRoot \
--baseDN dc=example,dc=com \
| grep newUnusedAttribute</userinput>
newUnusedAttribute.presence Index ...newUnusedAttribute.presence true...
</screen>
<para>If the newly indexed attribute has already been used, rebuild indexes
instead.</para>
</example>
</section>
<section xml:id="index-entry-limits">
<title>Changing Index Entry Limits</title>
<indexterm>
<primary>Indexes</primary>
<secondary>Entry limits</secondary>
</indexterm>
<para>As the number of entries in your directory grows, it can make sense
not to maintain indexes for particular values. For example, every entry
in the directory has the value <literal>top</literal> for the
<literal>objectClass</literal> attribute, so maintaining a list of entries
that match the filter <literal>(objectClass=top)</literal> is not a
reasonable use of resources. In a very, very large directory, the same can
be true for <literal>(givenName=John)</literal> and
<literal>(sn=Smith)</literal>.</para>
<para>In an index, each index key points to a list of entries that
are candidates to match. For the <literal>objectClass</literal> index key
that corresponds to <literal>=top</literal>, the list of entries can
include every entry in the directory.</para>
<para>OpenDJ directory server therefore defines an index entry limit. When
the number of entries that an index key points to exceeds the index entry
limit, OpenDJ stops maintaining the list of entries for that index key.</para>
<para>The default index entry limit value is 4000. 4000 is intended to be
large enough for most index keys, though it prevents OpenDJ from maintaining
indexes at any cost. You can use the <command>dbtest</command> command to
evaluate how well attributes are indexed, and consider whether to change
the index entry limit. Non-zero values in the "Undefined" column indicate
the number of index keys that have reached the limit and are no longer
maintained. The "Undefined keys" are then listed below.</para>
<informalexample><?dbfo pgwide="1"?>
<screen width="136">
$ <userinput>dbtest list-index-status --backendID userRoot --baseDN dc=example,dc=com</userinput>
<computeroutput>Index Name Index Type JE Database Name Index Valid Record Count Undefined 95% 90% 85%
---------------------------------------------------------------------------------------------------------------------------------------
id2children Index dc_example_dc_com_id2children true 2 1 0 0 0
id2subtree Index dc_example_dc_com_id2subtree true 2 2 0 0 0
uid.equality Index dc_example_dc_com_uid.equality true 10000 0 0 0 0
aci.presence Index dc_example_dc_com_aci.presence true 0 0 0 0 0
ds-sync-conflict.equality Index dc_example_dc_com_ds-sync-conflict.equality true 0 0 0 0 0
givenName.equality Index dc_example_dc_com_givenName.equality true 8605 0 0 0 0
givenName.substring Index dc_example_dc_com_givenName.substring true 19629 0 0 0 0
objectClass.equality Index dc_example_dc_com_objectClass.equality true 6 4 0 0 0
member.equality Index dc_example_dc_com_member.equality true 0 0 0 0 0
uniqueMember.equality Index dc_example_dc_com_uniqueMember.equality true 0 0 0 0 0
cn.equality Index dc_example_dc_com_cn.equality true 10000 0 0 0 0
cn.substring Index dc_example_dc_com_cn.substring true 86040 0 0 0 0
sn.equality Index dc_example_dc_com_sn.equality true 10000 0 0 0 0
sn.substring Index dc_example_dc_com_sn.substring true 32217 0 0 0 0
telephoneNumber.equality Index dc_example_dc_com_telephoneNumber.equality true 10000 0 0 0 0
telephoneNumber.substring Index dc_example_dc_com_telephoneNumber.substring true 73235 0 0 0 0
ds-sync-hist.ordering Index dc_example_dc_com_ds-sync-hist.ordering true 0 0 0 0 0
mail.equality Index dc_example_dc_com_mail.equality true 10000 0 0 0 0
mail.substring Index dc_example_dc_com_mail.substring true 31235 15 0 0 0
entryUUID.equality Index dc_example_dc_com_entryUUID.equality true 10002 0 0 0 0
Total: 20
Index: objectClass.equality
Undefined keys: [inetorgperson] [organizationalperson] [person] [top]
Index: id2children
Undefined keys: [2]
Index: mail.substring
Undefined keys: [.net] [@maild] [aildom] [ain.ne] [domain] [et] [ildoma] [in.net] [ldomai] [maildo] [main.n] [n.net] [net] [omain.] [t]
Index: id2subtree
Undefined keys: [1] [2]</computeroutput>
</screen>
</informalexample>
<para>In this case (for a directory with only about 10,000 entries) the
list of undefined keys is perfectly reasonable. Every user entry has the
object classes listed, and every user entry has a mail address ending in
to be used in search filters. The <literal>id2children</literal> and
<literal>id2subtree</literal> are for OpenDJ's internal use.</para>
<para>
For an explanation of the output
of the <command>dbtest list-index-status</command> command, see
<link xlink:show="new" xlink:href="reference#dbtest-1"
</para>
<para>If you do find the limit is too low for a certain key, you can change
the index entry limit on a per index basis.</para>
<example xml:id="change-index-entry-limit">
<title>Change Index Entry Limit</title>
<para>The following example changes the index entry limit for the
<literal>objectClass</literal> index, and then rebuilds the index for the
configuration change to take effect. The example is contrived, but the
steps are the same for any other index.</para>
<important>
<para>Changing the index entry limit significantly can result in
serious performance degradation. Be prepared to test performance
thoroughly before you roll out an index entry limit change in
production.</para>
</important>
<screen>
$ <userinput>dsconfig \
set-local-db-index-prop \
--port 4444 \
--hostname opendj.example.com \
--bindDN "cn=Directory Manager" \
--bindPassword password \
--backend-name userRoot \
--index-name objectClass \
--set index-entry-limit:5000 \
--trustAll \
--no-prompt</userinput>
$ <userinput>rebuild-index \
--port 4444 \
--hostname opendj.example.com \
--bindDN "cn=Directory Manager" \
--bindPassword password \
--baseDN dc=example,dc=com \
--index objectclass \
--start 0</userinput>
<computeroutput>Rebuild Index task 20110607160349596 scheduled to start Jun 7, 2011 4:03:49 PM</computeroutput>
</screen>
</example>
<para>Alternatively, you can configure the index entry limit for all
indexes stored in a backend by using the <command>dsconfig
set-backend-prop</command> command with the <option>--backend-name
<replaceable>backendName</replaceable> --set
index-entry-limit:<replaceable>limitValue</replaceable></option>
options.</para>
</section>
</section>
<section xml:id="verify-index">
<title>Verifying Indexes</title>
<indexterm>
<primary>Indexes</primary>
<secondary>Verifying</secondary>
</indexterm>
<para>
You can verify that indexes correspond to current directory data,
and that indexes do not contain errors by using the
<link
xlink:show="new"
xlink:href="reference#verify-index-1"
xlink:role="http://docbook.org/xlink/role/olink"
><command>verify-index</command></link> command.
</para>
<example xml:id="verify-index-example">
<title>Verify Index</title>
<para>The following example verifies the <literal>cn</literal> (common
name) index for completeness and for errors.</para>
<screen>
$ <userinput>verify-index \
--baseDN dc=example,dc=com \
--index cn \
--clean \
--countErrors</userinput>
msgID=9437595 msg=Local DB backend userRoot does not specify the number of
lock tables: defaulting to 97
msgID=9437594 msg=Local DB backend userRoot does not specify the number of
cleaner threads: defaulting to 24 threads
msg=Checked 1316 records and found 0 error(s) in 0 seconds
msgID=8388710 msg=Number of records referencing more than one entry: 315
msgID=8388711 msg=Number of records that exceed the entry limit: 0
msgID=8388713 msg=Maximum number of entries referenced by any
record is 32</computeroutput>
</screen>
<para>Ignore the messages regarding lock tables and cleaner threads. The
important information is whether any errors are found in the indexes.</para>
</example>
</section>
<section xml:id="default-indexes">
<title>Default Indexes</title>
<indexterm>
<primary>Indexes</primary>
<secondary>Default settings</secondary>
</indexterm>
<para>When you first install OpenDJ directory server and import your
data from LDIF, the following indexes are configured.</para>
<table pgwide="1" rules="none">
<title>Default Indexes</title>
<tgroup cols="7">
<colspec colnum="2" colname="c2" />
<colspec colnum="7" colname="c7" />
<thead>
<row>
<entry>Index</entry>
<entry>Approx.</entry>
<entry>Equality</entry>
<entry>Ordering</entry>
<entry>Presence</entry>
<entry>Substring</entry>
<entry>Entry Limit</entry>
</row>
</thead>
<tbody>
<row>
<entry><literal>aci</literal></entry>
<entry>-</entry>
<entry>-</entry>
<entry>-</entry>
<entry>Yes</entry>
<entry>-</entry>
<entry>4000</entry>
</row>
<row>
<entry><literal>cn</literal></entry>
<entry>-</entry>
<entry>Yes</entry>
<entry>-</entry>
<entry>-</entry>
<entry>Yes</entry>
<entry>4000</entry>
</row>
<row>
<entry><literal>dn2id</literal></entry>
<entry namest="c2" nameend="c7" align="center">Non-configurable
internal index</entry>
</row>
<row>
<entry><literal>ds-sync-conflict</literal></entry>
<entry>-</entry>
<entry>Yes</entry>
<entry>-</entry>
<entry>-</entry>
<entry>-</entry>
<entry>4000</entry>
</row>
<row>
<entry><literal>ds-sync-hist</literal></entry>
<entry>-</entry>
<entry>-</entry>
<entry>Yes</entry>
<entry>-</entry>
<entry>-</entry>
<entry>4000</entry>
</row>
<row>
<entry><literal>entryUUID</literal></entry>
<entry>-</entry>
<entry>Yes</entry>
<entry>-</entry>
<entry>-</entry>
<entry>-</entry>
<entry>4000</entry>
</row>
<row>
<entry><literal>givenName</literal></entry>
<entry>-</entry>
<entry>Yes</entry>
<entry>-</entry>
<entry>-</entry>
<entry>Yes</entry>
<entry>4000</entry>
</row>
<row>
<entry><literal>id2children</literal></entry>
<entry namest="c2" nameend="c7" align="center">Non-configurable
internal index</entry>
</row>
<row>
<entry><literal>id2subtree</literal></entry>
<entry namest="c2" nameend="c7" align="center">Non-configurable
internal index</entry>
</row>
<row>
<entry><literal>mail</literal></entry>
<entry>-</entry>
<entry>Yes</entry>
<entry>-</entry>
<entry>-</entry>
<entry>Yes</entry>
<entry>4000</entry>
</row>
<row>
<entry><literal>member</literal></entry>
<entry>-</entry>
<entry>Yes</entry>
<entry>-</entry>
<entry>-</entry>
<entry>-</entry>
<entry>4000</entry>
</row>
<row>
<entry><literal>objectClass</literal></entry>
<entry>-</entry>
<entry>Yes</entry>
<entry>-</entry>
<entry>-</entry>
<entry>-</entry>
<entry>4000</entry>
</row>
<row>
<entry><literal>sn</literal></entry>
<entry>-</entry>
<entry>Yes</entry>
<entry>-</entry>
<entry>-</entry>
<entry>Yes</entry>
<entry>4000</entry>
</row>
<row>
<entry><literal>telephone­Number</literal></entry>
<entry>-</entry>
<entry>Yes</entry>
<entry>-</entry>
<entry>-</entry>
<entry>Yes</entry>
<entry>4000</entry>
</row>
<row>
<entry><literal>uid</literal></entry>
<entry>-</entry>
<entry>Yes</entry>
<entry>-</entry>
<entry>-</entry>
<entry>-</entry>
<entry>4000</entry>
</row>
<row>
<entry><literal>unique­Member</literal></entry>
<entry>-</entry>
<entry>Yes</entry>
<entry>-</entry>
<entry>-</entry>
<entry>-</entry>
<entry>4000</entry>
</row>
</tbody>
</tgroup>
</table>
<para>When you create a JE backend using the <command>dsconfig</command>
command, OpenDJ creates the <literal>aci</literal> presence,
<literal>ds-sync-conflict</literal> equality,
<literal>ds-sync-hist</literal> ordering,
<literal>entryUUID</literal> equality, and
<literal>objectClass</literal> equality indexes automatically.</para>
</section>
</chapter>