chap-indexing.xml revision 51607ea01068c9047391e4c8b46bc9dbd0edb7fd
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark ! CCPL HEADER START
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark ! This work is licensed under the Creative Commons
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark ! Attribution-NonCommercial-NoDerivs 3.0 Unported License.
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark ! To view a copy of this license, visit
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark ! or send a letter to Creative Commons, 444 Castro Street,
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark ! Suite 900, Mountain View, California, 94041, USA.
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark ! You can also obtain a copy of the license at
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark ! See the License for the specific language governing permissions
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark ! and limitations under the License.
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark ! If applicable, add the following below this CCPL HEADER, with the fields
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark ! enclosed by brackets "[]" replaced with your own identifying information:
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark ! Portions Copyright [yyyy] [name of copyright owner]
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark ! CCPL HEADER END
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark ! Copyright 2011-2013 ForgeRock AS
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark xmlns='http://docbook.org/ns/docbook' version='5.0' xml:lang='en'
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark xsi:schemaLocation='http://docbook.org/ns/docbook http://docbook.org/xml/5.0/xsd/docbook.xsd'
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <indexterm>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </indexterm>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>OpenDJ provides several indexing schemes to speed up searches.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>When a client requests a directory search operation, the client sends
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark the server a filter expression such as
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <literal>(&(uid=*jensen*)(l=Stavanger))</literal>. The server then uses
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark applicable indexes to find entries with attribute values likely to match
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark the search. If no indexes are applicable, then the server potentially has
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark to go through all entries to look for candidate matches.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>Looking through all entries is resource-intensive for large directories.
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark For this reason, the <literal>unindexed-search</literal> privilege, allowing
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark users to request searches for which no applicable index exists, is reserved
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark for the directory root user by default.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>Rather than granting the <literal>unindexed-search</literal> privilege
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark to more users and client applications, you configure indexes to correspond
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark to the searches that clients need to perform.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>This chapter first describes index types, then demonstrates how to
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark index attribute values. This chapter also lists the default indexing
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark configuration for OpenDJ directory server.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>OpenDJ provides several different index types, each corresponding
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark to a different type of search.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <indexterm>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </indexterm>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>An approximate index is used to match values that "sound like" those
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark provided in the filter. An approximate index on <literal>cn</literal>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark allows clients to find people even when they misspell names as in the
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark following example.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <screen>$ ldapsearch --port 1389 --baseDN dc=example,dc=com "(cn~=Babs Jansen)" cn
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmarkdn: uid=bjensen,ou=People,dc=example,dc=com
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmarkcn: Barbara Jensen
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmarkcn: Babs Jensen</screen>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </section>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <indexterm>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </indexterm>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>An equality index is used to match values that correspond exactly
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark (though generally without case sensitivity) to the value provided in
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark the search filter. An equality index requires clients to match values
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark without wildcards or misspellings.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <screen>$ ldapsearch --port 1389 --baseDN dc=example,dc=com "(uid=bjensen)" mail
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmarkdn: uid=bjensen,ou=People,dc=example,dc=com
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmarkmail: bjensen@example.com</screen>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </section>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <indexterm>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </indexterm>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>An ordering index is used to match values for a filter that
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark specifies a range. The <literal>ds-sync-hist</literal> has an ordering
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark index by default because searches on that attribute often seek entries
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark with changes more recent than the last time a search was performed.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>The following example shows a search that specifies ranges.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <screen>$ ldapsearch --port 1389 --baseDN dc=example,dc=com
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark "(&(uidNumber>=1120)(roomNumber>=4500))" uid
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmarkdn: uid=charvey,ou=People,dc=example,dc=com
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmarkuid: charvey
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmarkdn: uid=eward,ou=People,dc=example,dc=com
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmarkdn: uid=mvaughan,ou=People,dc=example,dc=com
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmarkuid: mvaughan
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmarkdn: uid=pchassin,ou=People,dc=example,dc=com
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmarkuid: pchassin</screen>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </section>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <indexterm>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </indexterm>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>A presence index is used to match the fact that an attribute is
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark present on the entry, regardless of the value. The <literal>aci</literal>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark attribute is indexed for presence by default to allow quick retrieval
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark of entries with ACIs.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <screen>$ ldapsearch --port 1389 --baseDN dc=example,dc=com "(aci=*)" -
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmarkdn: dc=example,dc=com
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmarkdn: ou=People,dc=example,dc=com</screen>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </section>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <indexterm>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </indexterm>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>A substring index is used to match values specified with wildcards
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark in the filter. Substring indexes can be expensive to maintain, especially
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark for large attribute values.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <screen>$ ldapsearch --port 1389 --baseDN dc=example,dc=com "(cn=Barb*)" cn
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmarkdn: uid=bfrancis,ou=People,dc=example,dc=com
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmarkcn: Barbara Francis
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmarkdn: uid=bhal2,ou=People,dc=example,dc=com
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmarkcn: Barbara Hall
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmarkdn: uid=bjablons,ou=People,dc=example,dc=com
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmarkcn: Barbara Jablonski
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmarkdn: uid=bjensen,ou=People,dc=example,dc=com
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmarkcn: Barbara Jensen
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmarkcn: Babs Jensen
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmarkdn: uid=bmaddox,ou=People,dc=example,dc=com
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmarkcn: Barbara Maddox</screen>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </section>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <indexterm>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </indexterm>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>A VLV or browsing index are designed to help the server respond to
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark client applications that need virtual list view results, for example to
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark browse through a long list in a GUI. They also help the server respond
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark to clients that request server-side sorting of the search results.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>VLV indexes correspond to particular searches. Configure your
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark VLV indexes using the Control Panel, and copy the command-line
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark equivalent from the Details pane for the operation, if necessary.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </section>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <indexterm>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </indexterm>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>In some cases you need an index for a matching rule other than those
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark described above. For example, OpenDJ supports generalized time based
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark matching so applications can search for all times later than, or earlier
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark than a specified time.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </section>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </section>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <indexterm>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </indexterm>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>You modify index configurations using the <command>dsconfig</command>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark command. The configuration changes then take effect after you rebuild the
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark index according to the new configuration, using the
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <command>rebuild-index</command>. The <command>dsconfig
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --help-database</command> command lists subcommands for creating, reading,
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark updating, and deleting index configuration.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>Indexes are per directory backend rather than per suffix. To maintain
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark separate indexes for different suffixes on the same directory server, put
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark the suffixes in different backends.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>You can configure standard indexes from the Control Panel, and also
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark on the command line using the <command>dsconfig</command> command. After
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark you finish configuring the index, you must rebuild the index for the changes
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark to take effect.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>The following example creates a new substring index for
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <screen>$ dsconfig
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark create-local-db-index
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --port 4444
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --bindDN "cn=Directory Manager"
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --bindPassword password
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --backend-name userRoot
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --index-name description
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --set index-type:substring
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --trustAll
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --no-prompt</screen>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </example>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <indexterm>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </indexterm>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>The following example configures an approximate index for
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <screen>$ dsconfig
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark set-local-db-index-prop
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --port 4444
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --bindDN "cn=Directory Manager"
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --bindPassword password
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --backend-name userRoot
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --index-name cn
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --set index-type:approximate
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --trustAll
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --no-prompt</screen>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </example>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <indexterm>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </indexterm>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>The OpenDJ Control Panel New Index window does not help you set up
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark extensible matching rule indexes. Use the <command>dsconfig</command>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark command instead.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>The following example configures an extensible matching rule
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark index for "later than" and "earlier than" generalized time matching on
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <screen>$ dsconfig
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark create-local-db-index
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --port 4444
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --bindDN "cn=Directory Manager"
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --bindPassword password
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --backend-name userRoot
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --set index-type:extensible
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --set index-extensible-matching-rule:1.3.6.1.4.1.26027.1.4.5
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --set index-extensible-matching-rule:1.3.6.1.4.1.26027.1.4.6
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --index-name lastLoginTime
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --trustAll
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --no-prompt</screen>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </example>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </section>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <indexterm>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </indexterm>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>In the OpenDJ Control Panel, select Manage Indexes >
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark New VLV Index..., and then set up your VLV index using the New VLV
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark Index window.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <imageobject>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <imagedata fileref="images/create-vlv-index.png" format="PNG" />
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </imageobject>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </mediaobject>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>After you finish configuring your index and click OK, the Control
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark Panel prompts you to make the additional changes necessary to complete the
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark VLV index configuration, and then to build the index.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>You can also create the equivalent index configuration using the
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <screen>$ dsconfig
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark create-local-db-vlv-index
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --port 4444
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --bindDn "cn=Directory Manager"
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --bindPassword password
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --backend-name userRoot
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --index-name people-by-last-name
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --set base-dn:ou=People,dc=example,dc=com
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --set filter:"(|(givenName=*)(sn=*))"
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --set scope:single-level
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --set sort-order:"+sn +givenName"
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --trustAll
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --no-prompt</screen>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>When referring to a virtual list view (VLV) index after creation, you
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark must add <literal>vlv.</literal> as a prefix. In other words, if you named
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark the VLV index <literal>people-by-last-name</literal>, you refer to it as
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <literal>vlv.people-by-last-name</literal> when rebuilding indexes,
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark changing index properties such as the index entry limit, or verifying
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark indexes.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </section>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <indexterm>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </indexterm>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>After you change an index configuration, or when you find that
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark an index is corrupt, you can rebuild the index. When you rebuild indexes,
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark you specify the base DN of the data to index, and either the list of indexes
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark to rebuild or <option>--rebuildAll</option>. You can rebuild indexes while
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark the server is offline, or while the server is online. If you rebuild the
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark index while the server is online, then you must schedule the rebuild process
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark as a task.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>The following example rebuilds the <literal>cn</literal> index
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark immediately with the server online.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <screen>$ rebuild-index
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --port 4444
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --bindDN "cn=Directory Manager"
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --bindPassword password
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --baseDN dc=example,dc=com
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --index cn
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmarkRebuild Index task 20110607171639867 scheduled to start Jun 7, 2011 5:16:39 PM</screen>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </example>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>The following example rebuilds degraded indexes immediately with
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark the server online.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <screen>$ rebuild-index
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --port 4444
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --bindDN "cn=Directory Manager"
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --bindPassword password
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --baseDN dc=example,dc=com
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --rebuildDegraded
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark[31/Jan/2012:16:43:25 +0100] severity="NOTICE" msgCount=7 msgID=8847510
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark message="Due to changes in the configuration, index
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark dc_example_dc_com_description is currently operating in a degraded state
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark and must be rebuilt before it can be used"
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark[31/Jan/2012:16:43:25 +0100] severity="NOTICE" msgCount=8 msgID=8847591
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark message="Rebuild of all degraded indexes started with 160 total entries
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark to process"
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark[31/Jan/2012:16:43:25 +0100] severity="NOTICE" msgCount=10 msgID=8847493
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark message="Rebuild complete. Processed 160 entries in 0 seconds (average
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmarkRebuild Index task 20120131164324838 has been successfully completed</screen>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </example>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>When you add a new attribute as described in <link
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark xlink:role="http://docbook.org/xlink/role/olink"><citetitle>Updating
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark Directory Schema</citetitle></link>, and then create indexes for the new
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark attribute, the new indexes appear as degraded, even though the attribute
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark has not yet been used, and so indexes are sure to be empty, rather than
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark degraded.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>In this special case, you can safely use the
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <option>--clearDegradedState</option> option to avoid having to scan
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark the entire directory backend to rebuild the new, unused index. This
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark is shown in the following example, where an index has just been created
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <screen>$ dbtest
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark list-index-status
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --backendID userRoot
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --baseDN dc=example,dc=com
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark | grep newUnusedAttribute
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmarknewUnusedAttribute.equality Index ...newUnusedAttribute.equality false...
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmarknewUnusedAttribute.presence Index ...newUnusedAttribute.presence false...
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmarknewUnusedAttribute.substring Index ...newUnusedAttribute.substring false...
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark$ rebuild-index
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --port 4444
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --bindDN "cn=Directory Manager"
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --bindPassword password
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --baseDN dc=example,dc=com
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --clearDegradedState
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --index newUnusedAttribute
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmarkRebuild Index task 20130211175925012 scheduled to start Feb 11, 2013 5:59:25
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark list-index-status
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --backendID userRoot
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --baseDN dc=example,dc=com
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark | grep newUnusedAttribute
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmarknewUnusedAttribute.equality Index ...newUnusedAttribute.equality true...
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmarknewUnusedAttribute.presence Index ...newUnusedAttribute.presence true...
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmarknewUnusedAttribute.substring Index ...newUnusedAttribute.substring true...</screen>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>If the newly indexed attribute has already been used, rebuild indexes
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark instead.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </example>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </section>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <indexterm>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </indexterm>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>As the number of entries in your directory grows, it can make sense
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark not to maintain indexes for particular values. For example, every entry
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark in the directory has the value <literal>top</literal> for the
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <literal>objectClass</literal> attribute, so maintaining a list of entries
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark that match the filter <literal>(objectClass=top)</literal> is not a
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark reasonable use of resources. In a very, very large directory, the same can
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>In an index, each index key points to a list of entries that
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark are candidates to match. For the <literal>objectClass</literal> index key
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark that corresponds to <literal>=top</literal>, the list of entries can
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark include every entry in the directory.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>OpenDJ directory server therefore defines an index entry limit. When
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark the number of entries that an index key points to exceeds the index entry
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark limit, OpenDJ stops maintaining the list of entries for that index key.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>The default index entry limit value is 4000. 4000 is usually plenty
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark large for all index keys, except for <literal>objectClass</literal> indexes.
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark If you have clients performing searches with filters such as
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <literal>(objectClass=person)</literal>, you might suggest that they adjust
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark the search to be more specific, such as
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <literal>(&(mail=username@maildomain.net)(objectClass=person))</literal>,
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark so that the server can use an index, in this case equality for mail, to
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark limit the number of candidate entries to check for matches.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>You can change the index entry limit on a per index basis.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>The following example changes the index entry limit for the
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <literal>objectClass</literal> index, and then rebuilds the index for the
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark configuration change to take effect.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <screen>$ dsconfig
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark set-local-db-index-prop
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --port 4444
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --bindDN "cn=Directory Manager"
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --bindPassword password
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --backend-name userRoot
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --index-name objectClass
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --set index-entry-limit:5000
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --trustAll
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --no-prompt
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark$ rebuild-index
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --port 4444
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --bindDN "cn=Directory Manager"
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --bindPassword password
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --baseDN dc=example,dc=com
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --index objectclass
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmarkRebuild Index task 20110607160349596 scheduled to start Jun 7, 2011 4:03:49 PM</screen>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </example>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>Alternatively, you can configure the index entry limit for all
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark indexes stored in a backend by using the <command>dsconfig
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark set-backend-prop</command> command with the <option>--backend-name
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark index-entry-limit:<replaceable>limitValue</replaceable></option>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark options.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </section>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </section>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <indexterm>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </indexterm>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>You can verify that indexes correspond to current directory data,
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark and that indexes do not contain errors using the
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>The following example verifies the <literal>cn</literal> (common
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark name) index for completeness and for errors.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <screen>$ verify-index
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --baseDN dc=example,dc=com
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --index cn
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --countErrors
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark[07/Jun/2011:16:06:50 +0200] category=BACKEND severity=INFORMATION
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark msgID=9437595 msg=Local DB backend userRoot does not specify the number of
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark lock tables: defaulting to 97
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark[07/Jun/2011:16:06:50 +0200] category=BACKEND severity=INFORMATION
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark msgID=9437594 msg=Local DB backend userRoot does not specify the number of
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark cleaner threads: defaulting to 24 threads
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark[07/Jun/2011:16:06:51 +0200] category=JEB severity=NOTICE msgID=8847461
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark msg=Checked 1316 records and found 0 error(s) in 0 seconds
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark[07/Jun/2011:16:06:51 +0200] category=JEB severity=INFORMATION
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark msgID=8388710 msg=Number of records referencing more than one entry: 315
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark[07/Jun/2011:16:06:51 +0200] category=JEB severity=INFORMATION
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark msgID=8388711 msg=Number of records that exceed the entry limit: 0
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark[07/Jun/2011:16:06:51 +0200] category=JEB severity=INFORMATION
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark msgID=8388712 msg=Average number of entries referenced is 1.58/record
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark[07/Jun/2011:16:06:51 +0200] category=JEB severity=INFORMATION
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark msgID=8388713 msg=Maximum number of entries referenced by any
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark record is 32</screen>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>Ignore the messages regarding lock tables and cleaner threads. The
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark important information is whether any errors are found in the indexes.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </example>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </section>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <indexterm>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </indexterm>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>When searching, you can improve performance by making sure your search
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark is indexed as you expect. One way of checking is to request the
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <literal>debugsearchindex</literal> attribute in your results.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <screen>$ ldapsearch
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --port 1389
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --baseDN dc=example,dc=com
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark "(uid=bjensen)"
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark debugsearchindex
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmarkdn: cn=debugsearch
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmarkdebugsearchindex: filter=(uid=bjensen)[INDEX:uid.equality][COUNT:1]
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark final=[COUNT:1]</screen>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>When you request the <literal>debugsearchindex</literal> attribute,
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark instead of performing the search, OpenDJ returns debug information indicating
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark how it would process the search operation. In the example above you notice
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark OpenDJ hits the equality index for <literal>uid</literal> right away.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>A less exact search requires more work from OpenDJ. In the following
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark example OpenDJ would have to return 160 entries.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <screen>$ ldapsearch
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --port 1389
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark --baseDN dc=example,dc=com
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark debugsearchindex
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmarkdn: cn=debugsearch
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmarkdebugsearchindex: filter=(uid=*)[NOT-INDEXED] scope=wholeSubtree[COUNT:160]
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark final=[COUNT:160]</screen>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>By default OpenDJ rejects unindexed searches when the number of
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark candidate entries goes beyond the search or look-though limit.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </section>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <indexterm>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </indexterm>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>When you first install OpenDJ directory server and import your
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark data from LDIF, the following indexes are configured.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <entry namest="c2" nameend="c7" align="center">Non-configurable
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark internal index</entry>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <entry namest="c2" nameend="c7" align="center">Non-configurable
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark internal index</entry>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <entry namest="c2" nameend="c7" align="center">Non-configurable
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark internal index</entry>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <entry><literal>telephone­Number</literal></entry>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>When you create a JE backend using the <command>dsconfig</command>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark command, OpenDJ creates the <literal>aci</literal> presence,
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <literal>objectClass</literal> equality indexes automatically.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </section>