51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark ! CCPL HEADER START
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark ! This work is licensed under the Creative Commons
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark ! Attribution-NonCommercial-NoDerivs 3.0 Unported License.
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark ! To view a copy of this license, visit
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark ! or send a letter to Creative Commons, 444 Castro Street,
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark ! Suite 900, Mountain View, California, 94041, USA.
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark ! You can also obtain a copy of the license at
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark ! See the License for the specific language governing permissions
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark ! and limitations under the License.
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark ! If applicable, add the following below this CCPL HEADER, with the fields
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark ! enclosed by brackets "[]" replaced with your own identifying information:
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark ! Portions Copyright [yyyy] [name of copyright owner]
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark ! CCPL HEADER END
57d6342a74476c0bf2200992e778229d62ab1fa6mark ! Copyright 2011-2015 ForgeRock AS.
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark xmlns='http://docbook.org/ns/docbook' version='5.0' xml:lang='en'
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <indexterm><primary>Performance tuning</primary></indexterm>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>Server tuning refers to the art of adjusting server, JVM, and system
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark configuration to meet the service level performance requirements of directory
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark clients. In the optimal case you achieve service level performance
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark requirements without much tuning at all, perhaps only setting JVM runtime
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark options when installing OpenDJ.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>If you are reading this chapter, however, you are probably not
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark facing an optimal situation. Instead you are looking for trade offs that
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark maximize performance for clients given the constraints of your deployment.
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark This chapter therefore aims to provide suggestions on how to measure and
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark to improve directory service performance for better trade offs.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <title>Defining Performance Requirements & Constraints</title>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>Your key performance requirement is most likely to satisfy your
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark users or customers with the resources available to you. Before you can
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark solve potential performance problems, define what those users or customers
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark expect, and determine what resources you will have to satisfy their
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark expectations.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>Service-level agreement (SLA) is a formal name for what directory
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark client applications and the people who run them expect from your service in
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark terms of performance.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>SLAs might cover many aspects of the directory service. Whether or not
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark your SLA is formally defined, you ought to know what is expected, or at least
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark what you provide, in the following four areas.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <itemizedlist>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <listitem>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>Directory service <firstterm>response times</firstterm></para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>Directory service response times range from less than a
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark millisecond on average across a low latency connection on the same
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark network to however long it takes your network to deliver the response.
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark More important than average or best response times is the response time
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark distribution, because applications set timeouts based on worst case
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark scenarios. For example, a response time performance requirement might
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark be defined as, "Directory response times must average less than 10
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark milliseconds for all operations except searches returning more than 10
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark entries, with 99.9% of response times under 40 milliseconds."</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </listitem>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <listitem>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>Directory service <firstterm>throughput</firstterm></para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <indexterm>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </indexterm>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>Directory service throughput can range up to many thousands of
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark operations per second. In fact there is no upper limit for read operations
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark such as searches, because only write operations must be replicated. To
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark increase read throughput, simply add additional replicas. More important
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark than average throughput is peak throughput. You might have peak write
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark throughput in the middle of the night when batch jobs update entries in
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark bulk, and peak binds for a special event or first thing Monday morning.
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark For example, a throughput performance requirement might be expressed as,
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark "The directory service must sustain a mix of 5,000 operations per second
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark made up of 70% reads, 25% modifies, 3% adds, and 2% deletes."</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>Even better is to mimic the behavior of key operations for
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark performance testing, so that you understand the patterns of operations
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark in the throughput you need to provide.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </listitem>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <listitem>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>Directory service <firstterm>availability</firstterm></para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>OpenDJ is designed to let you build directory services that are
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark basically available, including during maintenance and even upgrade of
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark individual servers. Yet, in order to reach very high levels of
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark availability, you must make sure not only that the software is
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark designed for availability, but also that your operations execute in
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark such a way as to preserve availability. Availability requirements
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark can be as lax as best effort, or as stringent as 99.999% or more
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark uptime.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>Replication is the OpenDJ feature that allows you to build a
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark highly available directory service.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </listitem>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <listitem>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>Do not forget to make sure you understand and set expectations
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark about how you support your users when they run into trouble. Directory
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark services can perhaps help you turn password management into a self-service
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark visit to a web site, but some users no doubt still need to know what they
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark can expect if they need your help.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </listitem>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </itemizedlist>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>Writing down the SLA, even if your first version consists of
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark guesses, helps you reduce performance tuning from an open-ended project
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark to a clear set of measurable goals for a manageable project with a definite
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark outcome.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </section>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>With your SLA in hand, take inventory of the server, networks,
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark storage, people, and other resources at your disposal. Now is the time to
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark estimate whether it is possible to meet the requirements at all.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>If for example you are expected to serve more throughput than the
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark network can transfer, maintain high availability with only one physical
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark machine, store 100 GB of backups on a 50 GB partition, or provide 24/7
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark support all alone, no amount of tweaking available resources is likely to
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark fix the problem.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>When checking that the resources you have at least theoretically
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark suffice to meet your requirements, do not forget that high availability in
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark particular requires at least two of everything to avoid single points
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark of failure. Be sure to list the resources you expect to have, when and how
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark long you expect to have them, and why you need them. Also make note of
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark what is missing and why.</para>
0d0444587373bdecce45bb80de6c826eee238197mark In addition to the suggestions in this section, also read the section on
0d0444587373bdecce45bb80de6c826eee238197mark <link xlink:href="release-notes#prerequisites-hardware" xlink:show="new"
0d0444587373bdecce45bb80de6c826eee238197mark xlink:role="http://docbook.org/xlink/role/olink"><citetitle
0d0444587373bdecce45bb80de6c826eee238197mark >Hardware</citetitle></link> in the <citetitle>Release Notes</citetitle>.
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>Concerning server hardware, OpenDJ runs on systems with Java support,
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark and is therefore quite portable. That said, OpenDJ tends to perform best on
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark single-board, x86 systems due to low memory latency.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </section>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>OpenDJ is designed to work with local storage for the database,
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark not for network file systems such as NFS.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>High performance storage is essential if you need to handle high
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark write throughput.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>The Berkeley Java Edition DB works well with traditional disks as
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark long as the database cache size allows the DB to stay fully cached in
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark memory. This is the case because the database transaction log is append
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark only. When the DB is too big to stay cached in memory, however, then
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark cache misses lead to random disk access, slowing OpenDJ performance.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>You might mitigate this effect by using solid-state disks for
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark persistent storage, or for file system cache.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>Regarding database size on disk, if you have sustained write traffic
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark then the database grows to about twice its initial size on disk. This is
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark normal, and due to the way the database manages its logs. The size on disk
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark does not impact the DB cache size requirements.</para>
0d0444587373bdecce45bb80de6c826eee238197mark <xinclude:include href="/shared/para-write-barriers.xml" />
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </section>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </section>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </section>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>Even if you do not need high availability, you still need two of
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark everything, because your test environment needs to mimic your production
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark environment as closely as possible if you want to avoid nasty
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark surprises.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>In your test environment, you set up OpenDJ as you will later in
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark production, and then conduct experiments to determine how best to meet
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark the requirements defined in the SLA.</para>
57d6342a74476c0bf2200992e778229d62ab1fa6mark <para>Use <link xlink:show="new" xlink:href="reference#make-ldif-1"
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark xlink:role="http://docbook.org/xlink/role/olink">make-ldif</link> to generate
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark sample data that match what you expect to find in production.</para>
bf4ac85bb7f691982c1aaa24673c1527bd63815amark The OpenDJ LDAP Toolkit provides command-line tools
bf4ac85bb7f691982c1aaa24673c1527bd63815amark to help with basic performance testing.
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <itemizedlist>
bf4ac85bb7f691982c1aaa24673c1527bd63815amark <listitem>
bf4ac85bb7f691982c1aaa24673c1527bd63815amark >addrate</link> command measures add and delete throughput and response time.
bf4ac85bb7f691982c1aaa24673c1527bd63815amark </listitem>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <listitem>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark xlink:href="${coreDocBase}dev-guide#authrate-1">authrate</link> command
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark measures bind throughput and response time.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </listitem>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <listitem>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark xlink:href="${coreDocBase}dev-guide#modrate-1">modrate</link> command
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark measures modification throughput and response time.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </listitem>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <listitem>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark xlink:href="${coreDocBase}dev-guide#searchrate-1">searchrate</link> command
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark measures search throughput and response time.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </listitem>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </itemizedlist>
bf4ac85bb7f691982c1aaa24673c1527bd63815amark All these commands show you information about the response time distributions,
bf4ac85bb7f691982c1aaa24673c1527bd63815amark and allow you to perform tests at specific levels of throughput.
ec40cc0dc62425cea5d63fd9d984f8614479de25mark If you need additional precision when evaluating response times,
ec40cc0dc62425cea5d63fd9d984f8614479de25mark use the global configuration setting,
ec40cc0dc62425cea5d63fd9d984f8614479de25mark xlink:href="${configRefBase}global.html#etime-resolution"
ec40cc0dc62425cea5d63fd9d984f8614479de25mark to change elapsed processing time resolution
ec40cc0dc62425cea5d63fd9d984f8614479de25mark from milliseconds (default) to nanoseconds.
08248b5c5b494aff8d1922e8e0b5777796d7450dmark$ <userinput>dsconfig \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark set-global-configuration-prop \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark --port 4444 \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark --bindDN "cn=Directory Manager" \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark --bindPassword password \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark --set etime-resolution:nanoseconds \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark --no-prompt</userinput>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </section>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>When your tests show that OpenDJ performance is lacking even though
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark you have the right underlying network, hardware, storage, and system
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark resources in place, you can tweak OpenDJ performance in a number of ways.
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark This section mentions the most common tweaks.</para>
0d0444587373bdecce45bb80de6c826eee238197mark <xinclude:include href="/shared/sec-prerequisites-file-descriptors.xml" />
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>Default Java settings let you evaluate OpenDJ using limited system
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark resources. If you need high performance for production system, test with
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark the following JVM options. These apply to the Sun/Oracle JVM.</para>
ec40cc0dc62425cea5d63fd9d984f8614479de25mark To apply JVM settings for your server,
ec40cc0dc62425cea5d63fd9d984f8614479de25mark and apply the changes with the
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <variablelist>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <varlistentry>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <listitem>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </listitem>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </varlistentry>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <varlistentry>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <listitem>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>To use a heap larger than about 3.5 GB on a 64-bit system, use
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark this option.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </listitem>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </varlistentry>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <varlistentry>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <listitem>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>Set both minimum and maximum heap size to the same value to avoid
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark resizing. Leave space for the entire DB cache and more.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </listitem>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </varlistentry>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <varlistentry>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <listitem>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>Set the new generation size between 1-4 GB for high throughput
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark deployments, but leave enough overall JVM heap to avoid overlaps with
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark the space used for DB cache.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </listitem>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </varlistentry>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <varlistentry>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <term><option>-XX:MaxTenuringThreshold=1</option></term>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <listitem>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>Force OpenDJ to create only objects that have either a short
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark lifetime, or a long lifetime.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </listitem>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </varlistentry>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <varlistentry>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <listitem>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>The CMS garbage collector tends to give the best performance
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark characteristics. You might also consider the G1 garbage collector.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </listitem>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </varlistentry>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <varlistentry>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <listitem>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>Use these when diagnosing JVM tuning problems. You can turn them
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark off when everything is running smoothly.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </listitem>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </varlistentry>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <varlistentry>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <listitem>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>Java object pointers normally have the same size as native machine
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark pointers. If you run a small, but 64-bit JVM, then compressed object
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark pointers can save space. Set this option when you have a 64-bit JVM,
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <option>-Xmx</option> less than 32 GB, and Java SE 6u23 or later.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </listitem>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </varlistentry>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </variablelist>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </section>
fdda609cc5c9c83cfc4357255bbe3466b97dd53bmark <para>By default, OpenDJ compresses attribute descriptions and object class
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark sets to reduce data size. This is called compact encoding.</para>
ec40cc0dc62425cea5d63fd9d984f8614479de25mark By default, OpenDJ does not however compress entries
ec40cc0dc62425cea5d63fd9d984f8614479de25mark stored in its backend database.
ec40cc0dc62425cea5d63fd9d984f8614479de25mark If your entries hold values that compress well —
ec40cc0dc62425cea5d63fd9d984f8614479de25mark such as text, and not JPEG photos or MP3 audio —
ec40cc0dc62425cea5d63fd9d984f8614479de25mark you can gain space by setting the local DB backend property,
ec40cc0dc62425cea5d63fd9d984f8614479de25mark xlink:href="${configRefBase}local-db-backend.html#entries-compressed"
ec40cc0dc62425cea5d63fd9d984f8614479de25mark to <literal>true</literal> before you (re-)import data from LDIF.
ec40cc0dc62425cea5d63fd9d984f8614479de25mark OpenDJ compresses entries before writing them to the database.<footnote>
ec40cc0dc62425cea5d63fd9d984f8614479de25mark OpenDJ does not proactively rewrite all entries in the database
ec40cc0dc62425cea5d63fd9d984f8614479de25mark after you change the settings.
ec40cc0dc62425cea5d63fd9d984f8614479de25mark Instead, to force OpenDJ to compress all entries,
ec40cc0dc62425cea5d63fd9d984f8614479de25mark import the data from LDIF.
ec40cc0dc62425cea5d63fd9d984f8614479de25mark </footnote>
08248b5c5b494aff8d1922e8e0b5777796d7450dmark$ <userinput>dsconfig \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark set-backend-prop \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark --port 4444 \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark --bindDN "cn=Directory Manager" \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark --bindPassword password \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark --backend-name userRoot \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark --set entries-compressed:true \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark --trustAll \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark --no-prompt</userinput>
08248b5c5b494aff8d1922e8e0b5777796d7450dmark$ <userinput>import-ldif \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark --port 4444 \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark --bindDN "cn=Directory Manager" \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark --bindPassword password \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark --backendID userRoot \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark --includeBranch dc=example,dc=com \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark --start 0</userinput>
08248b5c5b494aff8d1922e8e0b5777796d7450dmark<computeroutput>Import task 20120917100628767 scheduled to start Sep 17, 2012 10:06:28 AM CEST</computeroutput>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </section>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <indexterm>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </indexterm>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>You can tweak OpenDJ to speed up import of large LDIF files.</para>
ec40cc0dc62425cea5d63fd9d984f8614479de25mark By default, the temporary directory used for scratch files is
ec40cc0dc62425cea5d63fd9d984f8614479de25mark where you installed OpenDJ.
ec40cc0dc62425cea5d63fd9d984f8614479de25mark to set this directory to a <literal>tmpfs</literal> file system,
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>In some cases, you can improve performance by using the
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <command>import-ldif</command> command to set the thread count larger than
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark the default, which is twice the number of CPUs.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>If you are certain your LDIF contains only valid entries with
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark correct syntax, because the LDIF was exported from OpenDJ with all checks
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark active for example, you can skip schema and DN validation. Use the
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <command>import-ldif</command> command to skip validation.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </section>
ec40cc0dc62425cea5d63fd9d984f8614479de25mark Database cache size is, by default, set as a percentage of the JVM heap
ec40cc0dc62425cea5d63fd9d984f8614479de25mark by using the backend property,
ec40cc0dc62425cea5d63fd9d984f8614479de25mark xlink:href="${configRefBase}local-db-backend.html#db-cache-percent"
ec40cc0dc62425cea5d63fd9d984f8614479de25mark Alternatively, you use the backend property,
ec40cc0dc62425cea5d63fd9d984f8614479de25mark xlink:href="${configRefBase}local-db-backend.html#db-cache-size"
ec40cc0dc62425cea5d63fd9d984f8614479de25mark to set the size.
ec40cc0dc62425cea5d63fd9d984f8614479de25mark If you set up multiple database backends,
ec40cc0dc62425cea5d63fd9d984f8614479de25mark the total percent of JVM heap used must remain less than 100,
ec40cc0dc62425cea5d63fd9d984f8614479de25mark and must leave space for other uses.
ec40cc0dc62425cea5d63fd9d984f8614479de25mark Default settings work for servers
ec40cc0dc62425cea5d63fd9d984f8614479de25mark with one user data backend JVM heaps up to 2 GB.
ec40cc0dc62425cea5d63fd9d984f8614479de25mark For heaps larger than 2 GB,
ec40cc0dc62425cea5d63fd9d984f8614479de25mark you can allocate a larger percentage of heap space to DB cache.
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>Depending on the size of your database, you have a choice to make
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark about database cache settings.</para>
ec40cc0dc62425cea5d63fd9d984f8614479de25mark By caching the entire database in the JVM heap,
ec40cc0dc62425cea5d63fd9d984f8614479de25mark you can get more deterministic response times and limit disk I/O.
ec40cc0dc62425cea5d63fd9d984f8614479de25mark Yet, caching the whole DB can require a very large JVM,
ec40cc0dc62425cea5d63fd9d984f8614479de25mark which you must pre-load on startup,
ec40cc0dc62425cea5d63fd9d984f8614479de25mark and which can result in long garbage collections
ec40cc0dc62425cea5d63fd9d984f8614479de25mark and a difficult-to-manage JVM.
ec40cc0dc62425cea5d63fd9d984f8614479de25mark Test database pre-load on startup by setting the
ec40cc0dc62425cea5d63fd9d984f8614479de25mark xlink:href="${configRefBase}local-db-backend.html#preload-time-limit"
ec40cc0dc62425cea5d63fd9d984f8614479de25mark for the backend.</para>
08248b5c5b494aff8d1922e8e0b5777796d7450dmark$ <userinput>dsconfig \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark set-backend-prop \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark --port 4444 \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark --bindDN "cn=Directory Manager" \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark --bindPassword password \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark --backend-name userRoot \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark --set preload-time-limit:30m \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark --trustAll \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark --no-prompt</userinput>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>Database pre-load is single-threaded, and loads each database one
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark at a time.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>By allowing file system cache to hold the portion of database that
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark does not fit in DB cache, you trade less deterministic and slightly slower
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark response times for not having to pre-load the DB and not having garbage
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark collection pauses with large JVMs. How you configure the file system cache
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark depends on your operating system.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </section>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>OpenDJ implements an entry cache. The entry cache is not designed to
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark cache every entry in your database, but is instead useful in cases where you
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark have a few, typically large entries that are regularly used. For example, if
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark you have a few large static groups and applications that regularly check
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark group membership, you could cache your group entries.</para>
08248b5c5b494aff8d1922e8e0b5777796d7450dmark$ <userinput>dsconfig \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark create-entry-cache \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark --port 4444 \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark --bindDN "cn=Directory Manager" \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark --bindPassword password \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark --cache-name "Large Group Entry Cache" \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark --type fifo \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark --set cache-level:1 \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark --set include-filter:"(ou=Large Static Groups)" \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark --set max-entries:10 \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark --set enabled:true \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark --trustAll \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark --no-prompt</userinput>
ec40cc0dc62425cea5d63fd9d984f8614479de25mark You can use the global setting,
ec40cc0dc62425cea5d63fd9d984f8614479de25mark xlink:href="${configRefBase}global.html#entry-cache-preload"
ec40cc0dc62425cea5d63fd9d984f8614479de25mark to force OpenDJ to load the entry cache as part of server startup.
08248b5c5b494aff8d1922e8e0b5777796d7450dmark$ <userinput>dsconfig \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark set-global-configuration-prop \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark --port 4444 \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark --bindDN "cn=Directory Manager" \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark --bindPassword password \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark --set entry-cache-preload:true \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark --no-prompt</userinput>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>By default, OpenDJ does not pre-load the entry cache.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </section>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>Debug logs trace the internal workings of OpenDJ, and therefore
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark generally should be used sparingly, especially in high performance
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark deployments.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>In general leave other logs active for production environments to
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark help troubleshoot any issues that arise.</para>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark <para>For OpenDJ servers handling very high throughput, however, such as
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark 100,000 operations per second or more, the access log constitute a
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark performance bottleneck, as each client request results in multiple access log
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark messages. Consider disabling the access log in such cases.</para>
08248b5c5b494aff8d1922e8e0b5777796d7450dmark$ <userinput>dsconfig \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark set-log-publisher-prop \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark --port 4444 \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark --bindDN "cn=Directory Manager" \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark --bindPassword password \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark --publisher-name "File-Based Access Logger" \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark --set enabled:false \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark --trustAll \
08248b5c5b494aff8d1922e8e0b5777796d7450dmark --no-prompt</userinput>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </section>
51607ea01068c9047391e4c8b46bc9dbd0edb7fdmark </section>