OPENDJ-808 (CR-1685) Implement a simple commons REST based HTTP connection handler Added support for ds-cfg-num-request-handlers in the HTTPConnectionHandler configuration to bring it at par with the LDAPConnectionHandler configuration. 02-config.ldif, HTTPConnectionHandlerConfiguration.xml, HTTPConnectionHandlerCfgDefn.properties: Added ds-cfg-num-request-handlers. HTTPConnectionHandler.java: Set the SelectorRunnersCount in grizzly based on the numRequestHandlers property. LDAPConnectionHandler.java, ConnectionHandler.java: Moved getNumRequestHandlers() from LDAPConnectionHandler to ConnectionHandler.

OPENDJ-858 (CR-1680) Add stats tracking to HTTP client connections Now monitoring total etime for each supported HTTP method. While implementing this, I noticed that HTTPRequestInfo was duplicating fields in HTTPClientConnection and adding little more. I moved the missing data in HTTPRequestInfo to HTTPClientConnection and transformed the HTTPRequestInfo class into an interface to keep zero coupling between logging and HTTPClientConnection. HTTPRequestInfo.java: Transformed this class into an interface. Renamed getRemoteHost() to getClientHost() and getRemoteAddress() to getClientAddress(). HTTPClientConnection.java Now implements HTTPRequestInfo. Moved several fields from HTTPRequestInfo here + added missing getters and setters. In sendResponse(), calling HTTPStatistics.updateRequestMonitoringData(). Added log(). TextHTTPAccessLogPublisher.java: Consequence of the changes to HTTPRequestInfo. Also added support for the "c-port", "s-ip", "s-computername", and "s-port" fields. CollectClientConnectionsFilter.java, SdkConnectionAdapter.java: Consequence of the changes to HTTPRequestInfo. HTTPStatistics.java: Renamed instance members nbRequests to requestMethodsTotalCount and nbRequestsTotalCount to requestsTotalCount. Added requestMethodsTotalTime instance member. Completed the constructor, clearStatistics(), getMonitorData(). Fixed clearStatistics(). Added addAll(), updateRequestMonitoringData(). 02-config.ldif: Added attributes for monitoring the total etime of HTTP requests. Renumbered some previously added attributes to follow what is done for LDAP operations etime.

OPENDJ-858 (CR-1651) Add stats tracking to HTTP client connections Added support for monitoring statistics of the HTTP connection handler: It monitors: total number of requests, and number of delete, get, patch, post and put requests. It also monitors the internal LDAP operations to display how the server is performing. HTTPConnectionHandler.java, HTTPClientConnection.java, CollectClientConnectionsFilter.java, SdkConnectionAdapter.java: Added support for statistics. config.ldif, HTTPConnectionHandlerConfiguration.xml, HTTPConnectionHandlerCfgDefn.properties: Added "keep-stats" property. 02-config.ldif: Added attributes "ds-mon-http-*" for monitoring HTTP statistics. LDAPStatistics.java: Changed getMonitorData() return type to List. Made createAttribute protected. HTTPStatsProbe.java, HTTPStatistics.java: ADDED

OPENDJ-902 (CR-1654) Add connectionID to the HTTP access log + move to extended log format HTTPConnectionHandler.java, HTTPClientConnection.java, CollectClientConnectionsFilter.java, SdkConnectionAdapter.java: Added support for statistics. 02-config.ldif, config.ldif, HTTPConnectionHandlerConfiguration.xml, HTTPConnectionHandlerCfgDefn.properties: Added "keep-stats" property. 02-config.ldif: Added attributes "ds-mon-http-*" for monitoring HTTP statistics. LDAPStatistics.java: Changed getMonitorData() return type to List. Made createAttribute protected. HTTPStatsProbe.java, HTTPStatistics.java: ADDED

OPENDJ-879 (CR-1642) Add HTTP access log Implemented the HTTP access logger by taking inspiration from other loggers. Configured the new logger everywhere other loggers are configured. Logged the HTTP request in several places to cover all cases (happy paths, errors, etc.). config.ldif, 02-config.ldif, HTTPAccessLogPublisherConfiguration.xml, FileBasedHTTPAccessLogPublisherConfiguration.xml, HTTPAccessLogPublisherCfgDefn.properties, FileBasedHTTPAccessLogPublisherCfgDefn.properties: ADDED Added 2 new objectClasses HTTPAccessLogPublisherConfiguration and FileBasedHTTPAccessLogPublisherConfiguration. config.properties: Added new error messages for the HTTP access logger. HTTPAccessLogger.java, HTTPAccessLogPublisher.java, TextHTTPAccessLogPublisher.java, HTTPRequestInfo.java: ADDED HTTPRequestInfo.log() prevents double logging. CollectClientConnectionsFilter.java: Logged the request info when HttpServletResponse.setStatus(), sendAuthenticationFailure() and onFailure() are called. Pushed more data to the HTTPRequestContext + switched to use the more specific HttpServletRequest/HttpServletResponse SdkConnectionAdapter.java: Logged the request info when close() is called. LoggerConfigManager.java, TestCaseUtils.java: Configured the HTTP access logger. InProcessServerController.java: TO BE REMOVED (by Matt on the native packaging branch) Sample log: localhost bjensen [03/May/2013:10:14:54 +0200] "GET /users/_queryFilter=true&_prettyPrint=true HTTP/1.1" 500 "curl/7.27.0" localhost bjensen [03/May/2013:10:15:05 +0200] "GET /users/_queryFilter=true&_prettyPrint=true HTTP/1.1" 200 "curl/7.27.0" localhost [03/May/2013:10:15:14 +0200] "GET /users/_queryFilter=true&_prettyPrint=true HTTP/1.1" 200 "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:20.0) Gecko/20100101 Firefox/20.0" localhost [03/May/2013:10:16:40 +0200] "GET /users/_queryFilter=true&_prettyPrint=true HTTP/1.1" 401 "curl/7.27.0" localhost [03/May/2013:10:16:50 +0200] "GET /users/_queryFilter=true&_prettyPrint=true HTTP/1.1" 200 "curl/7.27.0" localhost [03/May/2013:10:16:51 +0200] "GET /favicon.ico/null HTTP/1.1" 404 "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:20.0) Gecko/20100101 Firefox/20.0" localhost [03/May/2013:10:17:10 +0200] "GET /users/_queryFilter=true&_prettyPrint=true HTTP/1.1" 200 "curl/7.27.0"

OPENDJ-832 Leverage the work queue for processing requests received on the HTTP connection handler Limited the number of concurrent operations per client connection. SdkConnectionAdapter.java: Changed the queueing strategy to BoundedWorkQueueStrategy. BoundedWorkQueueStrategy.java: ADDED AbandonOperationWrapper.java, xtendedOperationWrapper.java, UnbindOperationWrapper.java: ADDED DirectoryServer.java: Extracted checkCanEnqueueRequest() from enqueueRequest(). Added tryEnqueueRequest(). WorkQueue.java: Pulled computeNumWorkerThreads() here from sub classes. Added abstract method trySubmitOperation(). Added abstract getter getNumWorkerThreads(). ParallelWorkQueue.java: Implemented trySubmitOperation() and getter getNumWorkerThreads(). Renamed getNumWorkerThreads() to computeNumWorkerThreads() + moved it to WorkQueue. TraditionalWorkQueue.java: Implemented trySubmitOperation() and getter getNumWorkerThreads(). Renamed getNumWorkerThreads() to computeNumWorkerThreads() + moved it to WorkQueue. Extacted overloaded method submitOperation(Operation, boolean) from submitOperation(Operation). ClientConnection.java, JmxClientConnection.java, HTTPClientConnection.java, InternalClientConnection.java, MockClientConnection.java, LDAPClientConnection.java: Added abstract method isConnectionValid() + implemented it in subclasses. 02-config.ldif, HTTPConnectionHandlerConfiguration.xml: Added property max-concurrent-ops-per-connection

OPENDJ-830 (CR-1538) Implement authentication and authorization for HTTP connection handler Added the possibility for the HTTP Connection Handler to accept unauthenticated requests. CollectClientConnectionsFilter.java: Extracted method getAuthenticationInfo() and added support for unauthenticated requests here. CollectClientConnectionsFilterTest.java: Added tests. HTTPConnectionHandler.java: Added acceptUnauthenticatedRequests(). config.ldif, 02-config.ldif, HTTPConnectionHandlerConfiguration.xml, HTTPConnectionHandlerCfgDefn.properties: Added property 'ds-cfg-authentication-required' with default 'true' to HTTP Connection Handler

OPENDJ-808 Implement a simple commons REST based HTTP connection handler Added support for using the Rest2LDAP servlet from OpenDJ HTTPConnectionHandler. HTTPConnectionHandler.java: Added support for using the Rest2LDAP servlet + removed the FakeServlet. Added the JSON_MAPPER constant to parse the JSON config file for Rest2LDAP. Added getClientConnectionsMap() getter. Added getConnectionFactory() method, heavily inspired from org.forgerock.opendj.rest2ldap.servlet.Rest2LDAPConnectionFactoryProvider.getConnectionFactory(). In startHttpServer(), silenced Grizzly logging + set the IOThreadStrategy to be single threaded + changed the access path to the servlet. CollectClientConnectionsFilter.java: Added support for using the Rest2LDAP servlet. Removed clientConnections instance member, now get this value from the HTTPConnectionHandler. config.ldif, 02-config.ldif, HTTPConnectionHandlerConfiguration.xml, HTTPConnectionHandlerCfgDefn.properties: Added ds-cfg-config-file property to ds-cfg-http-connection-handler objectClass. http-config.json: ADDED (copied from opendj-rest2ldap-servlet project) ivy.xml: Added dependencies to Rest2LDAP. ivysettings.xml: Added support for updating SNAPSHOTS.

OPENDJ-808 Implement a simple commons REST based HTTP connection handler Added the HTTPConnnectionHandler implemented using Grizzly. Added support for all configuration similar to the LDAPConnectionHandler. Missing support for ds-cfg-keep-stats and ds-cfg-num-request-handlers. config.ldif, 02-config.ldif: Added new entry for HTTP connection handler + defaults. HTTPConnectionHandlerCfgDefn.properties, HTTPConnectionHandlerConfiguration.xml: ADDED LDAPConnectionHandlerConfiguration.xml: Moved properties common to HTTPConnectionHandlerConfiguration.xml to Package.xml . protocol*.properties: Merged protocol messages between LDAP, HTTP and JMX. Made them non LDAP specific. Added 2 new protocol messages for HTTP. CollectClientConnectionsFilter.java, HTTPClientConnection.java, HTTPConnectionHandler.java, package-info.java: ADDED JmxClientConnection.java, JmxConnectionHandler.java, LDAPClientConnection.java: Used common protocol messages. LDAPConnectionHandler.java Used common protocol messages. Extracted method configureSSL() Simplified the code in getEnabledSSLCipherSuites() and getEnabledSSLProtocols() and protected against a multi threaded change ofg the SSL engine by using a local copy.

Fix OPENDJ-510 Add support for PBKDF2 password storage scheme

OPENDJ-473 CR-1252 Implement a task which allows the administrator to forcefully clear an index's degraded status.

Fix for OPENDJ-670. Add an objectclass for the statistics entries under cn=monitor. The schema for this objectclass is also added in 02-config.ldif.

Fix OPENDJ-668 Cannot configure ssl-cipher-suites on admin connector

Fix OPENDJ-665 Attribute Value Password Validator should implement check-substrings

Fix OPENDJ-649: Add supportedTLSCiphers and supportedTLSProtocols to RootDSE and system monitor

Fix OPENDJ-620 Enhance character set password validator to understand classes like 'All non-Latin characters'

Fix OPENDJ-562 Country String syntax should validate ISO 3166 codes Thanks to Manuel Gaupp for this contribution! The default mode for the syntax handler is strict.

Fix OPENDJ-558 Validation for JPEGSyntax values

Fix OPENDJ-520: Worker threads are too greedy when caching memory used for encoding/decoding entries and protocol messages * introduce global option max-internal-buffer-size (default 32KB) for managing the maximum size of internal buffers * use simpler ASN1Writer implementation for LDAP client connections * minor fix to SASL confidentiality and integrity for large packets.

Fix OPENDJ-482: Validation for the CertificateSyntax A huge thanks to Manuel Gaupp for this contribution. In addition to the original contributed change I have made the following alterations: * make the CertificateSyntax strictness configurable. By default the syntax will enforce the syntax, but this can be disabled if needed * add a new method "isHumanReadable" to AttributeSyntax and AttributeType classes in order to make it easier to determine whether or not a syntax/type is human readable. This can be used in order to determine whether or not an attribute value can be included in log messages, for example.

Implementaion of OPENDJ-441 - Operational attribute for password expiration time

Minor fix for OPENDJ-295: Dictionary password validator should verify that a password doesn't contain words, rather than match words. Change the check-substrings property to optional in order to provide backwards compatibility with OpenDJ 2.4.x.

Fix OPENDJ-409: Add support for optimistic concurrency control via ETag-like attributes Implement a new virtual attribute which automatically generates etags based on either the Adler-32 or CRC-32 algorithms.

Fix Issue OPENDJ-393 - OpenDJ config schema is not always well defined...

Fix OPENDJ-383 : Expose JE fileCacheSize property through OpenDJ configuration and admin Added optional ds-cfg-db-log-filecache-size attribute.

Patch for OPENDJ-221

Fix for OPENDJ-295 included an attribute type which already existed. Removing the new duplicate.

Patch for OPENDJ-295

Fix OPENDJ-324: Remove NDB backend implementation

OPENDJ-308: Implement access log filtering and configurable message format Add support for changing the format of the access log: * configurable timestamp format * enable logging of request and response control OIDS * change the format to a "combined" mode where operations are logged using a single log record rather than individual request and response records. Also fixed a bug in the time thread which prevented applications from deregistering time formatters.

OPENDJ-308: Implement access log filtering and configurable message format * Rename client-* properties to connection-* * Remove default behavior aliases since they clutter up the display of a filtering criteria in dsconfig interactive mode * Make protocol matching string based instead of enumeration: this will be forwards compatible with new protocols as they are added * Fix and optimize protocol and port matching.

OPENDJ-308: Implement access log filtering and configurable message format Pull up access log filtering support into abstract base class so that it can be re-used by the audit log.

Preparation work for OPENDJ-308: Implement access log filtering and configurable message format Add objectClass changes which got dropped in the previous commit for some reason.

Preparation work for OPENDJ-308: Implement access log filtering and configurable message format Implement log filtering based on connection, user, request, and response criteria. Also performed minor refactoring of AddressMask class in order to avoid performing unnecessary reverse lookups during log filtering.

Fix OPENDJ-308: Implement access log filtering and configurable message format * split access and audit log configuration, since filtering is not appropriate for the audit log * add initial filtering capabilities to the access log (based on operation type only for now).

Fix OPENDJ-115: Make replication connection timeouts and various monitoring intervals configurable Added connection-timeout property to replication synchronization provider configuration entry which is used for connection and SSL negotiation. Previously the timeout was hard-coded.

Fix OPENDJ-258: Integrate attribute cleanup plugin into OpenDJ (contributed by Profiq S.R.O) Many thanks to Nemanja Lukic and his colleagues at Profiq S.R.O for this kind contribution. The contribution has been modified as follows: *) use the admin framework for validating the attribute renaming mapping *) relax the constraint that attribute names must conform to RFC 4512 since non-compliant attribute names are good candidates for removing and renaming *) modify the rename target attribute validation to support attribute descriptions *) added schema element definitions for configuration attributes

Issue OPENDJ-262: Implement pass through authentication (PTA) Implement LDAP PTA password caching and enable tests.

Issue OPENDJ-262: Implement pass through authentication (PTA) * drop support for cached-password-min-age * rename cached-password-max-age to cached-password-ttl * add unit tests (disabled until feature implemented).

Issue OPENDJ-262: Implement pass through authentication (PTA) Add new properties for configuring password caching.

Issue OPENDJ-262: Implement pass through authentication (PTA) Add additional methods for configuring the mapped searched bind password (property, env, and file), similar to file based key and trust managers.

OPENDJ-262: Implement pass through authentication (PTA) Add configuration options for SSL, timeouts, and advanced TCP settings.

Minor refactoring work for OPENDJ-262: Implement pass through authentication (PTA) Add initial configuration and factory stubs.

Initial checkpoint of work for OPENDJ-262: Implement pass through authentication (PTA) Refactor PasswordPolicy implementation so that we now have a common parent type, "AuthenticatioPolicy", which will act as a common entry point for all authentication policies. In particular, password policies can be thought of as a specific type of authentication policy: one where authentication and the user's credentials are managed locally. PTA can be thought of as a non-local authentication policy: the password is managed by a third party. With this change we still only have one type of authentication policy and no attempt has been made to refactor code which uses it. More specifically, code such as the local backend WFE just type-casts the authentication policy to a password policy. Another implication of this change is that sub-entry based password policies now automatically inherit run-time changes made to the default password policy, whereas before a server restart was required.

Fix OPEND-250: Add an option to the JE backend for disabling the subordinate indexes id2children and id2subtree Add a boolean option "subordinate-indexes-enabled" to the JE backend configuration which can be used for disabling the subordinate indexes id2children and id2subtree.

Fix OPENDJ-247: Rename max-entries property in JE backend to something more clearly related to index analysis Renamed property to index-filter-analyzer-max-filters.

Fix OPENDJ-233: Integrate Samba password sync plugin into OpenDJ (contributed by Profiq S.R.O) Many thanks to Nemanja Lukic and his colleagues at Profiq S.R.O for this kind contribution. The contribution has been modified as follows: *) integrate as core feature of OpenDJ, not extension *) rename package names accordingly *) remove dependency on BouncyCastle.org MD4 implementation, by porting implementation to OpenDJ *) re-implement setOddParity method in order to avoid potential copyright issues with existing implementations. *) simplify configuration model and validation. TODO: *) support u: and dn: authid in password modify extended operation.

Partial fix for OPENDJ-194: Minor improvements to change log content and configuration Commit main functionality: * add support for wild-cards, and @objectClass notiation * add support for configuring additional attributes to be included with delete operations via ecl-include-for-deletes property. Requires protocol version upgrade which will be handled in final commit.

Fix OPENDJ-168: Enhance character set password validator to support optional character sets

Fix OPENDJ-112 - The Changelog virtual attribute appears in al entries, should only apply to rootDSE. The changes introduce a scope in the VirtualAttributeRule. The default is Whole_Subtree. The changelog virtual attributes are now registered with a BASE_OBJECT scope and "" base DNs, meaning the rootDSE only. Also each virtual attribute provider override the hasValue() method for optimization and avoiding computing the values and throwing them away.

Resolves issue OPENDJ-11 - Update the Berkeley DB Java Edition 4.1.6. Introduce 3 new parameters to tune the new DB-Cache evictor functionality

Update copyright for changed file.

Resolves Enhancement request OpenDJ-5: Support Linux md5 crypt storage for password This changes are adding support for the BSD MD5 crypt hash as part of the CRYPT password storage scheme. A new parameter has been added to the configuration of the storage scheme to select whether new passwords should be hashed with the unix algo (default) or the md5 one. When it comes to authentication, the scheme is able to detect the algo (based on the $1$ prefix) and match appropriately. Unit tests have been added, including test again passwords already hashed on Linux systems.

Refactor configuration of maximum persistent search resource limits

Implement a new resource limit policy which restricts the number of operations which can be performed during a time interval

First pass on solving issue #514, reducing Replication meta data, especially aged information. The default is to keep the replication meta data (historical information) for at least 1 day. Purging occurs on the fly when entries are modified, or via a task. Launching the task will be available through dsreplication in separate commit.

Implements a configurable limit in the number of persistent searches a server can handle.

Implements index analysis features. There are 2 components: Index Filter Analyzer : Part of DatabaseEnvironmentProvider, it gathers search filter statistics and displays the index filter, number of hits, max matching entries, and a message. Compound filters are broken down to their basic elements. f an index was not utilized while evaluating the search filter, max matching entries will be -1 and a diagnostic message will be included. The monitor entry also includes when the analyzer was enabled along with the number of total index and unindexed searches processed. Attribute Index Analysis, part of the dbtest tool. It cursors through all the indexes and finds the number of 100%, 95%, 90%, and 80% undefined index keys. It also displays the value of the undefined keys along with the total number of keys in the index. The process is currently single threaded and might not scale to 10 mil+ DBs. However, this is currently unavoidable without the ability for JE to cursor through the records in log order.

Implements a disk space thresholds feature, preventing the server from crashing or exiting of disks full. Each instance of the monitor have two thresholds: "low" and "full". Other components can register their own handlers if they wish to receive notifications when the thresholds are reached. It extends the MonitorProvider interface and utilizes the update interval mechanism to poll the free disk space. The current free space and state of all monitor instances are exposed through the cn=monitor interface under the cn=Disk Space Monitor branch. The only component using the monitor now is the JE backend. When the "low" threshold is reached, write operations will only be permitted to users with the BYPASS_LOCKDOWN privilege. When the "full" threshold is reached, the backend is placed in read-only mode. The default "low" and "full" thresholds are 100 MB and 10MB respectively. Along with the new feature, this patch also added two privileges: SERVER_LOCKDOWN and BYPASS_LOCKDOWN. Any user with the SERVER_LOCKDOWN privilege can put and take the server out of lockdown mode. Any user with the BYPASS_LOCKDOWN mode can bypass lockdown mode. This was restricted to only "root" users before. In addition, the MonitorProvider abstract class no longer extends DirectoryThread. It now uses a static single threaded ScheduledExecutorService to update provider states. Any provider that wishes to have its state updated periodically needs to register its updater runnable with the scheduleUpdate method.

Fixes issues #4552 #4557, making sure plugins and internal services are properly handling subtree move or delete. The changes particularly resolve problems raised by the community with the referential integrity and the isMemberOf plug-ins. Unit-tests have been updated to cover those cases

Fix issue #3404. Conflicting entries are now indexed by ds-sync-conflict attribute. This index is only updated when conflicts are detected and searched on for all deletes and modDN operations. Tested successfully against performance regression.

Fix an issue with the replication repair commands. The control was looked at after some checks it was supposed to skip. This prevented some entries to be repared. The changes are also updating the unit tests making sure the Replication Repair control is properly tested.

Add support for displaying replication CSNs in the Access Logs. The default is disabled, but the log of replication CSNs can be activated by setting the log-changenumber property in the Replication domain configuration.

Use cn as naming attribute for the WorkFlow and WorkFlowElements like other objects in OpenDS configuration

Fix for issues #3395 and #3998. The changes improves the replica initialization protocol, especially flow control and handling connection outage.

Fix for issue #4522 - ds-sync-hist attribute now has OctetString syntax to handle any kind of content.

Last batch of changes for this week. This adds support for the IETF based Password Policy for LDAP as SubEntry. Also resolves the following issues : - 4544 : initializeBackend() should not set JE env config params directly. - 4478 : ECL in draft compat mode / search lastchangenumber can be very long - 4538 : Virtual attributes not retrieved when entry cache configured - 4547 : Search Filter Matching differ for cn=Directory Manager and plain user. - 4514 : Logs shows unexpected message with replication monitoring data missing (Partial fix) - 4534 : Replication using security does not work after server restart - 4516 : SEVERE_ERROR: servers (...) have the same ServerId In addition, they also improve reliability and performance in various areas including CollectiveAttributes, Virtual Attributes and Subentries management, Schema loading, Replication...

- land Subentry Manager and Collective Attributes implementations; Merry XMAS to yall!

Fix#4262 ECL One should be able to disable ECL for each domain

In order to support a more clever algorithm for the DS to choose his RS, we introduce: - a weigth, which is an integer affected to each RS that combined with each others will define a percentage value which matches the number of DSs (compared with total number od DSs in the topology) that can be connected to the RS at a time in the topology. In these modif, this configuration of the weight is added as well as dynamic changes. Also transported in Topo messages. No modification of the connection algorithm yet - Also to support the future connection algorithm, these modifs introduces a Monitoring Publisher thread which is a thread that sens every 3 seconds a Monitoring message (unchanged format) to every DSs that are connected to him. These information will be used by the DSs to potentially reconnect to another RSs with a newer server state (info included in monitoring messages) The new connection algorithm will take into account: - group id - generation id - server states - locality (same VM) - weight (load)

- EXPERIMENTAL Parallel Work Queue implementation.

- CMT improvements to parallel request and response processing.

- Addition of ReplServerStartDSMsg now sent to a DS connecting to a RS in handshake phase instead of a ReplServerStartMsg. ReplServerStartDSMsg contains same thing as ReplServerStartMsg but also contains - replication server weight - number of ciurrently connected DS on the RS => both will be used for future new RS choice algorithm - Addition of a StopMsg sent: - when any entity (DS,RS) is closing a connection (sent just before) with a peer - when DS finishes phase 1 of handshake (was gathering RS info for RS choice so sent just after new ReplServerStartDSMsg is received) => both are used to distinguish between a proper connection closure (no message) and an unexpected one (error log) - Compatibility between protocol V4 and V3 (and before) - changed MonitorMsg to never be created with a protocol version - MonitorMsg now always sent with publish(msg, version) (publish method without version was used so bug) - TopologyMsg now always sent with publish(msg, version) (publish method without version was used so bug)

Entry attributes for ECL - Protocol V4

Rebuild index using new import engine.

Fix for issue 4239 setup on Windows 2008 set LDAP ds-cfg-listen-port to 389

Fix for #4098 Initializing a fractional domain through import-ldif when server is down does not work Fractional LDIF import plugin is now able to read fractional configuration from config backend if server is offline. => Initializing a fractional replica is possible even if server is offline.

Fix for issue 4178: Only base entry visible after on-line import

Some unit test fixes. Also, comment a unit test out until I figure out what the problem is.

These changes allow import-ldif to support multiple suffixes and fix some problems with the include/exclude options.

Support for External change log compatible with draft-good-ldap-changelog-04.txt , March 2003

Fix for 4169 : Ability to disable generation of historical information and 4170 : Replication should not generate HistoricalCsnOrdering index for replayed operation

Commit of new import code.

Fractional replication Info about the feature: https://www.opends.org/wiki/page/FractionalReplication Note: After configuration of a fractional domain, the domain goes into bad gen id status and must be initialized (to filter for configured attributes) through online full update or ldif import (limitation: for the moment only with server running) from a non fractional domain of the topology.

Issue 4075 : Provide implementations for the relative and partial time matching rules

issue 3961:Implement operational attribute 'governingstructurerule

Change aci and ds-cfg-global-aci equality matching rule to octetStringMatch instead of CaseIgnoreIA5EqualityMatchingRule. Issue 3312.

Fix for issue 3886 (nline import-ldif fails to use the skipfile option). Add a missing attribute in the schema for the online import task.

- simplify index configuration and add some configuration placeholders for future use.

Fix 3690 : SNMP: incorrect value for dsServerDescription entry Fix 3865 : Missing information for JMX and SNMP connection handlers in status output

Fix issue 3831 - add dsconfig support for NullBackend

Fix issue 3832 - Provide ergonomics style settings for common server properties

- land NDB Backend implementation.

Fix issue 3734 - Make network group policies extensible. See issue tracker for more details.

Integrating the changes related to Collation indexing support

- Assured replication (Safe Read) unit tests complete - Assured replication config correction - typos - enable less tests in precommit

fix issue with affinity-timeout

fix OIDs for affinity policy and timeout

add support of the client connection affinity

Integerated first part of the Fix for Issue# 262:Plugin for Collation/Internationalization

- [Issue 274] Recurring Tasks

* Ability to create monitor provider with hierarchical naming * Modification of Network Monitor provider names (includes hierarchical naming) * Provide basic Monitoring objects. * Add monitoring instrumentation for the connection handlers. * Add the ability to do a subtree search in the monitor backend.

These fixes add confidentiality/integrity to the SASL GSSAPI and DIGEST-MD5 mechanisms. The issue links: https://opends.dev.java.net/issues/show_bug.cgi?id=346 https://opends.dev.java.net/issues/show_bug.cgi?id=347 https://opends.dev.java.net/issues/show_bug.cgi?id=349 https://opends.dev.java.net/issues/show_bug.cgi?id=350 This page describes the changes: https://www.opends.org/wiki/page/SASLConnectionSecurityPhase1

The commit will bring the following features : - An updated version of the underlying database. BDB JE 3.3 is now used. - Attribute API refactoring providing a better abstraction and offering improved performances. - A new GUI called the Control-Panel to replace the Status-Panel: the specifications for this GUI are available on OpenDS Wiki and contains a link to a mockup. See <https://www.opends.org/wiki/page/ControlPanelUISpecification>. - Some changes in the replication protocol to implement "Assured Replication Mode". The specifications are on OpenDS Wiki at <https://www.opends.org/wiki/page/AssuredMode> and section 7 described some of the replication changes required to support this. Assured Replication is not finished, but the main replication protocol changes to support it are done. As explained by Gilles on an email on the Dev mailing list (http://markmail.org/message/46rgo3meq3vriy4a), with these changes the newer versions of OpenDS may not be able to replicate with OpenDS 1.0 instances. - Support for Service Tags on the platforms where the functionality is available and enabled. Specifications are published at <https://www.opends.org/wiki/page/OpenDSServiceTagEnabled>. For more information on Service Tags see <http://wikis.sun.com/display/ServiceTag/Sun+Service+Tag+FAQ>. - The Admin Connector service. In order to provide agentry of the OpenDS server at any time, a new service has been added, dedicated to the administration, configuration and monitoring of the server. An overview of the Admin Connector service and it's use is available on the OpenDS wiki <https://www.opends.org/wiki/page/ManagingAdministrationTrafficToTheServer> - Updates to the various command line tools to support the Admin Connector service. - Some internal re-architecting of the server to put the foundation of future developments such as virtual directory services. The new NetworkGroups and WorkFlow internal services which have been specified in <https://www.opends.org/wiki/page/BasicOperationRoutingThroughNetworkGroup> are now implemented. - Many bug fixes... This set of changes may have an impact on current clients and deployments, however it should be possible to upgrade existing 1.0 instances. Some of the changes impacting the clients include : - Scripts using the command line tools including dsconfig will need to be updated (usually to remove some no longer required options).

New top level object in RootConfiguration.xml : "extension", that allows to create a configuration object for an extension, that does not inherit from an existing configuration object.

Fix for issues 3255, 3265, and 3269: Issue 3255: Added ds-cfg-max-allowed-client-connections configuration parameter to prevent using up all file descriptors. By default the server will still allow unlimited number of connections just like before. The user will have to adjust this for their system. This might be automatically set for the OS in the future. Issue 3265: Subtree deletes and mod DN operations will now always use a JE transaction throughout the entire operation. This makes sure the operations are always executed atomically. Issue 3269: Added checks for canceled operations in JE backend code.

Fix for issue 3311 (import-ldif --templateFile does not work when the server is running) The fix consists on not assuming that the ldif file path is required to launch an import (the ds-task-import task definition has been modified for this) and allowing to create a task that imports contents based on a template.

Renumbered OID for the ds-task-export-include-operational-attributes AttributeType.

Fixed an issue where a NPE could be thrown when the server processes a mod DN operation with a new superior entry thats in the subtree to be moved. An informative error message is added so the user can fix the mistake. This patch also enables the debug logger to print the cause of an exception message as well as all the stack frames by default. It also removes the deadlock retry configuration parameter from the JE backend configuration definition since deadlocks can no longer occur in the JE backend. Fix for issue 3236

Fixes : 3143: snmp get requests 3141: snmp status information 3146: snmp unused configuration parameters

Fix for issue #2270 (export-ldif doesn't exclude operational attributes when it's a task) Add the following attribute into 'ds-task-export' attributeTypes: ( 1.3.6.1.4.1.26027.1.1.464 NAME 'ds-task-export-include-operational-attributes' SYNTAX 1.3.6.1.4.1.1466.115.121.1.7 SINGLE-VALUE X-ORIGIN 'OpenDS Directory Server' ) When export-ldif is ran as a task, we add the corresponding attribute value in the created task entry if the --excludeOperational is present

* Fix for Issue 218 : Monitor Information Available via SNMP * Update the nbproject to include the snmp source code

These changes remove the temporary file limitation from import-ldif. Several other changes were made also: - the cleaner is run at the end of the import - the cleaner is run periodically during import if database eviction is detected - the substring indexes are buffered to help boost performance during substring index processing - the import files have been moved into its own package org.opends.server.backends.jeb.importLDIF - the work threads do most of the processing - import aborts if a work thread throws a runtime exception - messages for the various stages of the import have been added (e.g. environment close) The only functionality missing is VLV index processing. Also, there is a 2G limit on the max entry ID value that can be used in a substring index. The following configuration attributes have been removed: - ds-cfg-import-temp-directory - ds-cfg-import-buffer-size - ds-cfg-import-pass-size The should be removed from your config.ldif file.

Sorry, fixed mistake by previous commit

Changed build repository for Java WebStart script

Updated the copyright statement to reflect that Sun owns the full copyright on the project files.

- rename and move [entry-]cache-preload from default entry cache configuration to global configuration. - abandon default entry cache configuration and objectclass which can be resurrected later if more related configuration attributes come in. - minor cleanup to preload messages.

Fix for 1288: Synchronization plugin should not create 10 listener threads for each baseDn

This fix removes the option to use nanosecond etime resolution as a JVM property. It is now a global configuration attribute. Due to the 10% throughput decrease I saw when using System.nanoTime, the default still uses System.currentTimeMillis to timestamp operations. System.nanoTime will only be used when the ds-cfg-etime-resolution attribute is set to nano-seconds. Fix for issue 2499

- [Issue 1595] need a fast way to prime the FileSystemEntryCache: Add generic entry cache pre-load mechanism which is backend and entry cache implementations independent.

- Use properly allocated OID for ds-cfg-cache-level instead of dummy.

- [Issue 2007] generic mechanism for using mulitple caches. - [Issue 2049] dsconfig is unable to configure the entry cache.

Fix 2424 - impact of generation ID on binary copy - add to the SetGenerationID task the possibility to clear the generation id in the topology by providing the attribute ds-task-reset-generation-id-new-value with value -1

s set of files provides the workflow configuration manual mode. Until now, the workflows were automatically configured-a wokflow was created for each base DN in the backends. When new suffixes were added or when a backend was added, the associated workflows were also created (and simillarly workflows were deleted as suffixes or backends were removed). With the manual mode, each and every workflow in the server must be defined explicitely in the configuration. By default, the server is running in automatic configuration mode. To have a server running with manual configuration mode one must set the attribute in cn=config: dn: cn=config ... ds-cfg-workflow-configuration-mode: auto|manual No attribute means "auto" mode. The workflow configuration consist of 3 parts: - the configuration of workfow elements - the configuration of workfows - the configuration of network groups The Workflow Elements - A workflow element is a basic task in a workflow. The workflow elements are organized in trees and the simplest tree is made of one element. For example, the workflow element that wraps a local backend is configured as follow: dn: ds-cfg-workflow-element-id=userRoot,cn=workflow elements,cn=config objectClass: top objectClass: ds-cfg-workflow-element objectClass: ds-cfg-local-backend-workflow-element ds-cfg-workflow-element-id: userRoot ds-cfg-enabled: true ds-cfg-java-class: org.opends.server.workflowelement.localbackend.LocalBackendWorkflowElement ds-cfg-backend: ds-cfg-backend-id=userRoot,cn=Backends,cn=config From an admin standpoint, the local backend workflow element is an aggregation of a single backend (attribute ds-cfg-backend). So we cannot disable/delete a backend as long as it is used by a local backend workflow element. The Workflows - A workflow is a chain of processing and it's targeting all the entries under a given baseDN. The processing is actually identified by the root node of the task tree described above. The configuration of a workflow looks like: dn: ds-cfg-workflow-id=userRoot,cn=workflows,cn=config objectClass: top objectClass: ds-cfg-workflow ds-cfg-workflow-id: userRoot ds-cfg-enabled: true ds-cfg-workflow-element: ds-cfg-workflow-element-id=userRoot,cn=workflow elements,cn=config ds-cfg-base-dn: dc=example,dc=com From an admin standpoint, the local workflow is an aggregation of a single elements (attribute ds-cfg-workflow-element). So we cannot disable/delete a workflow element as long as it is used by a local workflow. The Network Groups - A network group defines categories for client connection. The network group contains a set of workflows and each client operation is routed to one (or more) workflow(s). By default, the server create a default network group which contains all the workflows defined in the server. The default network group looks like: dn: ds-cfg-id=defaultNetworkGroup2,cn=network groups,cn=config objectClass: top objectClass: ds-cfg-network-group ds-cfg-id: defaultNetworkGroup2 ds-cfg-enabled: true ds-cfg-workflow: ds-cfg-workflow-id=adminRoot,cn=Workflows,cn=config ds-cfg-workflow: ds-cfg-workflow-id=ads-truststore,cn=Workflows,cn=config ds-cfg-workflow: ds-cfg-workflow-id=backup,cn=Workflows,cn=config ds-cfg-workflow: ds-cfg-workflow-id=config,cn=Workflows,cn=config ds-cfg-workflow: ds-cfg-workflow-id=monitor,cn=Workflows,cn=config ds-cfg-workflow: ds-cfg-workflow-id=schema,cn=Workflows,cn=config ds-cfg-workflow: ds-cfg-workflow-id=tasks,cn=Workflows,cn=config ds-cfg-workflow: ds-cfg-workflow-id=userRoot,cn=Workflows,cn=config From an admin standpoint, the network group is an aggregation of several workflows (attribute ds-cfg-workflow). So we cannot disable/delete a workflow as long as it is used by a network group. A unit test named WorkflowConfigurationTest tests the configuration of network groups, workflows and workflow elements.

Fix for issue 2505 (Provide monitoring information about the number of entries on a given base DN) The following changes are aimed to include a new attribute in the backend monitor entry to provide the number or entries per base DN. This information is used by the status, status-panel, dsreplication and graphical setup to be able to display the number of entries under a given base DN even when there are several base DNs defined in the same database. The way the new attribute appears in the monitoring entry is: ldapsearch -w s -p 1389 -b cn=monitor ds-backend-id=userRoot ds-base-dn-entry-countdn: cn=userRoot Backend,cn=monitor ds-base-dn-entry-count: 0 dc=ta ds-base-dn-entry-count: 1 dc=ti ds-base-dn-entry-count: 1 dc=to I prefer this approach because in my opinion having the number of entries is more important that the suffix. Anyway I think the discussion around the structure of the value is not crucial as long as the value is easy to parse (which is the case). The new attribute is defined in the schema and the new OID assigned to it is 1.3.6.1.4.1.26027.1.1.434. In order to have a minimum impact on performances, the method backend.numSubordinates is only called when there are several base DNs defined in the database.

Add configuration properties for the crypto manager's preferred digest algorithm, its preferred cipher and mac algorithms and the associated key lengths.

More changes for issue 466. - Defines the schema for secret keys. - Keeps the crypto manager secret key cache up to date with secret keys published in ADS.

Update the 02-config.ldif schema configuration file to compact the OID allocations after the earlier refactoring to use a more consistent naming for our configuration properties. I have also updated the OID registry on our wiki to reflect these changes.

Update the Get Symmetric Key extended operation handler so that it has its own object class and config definition rather than relying on the generic configuration. This makes it easier to use through dsconfig.

Merge branches/temp-admin@3208 onto trunk@3208.

Update the backend API to include new isIndexed methods that can be used to determine whether a given attribute is indexed in a specified manner, or that can be used to determine whether a specified filter is indexed. At present, all backends except the JE backend and the replication backend are considered always indexed for all operations. The JE backend is only considered indexed based on its index configuration. The replication backend is never considered indexed. Update the following components to make use of this new isIndexed capability: - The DSEE-compatible access control handler will now log a warning message at startup if it detects that there is no presence index for the aci attribute, which can make startup take a long time on a big database. - The group manager will now log a warning message at startup if any of the group implementation filters are unindexed, which can make startup take a long time on a big database. - The referential integrity plugin now requires that all of the attributes for which referential integrity is to be maintained must be configured with equality indexes. - The unique attribute plugin now requires that all of the attributes for which uniqueness is to be enforced must be configured with equality indexes. This commit also updates the LDIF backend so that it is possible to indicate via configuration whether its base DNs should be registered as public or private base DNs. The LDIF backend used as the admin root has been updated so that it is considered a private backend. The replication backend has also been updated so that it is considered a private backend.

Add support for an LDIF backend. Entries in this backend will be held in memory, and all read operations will be served from memory, but the underlying data will be stored in an LDIF file on disk and any write operation will cause that LDIF file to be updated. This backend supports all major operations, including moving/renaming non-leaf entries with the modify DN operation, and also supports the subtree delete control and LDIF import and export operations. Backup and restore operations are not currently supported.

Correct the schema definitions so that we use attribute names of ds-cfg-plugin-order-post-synchronization-* rather than ds-cfg-plugin-order-synchronization-*. OpenDS Issue Number: 2308

Add a new monitor provider that can be used to monitor memory usage within the JVM, including: - The total number of garbage collections (per memory pool) - The total duration of all garbage collections (per memory pool) - The averge duration for a garbage collection (per memory pool) - The duration of the most recent garbage collection (per memory pool) - The current size of each memory region - The size of each memory region after the last garbage collection

Add support for password storage schemes using AES, 3DES, RC4, and Blowfish. The AES, RC4, and Blowfish implementations all use 128-bit ciphers, and the 3DES implementation uses a 168-bit cipher. Note that while these password storage schemes are functional, they rely on the crypto manager, which is not fully implemented. The storage schemes are not exposed in the server configuration because the crypto manager does not have any mechanism to persist secret keys for symmetric encryption. Until the crypto manager provides persistence for these keys, passwords encoded using these schemes will not be usable after the server is restarted. Once the crypto manager implementation is complete, these schemes should be exposed in the server configuration. OpenDS Issue Numbers: 315, 316, 317, 318

Update the server to provide a basic framework for controlling when plugins will be invoked. There are two basic changes: - Add a new ds-cfg-invoke-for-internal-operations configuration attribute for all plugins, which indicates whether the plugin should be invoked for internal operations. If this is false, then the plugin will only be invoked for externally-requested operations. - Add four new plugin types: postSynchronizationAdd, postSynchronizationDelete, postSynchronizationModify, and postSynchronizationModifyDN. These allow a plugin to perform a limited set of processing for changes that are successfully applied through synchronization. The unique attribute plugin has also been updated to support the post-synchronization plugin types so that if a conflict is introduced concurrently on two different servers within the propagation delay, an administrative alert will be generated to indicate that manual intervention is required to address the problem. Finally, ensure that audit logging is enabled during the unit tests, and update the audit logger to include the connection ID and operation ID for the operation being logged. OpenDS Issue Number: 2057

- [Issue 1594] entry caches should have monitor information: implement generic entry cache monitor provider which allows any entry cache implementation current or future to provide a common or custom set of entry cache state data. update all existing entry cache implementations so that they provide their common state data to the entry cache monitor provider.

Update password storage scheme references in the server so that they use DNs rather than storage scheme names. This will allow better consistency in the configuration, since all other references between configuration objects are DN-based, and it will work better with the upcoming aggregation support. It also eliminates the need to know the storage scheme name, which is not obvious from looking at the configuration entry for the storage scheme, and can actually vary in some implementations depending on whether it's used with a user password or auth password syntax attribute. OpenDS Issue Number: 2155

Add a new LDIF connection handler, which can be used to watch for new LDIF files to be created in a directory, at which point the changes contained in those LDIF files will be processed by the server using internal operations. This can potentially be used to make configuration changes or invoke tasks if the server is unresponsive to external clients (e.g., all worker threads are busy). Note that in the default configuration, the LDIF connection handler is enabled but the target directory doesn't exist. In this configuration, it will simply not do anything until the target directory is created and one or more LDIF files are placed in it. This is a safeguard against an administrator inadvertently placing a file in that directory without understanding the potential consequences of that action. It also means that we can use it to perform debugging operations (e.g., "dump all") without having to restart the server if it becomes unresponsive. OpenDS Issue Number: 2216

- remove ds-cfg-je-database which is now obsolete.

Add a 7-bit clean plugin, which can be used to ensure that values for a specified set of attributes (optionally only for entries below a specified set of base DNs) are 7-bit clean. OpenDS Issue Number: 261

Implement support for a maximum blocked write time limit in the LDAP connection handler, which can be used to terminate client connections if an attempt to write data to the client has been blocked for too long. This will generally occur if the client has become unresponsive or there is a network outage. OpenDS Issue Number: 139

Commit plugin for maintaining referential integrity. Issue 257. New configuration attributes: - ds-cfg-referential-integrity-attribute-type Specify attribute types that referential integrity will be checked on; this is a mandatory attribute -ds-cfg-referential-integrity-base-dn Specify base DN that will limit scope of reference check; if not specified the server's public naming contexts are used - ds-cfg-referential-integrity-update-interval Specify update interval for background referential integrity processing; if update interval > 0 plugin performs background processing; default is 0 -ds-cfg-referential-integrity-log-file Specify log file location for update records when background processing is enabled; default is <instance>/logs/referint The plugin is disabled by default: dn: cn=Referential Integrity,cn=Plugins,cn=config objectClass: top objectClass: ds-cfg-plugin objectClass: ds-cfg-referential-integrity-plugin cn: Referential Integrity ds-cfg-plugin-class: org.opends.server.plugins.ReferentialIntegrityPlugin ds-cfg-plugin-enabled: false ds-cfg-plugin-type: postOperationDelete ds-cfg-plugin-type: postOperationModifyDN ds-cfg-plugin-type: subordinateModifyDN ds-cfg-referential-integrity-attribute-type: member ds-cfg-referential-integrity-attribute-type: uniqueMember

fix for #1733 & #845 - Initialization of replication

- [Issue 1481] Ability to set any JE property in config.ldif - [Issue 2031] provide core JE configurable attributes for FileSystemEntryCache - fixes to entry cache config manager for proper configuration, re-configuration, error reporting.

Update the server so that it has the ability to save a copy of its current configuration into a ".startok" file whenever it starts successfully. The start-ds script and DirectoryEnvironmentConfig class have been updated to expose an option to try to start the server using this "last known good" configuration rather than the active config file. OpenDS Issue Number: 1945

Issue 466 partial Update schema and config files with OIDs issued by Neil.

Create configuration definitions for "generic" objects in the server configuration. This also includes allocating object classes for each of these types. This will help make it easier for users to create new instances of these kinds of configuration objects because they will not need to know the fully-qualified name of the Java class that implements the associated logic.

Fix conflict from previous commit

Changes for replication security issues 511, 512, 608. There is a new backend representing the certificate trust store, which allows the setup code to query and populate the trust store over protocol. However, we are using blind trust until that piece of the setup code is ready. The encryption settings are currently global to the server instance in a new crypto manager config entry. Authentication will always be performed so the main setting is whether to use encryption or not. In the future (post 1.0) we will need to allow encryption to be configured on or off depending on which replication server we are connecting to but we need some discussion on the best way to specify that in configuration.

Add an SMTP account status notification handler, which can be configured to send e-mail messages whenever an account status notification is generated. The message can be sent to the end user impacted by the notification (based on an attribute in the user's entry) and/or a fixed set of recipients. The messages that will be generated are created from template files, which can include tokens that will be replced with things like: - The name of the notification type - The notification message - The DN of the target user's entry - Attribute values from the target user's entry - Values of account status notification properties, which may vary based on the type of notification This change also includes a fix for a problem that could allow password expiration warning messages to be sent to a client even if the bind was not successful. OpenDS Issue Number: 581

Change behavior of ds-cfg-unique-attribute-type attribute so that it is required (mandatory).

- [Issue 2030] new configuration attribute to enforce entry compression: allow for backend-compact-encoding property, enabled by default.

Added the JE backend configuration attribute to enable or disable compression when encoding entries for ID2Entry. When backend-compact-encoding is enabled, attribute type descriptions and object class sets will be encoded using a compressed form. This behavior is enabled by default. Fix for issue 2135

Add attribute uniqueness plugin implementation that provides single-server attribute uniqueness. The plugin has the following features: - provides ability to specify a group of attribute types that must have unique values; if no attribute types are specified then the plugin allows the operations to proceed with no checking - provides ability to specify a set of base DNs that limit the scope of the uniqueness checking; if no base DNs are specified the server's public naming contexts are used - allow changing of these configuration options without server restart - allows the uniqueness checking to span multiple base DNs; if the server's public naming contexts are used, then the specified attribute type values must be globally unique within the server Two configuration attributes have been added: 1. ds-cfg-unique-attribute-type used to specify the unique attribute type(s) 2. ds-cfg-unique-attribute-base-dn used specify the base DN(s) to limit the search scope A disabled plugin configuration has been added to the config.ldif file for the uid attribute: dn: cn=UID Unique Attribute ,cn=Plugins,cn=config objectClass: top objectClass: ds-cfg-plugin objectClass: ds-cfg-unique-attribute-plugin cn: UID Unique Attribute ds-cfg-plugin-class: org.opends.server.plugins.UniqueAttributePlugin ds-cfg-plugin-enabled: false ds-cfg-plugin-type: preOperationAdd ds-cfg-plugin-type: preOperationModify ds-cfg-plugin-type: preOperationModifyDN ds-cfg-unique-attribute-type: uid Issue 258.

Implement support for an identity mapper that can use regular expressions to transform the provided ID string before searching for the appropriate matching user in the server. This makes it possible, for example, to strip the realm portion of a Kerberos V principal to obtain just the username before searching for entries containing a uid attribute with that value. OpenDS Issue Number: 2087

Add support for a new type of plugin which can be used to detect changes and take some action whenever a subordinate entry is modified as a result of a modify DN operation that targets an entry that has one or more children (i.e., a subtree move or subtree rename operation). At present, subordinate modify DN plugins are not allowed to change the contents of the entry as it is being moved/renamed, but an appropriate API is in place if we decide to add this functionality in the future. This commit also includes a significant amount of cleanup for plugin result code, including corrections to copy-and-paste errors in the javadoc documentation. OpenDS Issue Number: 752

Update the task backend to provide a mechanism for sending e-mail messages to notify administrators whenever a given task has been completed. It is possible to specify a set of administrators that should be notified only if the task does not complete successfully, and/or a set of administrators that should be notified regardless of the task's success or failure. The basic framework for this capability has always been in place, and this change only provides the final implementation that actually generates and sends the e-mail message. This also includes a change to the way that log messages are handled so that any messages logged by a thread executing a task (or any DirectoryThread subclass created by a task thread) will be captured as part of that task. These messages will also be included in the notification e-mail message. OpenDS Issue Numbers: 2033, 2034

Update the server to provide an idle time limit configuration option that can be used to terminate client connections that have been idle for too long. This can be controlled on a server-wide default level using the ds-cfg-idle-time-limit configuration attribute in the cn=config entry, but it can also be overridden on a per-user level with the ds-rlim-idle-time-limit operational attribute in the user's entry. Note that while the idle time limit support is server wide, it needs help from the connection handler to provide an idle time for client connections. At this time, only the LDAP connection handler provides this capability. It is not appropriate for internal connections, and it is not seen as important for JMX connections at this time (although such support could be added in the future if the need arises). OpenDS Issue Number: 118

Provide valid OIDs for the ds-cfg-ssl-protocol and ds-cfg-ssl-cipher-suite configuration attributes and define them in the server schema, and make the names singular instead of plural. Also, update some of the configuration definitions to align the OIDs used in them with the OIDs used in the server schema.

Add two new configuration attributes, ds-cfg-enabled-alert-type and ds-cfg-disabled-alert-type to alert handler configuration entries. If a set of enabled alert types is given, then only alerts with one of those types will be passed to the associated alert handler. If a set of disabled alert types is given, then only alerts without one of those types will be passed to the associated alert handler. If both enabled and disabled lists are provided, then only alerts with a type on the enabled list and not on the disabled list will be processed. OpenDS Issue Number: 2027

Provide a configuration option that makes it possible for the administrator to control whether responses to failed bind operations include an error message that explains the problem. By default, failed bind operations will return a response that indicates the operation failed, but will not include any error message (although it will be written to the access log). However, if the administrator decides that the benefit of providing this information to the client outweighs any potential security risk, then this option may be used to include that message in the response to the client as well as having it written to the access log. OpenDS Issue Number: 1787

Provide a mechanism to disable privileges in the server if necessary. If a privilege is disabled, then the server will behave as if all users have that privilege. This can help improve compatibility with environments that expect a feature to always be available, or to only be governed by access control. OpenDS Issue Number: 1683

Implement support for restricting the set of tasks that can be invoked in the server. A new configuration attribute, ds-cfg-allowed-task, is now available in the cn=config entry, and any attempt to invoke a task whose fully-qualified class name is not included in this attribute will be rejected. OpenDS Issue Number: 2024

Add support for a new disconnect client task that can be used to allow an administrator to terminate a client connection if the need arises. The requester must have the disconnect-client privilege. The task entry should contain the ds-task-disconnect object class, which requires the ds-task-disconnect-connection-id attribute type and optionally allows the ds-task-disconnect-notify-client and ds-task-disconnect-message attribute types. Also, add support for a "Get Connection ID" extended operation, which allows a client to determine the connection ID associated with its connection in the server. OpenDS Issue Numbers: 429, 478, 2025

These set of changes implement VLV and filter capability to OpenDS: - A VLV index is defined by a name, base DN, search filter, search scope, sort order. A search request must match these parameters exactly to use the VLV index. - A VLV index made up of the entry IDs matching the definition criteria (above) and the corresponding attribute values that are part of the sort order in the sort order. This information is broken up into blocks of sorted sets. The block size can be configured through admin framework. Default block size is 4000. In the database, the sorted set is stored with the following format: 4 byte set size | entry IDs of 8 bytes each ... | attribute values of 16 bytes each ... - Each sorted set is keyed by the entry ID and attribute values of the largest entry in the sorted set. A special comparator (VLVKeyComparator) is used to sort the keys in the database in the order of the specified sort order. - When entries are added to the VLV index, its sort values are extracted and inserted into the sorted set whose key (also the largest entry in the set) is the smallest key that represents a entry that is greater or equal to the entry being inserted. If the sorted set exceeds the block size, it is divided in two and stored back into the database with the new key. In this implementation, a sorted set's key is never changed after it is created. - On importing from LDIF, each entry's sort values and ID is written out to a intermediate file in order. These files are later merged and inserted into the database. - Index rebuild and verify also works with VLV indexes. The verify job ensures that all the entries stored in the VLV index is in the correct order. - With this implementation, once a VLV index is created, it can not be changed without a rebuild. The server will NOT warn the user if the index has changed offline. Until a rebuild is done, it can return incorrect results. This should be fixed later. - Performance wise, modify, add, and delete performance will be degraded if the entry matches the indexing criteria. Searches not using the VLV index should not see any notable performance degradation. If the block size is set too big, there is a potential that a large number of updates will result in some JE lock timeouts since the few sorted sets are hotly contested. However, if the block size is too small, searches using the VLV control with offsets could be slow since there are more records to look through. This area need further investigation to determine the optimal default value. Fix for issue 38

This fixes issue 1971 and allows partial non-append imports of a backend: - When using import with backend IDs, the entire backend is only cleaned if the new clearBackend option is true or if the backend only contains one base DN. - If a backend's base DN is one of the include branch DNs, only that base DN will be cleared. - If the include branch DN is below one of the backend's base DNs, a temporary entry container will be created for the same backend base DN. All entries not in the include branch will be migrated to the temporary entry container. The entries from LDIF will then be imported. Lastly, All entries in the exclude branches will be migrated. The original entry container is then deleted and the temporary entry container renamed. - When using include branches, they all must be under one backend ID. If an exclude branch become orphaned by the LDIF import, the migrated entries will be rejected due to non parent. - The LDIFImport tool is updated to reflect the new behaviors and options. - The import task is updated to reflect the new behaviors. The options are checked when the task is initialized instead at run time to catch errors are soon as possible.

Add a general framework for OpenDS to send e-mail messages, and add an SMTP alert handler that can be used to send e-mail messages in response to administrative alerts generated within the server. OpenDS Issue Number: 221

Fix issue 2001 - unusual odyssey when attempting to create a virtual attribute configuration. The conflict-behavior was defined as being optional with a defined default value. However, the LDAP schema for the associated virtual attribute object class labeled the attribute as mandatory. This change updates the schema to align it with the XML definition.

Implement support for password history functionality. The password history can be maintained either based on the number of previous passwords to remember (e.g., a user cannot re-use any of his/her last five passwords), or the length of time the previous passwords have been retained (e.g., a user cannot re-use any password he/she has had within the last 365 days), or both. This commit also updates the manage-account tool so that it supports a public option for displaying the password history state values, and a hidden option for clearing a user's password history state. OpenDS Issue Number: 339

reverting the changes done in revision 2352 for issue 1916 because these changes do not preserve the enabled flag in the config.ldif and are not generic enough. See issue 1916 for the full discussion thread.

Update the password policy configuration to support a new attribute, ds-cfg-state-update-failure-policy. This attribute makes it possible to control how the server should handle failures that may occur when attempting to update password policy state information during a bind operation. This attribute allows the following values: - ignore -- If an otherwise successful bind attempt encounters a failure when trying to update the password policy state information for a user, then log an error message but allow that bind to succeed. - reactive -- If an otherwise successful bind attempt encounters a failure when trying to update the password policy state information for a user, then cause the bind to fail. - proactive -- If the server can detect ahead of time that the password policy state update could fail (e.g., if the entire server or target backend is in read-only mode) and it is known that a successful or failed bind attempt would need to update the password policy state information, then reject the bind before any processing is performed. If it gets past this phase and the attempt to update the state information later fails, then it will have the same behavior as the "reactive" policy. Note that bind attempts by root users will always be treated using the "ignore" policy to ensure that they are not locked out in the event of a significant problem (e.g., disk full). OpenDS Issue Number: 1810

Revert unrelated changes committed accidentally in revision 2359. The changes didn't break anything, but they weren't related to the issue being addressed by that commit and were not a complete fix for any other issue.

Update the bind processing code so that the ClientConnection.bindInProgress flag will not be unset between stages of a multi-stage SASL bind. OpenDS Issue Number: 1972

Fix issue 1934: define which JE index properties will be displayed by default by the dsconfig list-je-indexes sub-command. Also fix the JE index index-substring-length property LDAP schema: its attribute type was not included in its object class definition.

fix for issue 1916 : dsconfig create-synchronization-provider : unexpected error message Make non mandatory the enabled property of the synchronization provider. and change the definition of this property so that it have the value true as its default value.

Fix for issue 1321 : Synchronization changes aren't logged by default This adds a new configuration property for the accessLogPublishers : ds-cfg-suppress-synchronization-operations. It can be used to configure the behaviour of some log publisher with regards to the synchronization operations. When set to false (which is the default value) the access log publisher will log the synchronization operation. When set to true, the access log publisher will not log the synchronization operation. The property ds-cfg-suppress-internal-operations that control the logging of the internal operations now does not have any effect on the synchronization operation (so that one can configure the server to log all internal operations except the synchronization ones) Both the access log and the audit log implement this property. I've also added the string : "type=synchronization" at the end of the line corresponding to the operation request in the access log so that it is possible to differentiate the synchronzation operations from the other operations. example : [10/Jul/2007:15:29:34 +0200] ADD conn=-1 op=1314 msgID=1315 dn="cn=Natascha Pfifferling463, ou=Human Resources, dc=example,dc=com" type=synchronization [10/Jul/2007:15:29:34 +0200] ADD conn=-1 op=1314 msgID=1315 result="Success" etime=98 I could not find any unit test for the access and audit log and decided it was too complex to create new ones compared to the simplicity of this change...

Update the server's DIGEST-MD5 SASL mechanism handler so that it provides support for parsing the digest-uri element of the request. By default, no parsing is performed and any digest-uri value will be accepted. However, if the DIGEST-MD5 SASL mechanism handler is configured with the ds-cfg-server-fqdn attribute, then the digest-uri value will be expected to be "ldap/" followed by the value of that configuration attribute. OpenDS Issue Number: 1238

Bug# 1462:server deadlock on subtree delete Fix: Introduce a batch parameter to create the additional transactions for a new batch.

Define the configuration option "ds-cfg-strip-syntax-minimum-upper-bound" to strip the suggested minimum upper bound value from the schema definition attribute type description syntax oid. For example: attributeTypes: ( 2.5.4.41 NAME 'name' EQUALITY caseIgnoreMatch SUBSTR caseIgnoreSubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15{32768} X-ORIGIN 'RFC 4519' ) to attributeTypes: ( 2.5.4.41 NAME 'name' EQUALITY caseIgnoreMatch SUBSTR caseIgnoreSubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 X-ORIGIN 'RFC 4519' ) Setting this option to true will strip these values. The default behavior is to return the value. The server must be rebooted after the value has been changed. This is to provide a server work around for a bug in the JNDI getAttributeSyntaxDefinition method that incorrectly uses the value as part of the syntax name.

Update the way that the password policy import plugin encodes passwords during an LDIF import. The basic practice is as follows: - If the entry specifies a particular password policy, then the default password storage schemes for that policy will be used to encode the password. - If the entry does not specify a particular password policy, then the server will use the password scheme(s) specified in the ds-cfg-default-user-password-storage-scheme or ds-cfg-default-auth-password-storage-scheme attribute in the plugin configuration entry, if they are present. - If the configuration entry does not specify a default set of storage schemes, but it uses a password attribute that has the same syntax as the password attribute used by the server's default password policy, then the default password storage schemes from the server's default password policy will be used. - If all else fails, the server will use SSHA for passwords with the user password syntax, or SHA1 for passwords with the auth password syntax. OpenDS Issue Number: 849

Provide a way to control the order in which plugins are invoked. This implementation is similar to that used by DSEE 6. The "cn=Plugins,cn=config" configuration entry now supports a number of new configuration attributes (one per plugin type) that can be used to control the invocation order for plugins of that type. The plugin order specification should be a string that is a comma-delimited list of the names of the plugins in the order in which they should be invoked, and it should also include an asterisk to indicate the order in which any unmatched plugins should be invoked. For example: ds-cfg-plugin-order-pre-operation-add: Entry UUID, * This indicates that the "Entry UUID" plugin should be invoked before any other pre-operation add plugins. The plugin order is evaluated only at startup, and any problems or inconsistencies detected (e.g., the same plugin name listed twice in the order, or a plugin order that does not contain a wildcard character) will generate warning messages in the server's error log. OpenDS Issue Number: 253

This refactoring includes the following changes to the JE backend: - Extracted common interface DatabaseContainer from DN2ID, ID2Entry, etc... classes. - Moved database read and write methods from EntryContainer to DatabaseContainer. - Added index configuration to the XML based admin framework. - Removed redundant configuration objects (Config, IndexConfig). - Added exclusive/shared lock to EntryContainer. All access to an EntryContainer must acquire a lock before using the internal DatabaseContainers or making configuration changes. - Added the ability to add/remove/modify indexes with the backend online. Server will issue rebuild required warning when adding new indexes or sub-indexes (equality, substring, presence...). - Added the ability to change the index entry limit for both the backend and each index with the backend online. Server will issue rebuild required warning if the previous limit has been exceeded. - Added the ability to change entry compression and index substring length setting while the backend is online. - Added a persistent state database to each EntryContainer to persist backend configuration between server restarts. Server will issue rebuild required warning if a new index is added when the backend is offline. - Added a trusted flag to indexes so that non existent keys will not be interpreted as an empty entry ID set when an index is untrusted. An index is untrusted when it is added to an non-empty EntryContainer or an inconsistency is detected. Server will issue warning on startup to rebuild the index. - Fixed a issue where the LDIF import process stops responding if the temporary import dir is full or unwritable. Fix for issue 1480 1455 1575

Final part for issue 604 : replication needs to resolve naming conflicts These changes are the last part of the naming conflict resolution. It also introduce the ds-sync-conflict attribute that is used to mark the entries for which an automatic conflict resolution is not possible.

Fixed an issue where the error log doesn't actually log messages. Also improved the performance of the loggers when they are disabled. Fixed for issue 1780

[Issue 1117] Create an entry cache backed by DB+tmpfs

Major changes made to the logging framework. It should resolve the following issues: 1. Migrated configuration to the new admin framework. 2. Removed all dependencies on the JDK logger. (Issue 1503) 3. Added option to set the file permissions on all log files. (Issue 202) 4. Added option to write log files asynchronously. 5. Retention and rotation policies are now separate managed objects registered to the Directory Server. 6. Rotation and retention policies are not extensible. 6. Post-rotation actions are not yet implemented in this set of changes. 7. Tools and tasks can now use a custom log publisher that only picks up messages generated by a specific thread or thread group. 8. Debug logger no longer creates a log record object for every message. 9. Configurable Log File Paths (Issue 174) 10. Log Level Support by Category/Severity. This capability is limited for error logger. (Issue 177) 11. Support log file rotation (Issue 188) 12. Sized-based, Time-based rotation policies (Issues 190, 191) 13. Time-based, max size-based, file count-based retention policies (Issues 199, 201, 202) 14. Debug logger configurable via the admin framework (Issue 836)

Minor fixes to replication related schema definitions (e.g. missing attributes in object class definitions).

This is is the last part of the replication renaming (issue 1090) and is the renaming of the configuration. The change in the configurations are quite simples, basically the changelog string has been replaced by the replication-server and the synchronization string has been replaced by replication except when it is used as synchronization provider. The schema is a bit more complex because I've decided to keep the old objectclass names and the old attribute names as alias of the new names. This would allow the previous configuration to work, however my testing has shown that the admin framework does not seem to deal well with objectclass aliases and therefore this is not entirely true.

First set of modifications for issue 1090 : rename synchronization components Move from synchronization package name to replication.

This integrates the multi-master synchronization with the new admin framework (issue 1477) and makes possible to dynamically add or remove changelog server and synchronization domains in a running server (issue 639). It was necessary to slightly modify the configuration : - The objectclass ds-cfg-multimaster-synchronization-provider must be added to the entry : cn=Multimaster Synchronization,cn=Synchronization Providers,cn=config - the domains must be configured under cn=domains, cn=Multimaster Synchronization,cn=Synchronization Providers,cn=config instead of being directly under this entry. The synchronization.ldif file has been updated to reflect these changes. I will update the configuration doc in the wiki

Migrate the BackendConfigManager to the new configuration framework.

Issue 605 Total Update over protocol

Update the member virtual attribute implementation so that it provides a mechanism for preventing the entire member list from being returned, which can be a very expensive operation. When running with this configuration, the attribute will handle requests that determine whether a given user is a member of the group, but will not list the entire set of membership. OpenDS Issue Number: 1506

Add support for user-defined virtual attributes, in which the administrator may explicitly specify the values that should be used for a virtual attribute. This provides functionality similar to the Class of Service (CoS) facility in the Sun Java System Directory Server. OpenDS Issue Number: 538

Implement support for virtual static groups, which are entries which appear to be static groups but get their membership information from another group and present it through a virtual attribute. This can make it possible to use a dynamic group to actually define the set of membership, but still support applications which can only interact with static groups. OpenDS Issue Number: 425

Add a new password validator that can be used to require that passwords have a specified number of characters from various user-defined character sets. It is also possible to control whether passwords will be allowed to contain characters outside of any defined character set. OpenDS Issue Number: 337

Add support for two new password validators: - Issue #338: Prevent users from selecting a password that matches the value of any attribute (or a specified set of attributes) in that user's entry. - Issue #341: Prevent users from selecting a password that matches a value contained in a dictionary. Both validators support both forward and reverse matching, and for the dictionary password validator I have compiled a dictionary from public domain word lists.

Re-numbered OIDs added in revision 1607 that caused conflicts with OIDs alreadly assigned.

Added the following capabilities to OpenDS: - Index rebuilding capabilities. All indexes including system and attribute indexes can be rebuilt. Each index will be rebuilt by a seperate thread to increase performance. A max number of rebuild threads could be set to limit the resources used by large rebuild jobs. Partial rebuilds of attribute indexes could also be done by specifying the attribute index type after the attribute type (ie. sn.approximate). - Index rebuilding standalone tool. Rebuilding of attribute indexes could be done with the backend online. However, rebuilds including system indexes must be done with the backend offline. - Index rebuilding task. Rebuilding of attribute indexes are done with the backend online. Rebuilds that include system indexes will be performed after bring the backend offline. The user must have index-rebuild privilages to rebuild indexes. - Approxitae indexing capability. The value of the attribute will be normalized using the approximate maching rule of that attribute type. This is used as the key for the index. Approximate indexes are fully supported by the index verify, rebuild, and import jobs. - Fixed bug in build.xml where weave is enabled even if a test.* property is set. - Consolidated some common tool messages. - Consolidated some JE backend methods common to all tools. - Added unit tests for rebuild job and approximate indexes. Fix for issues 35, 39, 40, 41

Add initial support for a virtual attribute subsystem, and implement a few different kinds of virtual attributes. This commit addresses the following issues: - Issue #1475 -- General virtual attribute support - Issue #539 -- Support for the isMemberOf virtual attribute - Issue #544 -- Support for the entryDN virtual attribute - Issue #1056 -- Support for the subschemaSubentry virtual attribute - Issue #85 -- Support for the real attributes only control - Issue #86 -- Support for the virutal attributes only control In general, virtual attribute support consists of three parts: - An implementation of the org.opends.server.api.VirtualAttributeProvider class, which provides the logic for actually generating the values, providing support for various kinds of matching, and potentially the ability to process search operations involving the virtual attribute that might not otherwise be indexed. - The org.opends.server.types.VirtualAttribute class, which is a subclass of org.opends.server.types.Attribute and uses the virtual attribute provider to generate its values. - The org.opends.server.types.VirtualAttributeRule class, which associates a virtual attribute provider with a given attribute type, and also with a set of criteria that controls which entries should have the attribute. The virtual attribute rule currently supports the following criteria that can be used to decide whether an entry should have a given virtual attribute: - Zero or more base DNs. If any base DNs are provided, then any entry which falls below one of those base DNs will be a candidate to get the virtual attribute. If no base DNs are provided, then DIT location will not be taken into account when determining eligibility. - Zero or more group DNs. If any group DNs are provided, then any entry that belongs to one of the specified groups will be a candidate to get the virtual attribute. If no group DNs are provided, then group membership will not be taken into account when determining eligibility. - Zero or more search filters. If any filters are provided, then any entry that matches one of the specified filters will be a candidate to get the virtual attribute. If no filters are provided, then the contents of the entry will not be taken into account when determining eligibility. In addition to that criteria, virtual attribute rules define a conflict behavior, which controls how to behave when the entry already has one or more real values for the attribute. The conflict behavior can be "real-overrides-virtual" (to only show the real values), "virtual-overrides-real" (to only show the virtual values), or "merge-real-and-virtual" (to show both real and virtual values). The virtual attribute implementation has been designed so that there should be virtually no performance impact unless the attribute needs to be returned to the client or it is referenced in a search filter, and you can completely disable virtual attributes if you don't need them.

Back out the commit included in revision 1539 because it does not build properly (it appears to reference classes that are not in the repository).

[Issue #605] Total Update over protocol These changes implies : - new messages in the protocol and their tests - new logic in the SynchronizationDomain to handle the import and the export - new logic in the Changelog server to forward the messages for this feature - new tasks to trigger the total update Object class 1.3.6.1.4.1.26027.1.2.91: ds-task-initialize-from-remote-replica 1.3.6.1.4.1.26027.1.2.92: ds-task-initialize-remote-replica Attributes - same for both tasks : 1.3.6.1.4.1.26027.1.1.332: ds-task-initialize-domain-dn 1.3.6.1.4.1.26027.1.1.333: ds-task-initialize-replica-server-id 1.3.6.1.4.1.26027.1.1.334: ds-task-unprocessed-entry-count 1.3.6.1.4.1.26027.1.1.335: ds-task-processed-entry-count - a bunch of unit tests to tests the export and the import, on the producer side and on the consumer side

Add two new password validators to OpenDS: - A validator which enforces a restriction that passwords must have at least a specified number of unique characters (issue #1219). - A validator which enforces a restriction that passwords may not have any character which appears more than a specified number of times in a row (issue #1220).

Add a new password validator that determines whether a proposed new password is acceptable based on how similar it is to the user's current password. The processing uses the Levenshtein Distance algorithm to determine the number of changes required to convert the current password into the new password (a change may be either inserting a new character, removing an existing character, or replacing an existing character). Contributed By: Ales Novak OpenDS Issue Number: 340

Add support for: 1. global ACI implementation using the ds-cfg-global-aci attribute type 2. re-adds support for ACIs in the "cn=config" naming context

Update all CDDL headers in source files to remove a typo.