OPENDJ-1134 (CR-2355) Introduce a class in replication for encapsulating host+port combinations Now the replication code uses HostPort class for comparisons instead of "host:port" Strings. Changed all the HostPort variable/parameter names to *Address instead of *URL. Left all the relevant String variable/parameter names to *URL. HostPort.java: Removed LocalAddress(String), not used anymore. Moved getLocalAddress() here from ReplicationServer. Created localAddress() static method. Extracted method toInetSocketAddress() and moved it here. ReplicationServer.java: Created converting method toHostPorts() + used it everywhere replicationServerUrls was accessed. Replaced getNormalizedLocalURL() and normalizeServerURL() with simply calling HostPort.localAddress() and HostPort.valueOf(). In getConnectedRSUrls(), changed return type from Set<String> to Set<HostPort>. In connect() and disconnectRemovedReplicationServers(), used HostPort as a parameter instead of String. ReplicationServerDomain.java: In stopReplicationServers(), used HostPort as a parameter instead of String. ReplicationServerHandler.java: Extracted method toServerAddressURL(). ReplicationBroker.java: Consequence of the change to HostPort. ReplSessionSecurity.java: In isSslEncryption() removed unused parameter. Used StaticUtils.close(). ReplicationServerConfiguration.xml, ReplicationServerCfgDefn.properties: Added precisions on how to specify the "replication-server" property. ReplicationServerDynamicConfTest.java, ReplServerFakeConfiguration.java: Updated because this test was failing.

Partial fix for OPENDJ-951: Reduce size and frequency of replication MonitorMsg * increase default monitoring period to 60s * improve property documentation.

Fix OPENDJ-922: Replication window size is too small on high latency networks * set the default window size to 100K * indicate that the properties may be deprecated in future releases * tag properties as advanced. Reviewed manually by Ludo.

Fix for issue OPENDJ-214. Rebranded as OpenDJ and now uses product-name variable.

Fix OPENDJ-186: Change default values for LDAP request handlers, JE backend cache size, and replication purge delay See issue description for exact changes.

This is about refactoring the way the directory server chooses the replication server it will connect to. This also introduces a new (weighed) load balancing feature that spreads DS connections across the RSs, according to the RS weights defined by the administrator, Issue 4343: https://opends.dev.java.net/issues/show_bug.cgi?id=4343 The commit includes necessary modifications for implementing what is described in this document: https://www.opends.org/wiki/page/ReplicationServerSelection There is a little "implementation" section in this document that explains some important stuff. Also good to know for these code modifications: - The topology info related to RSs and kept by the ReplicationBroker is now fully kept in a new map of a new ReplicationServerInfo bag class. This map is updated upon reception of a TopologyMsg. - Protocol change: the TopologyMsg now includes the RS url in the RSInfo list - The dynamic change of the weight of a RS triggers a new TopologyMsg being fired, to support dynamic change of weights and automatic topology re-connections - SameGroupIdPoller thread has disappeared and its functionality is replaced by the mechanism that re-evaluates the more suitable RS (see section 5, in the document for more details)

- Corrected an error that made monitoring retrieval timeout - Allow 0 value for monitoring publisher period to allow disable it by configuration

In order to support a more clever algorithm for the DS to choose his RS, we introduce: - a weigth, which is an integer affected to each RS that combined with each others will define a percentage value which matches the number of DSs (compared with total number od DSs in the topology) that can be connected to the RS at a time in the topology. In these modif, this configuration of the weight is added as well as dynamic changes. Also transported in Topo messages. No modification of the connection algorithm yet - Also to support the future connection algorithm, these modifs introduces a Monitoring Publisher thread which is a thread that sens every 3 seconds a Monitoring message (unchanged format) to every DSs that are connected to him. These information will be used by the DSs to potentially reconnect to another RSs with a newer server state (info included in monitoring messages) The new connection algorithm will take into account: - group id - generation id - server states - locality (same VM) - weight (load)

- Addition of ReplServerStartDSMsg now sent to a DS connecting to a RS in handshake phase instead of a ReplServerStartMsg. ReplServerStartDSMsg contains same thing as ReplServerStartMsg but also contains - replication server weight - number of ciurrently connected DS on the RS => both will be used for future new RS choice algorithm - Addition of a StopMsg sent: - when any entity (DS,RS) is closing a connection (sent just before) with a peer - when DS finishes phase 1 of handshake (was gathering RS info for RS choice so sent just after new ReplServerStartDSMsg is received) => both are used to distinguish between a proper connection closure (no message) and an unexpected one (error log) - Compatibility between protocol V4 and V3 (and before) - changed MonitorMsg to never be created with a protocol version - MonitorMsg now always sent with publish(msg, version) (publish method without version was used so bug) - TopologyMsg now always sent with publish(msg, version) (publish method without version was used so bug)

Fix for 4169 : Ability to disable generation of historical information and 4170 : Replication should not generate HistoricalCsnOrdering index for replayed operation

The commit will bring the following features : - An updated version of the underlying database. BDB JE 3.3 is now used. - Attribute API refactoring providing a better abstraction and offering improved performances. - A new GUI called the Control-Panel to replace the Status-Panel: the specifications for this GUI are available on OpenDS Wiki and contains a link to a mockup. See <https://www.opends.org/wiki/page/ControlPanelUISpecification>. - Some changes in the replication protocol to implement "Assured Replication Mode". The specifications are on OpenDS Wiki at <https://www.opends.org/wiki/page/AssuredMode> and section 7 described some of the replication changes required to support this. Assured Replication is not finished, but the main replication protocol changes to support it are done. As explained by Gilles on an email on the Dev mailing list (http://markmail.org/message/46rgo3meq3vriy4a), with these changes the newer versions of OpenDS may not be able to replicate with OpenDS 1.0 instances. - Support for Service Tags on the platforms where the functionality is available and enabled. Specifications are published at <https://www.opends.org/wiki/page/OpenDSServiceTagEnabled>. For more information on Service Tags see <http://wikis.sun.com/display/ServiceTag/Sun+Service+Tag+FAQ>. - The Admin Connector service. In order to provide agentry of the OpenDS server at any time, a new service has been added, dedicated to the administration, configuration and monitoring of the server. An overview of the Admin Connector service and it's use is available on the OpenDS wiki <https://www.opends.org/wiki/page/ManagingAdministrationTrafficToTheServer> - Updates to the various command line tools to support the Admin Connector service. - Some internal re-architecting of the server to put the foundation of future developments such as virtual directory services. The new NetworkGroups and WorkFlow internal services which have been specified in <https://www.opends.org/wiki/page/BasicOperationRoutingThroughNetworkGroup> are now implemented. - Many bug fixes... This set of changes may have an impact on current clients and deployments, however it should be possible to upgrade existing 1.0 instances. Some of the changes impacting the clients include : - Scripts using the command line tools including dsconfig will need to be updated (usually to remove some no longer required options).

Updated the copyright statement to reflect that Sun owns the full copyright on the project files.

Commiting configuration XML files after adding info from the old config guide to be incorporated in the new generated config reference

Uncommitting last changes to configuration .xml files, second try

commiting configuration XML files after adding info from the old config guide to be incorporated in the new generated config reference

Advanced properties phase 1: tag advanced properties and perform various clean-up to the XML definitions: * clean up typos and any bad grammar in the synopsis/description elements * fixed common errors. For example, there were many places where the synopsis was repeated in the description (the description is only ever used in conjunction with the synopsis). There were also many places where the <adm:user-friendly-name> was being used in property definition description elements to refer to the property name, when it actually refers to the managed object name * split big synopsis (not sure of the plural form - synopsi?) where possible - there were lots of cases where we'd have a big synopsis but no description * overrode properties with sensible default values where appropriate (for example, I provided sensible default values for some of the attribute-type properties in the virtual attribute definitions) * generally made documentation more consistent across the definitions * removed unneeded XML attributes of the form 'mandatory="false"'. Attributes only need to be specified if their value is true as they default to false. Adding redundant attributes just makes the XML harder to read * removed any documentation that referred to the configuration in LDAP related terms (with one exception in the Schema backend). All documentation now refers to other properties using the property names, not their associated LDAP attribute.

Merge branches/temp-admin@3208 onto trunk@3208.

Correct a number of spelling errors in configuration definitions.

Add support for tagging managed object definitions so that similar types of managed object can be grouped together. Tagging will enable us to automatically generate more user-friendly documentation and administration tools as a result of them being easier to navigate and search. For example, an administration CLI will be able to split the available set of sub-commands into categories, thus making it easier for administrators to find the sub-command that they need. This change is implemented as follows: * provide an extensible way in which tags can be defined: the XML root configuration definition element now supports an "adm:tag-definition" element, which can be used as follows: <adm:tag-definition name="logging"> <adm:synopsis>Logging</adm:synopsis> </adm:tag-definition> * provide a means for tagging managed object definitions with zero or more tags using an "adm:tag" element, which can be used as follows: <adm:tag name="logging"/> * add support to the admin framework APIs for querying a managed object definition's tags * define an initial set of tags and tag managed object definitions appropriately (this is just an initial guess and is likely to change): * logging * replication (incl. mmr) * database (incl. caching) * security (authn and authz) * identity (user account management, pwp, etc) * core (connection handlers, virtual attributes, etc)

Enforce the limits on the server-id at the configuration framework level

Add support for specifying a "usage" string for string based property definitions which define a pattern constraint. The usage string is used when the syntax of a property needs to be displayed, for example in a CLI help screen or in server-side log messages when a bad property value is encountered. For example, a property whose values must be a valid UNIX mode could have the following string syntax definition: <adm:string> <adm:pattern> <adm:regex>^[0-7][0-7][0-7]$</adm:regex> <adm:usage>MODE</adm:usage> <adm:synopsis> An valid UNIX mode string. The mode string must contain three digits between zero and seven. </adm:synopsis> </adm:pattern> </adm:string> Existing string based properties have been updated with appropriate usage.

Add the xml definitions files missing in the previous commit use the admin framework facilities to do pattern matchin on the replication server addresses to make sure that it contain both a hostname and a port number. Also use the last : as the separator in case somebody would want to use a IPv6 address. Issue number 1579

Update all of the configuration definition XML files to include the CDDL header and copyright information.

Revert the addition of the copyright header to the configuration definitions because it breaks the XML validation.

Update all of the configuration definition XML files to include the CDDL header and copyright information.

This is is the last part of the replication renaming (issue 1090) and is the renaming of the configuration. The change in the configurations are quite simples, basically the changelog string has been replaced by the replication-server and the synchronization string has been replaced by replication except when it is used as synchronization provider. The schema is a bit more complex because I've decided to keep the old objectclass names and the old attribute names as alias of the new names. This would allow the previous configuration to work, however my testing has shown that the admin framework does not seem to deal well with objectclass aliases and therefore this is not entirely true.