Remove unused messages

Fix for OPENDJ-925, OPENDJ-926, and overall schema management. Review CR-1806. Back a few years ago, when we did the SVR4 package, we split the schema into read-only (Install) and writable (Instance). With the template model in 2.6, the whole schema is under config, and we should never edit/change the one in the template. So, I've removed all code related to the 2 locations for schema, reverted some I10N messages to their previous versions, and tidy up code and tests. All unit-tests are passing. And it's also possible to change the instance.loc and run setup and have a fully functioning server :-)

OPENDJ-902 (CR-1702) Add connectionID to the HTTP access log + move to extended log format OPENDJ-858 (CR-1702) Add stats tracking to HTTP client connections In HTTP access log, changed the name of the non standard "datetime" field to "x-datetime". Added the "x-etime" field that outputs as a number. Added validation for the log format that outputs error messages. FileBasedHTTPAccessLogPublisherConfiguration.xml, FileBasedHTTPAccessLogPublisherConfiguration.properties: Renamed the "datetime" field to "x-datetime" + added the "x-etime" field. Improved the description of the ds-cfg-log-format. TextHTTPAccessLogPublisher.java, config.properties: Extracted constants for all supported field names. Added ALL_SUPPORTED_FIELDS constant. Changed logFormatFields instance member from String[] to List<String>. Added validateLogFormat() which can output new error messages and called it from applyConfigurationChange() and initializeLogPublisher(). Added subtract(). HTTPClientConnection.java, HTTPRequestInfo.java: Added instance member totalProcessingTime. In sendResponse(), added to the totalProcessingTime + extracted method getProcessingTime(). Added getTotalProcessingTime().

Fix typo in message

OPENDJ-879 (CR-1642) Add HTTP access log Implemented the HTTP access logger by taking inspiration from other loggers. Configured the new logger everywhere other loggers are configured. Logged the HTTP request in several places to cover all cases (happy paths, errors, etc.). config.ldif, 02-config.ldif, HTTPAccessLogPublisherConfiguration.xml, FileBasedHTTPAccessLogPublisherConfiguration.xml, HTTPAccessLogPublisherCfgDefn.properties, FileBasedHTTPAccessLogPublisherCfgDefn.properties: ADDED Added 2 new objectClasses HTTPAccessLogPublisherConfiguration and FileBasedHTTPAccessLogPublisherConfiguration. config.properties: Added new error messages for the HTTP access logger. HTTPAccessLogger.java, HTTPAccessLogPublisher.java, TextHTTPAccessLogPublisher.java, HTTPRequestInfo.java: ADDED HTTPRequestInfo.log() prevents double logging. CollectClientConnectionsFilter.java: Logged the request info when HttpServletResponse.setStatus(), sendAuthenticationFailure() and onFailure() are called. Pushed more data to the HTTPRequestContext + switched to use the more specific HttpServletRequest/HttpServletResponse SdkConnectionAdapter.java: Logged the request info when close() is called. LoggerConfigManager.java, TestCaseUtils.java: Configured the HTTP access logger. InProcessServerController.java: TO BE REMOVED (by Matt on the native packaging branch) Sample log: localhost bjensen [03/May/2013:10:14:54 +0200] "GET /users/_queryFilter=true&_prettyPrint=true HTTP/1.1" 500 "curl/7.27.0" localhost bjensen [03/May/2013:10:15:05 +0200] "GET /users/_queryFilter=true&_prettyPrint=true HTTP/1.1" 200 "curl/7.27.0" localhost [03/May/2013:10:15:14 +0200] "GET /users/_queryFilter=true&_prettyPrint=true HTTP/1.1" 200 "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:20.0) Gecko/20100101 Firefox/20.0" localhost [03/May/2013:10:16:40 +0200] "GET /users/_queryFilter=true&_prettyPrint=true HTTP/1.1" 401 "curl/7.27.0" localhost [03/May/2013:10:16:50 +0200] "GET /users/_queryFilter=true&_prettyPrint=true HTTP/1.1" 200 "curl/7.27.0" localhost [03/May/2013:10:16:51 +0200] "GET /favicon.ico/null HTTP/1.1" 404 "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:20.0) Gecko/20100101 Firefox/20.0" localhost [03/May/2013:10:17:10 +0200] "GET /users/_queryFilter=true&_prettyPrint=true HTTP/1.1" 200 "curl/7.27.0"

OPENDJ-308: Implement access log filtering and configurable message format Improve filter configuration validation.

OPENDJ-308: Implement access log filtering and configurable message format Add support for changing the format of the access log: * configurable timestamp format * enable logging of request and response control OIDS * change the format to a "combined" mode where operations are logged using a single log record rather than individual request and response records. Also fixed a bug in the time thread which prevented applications from deregistering time formatters.

Initial checkpoint of work for OPENDJ-262: Implement pass through authentication (PTA) Refactor PasswordPolicy implementation so that we now have a common parent type, "AuthenticatioPolicy", which will act as a common entry point for all authentication policies. In particular, password policies can be thought of as a specific type of authentication policy: one where authentication and the user's credentials are managed locally. PTA can be thought of as a non-local authentication policy: the password is managed by a third party. With this change we still only have one type of authentication policy and no attempt has been made to refactor code which uses it. More specifically, code such as the local backend WFE just type-casts the authentication policy to a password policy. Another implication of this change is that sub-entry based password policies now automatically inherit run-time changes made to the default password policy, whereas before a server restart was required.

Implements a configurable limit in the number of persistent searches a server can handle.

issue 4102: Provide implementation for substitution syntax

3518:config.properties message has incomplete sentance

Fix issue 3734 - Make network group policies extensible. See issue tracker for more details.

Fix Issue 3540: Network Group: Uncaught java.lang.OutOfMemoryError while looping on calling dsconfig setting config properties Fix Issue 3541: NetGroup reqFilteringPolicy - shouldn't be able to allow and disallow at same time a attr, subtree

Fix 3534: Provide native Solaris packages (fix also 3533: upgrade fails starting server adding entry ds-cfg-key-id)

New top level object in RootConfiguration.xml : "extension", that allows to create a configuration object for an extension, that does not inherit from an existing configuration object.

Fix for issues 3255, 3265, and 3269: Issue 3255: Added ds-cfg-max-allowed-client-connections configuration parameter to prevent using up all file descriptors. By default the server will still allow unlimited number of connections just like before. The user will have to adjust this for their system. This might be automatically set for the OS in the future. Issue 3265: Subtree deletes and mod DN operations will now always use a JE transaction throughout the entire operation. This makes sure the operations are always executed atomically. Issue 3269: Added checks for canceled operations in JE backend code.

Updated the copyright statement to reflect that Sun owns the full copyright on the project files.

Fixed the config change listeners for all log publishers so they will prevent changing the log file permissions to an invalid value. The listeners will now reject changes that will prevent OpenDS from writing to the log files. Fix for issue 2107

- [Issue 2007] generic mechanism for using mulitple caches. - [Issue 2049] dsconfig is unable to configure the entry cache.

Fix for Issue #1961 (reviewed by Josu) * Remove instances of "Specifies the..." * Remove instances of "Indicates that..." * Reword instances of "Specifies to..." (which doesn't really make sense anyway) * Change instances of "This utility may be used..." to "This utility can be used..." as discussed in a previous issue * Change some erroneous instances of "may" to "might" or "can". This might sound like a pointless change but it has implications for localization - "may" is generally ambiguous.

s set of files provides the workflow configuration manual mode. Until now, the workflows were automatically configured-a wokflow was created for each base DN in the backends. When new suffixes were added or when a backend was added, the associated workflows were also created (and simillarly workflows were deleted as suffixes or backends were removed). With the manual mode, each and every workflow in the server must be defined explicitely in the configuration. By default, the server is running in automatic configuration mode. To have a server running with manual configuration mode one must set the attribute in cn=config: dn: cn=config ... ds-cfg-workflow-configuration-mode: auto|manual No attribute means "auto" mode. The workflow configuration consist of 3 parts: - the configuration of workfow elements - the configuration of workfows - the configuration of network groups The Workflow Elements - A workflow element is a basic task in a workflow. The workflow elements are organized in trees and the simplest tree is made of one element. For example, the workflow element that wraps a local backend is configured as follow: dn: ds-cfg-workflow-element-id=userRoot,cn=workflow elements,cn=config objectClass: top objectClass: ds-cfg-workflow-element objectClass: ds-cfg-local-backend-workflow-element ds-cfg-workflow-element-id: userRoot ds-cfg-enabled: true ds-cfg-java-class: org.opends.server.workflowelement.localbackend.LocalBackendWorkflowElement ds-cfg-backend: ds-cfg-backend-id=userRoot,cn=Backends,cn=config From an admin standpoint, the local backend workflow element is an aggregation of a single backend (attribute ds-cfg-backend). So we cannot disable/delete a backend as long as it is used by a local backend workflow element. The Workflows - A workflow is a chain of processing and it's targeting all the entries under a given baseDN. The processing is actually identified by the root node of the task tree described above. The configuration of a workflow looks like: dn: ds-cfg-workflow-id=userRoot,cn=workflows,cn=config objectClass: top objectClass: ds-cfg-workflow ds-cfg-workflow-id: userRoot ds-cfg-enabled: true ds-cfg-workflow-element: ds-cfg-workflow-element-id=userRoot,cn=workflow elements,cn=config ds-cfg-base-dn: dc=example,dc=com From an admin standpoint, the local workflow is an aggregation of a single elements (attribute ds-cfg-workflow-element). So we cannot disable/delete a workflow element as long as it is used by a local workflow. The Network Groups - A network group defines categories for client connection. The network group contains a set of workflows and each client operation is routed to one (or more) workflow(s). By default, the server create a default network group which contains all the workflows defined in the server. The default network group looks like: dn: ds-cfg-id=defaultNetworkGroup2,cn=network groups,cn=config objectClass: top objectClass: ds-cfg-network-group ds-cfg-id: defaultNetworkGroup2 ds-cfg-enabled: true ds-cfg-workflow: ds-cfg-workflow-id=adminRoot,cn=Workflows,cn=config ds-cfg-workflow: ds-cfg-workflow-id=ads-truststore,cn=Workflows,cn=config ds-cfg-workflow: ds-cfg-workflow-id=backup,cn=Workflows,cn=config ds-cfg-workflow: ds-cfg-workflow-id=config,cn=Workflows,cn=config ds-cfg-workflow: ds-cfg-workflow-id=monitor,cn=Workflows,cn=config ds-cfg-workflow: ds-cfg-workflow-id=schema,cn=Workflows,cn=config ds-cfg-workflow: ds-cfg-workflow-id=tasks,cn=Workflows,cn=config ds-cfg-workflow: ds-cfg-workflow-id=userRoot,cn=Workflows,cn=config From an admin standpoint, the network group is an aggregation of several workflows (attribute ds-cfg-workflow). So we cannot disable/delete a workflow as long as it is used by a network group. A unit test named WorkflowConfigurationTest tests the configuration of network groups, workflows and workflow elements.

issue 466 (partial) remove redundant server property BACKUP_PROPERTY_CIPHER_ALGORITHM The cipher algorithm used is encoded in the backup data prologue.

Merge branches/temp-admin@3208 onto trunk@3208.

Added messages to better indicate when a log file can not be opened due to possible file permission problems. A case where this can happen if the user configures a logger to set permission on the log file that will make it unwritable to OpenDS (ie, UNIX mode 555). This allows the user better realize the problem and take corrective action. Fix for issue 2107

entry cache cleanup: - remove obsolete methods, fields and messages. - aggregate filters evaluation for config phases. - simplify filters evaluation and error reporting. - use a single set of filter messages.

- [Issue 1481] Ability to set any JE property in config.ldif - [Issue 2031] provide core JE configurable attributes for FileSystemEntryCache - fixes to entry cache config manager for proper configuration, re-configuration, error reporting.

Update the server so that it has the ability to save a copy of its current configuration into a ".startok" file whenever it starts successfully. The start-ds script and DirectoryEnvironmentConfig class have been updated to expose an option to try to start the server using this "last known good" configuration rather than the active config file. OpenDS Issue Number: 1945

New tests for messaging plumbing to replace old ones: - Format string cannot end with '.' (QUICKSETUP and ADMIN_TOOLS excluded) - Categories cannot span multiple files and files cannot define messages in multiple categories Addional methods for MessageDescriptorRegistry to access descriptors by defining class Corrected some offending format strings Corrected message property file boilerplates and help comments

Correct a number of spelling errors identified in the message files.

This commit is a step toward getting OpenDS internationalized. There are still issues to be resolved before we can declare that we are internationalized but this commit covers the bulk of changes needed at this time. In general this commit converts any string values that might potentially be shown to an end user (client or administrator) to a new class called Message. This includes exception messages, error log messages, alert and email text, labels, and CLI output. Message's main interface includes methods for rendering itself as a string in the default locale using toString() or a specified locale toString(Locale). In addition to addressing localization, this would allow us to potentially support controls allowing clients to set a preferred locale for server communication or output log messages in different languages. Message extends CharSequence so it can be used in some places where strings are currently used with not code change (e.g. writing to a buffer) as well as allowing messages to be composed of one another. In order to create localized messages, instead of adding them to the *Messages.java class, you define them in the corresponding properties files. In general the property file keys are used to derive the new messages severity, category, and ordinal. There are directives that can be used in the properties file in order to avoid doing this however. For instance all the properties files used internally include the 'global.category' directive at the top of the file that instructs the generator to make all the following message belong to a single category. So for internal messages the property value key will be as follows: [SEVERITY]_[DESCRIPTION]_[ORDINAL] See the properties files themselves for more information. Once your messages are in the properties file you can use the ant target 'generatemessages' to create create MessageDescriptor objects that are referred to in the Java code. In the code you might think of these objects as replacing the int-valued message IDs. To create messages you call MessageDescriptor.get() which replaces MessageHandler.getMessage(). It might be helpful then to try to define your messages before referring to them in your code since until you run 'generatemessages' you won't have any Java-code with with to create your message objects. In order to support non-localizable text in messages (host names, DN's, etc.) Message's can be created from String values by calling Message.raw(String). In this way it is also possible to circumvent the intention of changing the APIs to use Message objects instead of Strings which is to support writing of localizable messages. This method is currently used in the code as a crutch for areas of the code that need internationalization (tagged with a 'TODO: i18n' comment). It might also be used by 3rd party developers that don't intend to localize their products. However for internal use we should not be using this method to create Messages for text that is locale sensitive. Additionally this code includes a MessageBuilder that can be used in much the same way that a StringBuilder can be used. It has a method toMessage() that can be used to generate a message object when needed. At this time this method just converts the buffer contents to a raw, unlocalizable message so this will need to addressed in the future. Some other issues: - While all of the unit tests pass after this commit, there may be issues in the code where we have no or minimal test coverage. This includes much of the setup and GUI tools packages for which I have not done any testing but will begin after this commit. - The new message related Java sources are contained in package org.opends.messages and organized in a new module src/module modeled somewhat after the admin module: src/messages/src (non-generated Java source files) src/messages/generated (generated source files) src/messages/messages (properties files for messages) If you run the code from within your IDE you will have to mark all messages/src, messages/generated and messages as source directories. - To generated the MessageDescriptor files you invoke the 'generatemessages' target. - If you run the code from within your IDE you will probably ne - This commit would change the plugin API. Here are the changes caught by DirectoryServerPluginTestCase: doShutdown(String)->doShutdown(Message) doPostDisconnect(PostDisconnectPluginResult,ClientConnection,DisconnectReason,int,String) -> doPostDisconnect(PostDisconnectPluginResult,ClientConnection,DisconnectReason,Message); - This commit aligns the entire code-base behind the new Formatter class (printf style formattings like %s) as opposed to the older MessageFormat style (using curly braces for message arguments). This allows us to provide more type safety for message arguments. So for instance you can include a %c argument specifier in your format string and the MessageDescriptor will enforce specification of a Character as message creation time. - Not all of the config framework has not been converted to use Messages. Perhaps as some point we might have a 'message' syntax for locale-sensitive textural data. I'm sure Matt will have ideas about this. - In addition to Message, MessageDescriptor and MessageBuilder, the org.opends.server.message package includes enums for Category and Severity that replace the definitions formerly found in MessageHandler. - The interface for the ErrorLogger has been simplified to allow logging of messages using a single Message parameter. The severity and category of the message will be derived from the Message itself instead of specified as arguments when a message is logged. - I will provide more documentation about this plumbing on the Wiki shortly. - I will expand on the included tests soon.