Importer.java revision 4765
3339N/A * The contents of this file are subject to the terms of the 3339N/A * Common Development and Distribution License, Version 1.0 only 3339N/A * (the "License"). You may not use this file except in compliance 3339N/A * You can obtain a copy of the license at 3339N/A * See the License for the specific language governing permissions 3339N/A * and limitations under the License. 3339N/A * When distributing Covered Code, include this CDDL HEADER in each 3339N/A * file and include the License file at 3339N/A * add the following below this CDDL HEADER, with the fields enclosed 3339N/A * by brackets "[]" replaced with your own identifying information: 3339N/A * Portions Copyright [yyyy] [name of copyright owner] 4495N/A * Copyright 2008-2009 Sun Microsystems, Inc. 4765N/A * Performs LDIF import and rebuild of indexes. 4591N/A //The executor service used for the sort tasks. 4591N/A //The executor service used for the index processing tasks. 4591N/A //Queue of free index buffers -- used to re-cycle index buffers; 4643N/A //Map of index keys to index buffers. Used to allocate sorted 4591N/A //index buffers to a index writer thread. 4591N/A //Map of DB containers to index managers. Used to start phase 2. 4591N/A //Futures used to indicate when the index file writers are done flushing 4591N/A //their work queues and have exited. End of phase one. 4591N/A //List of index file writer tasks. Used to signal stopIndexWriterTasks to the 4591N/A //index file writer tasks when the LDIF file has been done. 4643N/A //Map of DNs to Suffix objects. 3339N/A * Create a new import job with the specified ldif import config. 4649N/A * @param importConfiguration The LDIF import configuration. 4765N/A * @param localDBBackendCfg The local DB back-end configuration. 4765N/A * @param envConfig The JEB environment config. 4591N/A * @throws IOException If a problem occurs while opening the LDIF file for 4649N/A * @throws InitializationException If a problem occurs during initialization. 4765N/A * Return and import LDIF instance using the specified arguments. 4765N/A * @param importCfg The import config to use. 4765N/A * @param localDBBackendCfg The local DB backend config to use. 4765N/A * @param envCfg The JEB environment config to use. 4765N/A * @return A import LDIF instance. 4765N/A * @throws IOException If an I/O error occurs. 4765N/A * @throws InitializationException If the instance cannot be initialized. 4765N/A * Return an import rebuild index instance using the specified arguments. 4765N/A * @param rebuildCfg The rebuild config to use. 4765N/A * @param localDBBackendCfg The local DB backend config to use. 4765N/A * @param envCfg The JEB environment config to use. 4765N/A * @return An import rebuild index instance. 4765N/A * @throws IOException If an I/O error occurs. 4765N/A * @throws InitializationException If the instance cannot be initialized. 4765N/A * @throws JebException If a JEB exception occurs. 4765N/A * @throws ConfigException If the instance cannot be configured. 4591N/A * Return the suffix instance in the specified map that matches the specified 4591N/A * @param dn The DN to search for. 4591N/A * @param map The map to search. 4591N/A * @return The suffix instance that matches the DN, or null if no match is 4591N/A * Calculate buffer sizes and initialize JEB properties based on memory. 4591N/A * @param envConfig The environment config to use in the calculations. 4591N/A * @throws InitializationException If a problem occurs during calculation. 4643N/A // This entire base DN was explicitly excluded. Skip. 4649N/A There are no branches in the explicitly defined include list under 4649N/A this base DN. Skip this base DN all together. 4643N/A // Remove any overlapping include branches. 4649N/A // Remove any exclude branches that are not are not under a include 4643N/A // branch since they will be migrated as part of the existing entries 4643N/A // outside of the include branches anyways. 4643N/A // This entire base DN is explicitly included in the import with 4643N/A // no exclude branches that we need to migrate. Just clear the entry 4643N/A // Create a temp entry container 4765N/A * Rebuild the indexes using the specified rootcontainer. 4765N/A * @param rootContainer The rootcontainer to rebuild indexes in. 4765N/A * @throws ConfigException If a configuration error occurred. 4765N/A * @throws InitializationException If an initialization error occurred. 4765N/A * @throws IOException If an IO error occurred. 4765N/A * @throws JebException If the JEB database had an error. 4765N/A * @throws DatabaseException If a database error occurred. 4765N/A * @throws InterruptedException If an interrupted error occurred. 4765N/A * @throws ExecutionException If an execution error occurred. 4649N/A * Import a LDIF using the specified root container. 4591N/A * @param rootContainer The root container to use during the import. 4591N/A * @throws ConfigException If the import failed because of an configuration 4591N/A * @throws IOException If the import failed because of an IO error. 4591N/A * @throws InitializationException If the import failed because of an 4591N/A * @throws JebException If the import failed due to a database error. 4591N/A * @throws InterruptedException If the import failed due to an interrupted 4591N/A * @throws ExecutionException If the import failed due to an execution error. 4643N/A * @throws DatabaseException If the import failed due to a database error. 4643N/A //Make sure the unregistered EC for the base DN is the same as 4643N/A //the one in the import context. 4643N/A * Task used to migrate excluded branch. 4643N/A // This is the base entry for a branch that was excluded in the 4643N/A // import so we must migrate all entries in this branch over to 4643N/A // the new entry container. 4643N/A * Task to migrate existing entries. 4660N/A // This is the base entry for a branch that will be included 4660N/A // in the import so we don't want to copy the branch to the 4660N/A * Advance the cursor to next entry at the same level in the 4660N/A * skipping all the entries in this branch. 4660N/A * Set the next starting value to a value of equal length but 4660N/A * slightly greater than the previous DN. Since keys are 4660N/A * compared in reverse order we must set the first byte 4660N/A * No possibility of overflow here. 4643N/A * This task processes the LDIF file during phase 1. 4700N/A //Temporary messages until this code is cleaned up. 4700N/A "ancestorID check failed");
4700N/A //Temporary messages until this code is cleaned up. 4700N/A "ancestorID exception thrown");
4591N/A * The task reads the temporary index files and writes their results to the 4643N/A * This class is used to by a index DB merge thread performing DN processing 4643N/A * to keep track of the state of individual DN2ID index processing. 4591N/A * This task writes the temporary index files using the sorted buffers read 4591N/A * This task main function is to sort the index buffers given to it from 4591N/A * the import tasks reading the LDIF file. It will also create a index 4591N/A * file writer task and corresponding queue if needed. The sorted index 4591N/A * buffers are put on the index file writer queues for writing to a temporary 4591N/A * The buffer class is used to process a buffer from the temporary index files 4591N/A * during phase 2 processing. 4591N/A * The index manager class is used to carry information about index processing 4765N/A * The rebuild manager handles all rebuild index related tasks. 4765N/A * Create an instance of the rebuild index manager using the specified 4765N/A * @param rebuildConfig The rebuild configuration to use. 4765N/A * @param cfg The local DB configuration to use. 4765N/A * Initialize a rebuild manager to start rebuilding indexes. 4765N/A * @throws ConfigException If an configuration error occurred. 4765N/A * @throws InitializationException If an initialization error occurred. 4765N/A * @throws DatabaseException If an database error occurred. 4765N/A * @param startTime The time the rebuild started. 4765N/A * Perform the index rebuild. 4765N/A * @throws DatabaseException If an database error occurred. 4765N/A * @throws InterruptedException If an interrupted error occurred. 4765N/A * @throws ExecutionException If an Excecution error occurred. 4765N/A * @throws JebException If an JEB error occurred. 4765N/A * Return the number of entries processed by the rebuild manager. 4765N/A * @return The number of entries processed. 4765N/A * Return the total number of entries to process by the rebuild manager. 4765N/A * @return The total number for entries to process. 4765N/A * This class reports progress of the rebuild job at fixed intervals. 4765N/A * The number of records that had been processed at the time of the 4765N/A * previous progress report. 4765N/A * The time in milliseconds of the previous progress report. 4765N/A * The environment statistics at the time of the previous report. 4765N/A * Create a new verify progress task. 4765N/A * @throws DatabaseException An error occurred while accessing the JE 4765N/A * The action to be performed by this timer task. 3339N/A * This class reports progress of the import job at fixed intervals. 3339N/A * The number of entries that had been read at the time of the 3339N/A * previous progress report. 3339N/A * The time in milliseconds of the previous progress report. 3339N/A * The environment statistics at the time of the previous report. 4591N/A // Determines if eviction has been detected. 4591N/A // Entry count when eviction was detected. 3339N/A * Create a new import progress task. 4649N/A * The action to be performed by this timer task. 4649N/A // Unlikely to happen and not critical. 4591N/A * This class reports progress of the import job at fixed intervals. 4591N/A * The number of entries that had been read at the time of the 4591N/A * previous progress report. 4591N/A * The time in milliseconds of the previous progress report. 4591N/A * The environment statistics at the time of the previous report. 4591N/A // Determines if eviction has been detected. 4591N/A * Create a new import progress task. 4643N/A * @param indexMgrList List of index managers. 4765N/A * @param latestCount The latest count of entries processed in phase one. 4591N/A * The action to be performed by this timer task. 4591N/A // Unlikely to happen and not critical. 4643N/A * A class to hold information about the entry determined by the LDIF reader. 4643N/A * Return the suffix associated with the entry. 4643N/A * @return Entry's suffix instance; 4643N/A * Set the suffix instance associated with the entry. 4643N/A * @param suffix The suffix associated with the entry. 4643N/A * @param entryID The entry ID to set the entry ID to. 4643N/A * Return the entry ID associated with the entry. 4643N/A * @return The entry ID associated with the entry. 4643N/A * This class defines the individual index type available. 4649N/A * The sub-string index type. 4643N/A * The approximate index type. 4649N/A * The extensible sub-string index type. 4643N/A * The extensible shared index type. 4649N/A * This class is used as an index key for hash maps that need to 4649N/A * process multiple suffix index elements into a single queue and/or maps 4649N/A * based on both attribute type and index type 4649N/A * (ie., cn.equality, sn.equality,...). 4649N/A * It tries to perform some optimization if the index is a sub-string index. 4643N/A * Create index key instance using the specified attribute type, index type. 4643N/A * @param type The attribute type. 4643N/A * @param indexType The index type. 4765N/A * An equals method that uses both the attribute type and the index type. 4765N/A * @param obj the object to compare. 4765N/A * @return <CODE>true</CODE> if the objects are equal. 4649N/A * A hash code method that adds the hash codes of the attribute type and 4643N/A * index type and returns that value. 4643N/A * @return The combined hash values. 4643N/A * Return the attribute type. 4643N/A * @return The attribute type. 4643N/A * Return the index key name, which is the attribute type primary name, 4643N/A * a period, and the index type name. Used for building file names and 4643N/A * @return The index key name.