Importer.java revision 3598
3339N/A * The contents of this file are subject to the terms of the 3339N/A * Common Development and Distribution License, Version 1.0 only 3339N/A * (the "License"). You may not use this file except in compliance 3339N/A * You can obtain a copy of the license at 3339N/A * See the License for the specific language governing permissions 3339N/A * and limitations under the License. 3339N/A * When distributing Covered Code, include this CDDL HEADER in each 3339N/A * file and include the License file at 3339N/A * add the following below this CDDL HEADER, with the fields enclosed 3339N/A * by brackets "[]" replaced with your own identifying information: 3339N/A * Portions Copyright [yyyy] [name of copyright owner] 3339N/A * Copyright 2008 Sun Microsystems, Inc. 3339N/A * The tracer object for the debug logger. 3339N/A * The JE backend configuration. 3339N/A * The root container used for this import job. 3339N/A * The LDIF import configuration. 3339N/A * Map of base DNs to their import context. 3339N/A * The number of entries migrated. 3339N/A * The number of entries imported. 3339N/A * The number of milliseconds between job progress reports. 3339N/A * The progress report timer. 3339N/A //Number of entries import before checking if cleaning is needed after 3339N/A //eviction has been detected. 3339N/A //Minimum buffer amount to give to a buffer manager. 3339N/A //Total available memory for the buffer managers. 3339N/A //Memory size to be used for the DB cache in string format. 3339N/A //Used to do an initial clean after eviction has been detected. 3339N/A //A thread threw an Runtime exception stop the import. 3598N/A //Set to true if substring indexes are defined. 3339N/A * Create a new import job with the specified ldif import config. 3339N/A * @param ldifImportConfig The LDIF import config. 3598N/A * @param hasSubIndexes <CODE>True</CODE> If substring indexes are defined. 3339N/A * Start the worker threads. 3339N/A * @throws DatabaseException If a DB problem occurs. 3339N/A //Figure out how much buffer memory to give to each context. 3339N/A //Below min, use the min value. 3339N/A // Start a timer for the progress report. 3339N/A //Used to get at extra functionality such as eviction detected. 3339N/A * Import a ldif using the specified root container. 3339N/A * @param rootContainer The root container. 3339N/A * @throws DatabaseException If a DB error occurs. 3339N/A * @throws IOException If a IO error occurs. 3339N/A * @throws org.opends.server.backends.jeb.JebException If a JEB error occurs. 3339N/A * @throws DirectoryException If a directory error occurs. 3339N/A * @throws ConfigException If a configuration has an error. 3339N/A // Create an LDIF reader. Throws an exception if the file does not exist. 3339N/A // Make a note of the time we started. 3339N/A * Switch containers if the migrated entries were written to the temporary 3339N/A * @throws DatabaseException If a DB problem occurs. 3339N/A * @throws JebException If a JEB problem occurs. 3339N/A //Make sure the unregistered EC for the base DN is the same as 3339N/A //the one in the import context. 3339N/A "%s is not the same as the source entry container used " +
3339N/A * Create and log messages at the end of the successful import. 3339N/A * @param startTime The time the import started. 3339N/A * Run the cleaner if it is needed. 3339N/A * @param entriesRead The number of entries read so far. 3339N/A * @param evictEntryNumber The number of entries to run the cleaner after 3339N/A * @throws DatabaseException If a DB problem occurs. 3339N/A //Make sure work queue is empty before starting. 3339N/A * Run the cleaner, pausing the task thread output. 3339N/A * @param header Message to be printed before cleaning. 3339N/A * @throws DatabaseException If a DB problem occurs. 3339N/A //Need to force a checkpoint. 3339N/A //Actually clean the files. 3339N/A //This checkpoint removes the files if any were cleaned. 3339N/A * @throws JebException If a JEB problem occurs. 3339N/A * @throws DatabaseException If a DB problem occurs. 3339N/A * @throws IOException If an IO exception occurs. 3339N/A // Route it according to base DN. 3339N/A //If the progress task has noticed eviction proceeding, start running 3339N/A * Process an entry using the specified import context. 3339N/A * @param DNContext The import context. 3339N/A * @param entry The entry to process. 3339N/A //Add this DN to the pending map. 3339N/A * Add work item to specified import context's queue. 3339N/A * @param context The import context. 3339N/A * @param item The work item to add. 3339N/A * @return <CODE>True</CODE> if the the work item was added to the queue. 3339N/A // All worker threads died. We must stop now. 3339N/A * Wait until the work queue is empty. 3339N/A //Stop work threads telling them to skip substring flush. 3339N/A * @param abort <CODE>True</CODE> if stop work threads was called from an 3339N/A * @throws JebException if a Jeb error occurs. 3339N/A // Wait for each thread to stop. 3339N/A * Clean up after a successful import. 3339N/A * @throws DatabaseException If a DB error occurs. 3339N/A * @throws JebException If a Jeb error occurs. 3430N/A //Flush the buffer managers. 3339N/A * Uncaught exception handler. 3339N/A * @param t The thread working when the exception was thrown. 3339N/A * Get the entry limit exceeded counts from the indexes. 3339N/A * @return Count of the index with entry limit exceeded values. 3339N/A * Return an import context related to the specified DN. 3339N/A * @return An import context. 3339N/A * @throws DirectoryException If an directory error occurs. 3339N/A // The entry should not have been given to this backend. 3339N/A * Creates an import context for the specified entry container. 3339N/A * @param entryContainer The entry container. 3339N/A * @return Import context to use during import. 3339N/A * @throws DatabaseException If a database error occurs. 3339N/A * @throws JebException If a JEB error occurs. 3339N/A * @throws ConfigException If a configuration contains error. 3339N/A // This entire base DN was explicitly excluded. Skip. 3339N/A // There are no branches in the explicitly defined include list under 3339N/A // this base DN. Skip this base DN alltogether. 3339N/A // Remove any overlapping include branches. 3339N/A // Remvoe any exclude branches that are not are not under a include 3339N/A // branch since they will be migrated as part of the existing entries 3339N/A // outside of the include branches anyways. 3339N/A // This entire base DN is explicitly included in the import with 3339N/A // no exclude branches that we need to migrate. Just clear the entry 3339N/A // Create a temp entry container 3339N/A // Create an import context. 3339N/A // Set the include and exclude branches 3339N/A * Add specified context and entry to the work queue. 3339N/A * @param context The context related to the entry DN. 3339N/A * @param entry The entry to work on. 3339N/A * @return <CODE>True</CODE> if the element was added to the work queue. 3339N/A * Calculate the memory usage for the substring buffer and the DB cache. 3598N/A //If there are now substring indexes defined, set the DB cache 3598N/A //size to 60% and take a minimal substring buffer. 3339N/A * Return the string representation of the DB cache size. 3339N/A * @return DB cache size string. 3339N/A * Migrate any existing entries. 3339N/A * @throws JebException If a JEB error occurs. 3339N/A * @throws DatabaseException If a DB error occurs. 3339N/A * @throws DirectoryException If a directory error occurs. 3339N/A // This is the base entry for a branch that will be included 3339N/A // in the import so we don't want to copy the branch to the new 3339N/A * Advance the cursor to next entry at the same level in the DIT 3339N/A * skipping all the entries in this branch. 3339N/A * Set the next starting value to a value of equal length but 3339N/A * slightly greater than the previous DN. Since keys are compared 3339N/A * in reverse order we must set the first byte (the comma). 3339N/A * No possibility of overflow here. 3339N/A * Migrate excluded entries. 3339N/A * @throws JebException If a JEB error occurs. 3339N/A * @throws DatabaseException If a DB error occurs. 3339N/A * @throws DirectoryException If a directory error occurs. 3339N/A // This is the base entry for a branch that was excluded in the 3339N/A // import so we must migrate all entries in this branch over to 3339N/A // the new entry container. 3339N/A * This class reports progress of the import job at fixed intervals. 3339N/A * The number of entries that had been read at the time of the 3339N/A * previous progress report. 3339N/A * The time in milliseconds of the previous progress report. 3339N/A * The environment statistics at the time of the previous report. 3339N/A * The number of bytes in a megabyte. 3339N/A * Note that 1024*1024 bytes may eventually become known as a mebibyte(MiB). 3339N/A //Determines if the ldif is being read. 3339N/A //Determines if eviction has been detected. 3339N/A //Entry count when eviction was detected. 3339N/A * Create a new import progress task. 3339N/A * @throws DatabaseException If an error occurs in the JE database. 3339N/A * Return if reading the LDIF file. 3339N/A * Return value of evicting flag. 3339N/A * @return <CODE>True</CODE> if eviction is detected. 3339N/A * Return count of entries when eviction was detected. 3339N/A * @return The entry count when eviction was detected. 3339N/A * @param v The value to set the suspend value to. 3339N/A * The action to be performed by this timer task. 3339N/A // Unlikely to happen and not critical.