1b33c96954667ba382fa595baf7b31290bfdd517vboxsync * Copyright (C) 2004, 2008 Mark Adler, all rights reserved
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync * For conditions of distribution and use, see copyright notice in gzlog.h
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync * version 2.0, 25 Apr 2008
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync gzlog provides a mechanism for frequently appending short strings to a gzip
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync file that is efficient both in execution time and compression ratio. The
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync strategy is to write the short strings in an uncompressed form to the end of
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync the gzip file, only compressing when the amount of uncompressed data has
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync reached a given threshold.
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync gzlog also provides protection against interruptions in the process due to
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync system crashes. The status of the operation is recorded in an extra field
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync in the gzip file, and is only updated once the gzip file is brought to a
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync valid state. The last data to be appended or compressed is saved in an
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync auxiliary file, so that if the operation is interrupted, it can be completed
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync the next time an append operation is attempted.
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync gzlog maintains another auxiliary file with the last 32K of data from the
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync compressed portion, which is preloaded for the compression of the subsequent
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync data. This minimizes the impact to the compression ratio of appending.
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync Operations Concept:
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync Files (log name "foo"):
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync foo.gz -- gzip file with the complete log
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync foo.add -- last message to append or last data to compress
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync foo.dict -- dictionary of the last 32K of data for next compression
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync foo.temp -- temporary dictionary file for compression after this one
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync foo.lock -- lock file for reading and writing the other files
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync foo.repairs -- log file for log file recovery operations (not compressed)
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync gzip file structure:
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync - fixed-length (no file name) header with extra field (see below)
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync - compressed data ending initially with empty stored block
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync - uncompressed data filling out originally empty stored block and
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync subsequent stored blocks as needed (16K max each)
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync - gzip trailer
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync - no junk at end (no other gzip streams)
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync When appending data, the information in the first three items above plus the
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync foo.add file are sufficient to recover an interrupted append operation. The
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync extra field has the necessary information to restore the start of the last
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync stored block and determine where to append the data in the foo.add file, as
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync well as the crc and length of the gzip data before the append operation.
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync The foo.add file is created before the gzip file is marked for append, and
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync deleted after the gzip file is marked as complete. So if the append
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync operation is interrupted, the data to add will still be there. If due to
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync some external force, the foo.add file gets deleted between when the append
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync operation was interrupted and when recovery is attempted, the gzip file will
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync still be restored, but without the appended data.
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync When compressing data, the information in the first two items above plus the
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync foo.add file are sufficient to recover an interrupted compress operation.
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync The extra field has the necessary information to find the end of the
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync compressed data, and contains both the crc and length of just the compressed
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync data and of the complete set of data including the contents of the foo.add
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync Again, the foo.add file is maintained during the compress operation in case
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync of an interruption. If in the unlikely event the foo.add file with the data
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync to be compressed is missing due to some external force, a gzip file with
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync just the previous compressed data will be reconstructed. In this case, all
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync of the data that was to be compressed is lost (approximately one megabyte).
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync This will not occur if all that happened was an interruption of the compress
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync The third state that is marked is the replacement of the old dictionary with
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync the new dictionary after a compress operation. Once compression is
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync complete, the gzip file is marked as being in the replace state. This
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync completes the gzip file, so an interrupt after being so marked does not
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync result in recompression. Then the dictionary file is replaced, and the gzip
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync file is marked as completed. This state prevents the possibility of
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync restarting compression with the wrong dictionary file.
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync All three operations are wrapped by a lock/unlock procedure. In order to
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync gain exclusive access to the log files, first a foo.lock file must be
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync exclusively created. When all operations are complete, the lock is
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync released by deleting the foo.lock file. If when attempting to create the
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync lock file, it already exists and the modify time of the lock file is more
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync than five minutes old (set by the PATIENCE define below), then the old
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync lock file is considered stale and deleted, and the exclusive creation of
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync the lock file is retried. To assure that there are no false assessments
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync of the staleness of the lock file, the operations periodically touch the
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync lock file to update the modified date.
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync Following is the definition of the extra field with all of the information
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync required to enable the above append and compress operations and their
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync recovery if interrupted. Multi-byte values are stored little endian
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync (consistent with the gzip format). File pointers are eight bytes long.
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync The crc's and lengths for the gzip trailer are four bytes long. (Note that
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync the length at the end of a gzip file is used for error checking only, and
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync for large files is actually the length modulo 2^32.) The stored block
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync length is two bytes long. The gzip extra field two-byte identification is
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync "ap" for append. It is assumed that writing the extra field to the file is
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync an "atomic" operation. That is, either all of the extra field is written
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync to the file, or none of it is, if the operation is interrupted right at the
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync point of updating the extra field. This is a reasonable assumption, since
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync the extra field is within the first 52 bytes of the file, which is smaller
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync than any expected block size for a mass storage device (usually 512 bytes or
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync Extra field (35 bytes):
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync - Pointer to first stored block length -- this points to the two-byte length
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync of the first stored block, which is followed by the two-byte, one's
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync complement of that length. The stored block length is preceded by the
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync three-bit header of the stored block, which is the actual start of the
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync stored block in the deflate format. See the bit offset field below.
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync - Pointer to the last stored block length. This is the same as above, but
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync for the last stored block of the uncompressed data in the gzip file.
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync Initially this is the same as the first stored block length pointer.
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync When the stored block gets to 16K (see the MAX_STORE define), then a new
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync stored block as added, at which point the last stored block length pointer
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync is different from the first stored block length pointer. When they are
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync different, the first bit of the last stored block header is eight bits, or
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync one byte back from the block length.
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync - Compressed data crc and length. This is the crc and length of the data
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync that is in the compressed portion of the deflate stream. These are used
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync only in the event that the foo.add file containing the data to compress is
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync lost after a compress operation is interrupted.
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync - Total data crc and length. This is the crc and length of all of the data
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync stored in the gzip file, compressed and uncompressed. It is used to
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync reconstruct the gzip trailer when compressing, as well as when recovering
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync interrupted operations.
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync - Final stored block length. This is used to quickly find where to append,
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync and allows the restoration of the original final stored block state when
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync an append operation is interrupted.
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync - First stored block start as the number of bits back from the final stored
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync block first length byte. This value is in the range of 3..10, and is
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync stored as the low three bits of the final byte of the extra field after
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync subtracting three (0..7). This allows the last-block bit of the stored
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync block header to be updated when a new stored block is added, for the case
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync when the first stored block and the last stored block are the same. (When
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync they are different, the numbers of bits back is known to be eight.) This
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync also allows for new compressed data to be appended to the old compressed
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync data in the compress operation, overwriting the previous first stored
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync block, or for the compressed data to be terminated and a valid gzip file
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync reconstructed on the off chance that a compression operation was
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync interrupted and the data to compress in the foo.add file was deleted.
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync - The operation in process. This is the next two bits in the last byte (the
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync bits under the mask 0x18). The are interpreted as 0: nothing in process,
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync 1: append in process, 2: compress in process, 3: replace in process.
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync - The top three bits of the last byte in the extra field are reserved and
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync are currently set to zero.
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync Main procedure:
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync - Exclusively create the foo.lock file using the O_CREAT and O_EXCL modes of
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync the system open() call. If the modify time of an existing lock file is
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync more than PATIENCE seconds old, then the lock file is deleted and the
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync exclusive create is retried.
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync - Load the extra field from the foo.gz file, and see if an operation was in
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync progress but not completed. If so, apply the recovery procedure below.
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync - Perform the append procedure with the provided data.
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync - If the uncompressed data in the foo.gz file is 1MB or more, apply the
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync compress procedure.
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync - Delete the foo.lock file.
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync Append procedure:
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync - Put what to append in the foo.add file so that the operation can be
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync restarted if this procedure is interrupted.
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync - Mark the foo.gz extra field with the append operation in progress.
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync + Restore the original last-block bit and stored block length of the last
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync stored block from the information in the extra field, in case a previous
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync append operation was interrupted.
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync - Append the provided data to the last stored block, creating new stored
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync blocks as needed and updating the stored blocks last-block bits and
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync - Update the crc and length with the new data, and write the gzip trailer.
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync - Write over the extra field (with a single write operation) with the new
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync pointers, lengths, and crc's, and mark the gzip file as not in process.
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync Though there is still a foo.add file, it will be ignored since nothing
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync is in process. If a foo.add file is leftover from a previously
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync completed operation, it is truncated when writing new data to it.
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync - Delete the foo.add file.
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync Compress and replace procedures:
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync - Read all of the uncompressed data in the stored blocks in foo.gz and write
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync it to foo.add. Also write foo.temp with the last 32K of that data to
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync provide a dictionary for the next invocation of this procedure.
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync - Rewrite the extra field marking foo.gz with a compression in process.
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync * If there is no data provided to compress (due to a missing foo.add file
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync when recovering), reconstruct and truncate the foo.gz file to contain
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync only the previous compressed data and proceed to the step after the next
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync one. Otherwise ...
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync - Compress the data with the dictionary in foo.dict, and write to the
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync foo.gz file starting at the bit immediately following the last previously
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync compressed block. If there is no foo.dict, proceed anyway with the
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync compression at slightly reduced efficiency. (For the foo.dict file to be
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync missing requires some external failure beyond simply the interruption of
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync a compress operation.) During this process, the foo.lock file is
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync periodically touched to assure that that file is not considered stale by
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync another process before we're done. The deflation is terminated with a
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync non-last empty static block (10 bits long), that is then located and
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync written over by a last-bit-set empty stored block.
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync - Append the crc and length of the data in the gzip file (previously
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync calculated during the append operations).
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync - Write over the extra field with the updated stored block offsets, bits
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync back, crc's, and lengths, and mark foo.gz as in process for a replacement
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync of the dictionary.
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync @ Delete the foo.add file.
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync - Replace foo.dict with foo.temp.
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync - Write over the extra field, marking foo.gz as complete.
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync Recovery procedure:
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync - If not a replace recovery, read in the foo.add file, and provide that data
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync to the appropriate recovery below. If there is no foo.add file, provide
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync a zero data length to the recovery. In that case, the append recovery
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync restores the foo.gz to the previous compressed + uncompressed data state.
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync For the the compress recovery, a missing foo.add file results in foo.gz
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync being restored to the previous compressed-only data state.
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync - Append recovery:
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync - Pick up append at + step above
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync - Compress recovery:
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync - Pick up compress at * step above
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync - Replace recovery:
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync - Pick up compress at @ step above
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync - Log the repair with a date stamp in foo.repairs
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync#include <stdio.h> /* rename, fopen, fprintf, fclose */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync#include <string.h> /* strlen, strrchr, strcpy, strncpy, strcmp */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync#include <unistd.h> /* lseek, read, write, close, unlink, sleep, */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* ftruncate, fsync */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync/* Macro for debugging to deterministically force recovery operations */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync int gzlog_bail = 0; /* which point to bail at (1..8) */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync int gzlog_count = -1; /* number of times through to wait */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync# define BAIL(n) do { if (n == gzlog_bail && gzlog_count-- == 0) \
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync/* how old the lock file can be in seconds before considering it stale */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync/* maximum stored block size in Kbytes -- must be in 1..63 */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync/* number of stored Kbytes to trigger compression (must be >= 32 to allow
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync dictionary construction, and <= 204 * MAX_STORE, in order for >> 10 to
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync discard the stored block headers contribution of five bytes each) */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync/* size of a deflate dictionary (this cannot be changed) */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync/* values for the operation (2 bits) */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync/* macros to extract little-endian integers from an unsigned byte buffer */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync/* macros to store integers into a byte buffer in little-endian order */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync#define PUT2(p,a) do {(p)[0]=a;(p)[1]=(a)>>8;} while(0)
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync#define PUT4(p,a) do {PUT2(p,a);PUT2(p+2,a>>16);} while(0)
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync#define PUT8(p,a) do {PUT4(p,a);PUT4(p+4,a>>32);} while(0)
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync/* internal structure for log information */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync#define LOGID "\106\035\172" /* should be three non-zero characters */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync char id[4]; /* contains LOGID to detect inadvertent overwrites */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync int fd; /* file descriptor for .gz file, opened read/write */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync char *path; /* allocated path, e.g. "/var/log/foo" or "foo" */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync char *end; /* end of path, for appending suffices such as ".gz" */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync off_t first; /* offset of first stored block first length byte */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync int back; /* location of first block id in bits back from first */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync uint stored; /* bytes currently in last stored block */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync off_t last; /* offset of last stored block first length byte */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync ulong clen; /* length (modulo 2^32) of compressed data */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync ulong tlen; /* length (modulo 2^32) of total data */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync time_t lock; /* last modify time of our lock file */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync/* gzip header for gzlog */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync 0, 0, 0, 0, /* no modification time provided */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync 39, 0, 'a', 'p', 35, 0 /* extra field with "ap" subfield */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* 35 is EXTRA, 39 is EXTRA + 4 */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync/* initial gzip extra field content (52 == HEAD + EXTRA + 1) */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync 52, 0, 0, 0, 0, 0, 0, 0, /* offset of first stored block length */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync 52, 0, 0, 0, 0, 0, 0, 0, /* offset of last stored block length */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync 0, 0, 0, 0, 0, 0, 0, 0, /* compressed data crc and length */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync 0, 0, 0, 0, 0, 0, 0, 0, /* total data crc and length */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync 0, 0, /* final stored block data length */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync/* initial gzip data and trailer */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync 1, 0, 0, 0xff, 0xff, /* empty stored block (last) */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync 0, 0, 0, 0, /* crc */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync 0, 0, 0, 0 /* uncompressed length */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync/* Exclusively create foo.lock in order to negotiate exclusive access to the
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync foo.* files. If the modify time of an existing lock file is greater than
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync PATIENCE seconds in the past, then consider the lock file to have been
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync abandoned, delete it, and try the exclusive create again. Save the lock
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync file modify time for verification of ownership. Return 0 on success, or -1
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync on failure, usually due to an access restriction or invalid path. Note that
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync if stat() or unlink() fails, it may be due to another process noticing the
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync abandoned lock file a smidge sooner and deleting it, so those are not
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync flagged as an error. */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync while ((fd = open(log->path, O_CREAT | O_EXCL, 0644)) < 0) {
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync if (stat(log->path, &st) == 0 && time(NULL) - st.st_mtime > PATIENCE) {
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync sleep(2); /* relinquish the CPU for two seconds while waiting */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync/* Update the modify time of the lock file to now, in order to prevent another
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync task from thinking that the lock is stale. Save the lock file modify time
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync for verification of ownership. */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync/* Check the log file modify time against what is expected. Return true if
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync this is not our lock. If it is our lock, touch it to keep it. */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync if (stat(log->path, &st) || st.st_mtime != log->lock)
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync/* Unlock a previously acquired lock, but only if it's ours. */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync/* Check the gzip header and read in the extra field, filling in the values in
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync the log structure. Return op on success or -1 if the gzip header was not as
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync expected. op is the current operation in progress last written to the extra
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync field. This assumes that the gzip file has already been opened, with the
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync file descriptor log->fd. */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync read(log->fd, buf, HEAD + EXTRA) != HEAD + EXTRA ||
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync/* Write over the extra field contents, marking the operation as op. Use fsync
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync to assure that the device is written to, and in the requested order. This
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync operation, and only this operation, is assumed to be atomic in order to
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync assure that the log is recoverable in the event of an interruption at any
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync point in the process. Return -1 if the write to foo.gz failed. */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync/* Rewrite the last block header bits and subsequent zero bits to get to a byte
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync boundary, setting the last block bit if last is true, and then write the
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync remainder of the stored block header (length and one's complement). Leave
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync the file pointer after the end of the last stored block data. Return -1 if
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync there is a read or write failure on the foo.gz file */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* determine the locations of the bytes and bits to modify */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync len = back > 8 ? 2 : 1; /* bytes back from log->last */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync mask = 0x80 >> ((back - 1) & 7); /* mask for block last-bit */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* get the byte to modify (one or two back) into buf[0] -- don't need to
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync read the byte if the last-bit is eight bits back, since in that case
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync the entire byte will be modified */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync if (back != 8 && (lseek(log->fd, log->last - len, SEEK_SET) < 0 ||
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* change the last-bit of the last stored block as requested -- note
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync that all bits above the last-bit are set to zero, per the type bits
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync of a stored block being 00 and per the convention that the bits to
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync bring the stream to a byte boundary are also zeros */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync buf[2 - len] = (*buf & (mask - 1)) + (last ? mask : 0);
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* write the modified stored block header and lengths, move the file
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync pointer to after the last stored block data */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync return lseek(log->fd, log->last - len, SEEK_SET) < 0 ||
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync write(log->fd, buf + 2 - len, len + 4) != len + 4 ||
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync lseek(log->fd, log->stored, SEEK_CUR) < 0 ? -1 : 0;
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync/* Append len bytes from data to the locked and open log file. len may be zero
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync if recovering and no .add file was found. In that case, the previous state
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync of the foo.gz file is restored. The data is appended uncompressed in
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync deflate stored blocks. Return -1 if there was an error reading or writing
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync the foo.gz file. */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsynclocal int log_append(struct log *log, unsigned char *data, size_t len)
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* set the last block last-bit and length, in case recovering an
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync interrupted append, then position the file pointer to append to the
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* append, adding stored blocks and updating the offset of the last stored
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync block as needed, and update the total crc and length */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* append as much as we can to the last block */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* if we need to, add a new empty stored block */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* mark current block as not last */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* point to new, empty stored block */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* mark last block as last, update its length */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* write the new crc and length trailer, and truncate just in case (could
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync be recovering from partial append with a missing foo.add file) */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync (end = lseek(log->fd, 0, SEEK_CUR)) < 0 || ftruncate(log->fd, end))
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* write the extra field, marking the log file as done, delete .add file */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync unlink(log->path); /* ignore error, since may not exist */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync/* Replace the foo.dict file with the foo.temp file. Also delete the foo.add
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync file, since the compress operation may have been interrupted before that was
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync done. Returns 1 if memory could not be allocated, or -1 if reading or
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync writing foo.gz fails, or if the rename fails for some reason other than
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync foo.temp not existing. foo.temp not existing is a permitted error, since
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync the replace operation may have been interrupted after the rename is done,
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync but before foo.gz is marked as complete. */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* delete foo.add file */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync unlink(log->path); /* ignore error, since may not exist */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* rename foo.name to foo.dict, replacing foo.dict if it exists */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* mark the foo.gz file as done */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync/* Compress the len bytes at data and append the compressed data to the
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync foo.gz deflate data immediately after the previous compressed data. This
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync overwrites the previous uncompressed data, which was stored in foo.add
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync and is the data provided in data[0..len-1]. If this operation is
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync interrupted, it picks up at the start of this routine, with the foo.add
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync file read in again. If there is no data to compress (len == 0), then we
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync simply terminate the foo.gz file after the previously compressed data,
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync appending a final empty stored block and the gzip trailer. Return -1 if
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync reading or writing the log.gz file failed, or -2 if there was a memory
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync allocation failure. */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsynclocal int log_compress(struct log *log, unsigned char *data, size_t len)
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* compress and append compressed data */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* set up for deflate, allocating memory */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync if (deflateInit2(&strm, Z_DEFAULT_COMPRESSION, Z_DEFLATED, -15, 8,
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* read in dictionary (last 32K of data that was compressed) */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync if (fd >= 0) {
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* prime deflate with last bits of previous block, position write
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync pointer to write those bits and overwrite what follows */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync if (lseek(log->fd, log->first - (log->back > 8 ? 2 : 1),
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync read(log->fd, buf, 1) != 1 || lseek(log->fd, -1, SEEK_CUR) < 0) {
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* compress, finishing with a partial non-last empty static block */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync max = (((uint)0 - 1) >> 1) + 1; /* in case int smaller than size_t */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync deflate(&strm, len ? Z_NO_FLUSH : Z_PARTIAL_FLUSH);
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* find start of empty static block -- scanning backwards the first one
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync bit is the second bit of the block, if the last byte is zero, then
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync we know the byte before that has a one in the top bit, since an
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync empty static block is ten bits long */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync if ((log->first = lseek(log->fd, -1, SEEK_CUR)) < 0 ||
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync while ((*buf & ((uint)1 << (8 - log->back++))) == 0)
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync ; /* guaranteed to terminate, since *buf != 0 */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* update compressed crc and length */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* no data to compress -- fix up existing gzip stream */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* complete and truncate gzip stream */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync if (log_last(log, 1) || write(log->fd, buf, 8) != 8 ||
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync (end = lseek(log->fd, 0, SEEK_CUR)) < 0 || ftruncate(log->fd, end))
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* mark as being in the replace operation */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* execute the replace operation and mark the file as done */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync/* log a repair record to the .repairs file */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsynclocal void log_log(struct log *log, int op, char *record)
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync fprintf(rec, "%.24s %s recovery: %s\n", ctime(&now), op == APPEND_OP ?
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync "append" : (op == COMPRESS_OP ? "compress" : "replace"), record);
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync/* Recover the interrupted operation op. First read foo.add for recovering an
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync append or compress operation. Return -1 if there was an error reading or
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync writing foo.gz or reading an existing foo.add, or -2 if there was a memory
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync allocation failure. */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* log recovery */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* load foo.add file if expected and present */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync if (len != st.st_size || (data = malloc(st.st_size)) == NULL) {
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* recover the interrupted operation */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* log status */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* clean up */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync/* Close the foo.gz file (if open) and release the lock. */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync/* Open foo.gz, verify the header, and load the extra field contents, after
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync first creating the foo.lock file to gain exclusive access to the foo.*
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync files. If foo.gz does not exist or is empty, then write the initial header,
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync extra, and body content of an empty foo.gz log file. If there is an error
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync creating the lock file due to access restrictions, or an error reading or
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync writing the foo.gz file, or if the foo.gz file is not a proper log file for
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync this object (e.g. not a gzip file or does not contain the expected extra
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync field), then return true. If there is an error, the lock is released.
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync Otherwise, the lock is left in place. */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* release open file resource if left over -- can occur if lock lost
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync between gzlog_open() and gzlog_write() */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* negotiate exclusive access */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* open the log file, foo.gz */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* if new, initialize foo.gz with an empty log, delete old dictionary */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* verify log file and load extra field information */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* check for interrupted process and if so, recover */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* touch the lock file to prevent another process from grabbing it */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync/* See gzlog.h for the description of the external methods below */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* check arguments */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* allocate and initialize log structure */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* save path and end of path for name construction */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync log->path = malloc(n + 9); /* allow for ".repairs" */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* gain exclusive access and verify log file -- may perform a
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync recovery operation if needed */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* return pointer to log structure */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync/* gzlog_compress() return values:
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync 0: all good
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync -1: file i/o error (usually access issue)
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync -2: memory allocation failure
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync -3: invalid log pointer argument */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* check arguments */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync if (log == NULL || strcmp(log->id, LOGID) || len < 0)
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* see if we lost the lock -- if so get it again and reload the extra
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync field information (it probably changed), recover last operation if
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync necessary */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* create space for uncompressed data */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync len = ((size_t)(log->last - log->first) & ~(((size_t)1 << 10) - 1)) +
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* do statement here is just a cheap trick for error handling */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* read in the uncompressed data */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync read(log->fd, (char *)data + next, block) != block)
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync if (lseek(log->fd, 0, SEEK_CUR) != log->last + 4 + log->stored)
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* write the uncompressed data to the .add file */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync fd = open(log->path, O_WRONLY | O_CREAT | O_TRUNC, 0644);
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* write the dictionary for the next compress to the .temp file */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync fd = open(log->path, O_WRONLY | O_CREAT | O_TRUNC, 0644);
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync ret = write(fd, (char *)data + len - next, next) != next;
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* roll back to compressed data, mark the compress in progress */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* compress and append the data (clears mark) */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync } while (0);
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* broke out of do above on i/o error */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync/* gzlog_write() return values:
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync 0: all good
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync -1: file i/o error (usually access issue)
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync -2: memory allocation failure
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync -3: invalid log pointer argument */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsyncint gzlog_write(gzlog *logd, void *data, size_t len)
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* check arguments */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync if (log == NULL || strcmp(log->id, LOGID) || len < 0)
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* see if we lost the lock -- if so get it again and reload the extra
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync field information (it probably changed), recover last operation if
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync necessary */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* create and write .add file */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync fd = open(log->path, O_WRONLY | O_CREAT | O_TRUNC, 0644);
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* mark log file with append in progress */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* append data (clears mark) */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* check to see if it's time to compress -- if not, then done */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync if (((log->last - log->first) >> 10) + (log->stored >> 10) < TRIGGER)
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* time to compress */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync/* gzlog_close() return values:
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync -3: invalid log pointer argument */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* check arguments */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* close the log file and release the lock */
1b33c96954667ba382fa595baf7b31290bfdd517vboxsync /* free structure and return */