199767f8919635c4928607450d9e0abb932109ceToomas SoomeNetwork Working Group P. Deutsch
199767f8919635c4928607450d9e0abb932109ceToomas SoomeRequest for Comments: 1952 Aladdin Enterprises
199767f8919635c4928607450d9e0abb932109ceToomas SoomeCategory: Informational May 1996
199767f8919635c4928607450d9e0abb932109ceToomas Soome GZIP file format specification version 4.3
199767f8919635c4928607450d9e0abb932109ceToomas SoomeStatus of This Memo
199767f8919635c4928607450d9e0abb932109ceToomas Soome This memo provides information for the Internet community. This memo
199767f8919635c4928607450d9e0abb932109ceToomas Soome does not specify an Internet standard of any kind. Distribution of
199767f8919635c4928607450d9e0abb932109ceToomas Soome this memo is unlimited.
199767f8919635c4928607450d9e0abb932109ceToomas Soome The IESG takes no position on the validity of any Intellectual
199767f8919635c4928607450d9e0abb932109ceToomas Soome Property Rights statements contained in this document.
199767f8919635c4928607450d9e0abb932109ceToomas Soome Copyright (c) 1996 L. Peter Deutsch
199767f8919635c4928607450d9e0abb932109ceToomas Soome Permission is granted to copy and distribute this document for any
199767f8919635c4928607450d9e0abb932109ceToomas Soome purpose and without charge, including translations into other
199767f8919635c4928607450d9e0abb932109ceToomas Soome languages and incorporation into compilations, provided that the
199767f8919635c4928607450d9e0abb932109ceToomas Soome copyright notice and this notice are preserved, and that any
199767f8919635c4928607450d9e0abb932109ceToomas Soome substantive changes or deletions from the original are clearly
199767f8919635c4928607450d9e0abb932109ceToomas Soome A pointer to the latest version of this and related documentation in
199767f8919635c4928607450d9e0abb932109ceToomas Soome HTML format can be found at the URL
199767f8919635c4928607450d9e0abb932109ceToomas Soome <ftp://ftp.uu.net/graphics/png/documents/zlib/zdoc-index.html>.
199767f8919635c4928607450d9e0abb932109ceToomas Soome This specification defines a lossless compressed data format that is
199767f8919635c4928607450d9e0abb932109ceToomas Soome compatible with the widely used GZIP utility. The format includes a
199767f8919635c4928607450d9e0abb932109ceToomas Soome cyclic redundancy check value for detecting data corruption. The
199767f8919635c4928607450d9e0abb932109ceToomas Soome format presently uses the DEFLATE method of compression but can be
199767f8919635c4928607450d9e0abb932109ceToomas Soome easily extended to use other compression methods. The format can be
199767f8919635c4928607450d9e0abb932109ceToomas Soome implemented readily in a manner not covered by patents.
199767f8919635c4928607450d9e0abb932109ceToomas SoomeDeutsch Informational [Page 1]
199767f8919635c4928607450d9e0abb932109ceToomas SoomeRFC 1952 GZIP File Format Specification May 1996
199767f8919635c4928607450d9e0abb932109ceToomas SoomeTable of Contents
199767f8919635c4928607450d9e0abb932109ceToomas Soome 1. Introduction ................................................... 2
199767f8919635c4928607450d9e0abb932109ceToomas Soome 1.1. Purpose ................................................... 2
199767f8919635c4928607450d9e0abb932109ceToomas Soome 1.2. Intended audience ......................................... 3
199767f8919635c4928607450d9e0abb932109ceToomas Soome 1.3. Scope ..................................................... 3
199767f8919635c4928607450d9e0abb932109ceToomas Soome 1.4. Compliance ................................................ 3
199767f8919635c4928607450d9e0abb932109ceToomas Soome 1.5. Definitions of terms and conventions used ................. 3
199767f8919635c4928607450d9e0abb932109ceToomas Soome 1.6. Changes from previous versions ............................ 3
199767f8919635c4928607450d9e0abb932109ceToomas Soome 2. Detailed specification ......................................... 4
199767f8919635c4928607450d9e0abb932109ceToomas Soome 2.1. Overall conventions ....................................... 4
199767f8919635c4928607450d9e0abb932109ceToomas Soome 2.2. File format ............................................... 5
199767f8919635c4928607450d9e0abb932109ceToomas Soome 2.3. Member format ............................................. 5
199767f8919635c4928607450d9e0abb932109ceToomas Soome 2.3.1. Member header and trailer ........................... 6
199767f8919635c4928607450d9e0abb932109ceToomas Soome 2.3.1.1. Extra field ................................... 8
199767f8919635c4928607450d9e0abb932109ceToomas Soome 2.3.1.2. Compliance .................................... 9
199767f8919635c4928607450d9e0abb932109ceToomas Soome 3. References .................................................. 9
199767f8919635c4928607450d9e0abb932109ceToomas Soome 4. Security Considerations .................................... 10
199767f8919635c4928607450d9e0abb932109ceToomas Soome 5. Acknowledgements ........................................... 10
199767f8919635c4928607450d9e0abb932109ceToomas Soome 6. Author's Address ........................................... 10
199767f8919635c4928607450d9e0abb932109ceToomas Soome 7. Appendix: Jean-Loup Gailly's gzip utility .................. 11
199767f8919635c4928607450d9e0abb932109ceToomas Soome 8. Appendix: Sample CRC Code .................................. 11
199767f8919635c4928607450d9e0abb932109ceToomas Soome1. Introduction
199767f8919635c4928607450d9e0abb932109ceToomas Soome 1.1. Purpose
199767f8919635c4928607450d9e0abb932109ceToomas Soome The purpose of this specification is to define a lossless
199767f8919635c4928607450d9e0abb932109ceToomas Soome compressed data format that:
199767f8919635c4928607450d9e0abb932109ceToomas Soome * Is independent of CPU type, operating system, file system,
199767f8919635c4928607450d9e0abb932109ceToomas Soome and character set, and hence can be used for interchange;
199767f8919635c4928607450d9e0abb932109ceToomas Soome * Can compress or decompress a data stream (as opposed to a
199767f8919635c4928607450d9e0abb932109ceToomas Soome randomly accessible file) to produce another data stream,
199767f8919635c4928607450d9e0abb932109ceToomas Soome using only an a priori bounded amount of intermediate
199767f8919635c4928607450d9e0abb932109ceToomas Soome storage, and hence can be used in data communications or
199767f8919635c4928607450d9e0abb932109ceToomas Soome similar structures such as Unix filters;
199767f8919635c4928607450d9e0abb932109ceToomas Soome * Compresses data with efficiency comparable to the best
199767f8919635c4928607450d9e0abb932109ceToomas Soome currently available general-purpose compression methods,
199767f8919635c4928607450d9e0abb932109ceToomas Soome and in particular considerably better than the "compress"
199767f8919635c4928607450d9e0abb932109ceToomas Soome * Can be implemented readily in a manner not covered by
199767f8919635c4928607450d9e0abb932109ceToomas Soome patents, and hence can be practiced freely;
199767f8919635c4928607450d9e0abb932109ceToomas Soome * Is compatible with the file format produced by the current
199767f8919635c4928607450d9e0abb932109ceToomas Soome widely used gzip utility, in that conforming decompressors
199767f8919635c4928607450d9e0abb932109ceToomas Soome will be able to read data produced by the existing gzip
199767f8919635c4928607450d9e0abb932109ceToomas SoomeDeutsch Informational [Page 2]
199767f8919635c4928607450d9e0abb932109ceToomas SoomeRFC 1952 GZIP File Format Specification May 1996
199767f8919635c4928607450d9e0abb932109ceToomas Soome The data format defined by this specification does not attempt to:
199767f8919635c4928607450d9e0abb932109ceToomas Soome * Provide random access to compressed data;
199767f8919635c4928607450d9e0abb932109ceToomas Soome * Compress specialized data (e.g., raster graphics) as well as
199767f8919635c4928607450d9e0abb932109ceToomas Soome the best currently available specialized algorithms.
199767f8919635c4928607450d9e0abb932109ceToomas Soome 1.2. Intended audience
199767f8919635c4928607450d9e0abb932109ceToomas Soome This specification is intended for use by implementors of software
199767f8919635c4928607450d9e0abb932109ceToomas Soome to compress data into gzip format and/or decompress data from gzip
199767f8919635c4928607450d9e0abb932109ceToomas Soome The text of the specification assumes a basic background in
199767f8919635c4928607450d9e0abb932109ceToomas Soome programming at the level of bits and other primitive data
199767f8919635c4928607450d9e0abb932109ceToomas Soome representations.
199767f8919635c4928607450d9e0abb932109ceToomas Soome The specification specifies a compression method and a file format
199767f8919635c4928607450d9e0abb932109ceToomas Soome (the latter assuming only that a file can store a sequence of
199767f8919635c4928607450d9e0abb932109ceToomas Soome arbitrary bytes). It does not specify any particular interface to
199767f8919635c4928607450d9e0abb932109ceToomas Soome a file system or anything about character sets or encodings
199767f8919635c4928607450d9e0abb932109ceToomas Soome (except for file names and comments, which are optional).
199767f8919635c4928607450d9e0abb932109ceToomas Soome 1.4. Compliance
199767f8919635c4928607450d9e0abb932109ceToomas Soome Unless otherwise indicated below, a compliant decompressor must be
199767f8919635c4928607450d9e0abb932109ceToomas Soome able to accept and decompress any file that conforms to all the
199767f8919635c4928607450d9e0abb932109ceToomas Soome specifications presented here; a compliant compressor must produce
199767f8919635c4928607450d9e0abb932109ceToomas Soome files that conform to all the specifications presented here. The
199767f8919635c4928607450d9e0abb932109ceToomas Soome material in the appendices is not part of the specification per se
199767f8919635c4928607450d9e0abb932109ceToomas Soome and is not relevant to compliance.
199767f8919635c4928607450d9e0abb932109ceToomas Soome 1.5. Definitions of terms and conventions used
199767f8919635c4928607450d9e0abb932109ceToomas Soome byte: 8 bits stored or transmitted as a unit (same as an octet).
199767f8919635c4928607450d9e0abb932109ceToomas Soome (For this specification, a byte is exactly 8 bits, even on
199767f8919635c4928607450d9e0abb932109ceToomas Soome machines which store a character on a number of bits different
199767f8919635c4928607450d9e0abb932109ceToomas Soome from 8.) See below for the numbering of bits within a byte.
199767f8919635c4928607450d9e0abb932109ceToomas Soome 1.6. Changes from previous versions
199767f8919635c4928607450d9e0abb932109ceToomas Soome There have been no technical changes to the gzip format since
199767f8919635c4928607450d9e0abb932109ceToomas Soome version 4.1 of this specification. In version 4.2, some
199767f8919635c4928607450d9e0abb932109ceToomas Soome terminology was changed, and the sample CRC code was rewritten for
199767f8919635c4928607450d9e0abb932109ceToomas Soome clarity and to eliminate the requirement for the caller to do pre-
199767f8919635c4928607450d9e0abb932109ceToomas Soome and post-conditioning. Version 4.3 is a conversion of the
199767f8919635c4928607450d9e0abb932109ceToomas Soome specification to RFC style.
199767f8919635c4928607450d9e0abb932109ceToomas SoomeDeutsch Informational [Page 3]
199767f8919635c4928607450d9e0abb932109ceToomas SoomeRFC 1952 GZIP File Format Specification May 1996
199767f8919635c4928607450d9e0abb932109ceToomas Soome2. Detailed specification
199767f8919635c4928607450d9e0abb932109ceToomas Soome 2.1. Overall conventions
199767f8919635c4928607450d9e0abb932109ceToomas Soome In the diagrams below, a box like this:
199767f8919635c4928607450d9e0abb932109ceToomas Soome | | <-- the vertical bars might be missing
199767f8919635c4928607450d9e0abb932109ceToomas Soome represents one byte; a box like this:
199767f8919635c4928607450d9e0abb932109ceToomas Soome +==============+
199767f8919635c4928607450d9e0abb932109ceToomas Soome +==============+
199767f8919635c4928607450d9e0abb932109ceToomas Soome represents a variable number of bytes.
199767f8919635c4928607450d9e0abb932109ceToomas Soome Bytes stored within a computer do not have a "bit order", since
199767f8919635c4928607450d9e0abb932109ceToomas Soome they are always treated as a unit. However, a byte considered as
199767f8919635c4928607450d9e0abb932109ceToomas Soome an integer between 0 and 255 does have a most- and least-
199767f8919635c4928607450d9e0abb932109ceToomas Soome significant bit, and since we write numbers with the most-
199767f8919635c4928607450d9e0abb932109ceToomas Soome significant digit on the left, we also write bytes with the most-
199767f8919635c4928607450d9e0abb932109ceToomas Soome significant bit on the left. In the diagrams below, we number the
199767f8919635c4928607450d9e0abb932109ceToomas Soome bits of a byte so that bit 0 is the least-significant bit, i.e.,
199767f8919635c4928607450d9e0abb932109ceToomas Soome the bits are numbered:
199767f8919635c4928607450d9e0abb932109ceToomas Soome This document does not address the issue of the order in which
199767f8919635c4928607450d9e0abb932109ceToomas Soome bits of a byte are transmitted on a bit-sequential medium, since
199767f8919635c4928607450d9e0abb932109ceToomas Soome the data format described here is byte- rather than bit-oriented.
199767f8919635c4928607450d9e0abb932109ceToomas Soome Within a computer, a number may occupy multiple bytes. All
199767f8919635c4928607450d9e0abb932109ceToomas Soome multi-byte numbers in the format described here are stored with
199767f8919635c4928607450d9e0abb932109ceToomas Soome the least-significant byte first (at the lower memory address).
199767f8919635c4928607450d9e0abb932109ceToomas Soome For example, the decimal number 520 is stored as:
199767f8919635c4928607450d9e0abb932109ceToomas Soome +--------+--------+
199767f8919635c4928607450d9e0abb932109ceToomas Soome |00001000|00000010|
199767f8919635c4928607450d9e0abb932109ceToomas Soome +--------+--------+
199767f8919635c4928607450d9e0abb932109ceToomas Soome | + more significant byte = 2 x 256
199767f8919635c4928607450d9e0abb932109ceToomas Soome + less significant byte = 8
199767f8919635c4928607450d9e0abb932109ceToomas SoomeDeutsch Informational [Page 4]
199767f8919635c4928607450d9e0abb932109ceToomas SoomeRFC 1952 GZIP File Format Specification May 1996
199767f8919635c4928607450d9e0abb932109ceToomas Soome 2.2. File format
199767f8919635c4928607450d9e0abb932109ceToomas Soome A gzip file consists of a series of "members" (compressed data
199767f8919635c4928607450d9e0abb932109ceToomas Soome sets). The format of each member is specified in the following
199767f8919635c4928607450d9e0abb932109ceToomas Soome section. The members simply appear one after another in the file,
199767f8919635c4928607450d9e0abb932109ceToomas Soome with no additional information before, between, or after them.
199767f8919635c4928607450d9e0abb932109ceToomas Soome 2.3. Member format
199767f8919635c4928607450d9e0abb932109ceToomas Soome Each member has the following structure:
199767f8919635c4928607450d9e0abb932109ceToomas Soome +---+---+---+---+---+---+---+---+---+---+
199767f8919635c4928607450d9e0abb932109ceToomas Soome |ID1|ID2|CM |FLG| MTIME |XFL|OS | (more-->)
199767f8919635c4928607450d9e0abb932109ceToomas Soome +---+---+---+---+---+---+---+---+---+---+
199767f8919635c4928607450d9e0abb932109ceToomas Soome +---+---+=================================+
199767f8919635c4928607450d9e0abb932109ceToomas Soome | XLEN |...XLEN bytes of "extra field"...| (more-->)
199767f8919635c4928607450d9e0abb932109ceToomas Soome +---+---+=================================+
199767f8919635c4928607450d9e0abb932109ceToomas Soome +=========================================+
199767f8919635c4928607450d9e0abb932109ceToomas Soome |...original file name, zero-terminated...| (more-->)
199767f8919635c4928607450d9e0abb932109ceToomas Soome +=========================================+
199767f8919635c4928607450d9e0abb932109ceToomas Soome +===================================+
199767f8919635c4928607450d9e0abb932109ceToomas Soome |...file comment, zero-terminated...| (more-->)
199767f8919635c4928607450d9e0abb932109ceToomas Soome +===================================+
199767f8919635c4928607450d9e0abb932109ceToomas Soome +=======================+
199767f8919635c4928607450d9e0abb932109ceToomas Soome |...compressed blocks...| (more-->)
199767f8919635c4928607450d9e0abb932109ceToomas Soome +=======================+
199767f8919635c4928607450d9e0abb932109ceToomas Soome 0 1 2 3 4 5 6 7
199767f8919635c4928607450d9e0abb932109ceToomas Soome +---+---+---+---+---+---+---+---+
199767f8919635c4928607450d9e0abb932109ceToomas Soome | CRC32 | ISIZE |
199767f8919635c4928607450d9e0abb932109ceToomas Soome +---+---+---+---+---+---+---+---+
199767f8919635c4928607450d9e0abb932109ceToomas SoomeDeutsch Informational [Page 5]
199767f8919635c4928607450d9e0abb932109ceToomas SoomeRFC 1952 GZIP File Format Specification May 1996
199767f8919635c4928607450d9e0abb932109ceToomas Soome 2.3.1. Member header and trailer
199767f8919635c4928607450d9e0abb932109ceToomas Soome ID1 (IDentification 1)
199767f8919635c4928607450d9e0abb932109ceToomas Soome ID2 (IDentification 2)
199767f8919635c4928607450d9e0abb932109ceToomas Soome These have the fixed values ID1 = 31 (0x1f, \037), ID2 = 139
199767f8919635c4928607450d9e0abb932109ceToomas Soome (0x8b, \213), to identify the file as being in gzip format.
199767f8919635c4928607450d9e0abb932109ceToomas Soome CM (Compression Method)
199767f8919635c4928607450d9e0abb932109ceToomas Soome This identifies the compression method used in the file. CM
199767f8919635c4928607450d9e0abb932109ceToomas Soome = 0-7 are reserved. CM = 8 denotes the "deflate"
199767f8919635c4928607450d9e0abb932109ceToomas Soome compression method, which is the one customarily used by
199767f8919635c4928607450d9e0abb932109ceToomas Soome gzip and which is documented elsewhere.
199767f8919635c4928607450d9e0abb932109ceToomas Soome This flag byte is divided into individual bits as follows:
199767f8919635c4928607450d9e0abb932109ceToomas Soome bit 2 FEXTRA
199767f8919635c4928607450d9e0abb932109ceToomas Soome bit 4 FCOMMENT
199767f8919635c4928607450d9e0abb932109ceToomas Soome bit 5 reserved
199767f8919635c4928607450d9e0abb932109ceToomas Soome bit 6 reserved
199767f8919635c4928607450d9e0abb932109ceToomas Soome bit 7 reserved
199767f8919635c4928607450d9e0abb932109ceToomas Soome If FTEXT is set, the file is probably ASCII text. This is
199767f8919635c4928607450d9e0abb932109ceToomas Soome an optional indication, which the compressor may set by
199767f8919635c4928607450d9e0abb932109ceToomas Soome checking a small amount of the input data to see whether any
199767f8919635c4928607450d9e0abb932109ceToomas Soome non-ASCII characters are present. In case of doubt, FTEXT
199767f8919635c4928607450d9e0abb932109ceToomas Soome is cleared, indicating binary data. For systems which have
199767f8919635c4928607450d9e0abb932109ceToomas Soome different file formats for ascii text and binary data, the
199767f8919635c4928607450d9e0abb932109ceToomas Soome decompressor can use FTEXT to choose the appropriate format.
199767f8919635c4928607450d9e0abb932109ceToomas Soome We deliberately do not specify the algorithm used to set
199767f8919635c4928607450d9e0abb932109ceToomas Soome this bit, since a compressor always has the option of
199767f8919635c4928607450d9e0abb932109ceToomas Soome leaving it cleared and a decompressor always has the option
199767f8919635c4928607450d9e0abb932109ceToomas Soome of ignoring it and letting some other program handle issues
199767f8919635c4928607450d9e0abb932109ceToomas Soome of data conversion.
199767f8919635c4928607450d9e0abb932109ceToomas Soome If FHCRC is set, a CRC16 for the gzip header is present,
199767f8919635c4928607450d9e0abb932109ceToomas Soome immediately before the compressed data. The CRC16 consists
199767f8919635c4928607450d9e0abb932109ceToomas Soome of the two least significant bytes of the CRC32 for all
199767f8919635c4928607450d9e0abb932109ceToomas Soome bytes of the gzip header up to and not including the CRC16.
199767f8919635c4928607450d9e0abb932109ceToomas Soome [The FHCRC bit was never set by versions of gzip up to
199767f8919635c4928607450d9e0abb932109ceToomas Soome 1.2.4, even though it was documented with a different
199767f8919635c4928607450d9e0abb932109ceToomas Soome meaning in gzip 1.2.4.]
199767f8919635c4928607450d9e0abb932109ceToomas Soome If FEXTRA is set, optional extra fields are present, as
199767f8919635c4928607450d9e0abb932109ceToomas Soome described in a following section.
199767f8919635c4928607450d9e0abb932109ceToomas SoomeDeutsch Informational [Page 6]
199767f8919635c4928607450d9e0abb932109ceToomas SoomeRFC 1952 GZIP File Format Specification May 1996
199767f8919635c4928607450d9e0abb932109ceToomas Soome If FNAME is set, an original file name is present,
199767f8919635c4928607450d9e0abb932109ceToomas Soome terminated by a zero byte. The name must consist of ISO
199767f8919635c4928607450d9e0abb932109ceToomas Soome 8859-1 (LATIN-1) characters; on operating systems using
199767f8919635c4928607450d9e0abb932109ceToomas Soome EBCDIC or any other character set for file names, the name
199767f8919635c4928607450d9e0abb932109ceToomas Soome must be translated to the ISO LATIN-1 character set. This
199767f8919635c4928607450d9e0abb932109ceToomas Soome is the original name of the file being compressed, with any
199767f8919635c4928607450d9e0abb932109ceToomas Soome directory components removed, and, if the file being
199767f8919635c4928607450d9e0abb932109ceToomas Soome compressed is on a file system with case insensitive names,
199767f8919635c4928607450d9e0abb932109ceToomas Soome forced to lower case. There is no original file name if the
199767f8919635c4928607450d9e0abb932109ceToomas Soome data was compressed from a source other than a named file;
199767f8919635c4928607450d9e0abb932109ceToomas Soome for example, if the source was stdin on a Unix system, there
199767f8919635c4928607450d9e0abb932109ceToomas Soome is no file name.
199767f8919635c4928607450d9e0abb932109ceToomas Soome If FCOMMENT is set, a zero-terminated file comment is
199767f8919635c4928607450d9e0abb932109ceToomas Soome present. This comment is not interpreted; it is only
199767f8919635c4928607450d9e0abb932109ceToomas Soome intended for human consumption. The comment must consist of
199767f8919635c4928607450d9e0abb932109ceToomas Soome ISO 8859-1 (LATIN-1) characters. Line breaks should be
199767f8919635c4928607450d9e0abb932109ceToomas Soome denoted by a single line feed character (10 decimal).
199767f8919635c4928607450d9e0abb932109ceToomas Soome Reserved FLG bits must be zero.
199767f8919635c4928607450d9e0abb932109ceToomas Soome MTIME (Modification TIME)
199767f8919635c4928607450d9e0abb932109ceToomas Soome This gives the most recent modification time of the original
199767f8919635c4928607450d9e0abb932109ceToomas Soome file being compressed. The time is in Unix format, i.e.,
199767f8919635c4928607450d9e0abb932109ceToomas Soome seconds since 00:00:00 GMT, Jan. 1, 1970. (Note that this
199767f8919635c4928607450d9e0abb932109ceToomas Soome may cause problems for MS-DOS and other systems that use
199767f8919635c4928607450d9e0abb932109ceToomas Soome local rather than Universal time.) If the compressed data
199767f8919635c4928607450d9e0abb932109ceToomas Soome did not come from a file, MTIME is set to the time at which
199767f8919635c4928607450d9e0abb932109ceToomas Soome compression started. MTIME = 0 means no time stamp is
199767f8919635c4928607450d9e0abb932109ceToomas Soome XFL (eXtra FLags)
199767f8919635c4928607450d9e0abb932109ceToomas Soome These flags are available for use by specific compression
199767f8919635c4928607450d9e0abb932109ceToomas Soome methods. The "deflate" method (CM = 8) sets these flags as
199767f8919635c4928607450d9e0abb932109ceToomas Soome XFL = 2 - compressor used maximum compression,
199767f8919635c4928607450d9e0abb932109ceToomas Soome slowest algorithm
199767f8919635c4928607450d9e0abb932109ceToomas Soome XFL = 4 - compressor used fastest algorithm
199767f8919635c4928607450d9e0abb932109ceToomas Soome OS (Operating System)
199767f8919635c4928607450d9e0abb932109ceToomas Soome This identifies the type of file system on which compression
199767f8919635c4928607450d9e0abb932109ceToomas Soome took place. This may be useful in determining end-of-line
199767f8919635c4928607450d9e0abb932109ceToomas Soome convention for text files. The currently defined values are
199767f8919635c4928607450d9e0abb932109ceToomas SoomeDeutsch Informational [Page 7]
199767f8919635c4928607450d9e0abb932109ceToomas SoomeRFC 1952 GZIP File Format Specification May 1996
199767f8919635c4928607450d9e0abb932109ceToomas Soome 0 - FAT filesystem (MS-DOS, OS/2, NT/Win32)
199767f8919635c4928607450d9e0abb932109ceToomas Soome 2 - VMS (or OpenVMS)
199767f8919635c4928607450d9e0abb932109ceToomas Soome 5 - Atari TOS
199767f8919635c4928607450d9e0abb932109ceToomas Soome 6 - HPFS filesystem (OS/2, NT)
199767f8919635c4928607450d9e0abb932109ceToomas Soome 7 - Macintosh
199767f8919635c4928607450d9e0abb932109ceToomas Soome 8 - Z-System
199767f8919635c4928607450d9e0abb932109ceToomas Soome 10 - TOPS-20
199767f8919635c4928607450d9e0abb932109ceToomas Soome 11 - NTFS filesystem (NT)
199767f8919635c4928607450d9e0abb932109ceToomas Soome 13 - Acorn RISCOS
199767f8919635c4928607450d9e0abb932109ceToomas Soome 255 - unknown
199767f8919635c4928607450d9e0abb932109ceToomas Soome XLEN (eXtra LENgth)
199767f8919635c4928607450d9e0abb932109ceToomas Soome If FLG.FEXTRA is set, this gives the length of the optional
199767f8919635c4928607450d9e0abb932109ceToomas Soome extra field. See below for details.
199767f8919635c4928607450d9e0abb932109ceToomas Soome CRC32 (CRC-32)
199767f8919635c4928607450d9e0abb932109ceToomas Soome This contains a Cyclic Redundancy Check value of the
199767f8919635c4928607450d9e0abb932109ceToomas Soome uncompressed data computed according to CRC-32 algorithm
199767f8919635c4928607450d9e0abb932109ceToomas Soome used in the ISO 3309 standard and in section 8.1.1.6.2 of
199767f8919635c4928607450d9e0abb932109ceToomas Soome ITU-T recommendation V.42. (See http://www.iso.ch for
199767f8919635c4928607450d9e0abb932109ceToomas Soome ordering ISO documents. See gopher://info.itu.ch for an
199767f8919635c4928607450d9e0abb932109ceToomas Soome online version of ITU-T V.42.)
199767f8919635c4928607450d9e0abb932109ceToomas Soome ISIZE (Input SIZE)
199767f8919635c4928607450d9e0abb932109ceToomas Soome This contains the size of the original (uncompressed) input
199767f8919635c4928607450d9e0abb932109ceToomas Soome data modulo 2^32.
199767f8919635c4928607450d9e0abb932109ceToomas Soome 2.3.1.1. Extra field
199767f8919635c4928607450d9e0abb932109ceToomas Soome If the FLG.FEXTRA bit is set, an "extra field" is present in
199767f8919635c4928607450d9e0abb932109ceToomas Soome the header, with total length XLEN bytes. It consists of a
199767f8919635c4928607450d9e0abb932109ceToomas Soome series of subfields, each of the form:
199767f8919635c4928607450d9e0abb932109ceToomas Soome +---+---+---+---+==================================+
199767f8919635c4928607450d9e0abb932109ceToomas Soome |SI1|SI2| LEN |... LEN bytes of subfield data ...|
199767f8919635c4928607450d9e0abb932109ceToomas Soome +---+---+---+---+==================================+
199767f8919635c4928607450d9e0abb932109ceToomas Soome SI1 and SI2 provide a subfield ID, typically two ASCII letters
199767f8919635c4928607450d9e0abb932109ceToomas Soome with some mnemonic value. Jean-Loup Gailly
199767f8919635c4928607450d9e0abb932109ceToomas Soome <gzip@prep.ai.mit.edu> is maintaining a registry of subfield
199767f8919635c4928607450d9e0abb932109ceToomas Soome IDs; please send him any subfield ID you wish to use. Subfield
199767f8919635c4928607450d9e0abb932109ceToomas Soome IDs with SI2 = 0 are reserved for future use. The following
199767f8919635c4928607450d9e0abb932109ceToomas Soome IDs are currently defined:
199767f8919635c4928607450d9e0abb932109ceToomas SoomeDeutsch Informational [Page 8]
199767f8919635c4928607450d9e0abb932109ceToomas SoomeRFC 1952 GZIP File Format Specification May 1996
199767f8919635c4928607450d9e0abb932109ceToomas Soome SI1 SI2 Data
199767f8919635c4928607450d9e0abb932109ceToomas Soome ---------- ---------- ----
199767f8919635c4928607450d9e0abb932109ceToomas Soome 0x41 ('A') 0x70 ('P') Apollo file type information
199767f8919635c4928607450d9e0abb932109ceToomas Soome LEN gives the length of the subfield data, excluding the 4
199767f8919635c4928607450d9e0abb932109ceToomas Soome initial bytes.
199767f8919635c4928607450d9e0abb932109ceToomas Soome 2.3.1.2. Compliance
199767f8919635c4928607450d9e0abb932109ceToomas Soome A compliant compressor must produce files with correct ID1,
199767f8919635c4928607450d9e0abb932109ceToomas Soome ID2, CM, CRC32, and ISIZE, but may set all the other fields in
199767f8919635c4928607450d9e0abb932109ceToomas Soome the fixed-length part of the header to default values (255 for
199767f8919635c4928607450d9e0abb932109ceToomas Soome OS, 0 for all others). The compressor must set all reserved
199767f8919635c4928607450d9e0abb932109ceToomas Soome bits to zero.
199767f8919635c4928607450d9e0abb932109ceToomas Soome A compliant decompressor must check ID1, ID2, and CM, and
199767f8919635c4928607450d9e0abb932109ceToomas Soome provide an error indication if any of these have incorrect
199767f8919635c4928607450d9e0abb932109ceToomas Soome values. It must examine FEXTRA/XLEN, FNAME, FCOMMENT and FHCRC
199767f8919635c4928607450d9e0abb932109ceToomas Soome at least so it can skip over the optional fields if they are
199767f8919635c4928607450d9e0abb932109ceToomas Soome present. It need not examine any other part of the header or
199767f8919635c4928607450d9e0abb932109ceToomas Soome trailer; in particular, a decompressor may ignore FTEXT and OS
199767f8919635c4928607450d9e0abb932109ceToomas Soome and always produce binary output, and still be compliant. A
199767f8919635c4928607450d9e0abb932109ceToomas Soome compliant decompressor must give an error indication if any
199767f8919635c4928607450d9e0abb932109ceToomas Soome reserved bit is non-zero, since such a bit could indicate the
199767f8919635c4928607450d9e0abb932109ceToomas Soome presence of a new field that would cause subsequent data to be
199767f8919635c4928607450d9e0abb932109ceToomas Soome interpreted incorrectly.
199767f8919635c4928607450d9e0abb932109ceToomas Soome3. References
199767f8919635c4928607450d9e0abb932109ceToomas Soome [1] "Information Processing - 8-bit single-byte coded graphic
199767f8919635c4928607450d9e0abb932109ceToomas Soome character sets - Part 1: Latin alphabet No.1" (ISO 8859-1:1987).
199767f8919635c4928607450d9e0abb932109ceToomas Soome The ISO 8859-1 (Latin-1) character set is a superset of 7-bit
199767f8919635c4928607450d9e0abb932109ceToomas Soome ASCII. Files defining this character set are available as
199767f8919635c4928607450d9e0abb932109ceToomas Soome iso_8859-1.* in ftp://ftp.uu.net/graphics/png/documents/
199767f8919635c4928607450d9e0abb932109ceToomas Soome [2] ISO 3309
199767f8919635c4928607450d9e0abb932109ceToomas Soome [3] ITU-T recommendation V.42
199767f8919635c4928607450d9e0abb932109ceToomas Soome [4] Deutsch, L.P.,"DEFLATE Compressed Data Format Specification",
199767f8919635c4928607450d9e0abb932109ceToomas Soome available in ftp://ftp.uu.net/pub/archiving/zip/doc/
199767f8919635c4928607450d9e0abb932109ceToomas Soome [5] Gailly, J.-L., GZIP documentation, available as gzip-*.tar in
199767f8919635c4928607450d9e0abb932109ceToomas Soome [6] Sarwate, D.V., "Computation of Cyclic Redundancy Checks via Table
199767f8919635c4928607450d9e0abb932109ceToomas Soome Look-Up", Communications of the ACM, 31(8), pp.1008-1013.
199767f8919635c4928607450d9e0abb932109ceToomas SoomeDeutsch Informational [Page 9]
199767f8919635c4928607450d9e0abb932109ceToomas SoomeRFC 1952 GZIP File Format Specification May 1996
199767f8919635c4928607450d9e0abb932109ceToomas Soome [7] Schwaderer, W.D., "CRC Calculation", April 85 PC Tech Journal,
199767f8919635c4928607450d9e0abb932109ceToomas Soome [8] ftp://ftp.adelaide.edu.au/pub/rocksoft/papers/crc_v3.txt,
199767f8919635c4928607450d9e0abb932109ceToomas Soome describing the CRC concept.
199767f8919635c4928607450d9e0abb932109ceToomas Soome4. Security Considerations
199767f8919635c4928607450d9e0abb932109ceToomas Soome Any data compression method involves the reduction of redundancy in
199767f8919635c4928607450d9e0abb932109ceToomas Soome the data. Consequently, any corruption of the data is likely to have
199767f8919635c4928607450d9e0abb932109ceToomas Soome severe effects and be difficult to correct. Uncompressed text, on
199767f8919635c4928607450d9e0abb932109ceToomas Soome the other hand, will probably still be readable despite the presence
199767f8919635c4928607450d9e0abb932109ceToomas Soome of some corrupted bytes.
199767f8919635c4928607450d9e0abb932109ceToomas Soome It is recommended that systems using this data format provide some
199767f8919635c4928607450d9e0abb932109ceToomas Soome means of validating the integrity of the compressed data, such as by
199767f8919635c4928607450d9e0abb932109ceToomas Soome setting and checking the CRC-32 check value.
199767f8919635c4928607450d9e0abb932109ceToomas Soome5. Acknowledgements
199767f8919635c4928607450d9e0abb932109ceToomas Soome Trademarks cited in this document are the property of their
199767f8919635c4928607450d9e0abb932109ceToomas Soome respective owners.
199767f8919635c4928607450d9e0abb932109ceToomas Soome Jean-Loup Gailly designed the gzip format and wrote, with Mark Adler,
199767f8919635c4928607450d9e0abb932109ceToomas Soome the related software described in this specification. Glenn
199767f8919635c4928607450d9e0abb932109ceToomas Soome Randers-Pehrson converted this document to RFC and HTML format.
199767f8919635c4928607450d9e0abb932109ceToomas Soome6. Author's Address
199767f8919635c4928607450d9e0abb932109ceToomas Soome L. Peter Deutsch
199767f8919635c4928607450d9e0abb932109ceToomas Soome Aladdin Enterprises
199767f8919635c4928607450d9e0abb932109ceToomas Soome 203 Santa Margarita Ave.
199767f8919635c4928607450d9e0abb932109ceToomas Soome Menlo Park, CA 94025
199767f8919635c4928607450d9e0abb932109ceToomas Soome Phone: (415) 322-0103 (AM only)
199767f8919635c4928607450d9e0abb932109ceToomas Soome FAX: (415) 322-1734
199767f8919635c4928607450d9e0abb932109ceToomas Soome EMail: <ghost@aladdin.com>
199767f8919635c4928607450d9e0abb932109ceToomas Soome Questions about the technical content of this specification can be
199767f8919635c4928607450d9e0abb932109ceToomas Soome sent by email to:
199767f8919635c4928607450d9e0abb932109ceToomas Soome Jean-Loup Gailly <gzip@prep.ai.mit.edu> and
199767f8919635c4928607450d9e0abb932109ceToomas Soome Mark Adler <madler@alumni.caltech.edu>
199767f8919635c4928607450d9e0abb932109ceToomas Soome Editorial comments on this specification can be sent by email to:
199767f8919635c4928607450d9e0abb932109ceToomas Soome L. Peter Deutsch <ghost@aladdin.com> and
199767f8919635c4928607450d9e0abb932109ceToomas Soome Glenn Randers-Pehrson <randeg@alumni.rpi.edu>
199767f8919635c4928607450d9e0abb932109ceToomas SoomeDeutsch Informational [Page 10]
199767f8919635c4928607450d9e0abb932109ceToomas SoomeRFC 1952 GZIP File Format Specification May 1996
199767f8919635c4928607450d9e0abb932109ceToomas Soome7. Appendix: Jean-Loup Gailly's gzip utility
199767f8919635c4928607450d9e0abb932109ceToomas Soome The most widely used implementation of gzip compression, and the
199767f8919635c4928607450d9e0abb932109ceToomas Soome original documentation on which this specification is based, were
199767f8919635c4928607450d9e0abb932109ceToomas Soome created by Jean-Loup Gailly <gzip@prep.ai.mit.edu>. Since this
199767f8919635c4928607450d9e0abb932109ceToomas Soome implementation is a de facto standard, we mention some more of its
199767f8919635c4928607450d9e0abb932109ceToomas Soome features here. Again, the material in this section is not part of
199767f8919635c4928607450d9e0abb932109ceToomas Soome the specification per se, and implementations need not follow it to
199767f8919635c4928607450d9e0abb932109ceToomas Soome be compliant.
199767f8919635c4928607450d9e0abb932109ceToomas Soome When compressing or decompressing a file, gzip preserves the
199767f8919635c4928607450d9e0abb932109ceToomas Soome protection, ownership, and modification time attributes on the local
199767f8919635c4928607450d9e0abb932109ceToomas Soome file system, since there is no provision for representing protection
199767f8919635c4928607450d9e0abb932109ceToomas Soome attributes in the gzip file format itself. Since the file format
199767f8919635c4928607450d9e0abb932109ceToomas Soome includes a modification time, the gzip decompressor provides a
199767f8919635c4928607450d9e0abb932109ceToomas Soome command line switch that assigns the modification time from the file,
199767f8919635c4928607450d9e0abb932109ceToomas Soome rather than the local modification time of the compressed input, to
199767f8919635c4928607450d9e0abb932109ceToomas Soome the decompressed output.
199767f8919635c4928607450d9e0abb932109ceToomas Soome8. Appendix: Sample CRC Code
199767f8919635c4928607450d9e0abb932109ceToomas Soome The following sample code represents a practical implementation of
199767f8919635c4928607450d9e0abb932109ceToomas Soome the CRC (Cyclic Redundancy Check). (See also ISO 3309 and ITU-T V.42
199767f8919635c4928607450d9e0abb932109ceToomas Soome for a formal specification.)
199767f8919635c4928607450d9e0abb932109ceToomas Soome The sample code is in the ANSI C programming language. Non C users
199767f8919635c4928607450d9e0abb932109ceToomas Soome may find it easier to read with these hints:
199767f8919635c4928607450d9e0abb932109ceToomas Soome & Bitwise AND operator.
199767f8919635c4928607450d9e0abb932109ceToomas Soome ^ Bitwise exclusive-OR operator.
199767f8919635c4928607450d9e0abb932109ceToomas Soome >> Bitwise right shift operator. When applied to an
199767f8919635c4928607450d9e0abb932109ceToomas Soome unsigned quantity, as here, right shift inserts zero
199767f8919635c4928607450d9e0abb932109ceToomas Soome bit(s) at the left.
199767f8919635c4928607450d9e0abb932109ceToomas Soome ! Logical NOT operator.
199767f8919635c4928607450d9e0abb932109ceToomas Soome ++ "n++" increments the variable n.
199767f8919635c4928607450d9e0abb932109ceToomas Soome 0xNNN 0x introduces a hexadecimal (base 16) constant.
199767f8919635c4928607450d9e0abb932109ceToomas Soome Suffix L indicates a long value (at least 32 bits).
199767f8919635c4928607450d9e0abb932109ceToomas Soome /* Table of CRCs of all 8-bit messages. */
199767f8919635c4928607450d9e0abb932109ceToomas Soome unsigned long crc_table[256];
199767f8919635c4928607450d9e0abb932109ceToomas Soome /* Flag: has the table been computed? Initially false. */
199767f8919635c4928607450d9e0abb932109ceToomas Soome int crc_table_computed = 0;
199767f8919635c4928607450d9e0abb932109ceToomas Soome /* Make the table for a fast CRC. */
199767f8919635c4928607450d9e0abb932109ceToomas Soome void make_crc_table(void)
199767f8919635c4928607450d9e0abb932109ceToomas Soome unsigned long c;
199767f8919635c4928607450d9e0abb932109ceToomas SoomeDeutsch Informational [Page 11]
199767f8919635c4928607450d9e0abb932109ceToomas SoomeRFC 1952 GZIP File Format Specification May 1996
199767f8919635c4928607450d9e0abb932109ceToomas Soome for (n = 0; n < 256; n++) {
199767f8919635c4928607450d9e0abb932109ceToomas Soome c = (unsigned long) n;
199767f8919635c4928607450d9e0abb932109ceToomas Soome for (k = 0; k < 8; k++) {
199767f8919635c4928607450d9e0abb932109ceToomas Soome if (c & 1) {
199767f8919635c4928607450d9e0abb932109ceToomas Soome c = 0xedb88320L ^ (c >> 1);
199767f8919635c4928607450d9e0abb932109ceToomas Soome crc_table[n] = c;
199767f8919635c4928607450d9e0abb932109ceToomas Soome crc_table_computed = 1;
199767f8919635c4928607450d9e0abb932109ceToomas Soome Update a running crc with the bytes buf[0..len-1] and return
199767f8919635c4928607450d9e0abb932109ceToomas Soome the updated crc. The crc should be initialized to zero. Pre- and
199767f8919635c4928607450d9e0abb932109ceToomas Soome post-conditioning (one's complement) is performed within this
199767f8919635c4928607450d9e0abb932109ceToomas Soome function so it shouldn't be done by the caller. Usage example:
199767f8919635c4928607450d9e0abb932109ceToomas Soome unsigned long crc = 0L;
199767f8919635c4928607450d9e0abb932109ceToomas Soome while (read_buffer(buffer, length) != EOF) {
199767f8919635c4928607450d9e0abb932109ceToomas Soome crc = update_crc(crc, buffer, length);
199767f8919635c4928607450d9e0abb932109ceToomas Soome if (crc != original_crc) error();
199767f8919635c4928607450d9e0abb932109ceToomas Soome unsigned long update_crc(unsigned long crc,
199767f8919635c4928607450d9e0abb932109ceToomas Soome unsigned char *buf, int len)
199767f8919635c4928607450d9e0abb932109ceToomas Soome unsigned long c = crc ^ 0xffffffffL;
199767f8919635c4928607450d9e0abb932109ceToomas Soome if (!crc_table_computed)
199767f8919635c4928607450d9e0abb932109ceToomas Soome make_crc_table();
199767f8919635c4928607450d9e0abb932109ceToomas Soome for (n = 0; n < len; n++) {
199767f8919635c4928607450d9e0abb932109ceToomas Soome c = crc_table[(c ^ buf[n]) & 0xff] ^ (c >> 8);
199767f8919635c4928607450d9e0abb932109ceToomas Soome return c ^ 0xffffffffL;
199767f8919635c4928607450d9e0abb932109ceToomas Soome /* Return the CRC of the bytes buf[0..len-1]. */
199767f8919635c4928607450d9e0abb932109ceToomas Soome unsigned long crc(unsigned char *buf, int len)
199767f8919635c4928607450d9e0abb932109ceToomas Soome return update_crc(0L, buf, len);
199767f8919635c4928607450d9e0abb932109ceToomas SoomeDeutsch Informational [Page 12]