|
|
|
|
|
:mod:`zlib` --- Compression compatible with :program:`gzip`
|
|
|
===========================================================
|
|
|
|
|
|
.. module:: zlib
|
|
|
:synopsis: Low-level interface to compression and decompression routines compatible with
|
|
|
gzip.
|
|
|
|
|
|
|
|
|
For applications that require data compression, the functions in this module
|
|
|
allow compression and decompression, using the zlib library. The zlib library
|
|
|
has its own home page at http://www.zlib.net. There are known
|
|
|
incompatibilities between the Python module and versions of the zlib library
|
|
|
earlier than 1.1.3; 1.1.3 has a security vulnerability, so we recommend using
|
|
|
1.1.4 or later.
|
|
|
|
|
|
zlib's functions have many options and often need to be used in a particular
|
|
|
order. This documentation doesn't attempt to cover all of the permutations;
|
|
|
consult the zlib manual at http://www.zlib.net/manual.html for authoritative
|
|
|
information.
|
|
|
|
|
|
For reading and writing ``.gz`` files see the :mod:`gzip` module.
|
|
|
|
|
|
The available exception and functions in this module are:
|
|
|
|
|
|
|
|
|
.. exception:: error
|
|
|
|
|
|
Exception raised on compression and decompression errors.
|
|
|
|
|
|
|
|
|
.. function:: adler32(data[, value])
|
|
|
|
|
|
Computes an Adler-32 checksum of *data*. (An Adler-32 checksum is almost as
|
|
|
reliable as a CRC32 but can be computed much more quickly.) If *value* is
|
|
|
present, it is used as the starting value of the checksum; otherwise, a fixed
|
|
|
default value is used. This allows computing a running checksum over the
|
|
|
concatenation of several inputs. The algorithm is not cryptographically
|
|
|
strong, and should not be used for authentication or digital signatures. Since
|
|
|
the algorithm is designed for use as a checksum algorithm, it is not suitable
|
|
|
for use as a general hash algorithm.
|
|
|
|
|
|
This function always returns an integer object.
|
|
|
|
|
|
.. note::
|
|
|
To generate the same numeric value across all Python versions and
|
|
|
platforms use adler32(data) & 0xffffffff. If you are only using
|
|
|
the checksum in packed binary format this is not necessary as the
|
|
|
return value is the correct 32bit binary representation
|
|
|
regardless of sign.
|
|
|
|
|
|
.. versionchanged:: 2.6
|
|
|
The return value is in the range [-2**31, 2**31-1]
|
|
|
regardless of platform. In older versions the value is
|
|
|
signed on some platforms and unsigned on others.
|
|
|
|
|
|
.. versionchanged:: 3.0
|
|
|
The return value is unsigned and in the range [0, 2**32-1]
|
|
|
regardless of platform.
|
|
|
|
|
|
|
|
|
.. function:: compress(string[, level])
|
|
|
|
|
|
Compresses the data in *string*, returning a string contained compressed data.
|
|
|
*level* is an integer from ``0`` to ``9`` controlling the level of compression;
|
|
|
``1`` is fastest and produces the least compression, ``9`` is slowest and
|
|
|
produces the most. ``0`` is no compression. The default value is ``6``.
|
|
|
Raises the :exc:`error` exception if any error occurs.
|
|
|
|
|
|
|
|
|
.. function:: compressobj([level[, method[, wbits[, memlevel[, strategy]]]]])
|
|
|
|
|
|
Returns a compression object, to be used for compressing data streams that won't
|
|
|
fit into memory at once. *level* is an integer from
|
|
|
``0`` to ``9`` or ``-1``, controlling
|
|
|
the level of compression; ``1`` is fastest and produces the least compression,
|
|
|
``9`` is slowest and produces the most. ``0`` is no compression. The default
|
|
|
value is ``-1`` (Z_DEFAULT_COMPRESSION). Z_DEFAULT_COMPRESSION represents a default
|
|
|
compromise between speed and compression (currently equivalent to level 6).
|
|
|
|
|
|
*method* is the compression algorithm. Currently, the only supported value is
|
|
|
``DEFLATED``.
|
|
|
|
|
|
The *wbits* argument controls the size of the history buffer (or the
|
|
|
"window size") used when compressing data, and whether a header and
|
|
|
trailer is included in the output. It can take several ranges of values.
|
|
|
The default is 15.
|
|
|
|
|
|
* +9 to +15: The base-two logarithm of the window size, which
|
|
|
therefore ranges between 512 and 32768. Larger values produce
|
|
|
better compression at the expense of greater memory usage. The
|
|
|
resulting output will include a zlib-specific header and trailer.
|
|
|
|
|
|
* −9 to −15: Uses the absolute value of *wbits* as the
|
|
|
window size logarithm, while producing a raw output stream with no
|
|
|
header or trailing checksum.
|
|
|
|
|
|
* +25 to +31 = 16 + (9 to 15): Uses the low 4 bits of the value as the
|
|
|
window size logarithm, while including a basic :program:`gzip` header
|
|
|
and trailing checksum in the output.
|
|
|
|
|
|
*memlevel* controls the amount of memory used for internal compression state.
|
|
|
Valid values range from ``1`` to ``9``. Higher values using more memory,
|
|
|
but are faster and produce smaller output. The default is 8.
|
|
|
|
|
|
*strategy* is used to tune the compression algorithm. Possible values are
|
|
|
``Z_DEFAULT_STRATEGY``, ``Z_FILTERED``, and ``Z_HUFFMAN_ONLY``. The default
|
|
|
is ``Z_DEFAULT_STRATEGY``.
|
|
|
|
|
|
|
|
|
.. function:: crc32(data[, value])
|
|
|
|
|
|
.. index::
|
|
|
single: Cyclic Redundancy Check
|
|
|
single: checksum; Cyclic Redundancy Check
|
|
|
|
|
|
Computes a CRC (Cyclic Redundancy Check) checksum of *data*. If *value* is
|
|
|
present, it is used as the starting value of the checksum; otherwise, a fixed
|
|
|
default value is used. This allows computing a running checksum over the
|
|
|
concatenation of several inputs. The algorithm is not cryptographically
|
|
|
strong, and should not be used for authentication or digital signatures. Since
|
|
|
the algorithm is designed for use as a checksum algorithm, it is not suitable
|
|
|
for use as a general hash algorithm.
|
|
|
|
|
|
This function always returns an integer object.
|
|
|
|
|
|
.. note::
|
|
|
To generate the same numeric value across all Python versions and
|
|
|
platforms use crc32(data) & 0xffffffff. If you are only using
|
|
|
the checksum in packed binary format this is not necessary as the
|
|
|
return value is the correct 32bit binary representation
|
|
|
regardless of sign.
|
|
|
|
|
|
.. versionchanged:: 2.6
|
|
|
The return value is in the range [-2**31, 2**31-1]
|
|
|
regardless of platform. In older versions the value would be
|
|
|
signed on some platforms and unsigned on others.
|
|
|
|
|
|
.. versionchanged:: 3.0
|
|
|
The return value is unsigned and in the range [0, 2**32-1]
|
|
|
regardless of platform.
|
|
|
|
|
|
|
|
|
.. function:: decompress(string[, wbits[, bufsize]])
|
|
|
|
|
|
Decompresses the data in *string*, returning a string containing the
|
|
|
uncompressed data. The *wbits* parameter depends on
|
|
|
the format of *string*, and is discussed further below.
|
|
|
If *bufsize* is given, it is used as the initial size of the output
|
|
|
buffer. Raises the :exc:`error` exception if any error occurs.
|
|
|
|
|
|
.. _decompress-wbits:
|
|
|
|
|
|
The *wbits* parameter controls the size of the history buffer
|
|
|
(or "window size"), and what header and trailer format is expected.
|
|
|
It is similar to the parameter for :func:`compressobj`, but accepts
|
|
|
more ranges of values:
|
|
|
|
|
|
* +8 to +15: The base-two logarithm of the window size. The input
|
|
|
must include a zlib header and trailer.
|
|
|
|
|
|
* 0: Automatically determine the window size from the zlib header.
|
|
|
Only supported since zlib 1.2.3.5.
|
|
|
|
|
|
* −8 to −15: Uses the absolute value of *wbits* as the window size
|
|
|
logarithm. The input must be a raw stream with no header or trailer.
|
|
|
|
|
|
* +24 to +31 = 16 + (8 to 15): Uses the low 4 bits of the value as
|
|
|
the window size logarithm. The input must include a gzip header and
|
|
|
trailer.
|
|
|
|
|
|
* +40 to +47 = 32 + (8 to 15): Uses the low 4 bits of the value as
|
|
|
the window size logarithm, and automatically accepts either
|
|
|
the zlib or gzip format.
|
|
|
|
|
|
When decompressing a stream, the window size must not be smaller
|
|
|
than the size originally used to compress the stream; using a too-small
|
|
|
value may result in an :exc:`error` exception. The default *wbits* value
|
|
|
is 15, which corresponds to the largest window size and requires a zlib
|
|
|
header and trailer to be included.
|
|
|
|
|
|
*bufsize* is the initial size of the buffer used to hold decompressed data. If
|
|
|
more space is required, the buffer size will be increased as needed, so you
|
|
|
don't have to get this value exactly right; tuning it will only save a few calls
|
|
|
to :c:func:`malloc`. The default size is 16384.
|
|
|
|
|
|
|
|
|
.. function:: decompressobj([wbits])
|
|
|
|
|
|
Returns a decompression object, to be used for decompressing data streams that
|
|
|
won't fit into memory at once.
|
|
|
|
|
|
The *wbits* parameter controls the size of the history buffer (or the
|
|
|
"window size"), and what header and trailer format is expected. It has
|
|
|
the same meaning as `described for decompress() <#decompress-wbits>`__.
|
|
|
|
|
|
Compression objects support the following methods:
|
|
|
|
|
|
|
|
|
.. method:: Compress.compress(string)
|
|
|
|
|
|
Compress *string*, returning a string containing compressed data for at least
|
|
|
part of the data in *string*. This data should be concatenated to the output
|
|
|
produced by any preceding calls to the :meth:`compress` method. Some input may
|
|
|
be kept in internal buffers for later processing.
|
|
|
|
|
|
|
|
|
.. method:: Compress.flush([mode])
|
|
|
|
|
|
All pending input is processed, and a string containing the remaining compressed
|
|
|
output is returned. *mode* can be selected from the constants
|
|
|
:const:`Z_SYNC_FLUSH`, :const:`Z_FULL_FLUSH`, or :const:`Z_FINISH`,
|
|
|
defaulting to :const:`Z_FINISH`. :const:`Z_SYNC_FLUSH` and
|
|
|
:const:`Z_FULL_FLUSH` allow compressing further strings of data, while
|
|
|
:const:`Z_FINISH` finishes the compressed stream and prevents compressing any
|
|
|
more data. After calling :meth:`flush` with *mode* set to :const:`Z_FINISH`,
|
|
|
the :meth:`compress` method cannot be called again; the only realistic action is
|
|
|
to delete the object.
|
|
|
|
|
|
|
|
|
.. method:: Compress.copy()
|
|
|
|
|
|
Returns a copy of the compression object. This can be used to efficiently
|
|
|
compress a set of data that share a common initial prefix.
|
|
|
|
|
|
.. versionadded:: 2.5
|
|
|
|
|
|
Decompression objects support the following methods, and two attributes:
|
|
|
|
|
|
|
|
|
.. attribute:: Decompress.unused_data
|
|
|
|
|
|
A string which contains any bytes past the end of the compressed data. That is,
|
|
|
this remains ``""`` until the last byte that contains compression data is
|
|
|
available. If the whole string turned out to contain compressed data, this is
|
|
|
``""``, the empty string.
|
|
|
|
|
|
The only way to determine where a string of compressed data ends is by actually
|
|
|
decompressing it. This means that when compressed data is contained part of a
|
|
|
larger file, you can only find the end of it by reading data and feeding it
|
|
|
followed by some non-empty string into a decompression object's
|
|
|
:meth:`decompress` method until the :attr:`unused_data` attribute is no longer
|
|
|
the empty string.
|
|
|
|
|
|
|
|
|
.. attribute:: Decompress.unconsumed_tail
|
|
|
|
|
|
A string that contains any data that was not consumed by the last
|
|
|
:meth:`decompress` call because it exceeded the limit for the uncompressed data
|
|
|
buffer. This data has not yet been seen by the zlib machinery, so you must feed
|
|
|
it (possibly with further data concatenated to it) back to a subsequent
|
|
|
:meth:`decompress` method call in order to get correct output.
|
|
|
|
|
|
|
|
|
.. method:: Decompress.decompress(string[, max_length])
|
|
|
|
|
|
Decompress *string*, returning a string containing the uncompressed data
|
|
|
corresponding to at least part of the data in *string*. This data should be
|
|
|
concatenated to the output produced by any preceding calls to the
|
|
|
:meth:`decompress` method. Some of the input data may be preserved in internal
|
|
|
buffers for later processing.
|
|
|
|
|
|
If the optional parameter *max_length* is non-zero then the return value will be
|
|
|
no longer than *max_length*. This may mean that not all of the compressed input
|
|
|
can be processed; and unconsumed data will be stored in the attribute
|
|
|
:attr:`unconsumed_tail`. This string must be passed to a subsequent call to
|
|
|
:meth:`decompress` if decompression is to continue. If *max_length* is not
|
|
|
supplied then the whole input is decompressed, and :attr:`unconsumed_tail` is an
|
|
|
empty string.
|
|
|
|
|
|
|
|
|
.. method:: Decompress.flush([length])
|
|
|
|
|
|
All pending input is processed, and a string containing the remaining
|
|
|
uncompressed output is returned. After calling :meth:`flush`, the
|
|
|
:meth:`decompress` method cannot be called again; the only realistic action is
|
|
|
to delete the object.
|
|
|
|
|
|
The optional parameter *length* sets the initial size of the output buffer.
|
|
|
|
|
|
|
|
|
.. method:: Decompress.copy()
|
|
|
|
|
|
Returns a copy of the decompression object. This can be used to save the state
|
|
|
of the decompressor midway through the data stream in order to speed up random
|
|
|
seeks into the stream at a future point.
|
|
|
|
|
|
.. versionadded:: 2.5
|
|
|
|
|
|
|
|
|
.. seealso::
|
|
|
|
|
|
Module :mod:`gzip`
|
|
|
Reading and writing :program:`gzip`\ -format files.
|
|
|
|
|
|
http://www.zlib.net
|
|
|
The zlib library home page.
|
|
|
|
|
|
http://www.zlib.net/manual.html
|
|
|
The zlib manual explains the semantics and usage of the library's many
|
|
|
functions.
|
|
|
|
