303 lines
12 KiB
ReStructuredText
303 lines
12 KiB
ReStructuredText
|
||
:mod:`zlib` --- Compression compatible with :program:`gzip`
|
||
===========================================================
|
||
|
||
.. module:: zlib
|
||
:synopsis: Low-level interface to compression and decompression routines compatible with
|
||
gzip.
|
||
|
||
|
||
For applications that require data compression, the functions in this module
|
||
allow compression and decompression, using the zlib library. The zlib library
|
||
has its own home page at http://www.zlib.net. There are known
|
||
incompatibilities between the Python module and versions of the zlib library
|
||
earlier than 1.1.3; 1.1.3 has a security vulnerability, so we recommend using
|
||
1.1.4 or later.
|
||
|
||
zlib's functions have many options and often need to be used in a particular
|
||
order. This documentation doesn't attempt to cover all of the permutations;
|
||
consult the zlib manual at http://www.zlib.net/manual.html for authoritative
|
||
information.
|
||
|
||
For reading and writing ``.gz`` files see the :mod:`gzip` module.
|
||
|
||
The available exception and functions in this module are:
|
||
|
||
|
||
.. exception:: error
|
||
|
||
Exception raised on compression and decompression errors.
|
||
|
||
|
||
.. function:: adler32(data[, value])
|
||
|
||
Computes an Adler-32 checksum of *data*. (An Adler-32 checksum is almost as
|
||
reliable as a CRC32 but can be computed much more quickly.) If *value* is
|
||
present, it is used as the starting value of the checksum; otherwise, a fixed
|
||
default value is used. This allows computing a running checksum over the
|
||
concatenation of several inputs. The algorithm is not cryptographically
|
||
strong, and should not be used for authentication or digital signatures. Since
|
||
the algorithm is designed for use as a checksum algorithm, it is not suitable
|
||
for use as a general hash algorithm.
|
||
|
||
This function always returns an integer object.
|
||
|
||
.. note::
|
||
To generate the same numeric value across all Python versions and
|
||
platforms use adler32(data) & 0xffffffff. If you are only using
|
||
the checksum in packed binary format this is not necessary as the
|
||
return value is the correct 32bit binary representation
|
||
regardless of sign.
|
||
|
||
.. versionchanged:: 2.6
|
||
The return value is in the range [-2**31, 2**31-1]
|
||
regardless of platform. In older versions the value is
|
||
signed on some platforms and unsigned on others.
|
||
|
||
.. versionchanged:: 3.0
|
||
The return value is unsigned and in the range [0, 2**32-1]
|
||
regardless of platform.
|
||
|
||
|
||
.. function:: compress(string[, level])
|
||
|
||
Compresses the data in *string*, returning a string contained compressed data.
|
||
*level* is an integer from ``0`` to ``9`` controlling the level of compression;
|
||
``1`` is fastest and produces the least compression, ``9`` is slowest and
|
||
produces the most. ``0`` is no compression. The default value is ``6``.
|
||
Raises the :exc:`error` exception if any error occurs.
|
||
|
||
|
||
.. function:: compressobj([level[, method[, wbits[, memlevel[, strategy]]]]])
|
||
|
||
Returns a compression object, to be used for compressing data streams that won't
|
||
fit into memory at once. *level* is an integer from
|
||
``0`` to ``9`` or ``-1``, controlling
|
||
the level of compression; ``1`` is fastest and produces the least compression,
|
||
``9`` is slowest and produces the most. ``0`` is no compression. The default
|
||
value is ``-1`` (Z_DEFAULT_COMPRESSION). Z_DEFAULT_COMPRESSION represents a default
|
||
compromise between speed and compression (currently equivalent to level 6).
|
||
|
||
*method* is the compression algorithm. Currently, the only supported value is
|
||
``DEFLATED``.
|
||
|
||
The *wbits* argument controls the size of the history buffer (or the
|
||
"window size") used when compressing data, and whether a header and
|
||
trailer is included in the output. It can take several ranges of values.
|
||
The default is 15.
|
||
|
||
* +9 to +15: The base-two logarithm of the window size, which
|
||
therefore ranges between 512 and 32768. Larger values produce
|
||
better compression at the expense of greater memory usage. The
|
||
resulting output will include a zlib-specific header and trailer.
|
||
|
||
* −9 to −15: Uses the absolute value of *wbits* as the
|
||
window size logarithm, while producing a raw output stream with no
|
||
header or trailing checksum.
|
||
|
||
* +25 to +31 = 16 + (9 to 15): Uses the low 4 bits of the value as the
|
||
window size logarithm, while including a basic :program:`gzip` header
|
||
and trailing checksum in the output.
|
||
|
||
*memlevel* controls the amount of memory used for internal compression state.
|
||
Valid values range from ``1`` to ``9``. Higher values using more memory,
|
||
but are faster and produce smaller output. The default is 8.
|
||
|
||
*strategy* is used to tune the compression algorithm. Possible values are
|
||
``Z_DEFAULT_STRATEGY``, ``Z_FILTERED``, and ``Z_HUFFMAN_ONLY``. The default
|
||
is ``Z_DEFAULT_STRATEGY``.
|
||
|
||
|
||
.. function:: crc32(data[, value])
|
||
|
||
.. index::
|
||
single: Cyclic Redundancy Check
|
||
single: checksum; Cyclic Redundancy Check
|
||
|
||
Computes a CRC (Cyclic Redundancy Check) checksum of *data*. If *value* is
|
||
present, it is used as the starting value of the checksum; otherwise, a fixed
|
||
default value is used. This allows computing a running checksum over the
|
||
concatenation of several inputs. The algorithm is not cryptographically
|
||
strong, and should not be used for authentication or digital signatures. Since
|
||
the algorithm is designed for use as a checksum algorithm, it is not suitable
|
||
for use as a general hash algorithm.
|
||
|
||
This function always returns an integer object.
|
||
|
||
.. note::
|
||
To generate the same numeric value across all Python versions and
|
||
platforms use crc32(data) & 0xffffffff. If you are only using
|
||
the checksum in packed binary format this is not necessary as the
|
||
return value is the correct 32bit binary representation
|
||
regardless of sign.
|
||
|
||
.. versionchanged:: 2.6
|
||
The return value is in the range [-2**31, 2**31-1]
|
||
regardless of platform. In older versions the value would be
|
||
signed on some platforms and unsigned on others.
|
||
|
||
.. versionchanged:: 3.0
|
||
The return value is unsigned and in the range [0, 2**32-1]
|
||
regardless of platform.
|
||
|
||
|
||
.. function:: decompress(string[, wbits[, bufsize]])
|
||
|
||
Decompresses the data in *string*, returning a string containing the
|
||
uncompressed data. The *wbits* parameter depends on
|
||
the format of *string*, and is discussed further below.
|
||
If *bufsize* is given, it is used as the initial size of the output
|
||
buffer. Raises the :exc:`error` exception if any error occurs.
|
||
|
||
.. _decompress-wbits:
|
||
|
||
The *wbits* parameter controls the size of the history buffer
|
||
(or "window size"), and what header and trailer format is expected.
|
||
It is similar to the parameter for :func:`compressobj`, but accepts
|
||
more ranges of values:
|
||
|
||
* +8 to +15: The base-two logarithm of the window size. The input
|
||
must include a zlib header and trailer.
|
||
|
||
* 0: Automatically determine the window size from the zlib header.
|
||
Only supported since zlib 1.2.3.5.
|
||
|
||
* −8 to −15: Uses the absolute value of *wbits* as the window size
|
||
logarithm. The input must be a raw stream with no header or trailer.
|
||
|
||
* +24 to +31 = 16 + (8 to 15): Uses the low 4 bits of the value as
|
||
the window size logarithm. The input must include a gzip header and
|
||
trailer.
|
||
|
||
* +40 to +47 = 32 + (8 to 15): Uses the low 4 bits of the value as
|
||
the window size logarithm, and automatically accepts either
|
||
the zlib or gzip format.
|
||
|
||
When decompressing a stream, the window size must not be smaller
|
||
than the size originally used to compress the stream; using a too-small
|
||
value may result in an :exc:`error` exception. The default *wbits* value
|
||
is 15, which corresponds to the largest window size and requires a zlib
|
||
header and trailer to be included.
|
||
|
||
*bufsize* is the initial size of the buffer used to hold decompressed data. If
|
||
more space is required, the buffer size will be increased as needed, so you
|
||
don't have to get this value exactly right; tuning it will only save a few calls
|
||
to :c:func:`malloc`. The default size is 16384.
|
||
|
||
|
||
.. function:: decompressobj([wbits])
|
||
|
||
Returns a decompression object, to be used for decompressing data streams that
|
||
won't fit into memory at once.
|
||
|
||
The *wbits* parameter controls the size of the history buffer (or the
|
||
"window size"), and what header and trailer format is expected. It has
|
||
the same meaning as `described for decompress() <#decompress-wbits>`__.
|
||
|
||
Compression objects support the following methods:
|
||
|
||
|
||
.. method:: Compress.compress(string)
|
||
|
||
Compress *string*, returning a string containing compressed data for at least
|
||
part of the data in *string*. This data should be concatenated to the output
|
||
produced by any preceding calls to the :meth:`compress` method. Some input may
|
||
be kept in internal buffers for later processing.
|
||
|
||
|
||
.. method:: Compress.flush([mode])
|
||
|
||
All pending input is processed, and a string containing the remaining compressed
|
||
output is returned. *mode* can be selected from the constants
|
||
:const:`Z_SYNC_FLUSH`, :const:`Z_FULL_FLUSH`, or :const:`Z_FINISH`,
|
||
defaulting to :const:`Z_FINISH`. :const:`Z_SYNC_FLUSH` and
|
||
:const:`Z_FULL_FLUSH` allow compressing further strings of data, while
|
||
:const:`Z_FINISH` finishes the compressed stream and prevents compressing any
|
||
more data. After calling :meth:`flush` with *mode* set to :const:`Z_FINISH`,
|
||
the :meth:`compress` method cannot be called again; the only realistic action is
|
||
to delete the object.
|
||
|
||
|
||
.. method:: Compress.copy()
|
||
|
||
Returns a copy of the compression object. This can be used to efficiently
|
||
compress a set of data that share a common initial prefix.
|
||
|
||
.. versionadded:: 2.5
|
||
|
||
Decompression objects support the following methods, and two attributes:
|
||
|
||
|
||
.. attribute:: Decompress.unused_data
|
||
|
||
A string which contains any bytes past the end of the compressed data. That is,
|
||
this remains ``""`` until the last byte that contains compression data is
|
||
available. If the whole string turned out to contain compressed data, this is
|
||
``""``, the empty string.
|
||
|
||
The only way to determine where a string of compressed data ends is by actually
|
||
decompressing it. This means that when compressed data is contained part of a
|
||
larger file, you can only find the end of it by reading data and feeding it
|
||
followed by some non-empty string into a decompression object's
|
||
:meth:`decompress` method until the :attr:`unused_data` attribute is no longer
|
||
the empty string.
|
||
|
||
|
||
.. attribute:: Decompress.unconsumed_tail
|
||
|
||
A string that contains any data that was not consumed by the last
|
||
:meth:`decompress` call because it exceeded the limit for the uncompressed data
|
||
buffer. This data has not yet been seen by the zlib machinery, so you must feed
|
||
it (possibly with further data concatenated to it) back to a subsequent
|
||
:meth:`decompress` method call in order to get correct output.
|
||
|
||
|
||
.. method:: Decompress.decompress(string[, max_length])
|
||
|
||
Decompress *string*, returning a string containing the uncompressed data
|
||
corresponding to at least part of the data in *string*. This data should be
|
||
concatenated to the output produced by any preceding calls to the
|
||
:meth:`decompress` method. Some of the input data may be preserved in internal
|
||
buffers for later processing.
|
||
|
||
If the optional parameter *max_length* is non-zero then the return value will be
|
||
no longer than *max_length*. This may mean that not all of the compressed input
|
||
can be processed; and unconsumed data will be stored in the attribute
|
||
:attr:`unconsumed_tail`. This string must be passed to a subsequent call to
|
||
:meth:`decompress` if decompression is to continue. If *max_length* is not
|
||
supplied then the whole input is decompressed, and :attr:`unconsumed_tail` is an
|
||
empty string.
|
||
|
||
|
||
.. method:: Decompress.flush([length])
|
||
|
||
All pending input is processed, and a string containing the remaining
|
||
uncompressed output is returned. After calling :meth:`flush`, the
|
||
:meth:`decompress` method cannot be called again; the only realistic action is
|
||
to delete the object.
|
||
|
||
The optional parameter *length* sets the initial size of the output buffer.
|
||
|
||
|
||
.. method:: Decompress.copy()
|
||
|
||
Returns a copy of the decompression object. This can be used to save the state
|
||
of the decompressor midway through the data stream in order to speed up random
|
||
seeks into the stream at a future point.
|
||
|
||
.. versionadded:: 2.5
|
||
|
||
|
||
.. seealso::
|
||
|
||
Module :mod:`gzip`
|
||
Reading and writing :program:`gzip`\ -format files.
|
||
|
||
http://www.zlib.net
|
||
The zlib library home page.
|
||
|
||
http://www.zlib.net/manual.html
|
||
The zlib manual explains the semantics and usage of the library's many
|
||
functions.
|
||
|