Mostly by Toby Dickenson and Titus Brown.
Add an optional argument to a decompression object's decompress()
method. The argument specifies the maximum length of the return
value. If the uncompressed data exceeds this length, the excess data
is stored as the unconsumed_tail attribute. (Not to be confused with
unused_data, which is a separate issue.)
Difference from SF patch: Default value for unconsumed_tail is ""
rather than None. It's simpler if the attribute is always a string.
* fixes the zlib decompress sync flush bug as reported in bug #124981
* avoids repeat calls to (in|de)flateEnd when destroying (de)compression
objects
* raises exception when allocating unused_data fails
* fixes memory leak when allocating unused_data fails
* raises exception when allocating decompress data fails
* removes vestigial code from decompress flush now that decompression
returns all available data
* tidies code so object compress/decompress/flush routines are consistent
This involves changing the zlib build process to build zlib itself from sources, then use that library. Also updated are the comments to reflect the new official home of zlib, and add Windows specific notes regarding the build process.
a Z_BUF_ERROR while decompressing. If it is, assume that this means
the data being decompressed is bad and raise an exception, instead of
just assuming that Z_BUF_ERROR always means that more space is required.
and a couple of functions that were missed in the previous batches. Not
terribly tested, but very carefully scrutinized, three times.
All these were found by the little findkrc.py that I posted to python-dev,
which means there might be more lurking. Cases such as this:
long
func(a, b)
long a;
long b; /* flagword */
{
and other cases where the last ; in the argument list isn't followed by a
newline and an opening curly bracket. Regexps to catch all are welcome, of
course ;)
For more comments, read the patches@python.org archives.
For documentation read the comments in mymalloc.h and objimpl.h.
(This is not exactly what Vladimir posted to the patches list; I've
made a few changes, and Vladimir sent me a fix in private email for a
problem that only occurs in debug mode. I'm also holding back on his
change to main.c, which seems unnecessary to me.)
Without this, if inflate() returned Z_BUF_ERROR asking for more output
space, we would report the error; now, we increase the buffer size and
try again, just as for Z_OK.
is not an empty string, this means that you have arrived at the
end of the stream of compressed data, and the contents of .unused_data are
whatever follows the compressed stream.
decompressor object. This required adding a flag to the struct which is
true if initialisation was completed; on object destruction, deflateEnd()
is only called if the flag is true.
NOTE: There is still a bug of some sort in the behavior of zlib. In
at least one case, inflate returns Z_OK (which is typically
interpreted to mean that more output space is needed) when it has
finished inflating a buffer. This has been reported as a bug to the
zlib maintainers; we may need to change the Python interface.
(1) Use PyErr_NewException("module.class", NULL, NULL) to create the
exception object.
(2) Remove all calls to Py_FatalError(); instead, return or
ignore the errors -- the import code now checks PyErr_Occurred()
after calling a module's init function, so it's no longer a
fatal error for the initialization to fail.
Also did some small cleanups, e.g. removed unnecessary test for
"already initialized" from initfpectl(), and unified
initposix()/initnt().
I haven't checked this very thoroughly, so while the changes are
pretty trivial -- beware of untested code!
1. Fix bug in (de)compression objects. The final string resize used
zst.total_out to determine the length of the string, but the
(de)compression object will output data a little bit at a time, which
means total_out is not the string size. Fix: save original value of
total_out at the start of the call.
2. Be sure to Py_DECREF the result value if you exit with an
exception.
3. Use PyInt_FromLong instead of Py_BuildValue
4. include more constants from the zlib header file
5. Use PyErr_Format instead of using a local buffer and sprintf.
-- initialize length to DEFAULTALLOC and not 0
-- resize string before returning (to remove '\000' padding)
Also converted some compression routines to use PyString instead of
buffer.
Change default alloc size for uncompressing to 16K.
Remove comment about core dumps when an invalid window sizes is used.
This bug has been fixed in zlib 1.0.4.
Two new optional arguments to decompress, wbits and bufsize. wbits
specifies the window size and bufsize specifies the initial output
string size.
In decompression code -- decompress and decompressobj methods -- use a
Python string (and _PyString_Resize) to collect the uncompressed
stream. Replaces a separate buffer that was copied into a string.
Fix bug in decompress that caused it to always realloc the buffer when
it was finished decompressing.
Modernized handling of optional arguments to compressobj.
Updated doc strings.
Jeremy Hylton
Martin v. Löwis
Andrew M. Kuchling
Tim Peters
Mark Hammond
Thomas Wouters
Peter Schneider-Kamp
Guido van Rossum
Fred Drake
Barry Warsaw