Commit Graph

52 Commits

Author SHA1 Message Date
Ma Lin 251ffa9d2b
bpo-41486: Fix initial buffer size can't > UINT32_MAX in zlib module (GH-25738)
* Fix initial buffer size can't > UINT32_MAX in zlib module

After commit f9bedb630e, in 64-bit build,
if the initial buffer size > UINT32_MAX, ValueError will be raised.

These two functions are affected:
1. zlib.decompress(data, /, wbits=MAX_WBITS, bufsize=DEF_BUF_SIZE)
2. zlib.Decompress.flush([length])

This commit re-allows the size > UINT32_MAX.

* adds curly braces per PEP 7.

* Renames `Buffer_*` to `OutputBuffer_*` for clarity
2021-04-30 16:32:49 -07:00
Ma Lin f9bedb630e
bpo-41486: Faster bz2/lzma/zlib via new output buffering (GH-21740)
Faster bz2/lzma/zlib via new output buffering.
Also adds .readall() function to _compression.DecompressReader class
to take best advantage of this in the consume-all-output at once scenario.

Often a 5-20% speedup in common scenarios due to less data copying.

Contributed by Ma Lin.
2021-04-27 23:58:54 -07:00
Serhiy Storchaka 8cd1dbae32
bpo-41052: Fix pickling heap types implemented in C with protocols 0 and 1 (GH-22870) 2020-10-24 21:14:23 +03:00
Dong-hee Na 1937edd376
bpo-1635741: Port _lzma module to multiphase initialization (GH-19382) 2020-06-23 00:53:07 +09:00
Victor Stinner 4a21e57fe5
bpo-40268: Remove unused structmember.h includes (GH-19530)
If only offsetof() is needed: include stddef.h instead.

When structmember.h is used, add a comment explaining that
PyMemberDef is used.
2020-04-15 02:35:41 +02:00
Victor Stinner 62183b8d6d
bpo-40268: Remove explicit pythread.h includes (#19529)
Remove explicit pythread.h includes: it is always included
by Python.h.
2020-04-15 02:04:42 +02:00
Andy Lester 7668a8bc93
Use calloc-based functions, not malloc. (GH-19152) 2020-03-24 23:26:44 -05:00
Dong-hee Na 05e4a296ec
bpo-40024: Add PyModule_AddType() helper function (GH-19088) 2020-03-22 17:17:34 +01:00
animalize 4ffd05d7ec bpo-21872: fix lzma library decompresses data incompletely (GH-14048)
* 1. add test case with wrong behavior
* 2. fix bug when max_length == -1
* 3. allow b"" as valid input data for decompress_buf()
* 4. when max_length >= 0, let needs_input mechanism works
* add more asserts to test case
2019-09-12 15:20:37 +01:00
Jeroen Demeyer 530f506ac9 bpo-36974: tp_print -> tp_vectorcall_offset and tp_reserved -> tp_as_async (GH-13464)
Automatically replace
tp_print -> tp_vectorcall_offset
tp_compare -> tp_as_async
tp_reserved -> tp_as_async
2019-05-30 19:13:39 -07:00
Serhiy Storchaka d53fe5f407
bpo-36254: Fix invalid uses of %d in format strings in C. (GH-12264) 2019-03-13 22:59:55 +02:00
Serhiy Storchaka 0353b4eaaf
bpo-33138: Change standard error message for non-pickleable and non-copyable types. (GH-6239) 2018-10-31 02:28:07 +02:00
Alexey Izbyshev 3d4fabb2a4 bpo-35090: Fix potential division by zero in allocator wrappers (GH-10174)
* Fix potential division by zero in BZ2_Malloc()
* Avoid division by zero in PyLzma_Malloc()
* Avoid division by zero and integer overflow in PyZlib_Malloc()

Reported by Svace static analyzer.
2018-10-28 17:45:50 +01:00
Victor Stinner 9b7cf75721
bpo-33916: Fix bz2 and lzma init when called twice (GH-7843)
bz2, lzma: When Decompressor.__init__() is called twice, free the old
lock to not leak memory.
2018-06-23 10:35:23 +02:00
Oren Milman d019bc8319 bpo-31787: Prevent refleaks when calling __init__() more than once (GH-3995) 2018-02-13 19:28:33 +09:00
Antoine Pitrou a6a4dc816d bpo-31370: Remove support for threads-less builds (#3385)
* Remove Setup.config
* Always define WITH_THREAD for compatibility.
2017-09-07 18:56:24 +02:00
Ville Skyttä 49b2734bf1 Spelling fixes (#2902) 2017-08-03 09:00:59 +03:00
Serhiy Storchaka 88b2219358 Issue #27517: LZMA compressor and decompressor no longer raise exceptions if
given empty data twice.  Patch by Benjamin Fogle.
2016-10-31 08:31:13 +02:00
Serhiy Storchaka 04f17f103a Issue #27517: LZMA compressor and decompressor no longer raise exceptions if
given empty data twice.  Patch by Benjamin Fogle.
2016-10-31 08:30:09 +02:00
Serhiy Storchaka a12e7842a5 Issue #28275: Fixed possible use adter free in LZMADecompressor.decompress().
Original patch by John Leitch.
2016-09-27 20:23:41 +03:00
Serhiy Storchaka c0b7037d4f Issue #28275: Fixed possible use adter free in LZMADecompressor.decompress().
Original patch by John Leitch.
2016-09-27 20:14:26 +03:00
Benjamin Peterson af580dff4a replace PY_LONG_LONG with long long 2016-09-06 10:46:49 -07:00
Serhiy Storchaka 2954f83999 - Issue #27332: Fixed the type of the first argument of module-level functions
generated by Argument Clinic.  Patch by Petr Viktorin.
2016-07-07 18:20:03 +03:00
Serhiy Storchaka 1a2b24f02d Issue #27332: Fixed the type of the first argument of module-level functions
generated by Argument Clinic.  Patch by Petr Viktorin.
2016-07-07 17:35:15 +03:00
Martin Panter 0b7d84de6b Issue #27171: Merge typo fixes from 3.5 2016-06-02 10:11:18 +00:00
Martin Panter e26da7c03a Issue #27171: Fix typos in documentation, comments, and test function names 2016-06-02 10:07:09 +00:00
Serhiy Storchaka 7a9579c0ce Got rid of redundand "self" parameter declarations.
Argument Clinic is now able to infer all needed information.
2016-05-02 13:45:20 +03:00
Serhiy Storchaka 48842714b9 Issue #22570: Renamed Py_SETREF to Py_XSETREF. 2016-04-06 09:45:48 +03:00
Serhiy Storchaka 4a1e70fc31 Issue #20440: Applied yet one patch for using Py_SETREF.
The patch is automatically generated, it replaces the code that uses Py_CLEAR.
2015-12-27 12:36:18 +02:00
Larry Hastings 89964c48d1 Issue #23944: Argument Clinic now wraps long impl prototypes at column 78. 2015-04-14 18:07:59 -04:00
Serhiy Storchaka 1009bf18b3 Issue #23501: Argumen Clinic now generates code into separate files by default. 2015-04-03 23:53:51 +03:00
Serhiy Storchaka 009b811d67 Removed unintentional trailing spaces in non-external and non-generated C files. 2015-03-18 21:53:15 +02:00
Serhiy Storchaka 79d8f3f123 Regenerated Argument Clinic checksums. 2015-02-20 12:46:11 +02:00
Antoine Pitrou 26795baaa8 Issue #15955: Add an option to limit output size when decompressing LZMA data.
Patch by Nikolaus Rath and Martin Panter.
2015-01-17 16:22:18 +01:00
Victor Stinner 049e509a9f Issue #22207: Fix "comparison between signed and unsigned integers" warning in
test checking for integer overflow on Py_ssize_t type: cast explicitly to
size_t.
2014-08-17 22:20:00 +02:00
Larry Hastings 581ee3618c Issue #20326: Argument Clinic now uses a simple, unique signature to
annotate text signatures in docstrings, resulting in fewer false
positives.  "self" parameters are also explicitly marked, allowing
inspect.Signature() to authoritatively detect (and skip) said parameters.

Issue #20326: Argument Clinic now generates separate checksums for the
input and output sections of the block, allowing external tools to verify
that the input has not changed (and thus the output is not out-of-date).
2014-01-28 05:00:08 -08:00
Larry Hastings f256c22f34 Fix for catestrophic errors in previous checkin (Argument Clinic rollup patch). 2014-01-25 21:30:37 -08:00
Larry Hastings c20472640c Issue #20390: Small fixes and improvements for Argument Clinic. 2014-01-25 20:43:29 -08:00
Serhiy Storchaka 98c779e8da Issue #20193: The _lzma module now uses Argument Clinic.
LZMACompressor.__init__ is left not converted.
2014-01-25 14:02:29 +02:00
Nadeem Vawda e6514f533e #19395: Raise exception when pickling a (BZ2|LZMA)(Compressor|Decompressor).
The underlying C libraries provide no mechanism for serializing compressor and
decompressor objects, so actually pickling these classes is impractical.
Previously, these objects would be pickled without error, but attempting to use
a deserialized instance would segfault the interpreter.
2013-10-28 21:41:24 +01:00
Nadeem Vawda 3797065ac5 #19395: Raise exception when pickling a (BZ2|LZMA)(Compressor|Decompressor).
The underlying C libraries provide no mechanism for serializing compressor and
decompressor objects, so actually pickling these classes is impractical.
Previously, these objects would be pickled without error, but attempting to use
a deserialized instance would segfault the interpreter.
2013-10-28 21:35:23 +01:00
Victor Stinner b7f1f65f1c Issue #18227: "Free" function of bz2, lzma and zlib modules has no return value (void) 2013-07-07 17:10:34 +02:00
Victor Stinner 5064a52bcb Issue #18227: Use PyMem_RawAlloc() in bz2, lzma and zlib modules 2013-07-07 16:50:27 +02:00
Nadeem Vawda a425c3d5a2 Make lzma.{encode,decode}_filter_properties private.
These functions were originally added to support LZMA compression in the zipfile
module, and are not of interest for the majority of users.

They can be made public in 3.4 if there is user interest, but in the meanwhile,
I've opted to present a smaller, simpler API for the module's initial release.
2012-06-21 23:36:48 +02:00
Martin v. Löwis d1b7f3993a Silence VS 2010 warning on loss of precision (_int64 -> _int32).
This is safe because the actual value is already range-checked.
2012-05-15 14:06:21 +02:00
Ross Lagerwall 4d688e3275 Remove uninitialized compiler warning. 2012-05-12 08:30:33 +02:00
Nadeem Vawda 486a0451f3 Fix buildbot failures in test_lzma. 2012-05-07 00:40:57 +02:00
Nadeem Vawda bc459bb484 Rename lzma.check_is_supported() to is_check_supported() to avoid grammatical confusion. 2012-05-06 23:01:51 +02:00
Nadeem Vawda f55b329edc Add lzma.{encode,decode}_filter_properties(). 2012-05-06 23:01:27 +02:00
Nadeem Vawda 54c74ece28 Fix formatting of preset values for exception messages in _lzmamodule.c. 2012-05-06 13:35:47 +02:00