Commit Graph

46 Commits

Author SHA1 Message Date
Ruben Vorderman eae7dad402
gh-95534: Improve gzip reading speed by 10% (#97664)
Change summary:
+ There is now a `gzip.READ_BUFFER_SIZE` constant that is 128KB. Other programs that read in 128KB chunks: pigz and cat. So this seems best practice among good programs. Also it is faster than 8 kb chunks.
+ a zlib._ZlibDecompressor was added. This is the _bz2.BZ2Decompressor ported to zlib. Since the zlib.Decompress object is better for in-memory decompression, the _ZlibDecompressor is hidden. It only makes sense in file decompression, and that is already implemented now in the gzip library. No need to bother the users with this.
+ The ZlibDecompressor uses the older Cpython arrange_output_buffer functions, as those are faster and more appropriate for the use case. 
+ GzipFile.read has been optimized. There is no longer a `unconsumed_tail` member to write back to padded file. This is instead handled by the ZlibDecompressor itself, which has an internal buffer. `_add_read_data` has been inlined, as it was just two calls.

EDIT: While I am adding improvements anyway, I figured I could add another one-liner optimization now to the python -m gzip application. That read chunks in io.DEFAULT_BUFFER_SIZE previously, but has been updated now to use READ_BUFFER_SIZE chunks.
2022-10-16 19:10:58 -07:00
Erlend E. Aasland f07adf82f3
gh-90928: Improve static initialization of keywords tuple in AC (#95907) 2022-08-13 12:09:40 +02:00
Eric Snow 6f6a4e6cc5
gh-90928: Statically Initialize the Keywords Tuple in Clinic-Generated Code (gh-95860)
We only statically initialize for core code and builtin modules.  Extension modules still create
the tuple at runtime.  We'll solve that part of interpreter isolation separately.

This change includes generated code. The non-generated changes are in:

* Tools/clinic/clinic.py
* Python/getargs.c
* Include/cpython/modsupport.h
* Makefile.pre.in (re-generate global strings after running clinic)
* very minor tweaks to Modules/_codecsmodule.c and Python/Python-tokenize.c

All other changes are generated code (clinic, global strings).
2022-08-11 15:25:49 -06:00
Victor Stinner b270b82f11
gh-91320: Argument Clinic uses _PyCFunction_CAST() (#32210)
Replace "(PyCFunction)(void(*)(void))func" cast with
_PyCFunction_CAST(func).
2022-05-03 20:25:41 +02:00
Serhiy Storchaka a055dac0b4
gh-91583: AC: Fix regression for functions with defining_class (GH-91739)
Argument Clinic now generates the same efficient code as before
adding the defining_class parameter.
2022-04-30 13:15:02 +03:00
Gregory P. Smith 9d1c4d69db
bpo-38256: Fix binascii.crc32() when inputs are 4+GiB (GH-32000)
When compiled with `USE_ZLIB_CRC32` defined (`configure` sets this on POSIX systems), `binascii.crc32(...)` failed to compute the correct value when the input data was >= 4GiB. Because the zlib crc32 API is limited to a 32-bit length.

This lines it up with the `zlib.crc32(...)` implementation that doesn't have that flaw.

**Performance:** This also adopts the same GIL releasing for larger inputs logic that `zlib.crc32` has, and causes the Windows build to always use zlib's crc32 instead of our slow C code as zlib is a required build dependency on Windows.
2022-03-20 12:28:15 -07:00
Ruben Vorderman ea23e7820f
bpo-43613: Faster implementation of gzip.compress and gzip.decompress (GH-27941)
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2021-09-02 17:02:59 +02:00
Mohamed Koubaa 1aaa21ff81
bpo-1635741 port zlib module to multi-phase init (GH-21995)
Port the zlib extension module to multi-phase initialization (PEP 489).
2020-09-07 10:27:55 +02:00
Serhiy Storchaka 5f4b229df7
bpo-40792: Make the result of PyNumber_Index() always having exact type int. (GH-20443)
Previously, the result could have been an instance of a subclass of int.

Also revert bpo-26202 and make attributes start, stop and step of the range
object having exact type int.

Add private function _PyNumber_Index() which preserves the old behavior
of PyNumber_Index() for performance to use it in the conversion functions
like PyLong_AsLong().
2020-05-28 10:33:45 +03:00
Serhiy Storchaka 578c3955e0
bpo-37999: No longer use __int__ in implicit integer conversions. (GH-15636)
Only __index__ should be used to make integer conversions lossless.
2020-05-26 18:43:38 +03:00
Rémi Lapeyre 4901fe274b bpo-37034: Display argument name on errors with keyword arguments with Argument Clinic. (GH-13593) 2019-08-29 17:49:08 +03:00
Serhiy Storchaka 3191391515
bpo-36127: Argument Clinic: inline parsing code for keyword parameters. (GH-12058) 2019-03-14 10:32:22 +02:00
Serhiy Storchaka 4fa9591025
bpo-35582: Argument Clinic: inline parsing code for positional parameters. (GH-11313) 2019-01-11 16:01:14 +02:00
Serhiy Storchaka 32d96a2b5b
bpo-23867: Argument Clinic: inline parsing code for a single positional parameter. (GH-9689) 2018-12-25 13:23:47 +02:00
Serhiy Storchaka 4a934d490f
bpo-33012: Fix invalid function cast warnings with gcc 8 in Argument Clinic. (GH-6748)
Fix invalid function cast warnings with gcc 8
for method conventions different from METH_NOARGS, METH_O and
METH_VARARGS in Argument Clinic generated code.
2018-11-27 11:27:36 +02:00
Zackery Spytz d2cbfffc84 bpo-25007: Add copy protocol support to zlib compressors and decompressors (GH-7940) 2018-06-27 21:04:51 +03:00
Serhiy Storchaka a5552f023e
bpo-32240: Add the const qualifier to declarations of PyObject* array arguments. (#4746) 2017-12-15 13:11:11 +02:00
Tal Einat 4f57409a2f bpo-31926: fix missing *_METHODDEF statements by argument clinic (#4230)
When a single .c file contains several functions and/or methods with
the same name, a safety _METHODDEF #define statement is generated
only for one of them.

This fixes the bug by using the full name of the function to avoid
duplicates rather than just the name.
2017-11-03 11:09:00 +02:00
Serhiy Storchaka 6969eaf468 bpo-29464: Rename METH_FASTCALL to METH_FASTCALL|METH_KEYWORDS and make (#1955)
the bare METH_FASTCALL be used for functions with positional-only
parameters.
2017-07-03 21:20:15 +03:00
Sylvain 7445381c60 bpo-30600: Fix error messages (condition order in Argument Clinic) (#2051)
The function '_PyArg_ParseStack()' and
'_PyArg_UnpackStack' were failing (with error
"XXX() takes Y argument (Z given)") before
the function '_PyArg_NoStackKeywords()' was called.
Thus, the latter did not raise its more meaningful
error : "XXX() takes no keyword arguments".
2017-06-10 07:51:48 +03:00
Victor Stinner 259f0e4437 Run Argument Clinic: METH_VARARGS=>METH_FASTCALL
Issue #29286. Run Argument Clinic to get the new faster METH_FASTCALL calling
convention for functions using only positional arguments.
2017-01-17 01:35:17 +01:00
Victor Stinner 3e1fad6913 Rename _PyArg_ParseStack to _PyArg_ParseStackAndKeywords
Issue #29286.
2017-01-17 01:29:01 +01:00
Victor Stinner 37e4ef7b17 Issue #27810: Rerun Argument Clinic on all modules 2016-09-09 20:00:13 -07:00
Serhiy Storchaka 15f3228b7c Issue #16764: Support keyword arguments to zlib.decompress(). Patch by
Xiang Zhang.
2016-08-15 10:06:16 +03:00
Serhiy Storchaka 9171a8b4ce Issue #27574: Decreased an overhead of parsing keyword arguments in functions
implemented with using Argument Clinic.
2016-08-14 10:52:18 +03:00
Martin Panter 525a949251 Issue #27130: Merge zlib 64-bit fixes from 3.5 2016-07-23 03:39:49 +00:00
Martin Panter 84544c1020 Issue #27130: Fix handling of buffers exceeding UINT_MAX in “zlib” module
Patch by Xiang Zhang.
2016-07-23 03:02:07 +00:00
Serhiy Storchaka 2954f83999 - Issue #27332: Fixed the type of the first argument of module-level functions
generated by Argument Clinic.  Patch by Petr Viktorin.
2016-07-07 18:20:03 +03:00
Serhiy Storchaka 1a2b24f02d Issue #27332: Fixed the type of the first argument of module-level functions
generated by Argument Clinic.  Patch by Petr Viktorin.
2016-07-07 17:35:15 +03:00
Serhiy Storchaka 95657cdd40 Issue #26243: Only the level argument to zlib.compress() is keyword argument
now.  The first argument is positional-only.
2016-06-25 22:43:05 +03:00
Serhiy Storchaka 5dee6551e2 Issue #26305: Argument Clinic now uses braces in C code as required by PEP 7. 2016-06-09 16:16:06 +03:00
Martin Panter f4affb71bc Issue #5784: Merge zlib from 3.5 2016-05-27 08:00:24 +00:00
Martin Panter 0fdf41d847 Issue #5784: Expand documentation and tests for zlib wbits parameter
Based on documentation by AM Kuchling.
2016-05-27 07:32:11 +00:00
Martin Panter b0cb42dfdb Issue 26243: Forgot to update zlib doc strings in Argument Clinic 2016-02-10 10:45:54 +00:00
Martin Panter 1fe0d13d12 Issue #26243: zlib.compress() keyword argument support by Aviv Palivoda 2016-02-10 10:06:36 +00:00
Martin Panter 567d513b9b Issue #26244: Clarify default zlib compression level in documentation
Based on patch by Aviv Palivoda.
2016-02-03 07:06:33 +00:00
Martin Panter e99e97762c Issue #25626: Change zlib to accept Py_ssize_t and cap to UINT_MAX
The underlying zlib library stores sizes in “unsigned int”. The corresponding
Python parameters are all sizes of buffers filled in by zlib, so it is okay
to reduce higher values to the UINT_MAX internal cap. OverflowError is still
raised for sizes that do not fit in Py_ssize_t.

Sizes are now limited to Py_ssize_t rather than unsigned long, because Python
byte strings cannot be larger than Py_ssize_t. Previously this could result
in a SystemError on 32-bit platforms.

This resolves a regression in the gzip module when reading more than UINT_MAX
or LONG_MAX bytes in one call, introduced by revision 62723172412c.
2015-11-20 08:13:35 +00:00
Serhiy Storchaka 247789cee9 Issue #24007: Argument Clinic now writes the format of PyArg_Parse*() at the
same line as function name.
2015-04-24 00:40:51 +03:00
Larry Hastings 89964c48d1 Issue #23944: Argument Clinic now wraps long impl prototypes at column 78. 2015-04-14 18:07:59 -04:00
Serhiy Storchaka 92e8af67a8 Issue #23492: Argument Clinic now generates argument parsing code with
PyArg_Parse instead of PyArg_ParseTuple if possible.
2015-04-04 00:12:11 +03:00
Larry Hastings 0759f84d62 Issue #23500: Argument Clinic is now smarter about generating the "#ifndef"
(empty) definition of the methoddef macro: it's only generated once, even
if Argument Clinic processes the same symbol multiple times, and it's emitted
at the end of all processing rather than immediately after the first use.
2015-04-03 13:09:02 -07:00
Larry Hastings 2623c8c23c Issue #20530: Argument Clinic's signature format has been revised again.
The new syntax is highly human readable while still preventing false
positives.  The syntax also extends Python syntax to denote "self" and
positional-only parameters, allowing inspect.Signature objects to be
totally accurate for all supported builtins in Python 3.4.
2014-02-08 22:15:29 -08:00
Serhiy Storchaka 54c1391ba9 Issue #20489: Explicitly qualified expressions for default values in methods. 2014-02-05 13:34:01 +02:00
Larry Hastings 7726ac9163 #Issue 20456: Several improvements and bugfixes for Argument Clinic,
including correctly generating code for Clinic blocks inside C
preprocessor conditional blocks.
2014-01-31 22:03:12 -08:00
Larry Hastings 581ee3618c Issue #20326: Argument Clinic now uses a simple, unique signature to
annotate text signatures in docstrings, resulting in fewer false
positives.  "self" parameters are also explicitly marked, allowing
inspect.Signature() to authoritatively detect (and skip) said parameters.

Issue #20326: Argument Clinic now generates separate checksums for the
input and output sections of the block, allowing external tools to verify
that the input has not changed (and thus the output is not out-of-date).
2014-01-28 05:00:08 -08:00
Serhiy Storchaka 2c5ddbe030 Issue #20193: The zlib module now uses Argument Clinic. 2014-01-27 00:03:31 +02:00