`urllib.unquote_to_bytes` and `urllib.unquote` could both potentially generate `O(len(string))` intermediate `bytes` or `str` objects while computing the unquoted final result depending on the input provided. As Python objects are relatively large, this could consume a lot of ram.
This switches the implementation to using an expanding `bytearray` and a generator internally instead of precomputed `split()` style operations.
Microbenchmarks with some antagonistic inputs like `mess = "\u0141%%%20a%fe"*1000` show this is 10-20% slower for unquote and unquote_to_bytes and no different for typical inputs that are short or lack much unicode or % escaping. But the functions are already quite fast anyways so not a big deal. The slowdown scales consistently linear with input size as expected.
Memory usage observed manually using `/usr/bin/time -v` on `python -m timeit` runs of larger inputs. Unittesting memory consumption is difficult and does not seem worthwhile.
Observed memory usage is ~1/2 for `unquote()` and <1/3 for `unquote_to_bytes()` using `python -m timeit -s 'from urllib.parse import unquote, unquote_to_bytes; v="\u0141%01\u0161%20"*500_000' 'unquote_to_bytes(v)'` as a test.
An earlier commit only defined check_ticks_per_second() when HAVE_TIMES is defined. However, we also need it when HAVE_CLOCK is defined. This primarily affects Windows.
https://github.com/python/cpython/issues/81057
Stack effects can now have a type, e.g. `inst(X, (left, right -- jump/uint64_t)) { ... }`.
Instructions converted to the non-legacy format:
* COMPARE_OP
* COMPARE_OP_FLOAT_JUMP
* COMPARE_OP_INT_JUMP
* COMPARE_OP_STR_JUMP
* STORE_ATTR
* DELETE_ATTR
* STORE_GLOBAL
* STORE_ATTR_INSTANCE_VALUE
* STORE_ATTR_WITH_HINT
* STORE_ATTR_SLOT, and complete the store_attr family
* Complete the store_subscr family: STORE_SUBSCR{,DICT,LIST_INT}
(STORE_SUBSCR was alread half converted,
but wasn't using cache effects yet.)
* DELETE_SUBSCR
* PRINT_EXPR
* INTERPRETER_EXIT (a bit weird, ends in return)
* RETURN_VALUE
* GET_AITER (had to restructure it some)
The original had mysterious `SET_TOP(NULL)` before `goto error`.
I assume those just account for `obj` having been decref'ed,
so I got rid of them in favor of the cleanup implied by `ERROR_IF()`.
* LIST_APPEND (a bit unhappy with it)
* SET_ADD (also a bit unhappy with it)
Various other improvements/refactorings as well.
test.test_ast_recursion_limit() now uses infinite_recursion() of
test.support to prevent crashes on debug builds.
Before this change, the test crashed on ARM64 Windows 3.x buildbot
worker which builds Python in debug mode.
When checking for auto() instances, only top-level usage is supported,
which means either alone or as part of a regular tuple. Other
containers, such as lists, dicts, or namedtuples, will not have auto()
transformed into a value.
A few TCP socket options have been added to the Linux kernel these last
few years.
This commit adds all the ones available in Linux 6.0:
https://elixir.bootlin.com/linux/v6.0/source/include/uapi/linux/tcp.h#L91
While at it, the TCP_FASTOPEN option has been moved lower in the list
just to keep the same order as in tcp.h to ease future synchronisations.
Signed-off-by: Matthieu Baerts <matthieu.baerts@tessares.net>
The Py_CLEAR(), Py_SETREF() and Py_XSETREF() macros now only evaluate
their arguments once. If an argument has side effects, these side
effects are no longer duplicated.
Use temporary variables to avoid duplicating side effects of macro
arguments. If available, use _Py_TYPEOF() to avoid type punning.
Otherwise, use memcpy() for the assignment to prevent a
miscompilation with strict aliasing caused by type punning.
Add _Py_TYPEOF() macro: __typeof__() on GCC and clang.
Add test_py_clear() and test_py_setref() unit tests to _testcapi.
Shantanu
Yesung(Isaac) Lee
Beweeted
Gregory P. Smith
DarioDaF
Brad Wolfe
busywhitespace
Fantix King
Kai Zhang
ram vikram singh
Eric Snow
Raymond Hettinger
andrei kulakov
Shreyan Avigyan
Mark Shannon
Christian Rendina
Ken Jin
Kumar Aditya
Stanley
Guido van Rossum
Jia Junjie
Victor Stinner
Ethan Furman
Matthieu Baerts
Takeshi KOMIYA
Dong-hee Na
Nikita Sobolev
Irit Katriel
Carl Meyer
Matěj Cepl
Pablo Galindo Salgado
Thomas Wouters