EnumMeta double-checks that `__repr__`, `__str__`, `__format__`, and `__reduce_ex__` are not the same as `object`'s, and replaces them if they are -- even if that replacement was intentionally done in the Enum being constructed. This patch fixes that.
Partially revert commit ac46eb4ad6662cf6d771b20d8963658b2186c48c:
"bpo-38113: Update the Python-ast.c generator to PEP384 (gh-15957)".
Using a module state per module instance is causing subtle practical
problems.
For example, the Mercurial project replaces the __import__() function
to implement lazy import, whereas Python expected that "import _ast"
always return a fully initialized _ast module.
Add _PyAST_Fini() to clear the state at exit.
The _ast module has no state (set _astmodule.m_size to 0). Remove
astmodule_traverse(), astmodule_clear() and astmodule_free()
functions..
(cherry picked from commit e5fbe0cbd4)
Co-authored-by: Victor Stinner <vstinner@python.org>
Argument script_exec_args is usually an absolute file name,
but twice has form ['-m', 'module_name'].
(cherry picked from commit 7e711ead26)
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
* Add failing test.
* bpo-29590: fix stack trace for gen.throw() with yield from (GH-NNNN)
When gen.throw() is called on a generator after a "yield from", the
intermediate stack trace entries are lost. This commit fixes that.
(cherry picked from commit 8b33961e4b)
Co-authored-by: Chris Jerdonek <chris.jerdonek@gmail.com>
Simply closing the event loop isn't enough to avoid warnings. If we
don't also shut down the event loop's default executor, it sometimes
logs a "dangling thread" warning.
Follow-up to GH-22017
(cherry picked from commit be435ae2b0)
Co-authored-by: Ben Darnell <ben@bendarnell.com>
Co-authored-by: Ben Darnell <ben@bendarnell.com>
* bpo-41696: Fix handling of debug mode in asyncio.run
This allows PYTHONASYNCIODEBUG or -X dev to enable asyncio debug mode
when using asyncio.run
* 📜🤖 Added by blurb_it.
Co-authored-by: hauntsaninja <>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
(cherry picked from commit 0770ad948c)
Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
Stopping and restarting a proactor event loop on windows can lead to
spurious errors logged (ConnectionResetError while reading from the
self pipe). This fixes the issue by ensuring that we don't attempt
to start multiple copies of the self-pipe reading loop.
(cherry picked from commit ea5a6363c3)
Co-authored-by: Ben Darnell <ben@bendarnell.com>
Co-authored-by: Ben Darnell <ben@bendarnell.com>
Currently, empty sequences in gather rules make the conditional for
gather rules fail as empty sequences evaluate as "False". We need to
explicitly check for "None" (the failure condition) to avoid false
negatives.
(cherry picked from commit e55a0e9)
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
When allocating MemoryError classes, there is some logic to use
pre-allocated instances in a freelist only if the type that is being
allocated is not a subclass of MemoryError. Unfortunately in the
destructor this logic is not present so the freelist is altered even
with subclasses of MemoryError..
(cherry picked from commit 9b648a95cc)
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
* Fixed `f-string/str.format` error description when using two `,` in format specifier.
Co-authored-by: millefalcon <hanish0019@hmail.com>
(cherry picked from commit 0d6aa7f0ee)
Co-authored-by: han-solo <hanish0019@gmail.com>
Co-authored-by: han-solo <hanish0019@gmail.com>
Currently, if `asyncio.wait_for()` itself is cancelled it will always
raise `CancelledError` regardless if the underlying task is still
running. This is similar to a race with the timeout, which is handled
already.
(cherry picked from commit a2118a1462)
Co-authored-by: Elvis Pranskevichus <elvis@magic.io>
When I was fixing bpo-32751 back in GH-7216 I missed the case when
*timeout* is zero or negative. This takes care of that.
Props to @aaliddell for noticing the inconsistency.
(cherry picked from commit c517fc7121)
Co-authored-by: Elvis Pranskevichus <elvis@magic.io>
- AIX has extreme over-allocation that is in no relation to the physical
RAM and swap.
(cherry picked from commit 39dab24621)
Authored-by: Stefan Krah <skrah@bytereef.org>
Prior to this change, attempting to subclass the C implementation of
zoneinfo.ZoneInfo gave the following error:
TypeError: unbound method ZoneInfo.__init_subclass__() needs an argument
https://bugs.python.org/issue41025
(cherry picked from commit 87d8287865)
Co-authored-by: Paul Ganssle <paul@ganssle.io>
regrtest_unraisable_hook() temporarily replaces sys.stderr with
sys.__stderr__ to help to display errors when a test captures stderr.
(cherry picked from commit 701b63894f)
The issue raised by recursive evaluation is infinite recursion with
recursive types. In that case, only the first recursive ForwardRef is
evaluated.
(cherry picked from commit 653f420b53)
Co-authored-by: wyfo <joperez@hotmail.fr>
Walk down the MRO backwards to find the type that originally defined the final `tp_setattro`, then make sure we are not jumping over intermediate C-level bases with the Python-level call.
Automerge-Triggered-By: @gvanrossum
(cherry picked from commit c53b310e59)
Co-authored-by: scoder <stefan_ml@behnel.de>
reject control chars in http method in http.client.putrequest to prevent http header injection
(cherry picked from commit 8ca8a2e8fb)
Co-authored-by: AMIR <31338382+amiremohamadi@users.noreply.github.com>
Avoid infinite loop when reading specially crafted TAR files using the tarfile module
(CVE-2019-20907).
(cherry picked from commit 5a8d121a1f)
Co-authored-by: Rishi <rishi_devan@mail.com>
Automerge-Triggered-By: @encukou
3.8.3 had a regression where compiling with
ast.PyCF_ALLOW_TOP_LEVEL_AWAIT woudl agressively mark things are
coroutine even if there were not.
(cherry picked from commit bd46174)
Co-authored-by: Matthias Bussonnier <bussonniermatthias@gmail.com>
Co-authored-by: Matthias Bussonnier <bussonniermatthias@gmail.com>
Also enables using debug build of `python3_d.dll`
Reference: CVE-2020-15523
(cherry picked from commit dcbaa1b49c)
Co-authored-by: Steve Dower <steve.dower@python.org>
The __hash__() methods of classes IPv4Interface and IPv6Interface had issue
of generating constant hash values of 32 and 128 respectively causing hash collisions.
The fix uses the hash() function to generate hash values for the objects
instead of XOR operation
(cherry picked from commit b30ee26e36)
Co-authored-by: Ravi Teja P <rvteja92@gmail.com>
Fix also a resource warning when store counts and module info.
(cherry picked from commit 04cdeb7a56)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Prefix the error message with `fstring: `, when parsing an f-string expression throws a `SyntaxError`.
(cherry picked from commit 2e0a920e9e)
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
Signed-off-by: Christian Heimes <christian@python.org>
Automerge-Triggered-By: @tiran.
(cherry picked from commit bb6ec14479)
Co-authored-by: Christian Heimes <christian@python.org>
* Fix support of non-ASCII names in functions OpenDatabase()
and init_database().
* Fix support of non-ASCII SQL in method Database.OpenView().
(cherry picked from commit 55939b1708)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Unexpected errors in calling the __iter__ method are no longer
masked by TypeError in csv.reader(), csv.writer.writerow() and
csv.writer.writerows().
(cherry picked from commit c88239f864)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Unexpected errors in calling the __iter__ method are no longer
masked by TypeError in the "in" operator and functions
operator.contains(), operator.indexOf() and operator.countOf().
(cherry picked from commit cafe1b6e9d)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
`GET_INVALID_TARGET` might unexpectedly return `NULL`, which if not
caught will cause a SEGFAULT. Therefore, this commit introduces a new
inline function `RAISE_SYNTAX_ERROR_INVALID_TARGET` that always
checks for `GET_INVALID_TARGET` returning NULL and can be used in
the grammar, replacing the long C ternary operation used till now.
(cherry picked from commit 6c4e0bd974)
Automerge-Triggered-By: @pablogsal
skip_if_broken_multiprocessing_synchronize() only attempts for create
a semaphore on Linux to fix multiprocessing
test_resource_tracker_reused() on macOS.
(cherry picked from commit 3358da4054)
Co-authored-by: Victor Stinner <vstinner@python.org>
* bpo-40334: Produce better error messages on invalid targets (GH-20106)
The following error messages get produced:
- `cannot delete ...` for invalid `del` targets
- `... is an illegal 'for' target` for invalid targets in for
statements
- `... is an illegal 'with' target` for invalid targets in
with statements
Additionally, a few `cut`s were added in various places before the
invocation of the `invalid_*` rule, in order to speed things
up.
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
(cherry picked from commit 01ece63d42)
On Linux, skip tests using multiprocessing if the current user cannot
create a file in /dev/shm/ directory. Add the
skip_if_broken_multiprocessing_synchronize() function to the
test.support module.
(cherry picked from commit ddbeb2f3e0)
…-20937)
test_hashlib emits some warnings when it cannot find some hashes
as it assumes they failed to compile. Since we can disable hashes
through configure, we emit the warnings only in the case that we
did not intentionaly disable them.
Automerge-Triggered-By: @tiran
(cherry picked from commit 236a0f5)
Co-authored-by: stratakis <cstratak@redhat.com>
Fix test_copyreg when numpy is installed: test.pickletester now
saves/restores warnings.filters when importing numpy, to ignore
filters installed by numpy.
Add the save_restore_warnings_filters() function to the
test.support.warnings_helper module.
(cherry picked from commit 8362893e3f)
When a file ends with a line that contains a line continuation character
the text of the emitted SyntaxError is empty, contrary to the old
parser, where the error text contained the text of the last line.
(cherry picked from commit 113e2b0a07)
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
ensurepip optionally installs or upgrades 'pip' and 'setuptools' using
the version of those modules bundled with Python. The internal PIP
installation routine by default temporarily uses its cache, if it
exists. This is undesirable as Python builds and installations may be
independent of the user running the build, whilst PIP cache location
is dependent on the user's environment and outside of the build
environment.
At the same time, there's no value in using the cache while installing
bundled modules.
This change disables PIP caching when used in ensurepip.
(cherry picked from commit 4a3a682b12)
Co-authored-by: Krzysztof Konopko <kkonopko@users.noreply.github.com>
In Python 3.7 the behavior of parse_multipart changed requiring CONTENT-LENGTH
header, this fix remove this header as required and fix FieldStorage
read_lines_to_outerboundary, by not using limit when it's negative,
since by default it's -1 if not content-length and keeps substracting what
was read from the file object.
Also added a test case for this problem.
(cherry picked from commit d8cf3514dd)
Co-authored-by: roger <rogerduran@gmail.com>
Using a log2n way to fill a much smaller buffer, and receiving in a cleaner way with EOF.
The failing test was reproducible using the following command thanks to @aeros :
```bash
./python -m test test_asyncio.test_sock_lowlevel --match test_sock_client_racing -j100 -F -v
```
According to test results, we may still need to bump the timeout:
5aad027db9/Lib/test/test_asyncio/test_sock_lowlevel.pyGH-L256-L257
(cherry picked from commit 8f04a84755)
Co-authored-by: Fantix King <fantix.king@gmail.com>
Remote host cyrus.andrew.cmu.edu is blocking incoming connections and is
causing test suite to fail.
Signed-off-by: Christian Heimes <christian@python.org>
Followup of bpo-40854, there is one remaining usage of PLATLIBDIR
which should be replaced by config->platlibdir.
test_sys checks that sys.platlibdir attribute exists and is a string.
Update Makefile: getpath.c and sysmodule.c no longer need PLATLIBDIR
macro, PyConfig.platlibdir member is used instead.
Co-authored-by: Sandro Mani <manisandro@gmail.com>
(cherry picked from commit d72b9644a3)
The GDB provided by HPE on HP-UX contains a modified version string. Therefore
the tests fail. Adapt the regex to match that string.
Patch by Michael Osipov.
Co-Authored-by: Michael Osipov <michael.osipov@siemens.com>
(cherry picked from commit b2dca49ca3)
Co-authored-by: Victor Stinner <vstinner@python.org>
A line with only a line continuation character should be considered
a blank line at tokenizer level so that only a single NEWLINE token
gets emitted. The old parser was working around the issue, but the
new parser threw a `SyntaxError` for valid input. For example,
an empty line following a line continuation character was interpreted
as a `SyntaxError`.
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
(cherry picked from commit 896f4cf63f)
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
test_repl.test_close_stdin() now calls
support.suppress_msvcrt_asserts() to fix the test on Windows.
* Move suppress_msvcrt_asserts() from test.libregrtest.setup to
test.support. Make its verbose parameter optional: verbose=False by
default.
* SuppressCrashReport now uses SetErrorMode() of the msvcrt module,
rather than using ctypes.
* Remove also an unused variable (deadline) in wait_process().
(cherry picked from commit f6e58aefde)
Export explicitly the Py_GetArgcArgv() function to the C API and
document the function. Previously, it was exported implicitly which
no longer works since Python is built with -fvisibility=hidden.
* Add PyConfig._orig_argv member.
* Py_InitializeFromConfig() no longer calls _PyConfig_Write() twice.
* PyConfig_Read() no longer initializes Py_GetArgcArgv(): it is now
_PyConfig_Write() responsibility.
* _PyConfig_Write() result type becomes PyStatus instead of void.
* Write an unit test on Py_GetArgcArgv().
(cherry picked from commit e81f6e687d)
* Refresh importlib.metadata from importlib_metadata 1.6.1.
* 📜🤖 Added by blurb_it.
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
(cherry picked from commit 161541ab45)
Co-authored-by: Jason R. Coombs <jaraco@jaraco.com>
Co-authored-by: Jason R. Coombs <jaraco@jaraco.com>
* bpo-40807: Show warnings once from codeop._maybe_compile
* Move catch_warnings
* news
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
(cherry picked from commit 052d3fc090)
Co-authored-by: Cheryl Sabella <cheryl.sabella@gmail.com>
Skip some :mod:`multiprocessing` tests when MD5 hash digest is blocked.
Signed-off-by: Christian Heimes <christian@python.org>
(cherry picked from commit b022e5cffb)
Co-authored-by: Christian Heimes <christian@python.org>
* bpo-40826: Add _Py_EnsureTstateNotNULL() macro (GH-20571)
Add _Py_EnsureTstateNotNULL(tstate) macro: call Py_FatalError() if
tstate is NULL, the error message contains the current function name.
(cherry picked from commit 3026cad59b)
* bpo-40826: PyOS_InterruptOccurred() requires GIL (GH-20578)
PyOS_InterruptOccurred() now fails with a fatal error if it is called
with the GIL released.
(cherry picked from commit cbe1296922)
* bpo-40826: Fix GIL usage in PyOS_Readline() (GH-20579)
Fix GIL usage in PyOS_Readline(): lock the GIL to set an exception.
Pass tstate to my_fgets() and _PyOS_WindowsConsoleReadline(). Cleanup
these functions.
(cherry picked from commit c353764fd5)
* bpo-40826: Add _PyOS_InterruptOccurred(tstate) function (GH-20599)
my_fgets() now calls _PyOS_InterruptOccurred(tstate) to check for
pending signals, rather calling PyOS_InterruptOccurred().
my_fgets() is called with the GIL released, whereas
PyOS_InterruptOccurred() must be called with the GIL held.
test_repl: use text=True and avoid SuppressCrashReport in
test_multiline_string_parsing().
Fix my_fgets() on Windows: fgets(fp) does crash if fileno(fp) is closed.
(cherry picked from commit fa7ab6aa0f)
* bpo-40630: Add tracemalloc.reset_peak (GH-20102, cherrypick 8b62644)
The reset_peak function sets the peak memory size to the current size,
representing a resetting of that metric. This allows for recording the
peak of specific sections of code, ignoring other code that may have
had a higher peak (since the most recent `tracemalloc.start()` or
tracemalloc.clear_traces()` call).
* Adjust docs to point to 3.9
The topological sort functionality that was introduced initially in the
functools module has been moved to a new graphlib module to
better accommodate the new tools and keep the original scope of the
functools module.
(cherry picked from commit 2f172d8f15)
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
* Fix failure of _Py_dg_dtoa to remove trailing zeros
* Add regression test and news entry
* Add explanation about why it's safe to strip trailing zeros
* Make code safer, clean up comments, add change note at top of file
* Nitpick: avoid implicit int-to-float conversion in tests
(cherry picked from commit 895c9c1d43)
Co-authored-by: Mark Dickinson <mdickinson@enthought.com>
* bpo-39040: Fix parsing of email headers with encoded-words inside a quoted string.
It is fairly common to find malformed mime headers (especially content-disposition
headers) where the parameter values, instead of being encoded to RFC
standards, are "encoded" by doing RFC 2047 "encoded word" encoding, and
then enclosing the whole thing in quotes. The processing of these malformed
headers was incorrectly leaving the spaces between encoded words in the decoded
text (whitespace between adjacent encoded words is supposed to be stripped on
decoding). This changeset fixes the encoded word processing inside quoted strings
(bare-quoted-string) to do correct RFC 2047 decoding by stripping that
whitespace.
(cherry picked from commit 21017ed904)
Co-authored-by: Abhilash Raj <maxking@users.noreply.github.com>
Skip new "racing" socket tests which fail randomly until someone fix
them, to ease analysis of buildbot failures (skip tests which are
known to be broken/unstable).
(cherry picked from commit 84ee7e1573)
Co-authored-by: Victor Stinner <vstinner@python.org>
hashlib.compare_digest uses OpenSSL's CRYPTO_memcmp() function
when OpenSSL is available.
Note: The _operator module is a builtin module. I don't want to add
libcrypto dependency to libpython. Therefore I duplicated the wrapper
function and added a copy to _hashopenssl.c..
(cherry picked from commit db5aed931f)
Co-authored-by: Christian Heimes <christian@python.org>
struct.error is now raised if there is a null character in a struct
format string.
(cherry picked from commit 3f59b55316)
Co-authored-by: Zackery Spytz <zspytz@gmail.com>
When a `SyntaxError` in the expression part of a fstring is found,
the filename attribute of the `SyntaxError` is always `<fstring>`.
With this commit, it gets changed to always have the name of the file
the fstring resides in.
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>.
(cherry picked from commit f7b1e46156)
This updates _PyErr_ChainStackItem() to use _PyErr_SetObject()
instead of _PyErr_ChainExceptions(). This prevents a hang in
certain circumstances because _PyErr_SetObject() performs checks
to prevent cycles in the exception context chain while
_PyErr_ChainExceptions() doesn't.
(cherry picked from commit 7c30d12bd5)
Co-authored-by: Chris Jerdonek <chris.jerdonek@gmail.com>
The error message, generated for a non-parenthesized generator expression
in function calls, was still the generic `invalid syntax`, when the generator expression wasn't appearing as the first argument in the call. With this patch, even on input like `f(a, b, c for c in d, e)`, the correct error message gets produced.
(cherry picked from commit ae14583302)
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
Allows contextvars from the main thread to be accessed in the separate thread used in `asyncio.to_thread()`. See the [discussion](https://github.com/python/cpython/pull/20143GH-discussion_r427808225) in GH-20143 for context.
Automerge-Triggered-By: @aeros
(cherry picked from commit 0f56263e62)
Co-authored-by: Kyle Stanley <aeros167@gmail.com>
Implements `asyncio.to_thread`, a coroutine for asynchronously running IO-bound functions in a separate thread without blocking the event loop. See the discussion starting from [here](https://github.com/python/cpython/pull/18410GH-issuecomment-628930973) in GH-18410 for context.
Automerge-Triggered-By: @aeros
(cherry picked from commit cc2bbc2227)
Co-authored-by: Kyle Stanley <aeros167@gmail.com>
Make the the following imports lazy in test.support:
* bz2
* gzip
* lzma
* resource
* zlib
The following test.support decorators now need to be called
with parenthesis:
* @support.requires_bz2
* @support.requires_gzip
* @support.requires_lzma
* @support.requires_zlib
For example, "@requires_zlib" becomes "@requires_zlib()".
When unparsing a non-empty tuple, the parentheses can be safely
omitted if there aren't any elements that explicitly require them (such as starred expressions).
Unprintable characters such as `\x00` weren't correctly roundtripped
due to not using default string repr when generating docstrings. This
patch correctly encodes all unprintable characters (except `\n` and `\t`, which
are commonly used for formatting, and found unescaped).
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
Co-authored-by: Batuhan Taskaya <isidentical@gmail.com>
Miss Islington (bot)
Łukasz Langa
Ethan Furman
Pablo Galindo
Victor Stinner
Terry Jan Reedy
Steve Dower
Karthikeyan Singaravelan
Berker Peksag
Serhiy Storchaka
Lysandros Nikolaou
Guido van Rossum
Christian Heimes
Dong-hee Na
stratakis
Huon Wilson
Hai Shi
Batuhan Taskaya
CyberSaxosTiGER
Irit Katriel