Fix a race condition in "make regen-all" when make -jN option is used
to run jobs in parallel. The clinic.py script now only use atomic
write to write files. Moveover, generated files are now left
unchanged if the content does not change, to not change the file
modification time.
The "make regen-all" command runs "make clinic" and "make
regen-importlib" targets:
* "make regen-importlib" builds object files (ex: Modules/_weakref.o)
from source files (ex: Modules/_weakref.c) and clinic files (ex:
Modules/clinic/_weakref.c.h)
* "make clinic" always rewrites all clinic files
(ex: Modules/clinic/_weakref.c.h)
Since there is no dependency between "clinic" and "regen-importlib"
Makefile targets, these two targets can be run in parallel. Moreover,
half of clinic.py file writes are not atomic and so there is a race
condition when "make regen-all" runs jobs in parallel using make -jN
option (which can be passed in MAKEFLAGS environment variable).
Fix clinic.py to make all file writes atomic:
* Add write_file() function to ensure that all file writes are
atomic: write into a temporary file and then use os.replace().
* Moreover, write_file() doesn't recreate or modify the file if the
content does not change to avoid modifying the file modification
file.
* Update test_clinic to verify these assertions with a functional
test.
* Remove Clinic.force attribute which was no longer used, whereas
Clinic.verify remains useful.
(cherry picked from commit 8fba9523cf)
(cherry picked from commit c53c3f4000)
Co-authored-by: Victor Stinner <vstinner@python.org>
skip test_min_max_version_mismatch when TLS 1.0 is not available
Signed-off-by: Christian Heimes <christian@python.org>
(cherry picked from commit ce04e7105b)
Co-authored-by: Christian Heimes <christian@python.org>
The onerror is supposed to be called with failed function, but in this case lstat is wrongly used instead of open.
Not sure if this needs bug or not...
Automerge-Triggered-By: GH:hynek
(cherry picked from commit e59b2deffd)
Co-authored-by: Michal Čihař <michal@cihar.com>
The overflow occurs under some circumstances when a task or future
recursively returns itself.
Co-authored-by: Kyle Stanley <aeros167@gmail.com>
(cherry picked from commit 42d873c63a)
Co-authored-by: Andrew Svetlov <andrew.svetlov@gmail.com>
It is not clear why this can happen, but several users have mentioned
getting this exception on macOS.
(cherry picked from commit 23831a7a90)
Co-authored-by: Ronald Oussoren <ronaldoussoren@mac.com>
Co-authored-by: Ronald Oussoren <ronaldoussoren@mac.com>
* Prevent some possible DoS attacks via providing invalid Plist files
with extremely large number of objects or collection sizes.
* Raise InvalidFileException for too large bytes and string size instead of returning garbage.
* Raise InvalidFileException instead of ValueError for specific invalid datetime (NaN).
* Raise InvalidFileException instead of TypeError for non-hashable dict keys.
* Add more tests for invalid Plist files.
(cherry picked from commit 34637a0ce2)
They were occurring with both repeated 'force-calltip' invocations and by typing parentheses
in expressions, strings, and comments in the argument code.
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
(cherry picked from commit da7bb7b4d7)
Co-authored-by: Tal Einat <taleinat+github@gmail.com>
[[bpo-29566]()]() notes that binhex.binhex uses inconsistent line endings (both Unix and MacOS9 line endings are used). This PR changes this to use the MacOS9 line endings everywhere.
(cherry picked from commit 2165cea548)
Co-authored-by: Ronald Oussoren <ronaldoussoren@mac.com>
Automerge-Triggered-By: GH:ronaldoussoren
* When the parameters argument is a list, correctly handle the case
of changing it during iteration.
* When the parameters argument is a custom sequence, no longer
override an exception raised in ``__len__()``.
(cherry picked from commit 0b419b7910)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
shutil.which will not return None anymore for empty str in PATHEXT
Empty PATHEXT will now be defaulted to _WIN_DEFAULT_PATHEXT
(cherry picked from commit da6f098188)
Co-authored-by: Christopher Marchfelder <marchfelder@googlemail.com>
tarfile writes full path to FNAME field of GZIP format instead of just basename if user specified absolute path. Some archive viewers may process file incorrectly. Also it creates security issue because anyone can know structure of directories on system and know username or other personal information.
RFC1952 says about FNAME:
This is the original name of the file being compressed, with any directory components removed.
So tarfile must remove directory names from FNAME and write only basename of file.
Automerge-Triggered-By: @jaraco
(cherry picked from commit 22748a83d9)
Co-authored-by: Artem Bulgakov <ArtemSBulgakov@ya.ru>
Use wide-char _W_* fields of lconv structure on Windows
Remove "ps_AF" from test__locale.known_numerics on Windows
(cherry picked from commit f2312037e3)
Co-authored-by: TIGirardi <tiagoigirardi@gmail.com>
(cherry picked from commit 3185267400)
Co-authored-by: Ronald Oussoren <ronaldoussoren@mac.com>
Co-authored-by: Ronald Oussoren <ronaldoussoren@mac.com>
Reduces the rate at which the ENV CHANGED failure occurs in test_asyncio SSL tests (due to unclosed transport), but does not 100% resolve it.
(cherry picked from commit de73d432bb)
Co-authored-by: Justin Turner Arthur <justinarthur@gmail.com>
This PR replaces GH-1977. The reason for the replacement is two-fold.
The fix itself is different is that if the CTE header doesn't exist in the original message, it is inserted. This is important because the new CTE could be quoted-printable whereas the original is implicit 8bit.
Also the tests are different. The test_nonascii_as_string_without_cte test in GH-1977 doesn't actually test the issue in that it passes without the fix. The test_nonascii_as_string_without_content_type_and_cte test is improved here, and even though it doesn't fail without the fix, it is included for completeness.
Automerge-Triggered-By: @warsaw
(cherry picked from commit bf838227c3)
Co-authored-by: Mark Sapiro <mark@msapiro.net>
_get_socket() already prints a debug message for the host and port.
https://bugs.python.org/issue32793
Automerge-Triggered-By: @maxking
(cherry picked from commit 46a7564578)
Co-authored-by: Zackery Spytz <zspytz@gmail.com>
Co-authored-by: Zackery Spytz <zspytz@gmail.com>
The test now waits until all threads complete to avoid leaking
running threads.
Also, use regular threads rather than daemon threads.
(cherry picked from commit 13ff396c01)
(cherry picked from commit f5393dc2a0)
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Victor Stinner <vstinner@python.org>
This fixes the test failure with Tk 6.8.10 which is caused by changes to how Tk rounds the `from`, `to` and `tickinterval` arguments. This PR uses `noconv` if the patchlevel is greater than or equal to 8.6.10 (credit to Serhiy for this idea as it is much simpler than what I previously proposed).
Going into more detail for those who want it, the Tk change was made in [commit 591f68c](591f68cb38) and means that the arguments listed above are rounded relative to the value of `from`. However, when rounding the `from` argument ([line 623](591f68cb38/generic/tkScale.cGH-L623)), it is rounded relative to itself (i.e. rounding `0`) and therefore the assigned value for `from` is always what is given (no matter what values of `from` and `resolution`).
Automerge-Triggered-By: @pablogsal
(cherry picked from commit aecf036738)
Co-authored-by: E-Paine <63801254+E-Paine@users.noreply.github.com>
Call urllib.request.urlcleanup() to reset the global
urllib.request._opener.
(cherry picked from commit 1fce240d6c)
Co-authored-by: Victor Stinner <vstinner@python.org>
Enable recursion checks which were disabled when get __bases__ of
non-type objects in issubclass() and isinstance() and when intern
strings. It fixes a stack overflow when getting __bases__ leads
to infinite recursion.
Originally recursion checks was disabled for PyDict_GetItem() which
silences all errors including the one raised in case of detected
recursion and can return incorrect result. But now the code uses
PyDict_GetItemWithError() and PyDict_SetDefault() instead.
(cherry picked from commit 9ece9cd65c)
'Python Shell' may have contributed to some beginners confusing 'IDLE' with ' Python'.
(cherry picked from commit 05cc881cbc)
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
The marker was added to the language in 3.8 and
3.7 only gets security patches.
(cherry picked from commit 40a0625792)
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
EnumMeta double-checks that `__repr__`, `__str__`, `__format__`, and `__reduce_ex__` are not the same as `object`'s, and replaces them if they are -- even if that replacement was intentionally done in the Enum being constructed. This patch fixes that.
Automerge-Triggered-By: @ethanfurman
(cherry picked from commit 22415ad625)
Co-authored-by: Ethan Furman <ethan@stoneleaf.us>
Fix a race condition in the call_soon_threadsafe() method of
asyncio.ProactorEventLoop: do nothing if the self-pipe socket has
been closed.
(cherry picked from commit 1b0f0e3d7d)
Co-authored-by: Victor Stinner <vstinner@python.org>
Argument script_exec_args is usually an absolute file name,
but twice has form ['-m', 'module_name'].
(cherry picked from commit 7e711ead26)
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
This is a trivial PR to fix a typo in a docstring in typing.py. From reverences -> references.
(cherry picked from commit 84ef33c511)
Co-authored-by: Graham Bleaney <gbleaney@gmail.com>
Automerge-Triggered-By: @Mariatta
The error is exposed on non-UTF-8 locales.
(cherry picked from commit 67987acd5d)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
(cherry picked from commit c73ee5acc9)
Simply closing the event loop isn't enough to avoid warnings. If we
don't also shut down the event loop's default executor, it sometimes
logs a "dangling thread" warning.
Follow-up to GH-22017
(cherry picked from commit be435ae2b0)
Co-authored-by: Ben Darnell <ben@bendarnell.com>
Co-authored-by: Ben Darnell <ben@bendarnell.com>
* bpo-41696: Fix handling of debug mode in asyncio.run
This allows PYTHONASYNCIODEBUG or -X dev to enable asyncio debug mode
when using asyncio.run
* 📜🤖 Added by blurb_it.
Co-authored-by: hauntsaninja <>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
(cherry picked from commit 0770ad948c)
Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
Stopping and restarting a proactor event loop on windows can lead to
spurious errors logged (ConnectionResetError while reading from the
self pipe). This fixes the issue by ensuring that we don't attempt
to start multiple copies of the self-pipe reading loop.
(cherry picked from commit ea5a6363c3)
Co-authored-by: Ben Darnell <ben@bendarnell.com>
Co-authored-by: Ben Darnell <ben@bendarnell.com>
When allocating MemoryError classes, there is some logic to use
pre-allocated instances in a freelist only if the type that is being
allocated is not a subclass of MemoryError. Unfortunately in the
destructor this logic is not present so the freelist is altered even
with subclasses of MemoryError..
(cherry picked from commit 9b648a95cc)
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>.
(cherry picked from commit 87e91ae2e5)
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
When I was fixing bpo-32751 back in GH-7216 I missed the case when
*timeout* is zero or negative. This takes care of that.
Props to @aaliddell for noticing the inconsistency..
(cherry picked from commit c517fc7121)
Currently, if `asyncio.wait_for()` itself is cancelled it will always
raise `CancelledError` regardless if the underlying task is still
running. This is similar to a race with the timeout, which is handled
already.
(cherry picked from commit a2118a1462)
Co-authored-by: Elvis Pranskevichus <elvis@magic.io>
Fix grammar in BaseTransport.close docstring.
https://bugs.python.org/issue41572
Signed-off-by: Cleber Rosa <crosa@redhat.com>
(cherry picked from commit 1afb42cfa8)
Co-authored-by: Cleber Rosa <cleber.gnu@gmail.com>
A comment in the venv `activate` script (as well as `activate.csh` and `activate.fish`) referencing Aspen magic directories lists a "further information" URL for Aspen at the zetadev.com website. zetadev.com changed ownership in 2019, and now redirects to a server in China with an expired security certificate.
Out of an abundance of caution, while not changing the _code_ for the activate scripts, this PR updates the URL in those comments to reference Aspen's new documentation home at https://aspen.io/.
No issue created, as I suspect this falls within the definition of a "trivial" change. Please let me know if I'm wrong about that, and I'll open the necessary issue(s).
While filed against the 3.8 branch, strictly speaking this is not a backported PR. The comment in question was entirely removed from the script between Python 3.8 and 3.9.
(IMHO this _should_ probably be backported to 3.7 and 3.6, as well. I'll happily file those PRs if needed.)
Automerge-Triggered-By: @vsajip
The error message was missing space between the action "acquire" and "_wait_semaphore" which is an attribute for instances of Condition.
(cherry picked from commit 99c0ee3c89)
Co-authored-by: Allen <64019758+aboddie@users.noreply.github.com>
asyncio.AbstractEventLoop.run_in_executor should be a method that returns an asyncio Future, not an async method.
This matches the concrete implementations, and the documentation better.
(cherry picked from commit 29f84294d8)
Co-authored-by: James Weaver <james.barrett@bbc.co.uk>
Fix the repression introduced by the initial regression fix.
(cherry picked from commit c818b15fa5)
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
(cherry picked from commit f24430f154)
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
test_run method test_fatal_error failed when run twice, as with
python -m test -m test_fatal_error test_idle test_idle
because func.called was not reinitialized to 0.
This bug caused a failure on a refleak buildbot.
(cherry picked from commit 416f0b71ba)
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
A message box pops up when an unexpected error stops the run process. Tell users it is likely a random glitch, but report it if not.
(cherry picked from commit f2e161c279)
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
Miss Islington (bot)
Serhiy Storchaka
Jakub Stasiak
Irit Katriel
Anthony Sottile
Jason R. Coombs
Dong-hee Na
Pablo Galindo
Thomas Grainger
Łukasz Langa
Terry Jan Reedy
Ethan Furman
Mariatta
Victor Stinner
Elvis Pranskevichus
Frank Dana