multiprocessing.Queue.join_thread() now waits until the thread
completes, even if the thread was started by the same process which
created the queue.
Fix the following warning which occurs randomly when running
test_handle_called_with_mp_queue of test_logging.QueueListenerTest:
Warning -- threading_cleanup() failed to cleanup -1 threads after 4 sec (count: 0, dangling: 1)
(cherry picked from commit 3b69d911c5)
If history-length is set in .inputrc, and the history file is double the
history size (or more), history_get(N) returns NULL, and python
segfaults. Fix that by checking for NULL return value.
It seems that the root cause is incorrect handling of bigger history in
readline, but Python should not segfault even if readline returns
unexpected value.
This issue affects only GNU readline. When using libedit emulation
system history size option does not work.
Leading whitespace was incorrectly dropped during folding of certain lines in the _header_value_parser's folding algorithm. This makes the whitespace handling code consistent.
* [3.6] bpo-30703: Improve signal delivery (GH-2415)
* Improve signal delivery
Avoid using Py_AddPendingCall from signal handler, to avoid calling signal-unsafe functions.
* Remove unused function
* Improve comments
* Add stress test
* Adapt for --without-threads
* Add second stress test
* Add NEWS blurb
* Address comments @haypo.
(cherry picked from commit c08177a1cc)
* bpo-30796: Fix failures in signal delivery stress test (#2488)
* bpo-30796: Fix failures in signal delivery stress test
setitimer() can have a poor minimum resolution on some machines,
this would make the test reach its deadline (and a stray signal
could then kill a subsequent test).
* Make sure to clear the itimer after the test
Split TextViewer class into ViewWindow, ViewFrame, and TextFrame classes so that instances
of the latter two can be placed with other widgets within a multiframe window.
Patch by Cheryl Sabella.
(cherry picked from commit 42bc8be)
* Add 'parens' style to highlight both opener and closer.
* Make 'default' style, which is not default, a synonym for 'opener'.
* Make time-delay work the same with all styles.
* Add help for config dialog extensions tab, including parenmatch.
* Add new tests.
Original patch by Charles Wohlganger.
(cherry picked from commit fae2c35)
This happened because shortcut has a class binding and 'break' was not returned.
Fix other potential conflicts between IDLE and default key bindings.
* Add news item
* Update NEWS
(cherry picked from commit 213ce12)
New error condition paths were introduced, which did not decrement
`key2` and `val2` objects. Therefore, decrement references before
jumping to the error label.
Signed-off-by: Eric N. Vander Weele <ericvw@gmail.com>
(cherry picked from commit a7874c73c0)
* bpo-30765: Avoid blocking when PyThread_acquire_lock() is asked not to lock
This is especially important if PyThread_acquire_lock() is called reentrantly
(for example from a signal handler).
* Update 2017-06-26-14-29-50.bpo-30765.Q5iBmf.rst
* Avoid core logic when taking the mutex failed
(cherry picked from commit f84ac420c2)
Verify user-entered key sequences by trying to bind them with tk.
Add tests for all 3 validation functions.
Original patch by G Polo. Tests added by Cheryl Sabella.
(cherry picked from commit 8c78aa7)
Bug didn't manifest itself when importing a module with source as .py files are always the first on the search path. The issue only showed up in bytecode-only packages where the calculated file path would be ``__init__.py/__init__.pyc``.
Patch by Alexandru Ardelean.
(cherry picked from commit c38e32a100)
CPython workflow is changing! We're going to start using "blurb"
to manage Misc/NEWS entries:
https://github.com/python/core-workflow
(This will be a big win for release managers, honest.)
This checkin simply populates the "Misc/NEWS.d" subdirectory tree
so that people can start putting their news entries in there.
No other changes (yet).
The current regex based splitting produces a wrong result. For example::
http://abc#@def
Web browsers parse that URL as ``http://abc/#@def``, that is, the host
is ``abc``, the path is ``/``, and the fragment is ``#@def``.
(cherry picked from commit 90e01e50ef)
The Windows build now depends on Python 3.6 to fetch externals, but it will be downloaded via NuGet (which is downloaded via PowerShell) if it is not available via `py -3.6`. This means the only thing that must be installed on a modern Windows box to do a full build of CPython with all extensions is Visual Studio.
Also fixes an outdated note about _lzma in PCbuild/readme.txt
(cherry-picked from commit 51599e2bdd)
unittest.TestCase.assertRaises() now manually breaks a
reference cycle to not keep objects alive longer than expected.
(cherry picked from commit bbd3cf8f1e)
the original logic was just comparing the network address
but this is wrong because if the network address is equal then
we need to compare the ip address for breaking the tie
add more ip_interface comparison tests.
(cherry picked from commit 7bd8d3e794)
* bpo-29591: Upgrade Modules/expat to libexpat 2.2
* bpo-29591: Restore Python changes on expat
* bpo-29591: Remove expat config of unsupported platforms
Remove the configuration (Modules/expat/*config.h) of unsupported
platforms:
* Amiga
* MacOS Classic on PPC32
* Open Watcom
* bpo-29591: Remove useless XML_HAS_SET_HASH_SALT
The XML_HAS_SET_HASH_SALT define of Modules/expat/expat.h became
useless since our local expat copy was upgrade to expat 2.1 (it's now
expat 2.2.0).
(cherry picked from commit 23ec4b57e1)
* bpo-24484: Avoid race condition in multiprocessing cleanup
The finalizer registry can be mutated while inspected by multiprocessing
at process exit.
* Use test.support.start_threads()
* Add Misc/NEWS.
(cherry picked from commit 1eb6c0074d)
* Move co_extra_freefuncs to interpreter state to avoid crashes in
multi-threaded scenarios involving deletion of code objects
* Don't require that extra be zero initialized
* Build test list instead of defining empty test class
* Ensure extra is always assigned on success
* Keep the old fields in the thread state object, just don't use them
Add new linked list of code extra objects on a per-interpreter basis
so that interpreter state size isn't changed
* Rename __PyCodeExtraState_Get and add comment about it going away in 3.7
Fix sort order of import's in test_code.py
* Remove an extraneous space
* Remove docstrings for comments
* Touch up formatting
* Fix casing of coextra local
* Fix casing of another variable
* Prefix PyCodeExtraState with __ to match C API for getting it
* Update NEWS file for bpo-30604
This PR contains two updates to typing module:
- Support ContextManager on all versions (original PR by Jelle Zijlstra).
- Add generic AsyncContextManager..
(cherry picked from commit 29fda8db16)
* bpo-29406: asyncio SSL contexts leak sockets after calling close with certain servers (#409)
(cherry picked from commit a608d2d5a7)
* [3.6] bpo-29406: asyncio SSL contexts leak sockets after calling close with certain servers (GH-409)
* asyncio SSL contexts leak sockets after calling close with certain servers
* cleanup _shutdown_timeout_handle on _fatal_error.
(cherry picked from commit a608d2d5a7)
contextlib.AbstractContextManager now supports anti-registration
by setting __enter__ = None or __exit__ = None, following the pattern
introduced in bpo-25958..
(cherry picked from commit 57161aac5e)
If we have a chain of generators/coroutines that are 'yield from'ing
each other, then resuming the stack works like:
- call send() on the outermost generator
- this enters _PyEval_EvalFrameDefault, which re-executes the
YIELD_FROM opcode
- which calls send() on the next generator
- which enters _PyEval_EvalFrameDefault, which re-executes the
YIELD_FROM opcode
- ...etc.
However, every time we enter _PyEval_EvalFrameDefault, the first thing
we do is to check for pending signals, and if there are any then we
run the signal handler. And if it raises an exception, then we
immediately propagate that exception *instead* of starting to execute
bytecode. This means that e.g. a SIGINT at the wrong moment can "break
the chain" – it can be raised in the middle of our yield from chain,
with the bottom part of the stack abandoned for the garbage collector.
The fix is pretty simple: there's already a special case in
_PyEval_EvalFrameEx where it skips running signal handlers if the next
opcode is SETUP_FINALLY. (I don't see how this accomplishes anything
useful, but that's another story.) If we extend this check to also
skip running signal handlers when the next opcode is YIELD_FROM, then
that closes the hole – now the exception can only be raised at the
innermost stack frame.
This shouldn't have any performance implications, because the opcode
check happens inside the "slow path" after we've already determined
that there's a pending signal or something similar for us to process;
the vast majority of the time this isn't true and the new check
doesn't run at all..
(cherry picked from commit ab4413a7e9)
On Windows, subprocess.Popen.communicate() now also ignore EINVAL
on stdin.write() if the child process is still running but closed the
pipe.
(cherry picked from commit d52aa31378)
* Fix bpo-30584
* Adding a comment mentionning the bpo and explaining what is the identifier
* Add Denis Osipov to Misc/ACKS
(cherry picked from commit 897bba7563)
At the time when an abstract base class' __init_subclass__ runs,
ABCMeta.__new__ has not yet finished running, so in the presence of
__init_subclass__, inspect.isabstract() can no longer depend only on
TPFLAGS_IS_ABSTRACT.
(cherry picked from commit fcfe80ec25)
Many metaclasses in the standard library don't play nice with
__init_subclass__. This bug makes ABCMeta in particular with
__init_subclass__, which is an 80/20 solution for me personally.
AFAICT, a general solution to this problem requires updating all
metaclasses in the standard library to make sure they pass **kwargs to
type.__new__, whereas this PR only fixes ABCMeta. For context, see
https://bugs.python.org/issue29581.
* added a test combining ABCMeta and __init_subclass__
* Added NEWS item
(cherry picked from commit bd583ef985)
* [3.6] bpo-29581: Make ABCMeta.__new__ pass **kwargs to type.__new__ (GH-527)
Many metaclasses in the standard library don't play nice with
__init_subclass__. This bug makes ABCMeta in particular with
__init_subclass__, which is an 80/20 solution for me personally.
AFAICT, a general solution to this problem requires updating all
metaclasses in the standard library to make sure they pass **kwargs to
type.__new__, whereas this PR only fixes ABCMeta. For context, see
https://bugs.python.org/issue29581.
* added a test combining ABCMeta and __init_subclass__
* Added NEWS item.
(cherry picked from commit bd583ef985)
* **kwargs -> ``kwargs`` in attempts to fix the Travis build.
* Quote the **kwargs
* bpo-30557: faulthandler now correctly filters and displays exception codes on Windows (#1924)
* bpo-30557: faulthandler now correctly filters and displays exception codes on Windows
* Adds test for non-fatal exceptions.
* Adds bpo number to comment.
* bpo-30557: Fix test_faulthandler (#1969)
On Windows 8, 8.1 and 10 at least, the exit code is the exception
code (no bit is cleared).
The patch for bpo-30052 changed the preferred link target
for :func:`bytes` and :func`bytearray` references to be the
respective type definitions rather than the corresponding
builtin function entries.
This patch changes the daily documentation builds to disable
the output caching in Sphinx, in order to ensure that
cross-reference changes like this one are reliably picked
up and applied automatically after merging.
(cherry picked from commit 7a82f9c2b9)
* bpo-30414: multiprocesing.Queue._feed do not break from main loop on exc
Queue background running thread was not handling exceptions correctly.
Any exception occurred inside thread (putting unpickable object) cause
feeder to finish running. After that every message put into queue is
silently ignored.
* bpo-30414: multiprocesing.Queue._feed do not break from main loop on exc
Queue background running thread was not handling exceptions correctly.
Any exception occurred inside thread (putting unpickable object) cause
feeder to finish running. After that every message put into queue is
silently ignored.
(cherry picked from commit bc50f03db4)
test_thread: setUp() now uses support.threading_setup() and
support.threading_cleanup() to wait until threads complete to avoid
random side effects on following tests.
Co-Authored-By: Grzegorz Grzywacz <grzegorz.grzywacz@nazwa.pl>
(cherry picked from commit 79ef7f8e88)
Catch the Windows socket WSAEINVAL error (code 10022) in imaplib and
poplib on shutdown(SHUT_RDWR): An invalid operation was attempted
This error occurs sometimes on SSL connections.
(cherry picked from commit 83a2c28798)
* Use explicit numbering for footnotes referred by explicit number.
* Restore missed footnote reference in stdtypes.rst.
* Fix literal strings formatting in howto/urllib2.rst.
* Update susp-ignored.csv for zipapp.rst.
* Fix suspicious mark up in Misc/NEWS..
(cherry picked from commit d97b7dc94b)
Warnings emitted when compile a regular expression now always point
to the line in the user code. Previously they could point into inners
of the re module if emitted from inside of groups or conditionals..
(cherry picked from commit c7ac7280c3)
* python-gdb.py supports method-wrapper
bpo-29367: python-gdb.py now supports also method-wrapper (wrapperobject)
objects.
(cherry picked from commit 611083331d)
* Update and enhance python-gdb.py
bpo-29259: Detect PyCFunction is the current frame, not only in the
older frame.
when there are no more `await` or `yield (from)` before return in coroutine,
cancel was ignored.
example:
async def coro():
asyncio.Task.current_task().cancel()
return 42
...
res = await coro() # should raise CancelledError
(cherry picked from commit 991adca012)
* bpo-28787: Fix out of tree --with-dtrace builds
* Unsilence directory creation
* Add Misc/NEWS and Misc/ACKS entries.
(cherry picked from commit f6eae5bf1c)
Now allowed several subsequential inline modifiers at the start of the
pattern (e.g. '(?i)(?s)...'). In verbose mode whitespaces and comments
now are allowed before and between inline modifiers (e.g.
'(?x) (?i) (?s)...')..
(cherry picked from commit 305ccbe27e)
* bpo-29243: Fix Makefile with respect to --enable-optimizations (#1478)
* bpo-29243: Fix Makefile with respect to --enable-optimizations
When using the Profile Guided Optimization (./configure --enable-optimizations)
Python is built not only during `make` but rebuilt again during `make test`,
`make install` and others. This patch fixes the issue.
Note that this fix produces no change at all in the Makefile if configure is
run witout --enable-optimizations.
* !squash
(cherry picked from commit a1054c3b00)
* [3.6] bpo-29243: Fix Makefile with respect to --enable-optimizations (GH-1478)
* bpo-29243: Fix Makefile with respect to --enable-optimizations
When using the Profile Guided Optimization (./configure --enable-optimizations)
Python is built not only during `make` but rebuilt again during `make test`,
`make install` and others. This patch fixes the issue.
Note that this fix produces no change at all in the Makefile if configure is
run witout --enable-optimizations.
* !squash.
(cherry picked from commit a1054c3b00)
It was possible to get a core dump by using uninitialized
_json objects. Now __new__ methods create initialized objects.
__init__ methods are removed..
(cherry picked from commit 76a3e51a40)
* bpo-23404: make touch becomes make regen-all (#1405)
Don't rebuild generated files based on file modification time
anymore, the action is now explicit. Replace "make touch"
with "make regen-all".
Changes:
* Remove "make touch", Tools/hg/hgtouch.py and .hgtouch
* Add a new "make regen-all" command to rebuild all generated files
* Add subcommands to only generate specific files:
- regen-ast: Include/Python-ast.h and Python/Python-ast.c
- regen-grammar: Include/graminit.h and Python/graminit.c
- regen-importlib: Python/importlib_external.h and Python/importlib.h
- regen-opcode: Include/opcode.h
- regen-opcode-targets: Python/opcode_targets.h
- regen-typeslots: Objects/typeslots.inc
* Rename PYTHON_FOR_GEN to PYTHON_FOR_REGEN
* pgen is now only built by by "make regen-grammar"
* Add $(srcdir)/ prefix to paths to source files to handle correctly
compilation outside the source directory
Note: $(PYTHON_FOR_REGEN) is no more used nor needed by "make"
default target building Python.
(cherry picked from commit a5c62a8e9f)
* bpo-30273: Update sysconfig (#1464)
The AST_H_DIR variable was removed from Makefile.pre.in by the commit
a5c62a8e9f (bpo-23404).
AST_H_DIR was hardcoded to "Include", so replace the removed variable
by its content.
Remove also ASDLGEN variable from sysconfig example since this
variable was also removed.
(cherry picked from commit b109a1d336)
They now work when delete replaced attribute or item inside the with
statement. The old value of the attribute or item (or None if it doesn't
exist) now will be assigned to the target of the "as" clause, if there is
one.
(cherry picked from commit d1a1def7bf)
contextlib._GeneratorContextManager.__exit__ includes a special case to deal with
PEP 479 RuntimeErrors created when `StopIteration` is thrown into the context
manager body.
Previously this check was too permissive, and undid one level of chaining on *all*
RuntimeError instances, not just those that wrapped a StopIteration instance.
(cherry picked from commit 00c75e9a45)
The reference to administrative data was confusing to readers,
so this simplifies the note to explain that deep copying may copy
more then you intended, such as data that you expected to be
shared between copies.
(cherry picked from commit 19e0494256)
* bpo-29942: Fix the use of recursion in itertools.chain.from_iterable.
Fix the use of recursion in itertools.chain.from_iterable. Using recursion
is unnecessary, and can easily cause stack overflows, especially when
building in low optimization modes or with Py_DEBUG enabled.
(cherry picked from commit 5466d4af5f)
The presence of the ``--enable-optimizations`` flag is indicated by the
value of ``$enableval``, but the configure script was checking ``$withval``,
resulting in the ``--enable-optimizations`` flag being effectively ignored.
(cherry picked from commit 8cea5929f5)
* bpo-29861: release references to multiprocessing Pool tasks (#743)
* bpo-29861: release references to multiprocessing Pool tasks
Release references to tasks, their arguments and their results as soon
as they are finished, instead of keeping them alive until another task
arrives.
* Comments in test
(cherry picked from commit 8988945cdc)
* Fix Misc/NEWS (hopefully)
Directory and zipfile execution previously added
the parent directory of the directory or zipfile
as sys.path[0] and then subsequently overwrote
it with the directory or zipfile itself.
This caused problems in isolated mode, as it
overwrote the "stdlib as a zip archive" entry
in sys.path, as the parent directory was
never added.
The attempted fix to that issue in bpo-29319
created the opposite problem in *non*-isolated
mode, by potentially leaving the parent
directory on sys.path instead of overwriting it.
This change fixes the root cause of the problem
by removing the whole "add-and-overwrite" dance
for sys.path[0], and instead simply never adds
the parent directory to sys.path in the first
place.
(cherry picked from commit d2977a3ae2)
(cherry picked from commit c60948464f)
Directory and zipfile execution previously added
the parent directory of the directory or zipfile
as sys.path[0] and then subsequently overwrote
it with the directory or zipfile itself.
This caused problems in isolated mode, as it
overwrote the "stdlib as a zip archive" entry
in sys.path, as the parent directory was
never added.
The attempted fix to that issue in bpo-29319
created the opposite problem in *non*-isolated
mode, by potentially leaving the parent
directory on sys.path instead of overwriting it.
This change fixes the root cause of the problem
by removing the whole "add-and-overwrite" dance
for sys.path[0], and instead simply never adds
the parent directory to sys.path in the first
place.
(cherry picked from commit d2977a3ae2)
Skip some tests of select.poll when running on macOS due to unresolved
issues with the underlying system poll function on some macOS versions.
(cherry picked from commit de04644627)
* bpo-27593: Get SCM build info from git instead of hg. (#446)
sys.version and the platform module python_build(),
python_branch(), and python_revision() functions now use
git information rather than hg when building from a repo.
Based on original patches by Brett Cannon and Steve Dower.
(cherry picked from commit 5c4b0d063a)
Serhiy Storchaka
terryjreedy
Victor Stinner
Nir Soffer
Ned Deily
Joel Hillacre
Yury Selivanov
Masayuki Yamamoto
Nick Coghlan
Antoine Pitrou
Emily Morehouse
Dong-hee Na
Brett Cannon
larryhastings
Steve Dower
Mariatta
Zachary Ware
Dino Viehland
Antoine Pietri
Denis Osipov
Nate
Xiang Zhang
csabella
T. Wouters
Łukasz Langa
INADA Naoki
stratakis
torsava
cocoatomo
Christophe Zeitouny
Berker Peksag
Michael Seifert
Yury Selivanov
orenmn
Benjamin Peterson
Petr Motejlek