Commit Graph

19697 Commits

Author SHA1 Message Date
Mariatta 292b421d48 bpo-28837: Fix lib2to3 handling of map/zip/filter calls when followed with a 'trailer', e.g. zip()[x] (GH-24) (GH-2235)
(cherry picked from commit 93b4b47e3a)
2017-06-15 19:56:52 -07:00
Victor Stinner 50dbf577e1 bpo-23890: Fix ref cycle in TestCase.assertRaises (#858)
unittest.TestCase.assertRaises() now manually breaks a
reference cycle to not keep objects alive longer than expected.
(cherry picked from commit bbd3cf8f1e)
2017-06-16 00:18:15 +02:00
Dong-hee Na e45ea377b8 bpo-30149: Fix partialmethod without explicit self parameter (#1308) (#1662) 2017-06-15 17:41:57 +03:00
Serhiy Storchaka cf58dfb44c [3.6] bpo-29931 fix __lt__ check in ipaddress.ip_interface for both v4 and v6. (GH-879) (#2217)
the original logic was just comparing the network address
but this is wrong because if the network address is equal then
we need to compare the ip address for breaking the tie

add more ip_interface comparison tests.
(cherry picked from commit 7bd8d3e794)
2017-06-15 17:16:38 +03:00
Serhiy Storchaka 523a243840 [3.6] bpo-30605: Fix compiling binary regexs with BytesWarnings enabled. (GH-2016) (#2214)
Running our unit tests with `-bb` enabled triggered this failure..
(cherry picked from commit 171b9a354e)
2017-06-15 16:55:22 +03:00
Victor Stinner 86b95370c4 bpo-29591: Upgrade Modules/expat to libexpat 2.2 (#2164) (#2200)
* bpo-29591: Upgrade Modules/expat to libexpat 2.2

* bpo-29591: Restore Python changes on expat

* bpo-29591: Remove expat config of unsupported platforms

Remove the configuration (Modules/expat/*config.h) of unsupported
platforms:

* Amiga
* MacOS Classic on PPC32
* Open Watcom

* bpo-29591: Remove useless XML_HAS_SET_HASH_SALT

The XML_HAS_SET_HASH_SALT define of Modules/expat/expat.h became
useless since our local expat copy was upgrade to expat 2.1 (it's now
expat 2.2.0).

(cherry picked from commit 23ec4b57e1)
2017-06-15 01:26:16 +02:00
Antoine Pitrou 6fd0345995 [3.6] bpo-24484: Avoid race condition in multiprocessing cleanup (GH-2159) (#2166)
* bpo-24484: Avoid race condition in multiprocessing cleanup

The finalizer registry can be mutated while inspected by multiprocessing
at process exit.

* Use test.support.start_threads()

* Add Misc/NEWS.
(cherry picked from commit 1eb6c0074d)
2017-06-13 17:51:26 +02:00
Dino Viehland 2997fec01e [3.6] bpo-30604: Move co_extra_freefuncs to interpreter state to avoid crashes in threads (#2015)
* Move co_extra_freefuncs to interpreter state to avoid crashes in
multi-threaded scenarios involving deletion of code objects

* Don't require that extra be zero initialized

* Build test list instead of defining empty test class

* Ensure extra is always assigned on success

* Keep the old fields in the thread state object, just don't use them
Add new linked list of code extra objects on a per-interpreter basis
  so that interpreter state size isn't changed

* Rename __PyCodeExtraState_Get and add comment about it going away in 3.7
Fix sort order of import's in test_code.py

* Remove an extraneous space

* Remove docstrings for comments

* Touch up formatting

* Fix casing of coextra local

* Fix casing of another variable

* Prefix PyCodeExtraState with __ to match C API for getting it

* Update NEWS file for bpo-30604
2017-06-12 21:46:35 -04:00
Antoine Pietri 8399a177de [3.6] bpo-30177: add NEWS entry (#2134) 2017-06-12 12:23:24 -04:00
Serhiy Storchaka d89dc844d2 [3.6] bpo-28994: Fixed errors handling in atexit._run_exitfuncs(). (GH-2034) (#2121)
The traceback no longer displayed for SystemExit raised in a callback registered by atexit..
(cherry picked from commit 3fd54d4a7e)
2017-06-12 09:02:13 +03:00
Yury Selivanov 83d30bd667 Revert "[3.6] bpo-29406: asyncio SSL contexts leak sockets after calling close with certain servers (GH-409) (#2062)" (#2112)
This reverts commit 6e14fd2a14.
2017-06-11 16:46:45 +02:00
Yury Selivanov 176f2ebdad bpo-30508: Don't log exceptions if Task/Future "cancel()" method was called. (#2109) 2017-06-11 14:00:14 +00:00
terryjreedy 82e606785e [3.6] Add IDLE news items (GH-2090) (#2096)
(cherry-pick from c94c41c)
2017-06-10 23:15:08 -04:00
Mariatta 8d15b19be6 bpo-28556: Updates to typing module (GH-2076) (GH-2087)
This PR contains two updates to typing module:

- Support ContextManager on all versions (original PR by Jelle Zijlstra).
- Add generic AsyncContextManager..
(cherry picked from commit 29fda8db16)
2017-06-10 14:41:00 -07:00
Yury Selivanov 6e14fd2a14 [3.6] bpo-29406: asyncio SSL contexts leak sockets after calling close with certain servers (GH-409) (#2062)
* bpo-29406: asyncio SSL contexts leak sockets after calling close with certain servers (#409)

(cherry picked from commit a608d2d5a7)

* [3.6] bpo-29406: asyncio SSL contexts leak sockets after calling close with certain servers (GH-409)

* asyncio SSL contexts leak sockets after calling close with certain servers

* cleanup _shutdown_timeout_handle on _fatal_error.
(cherry picked from commit a608d2d5a7)
2017-06-10 10:00:45 -04:00
Mariatta 753422f6e3 bpo-30266: support "= None" pattern in AbstractContextManager (GH-1448) (GH-2054)
contextlib.AbstractContextManager now supports anti-registration
by setting __enter__ = None or __exit__ = None, following the pattern
introduced in bpo-25958..
(cherry picked from commit 57161aac5e)
2017-06-09 20:36:28 -07:00
Yury Selivanov fe9c7a0fd3 Break circular references when closing SSLTransport objects (#981) (#2049) 2017-06-09 19:14:35 -04:00
Yury Selivanov 7a16a4535d Closing transport during handshake process leaks socket (#480) (#2044) 2017-06-09 18:33:31 -04:00
Yury Selivanov fa7f519113 Fix waiter cancellation in asyncio.Lock (#1031) (#2037)
Avoid a deadlock when the waiter who is about to take the lock is
cancelled

Issue #27585
2017-06-09 17:07:48 -04:00
Yury Selivanov e89f95bfd0 [3.6] bpo-30039: Don't run signal handlers while resuming a yield from stack (GH-1081) (#1640)
If we have a chain of generators/coroutines that are 'yield from'ing
each other, then resuming the stack works like:

- call send() on the outermost generator
- this enters _PyEval_EvalFrameDefault, which re-executes the
  YIELD_FROM opcode
- which calls send() on the next generator
- which enters _PyEval_EvalFrameDefault, which re-executes the
  YIELD_FROM opcode
- ...etc.

However, every time we enter _PyEval_EvalFrameDefault, the first thing
we do is to check for pending signals, and if there are any then we
run the signal handler. And if it raises an exception, then we
immediately propagate that exception *instead* of starting to execute
bytecode. This means that e.g. a SIGINT at the wrong moment can "break
the chain" – it can be raised in the middle of our yield from chain,
with the bottom part of the stack abandoned for the garbage collector.

The fix is pretty simple: there's already a special case in
_PyEval_EvalFrameEx where it skips running signal handlers if the next
opcode is SETUP_FINALLY. (I don't see how this accomplishes anything
useful, but that's another story.) If we extend this check to also
skip running signal handlers when the next opcode is YIELD_FROM, then
that closes the hole – now the exception can only be raised at the
innermost stack frame.

This shouldn't have any performance implications, because the opcode
check happens inside the "slow path" after we've already determined
that there's a pending signal or something similar for us to process;
the vast majority of the time this isn't true and the new check
doesn't run at all..
(cherry picked from commit ab4413a7e9)
2017-06-09 17:06:39 -04:00
Victor Stinner b319d09ee4 bpo-30418: Popen.communicate() always ignore EINVAL (#2002) (#2004)
On Windows, subprocess.Popen.communicate() now also ignore EINVAL
on stdin.write() if the child process is still running but closed the
pipe.
(cherry picked from commit d52aa31378)
2017-06-08 23:14:07 +02:00
Denis Osipov ca1b66fd05 [3.6] bpo-30584: Fix test_os fails on non-English Windows (GH-1980) (#1999)
* Fix bpo-30584

* Adding a comment mentionning the bpo and explaining what is the identifier

* Add Denis Osipov to Misc/ACKS
(cherry picked from commit 897bba7563)
2017-06-08 14:02:05 +02:00
Antoine Pietri ceabf9acf0 bpo-30177: pathlib: include the full path in resolve(strict=False) (#1893) (#1985) 2017-06-07 10:18:56 -07:00
Nate 09b6c0c71e [3.6] bpo-29822: make inspect.isabstract() work during __init_subclass__ (#1979)
At the time when an abstract base class' __init_subclass__ runs,
ABCMeta.__new__ has not yet finished running, so in the presence of
__init_subclass__, inspect.isabstract() can no longer depend only on
TPFLAGS_IS_ABSTRACT.
(cherry picked from commit fcfe80ec25)
2017-06-07 07:21:34 +03:00
Nate 6fb12b5c43 bpo-29581: bpo-29581: Make ABCMeta.__new__ pass **kwargs to type.__new__ (GH-527) (GH-1282)
Many metaclasses in the standard library don't play nice with
__init_subclass__. This bug makes ABCMeta in particular with
__init_subclass__, which is an 80/20 solution for me personally.
AFAICT, a general solution to this problem requires updating all
metaclasses in the standard library to make sure they pass **kwargs to
type.__new__, whereas this PR only fixes ABCMeta. For context, see
https://bugs.python.org/issue29581.

* added a test combining ABCMeta and __init_subclass__
* Added NEWS item

(cherry picked from commit bd583ef985)

* [3.6] bpo-29581: Make ABCMeta.__new__ pass **kwargs to type.__new__ (GH-527)

Many metaclasses in the standard library don't play nice with
__init_subclass__. This bug makes ABCMeta in particular with
__init_subclass__, which is an 80/20 solution for me personally.
AFAICT, a general solution to this problem requires updating all
metaclasses in the standard library to make sure they pass **kwargs to
type.__new__, whereas this PR only fixes ABCMeta. For context, see
https://bugs.python.org/issue29581.

* added a test combining ABCMeta and __init_subclass__
* Added NEWS item.
(cherry picked from commit bd583ef985)

* **kwargs -> ``kwargs`` in attempts to fix the Travis build.

* Quote the **kwargs
2017-06-06 17:31:03 -07:00
Steve Dower 2bafc0dcca [3.6] bpo-30557: faulthandler now correctly filters and displays exception … (#1960)
* bpo-30557: faulthandler now correctly filters and displays exception codes on Windows (#1924)

* bpo-30557: faulthandler now correctly filters and displays exception codes on Windows

* Adds test for non-fatal exceptions.

* Adds bpo number to comment.

* bpo-30557: Fix test_faulthandler (#1969)

On Windows 8, 8.1 and 10 at least, the exit code is the exception
code (no bit is cleared).
2017-06-06 13:47:14 -07:00
Nick Coghlan 798cfb2123 bpo-30052: Always regenerate cross-references (GH-1339) (GH-1921)
The patch for bpo-30052 changed the preferred link target
for :func:`bytes` and :func`bytearray` references to be the
respective type definitions rather than the corresponding
builtin function entries.

This patch changes the daily documentation builds to disable
the output caching in Sphinx, in order to ensure that
cross-reference changes like this one are reliably picked
up and applied automatically after merging.
(cherry picked from commit 7a82f9c2b9)
2017-06-05 19:19:37 -07:00
Xiang Zhang 95b4da2be4 bpo-30378: Fix the problem that SysLogHandler can't handle IPv6 addresses (#1676) (#1903) 2017-06-01 22:20:27 +08:00
csabella 00b381b9a7 bpo-17188: DOC: Document 'from None' in raise statement (GH-1671) (GH-1867)
Original patch by Dennis Mårtensson.
(cherry picked from commit 763557eac0)
2017-05-30 13:27:39 -07:00
Mariatta 94d8261d1c [3.6] bpo-29960 _random.Random corrupted on exception in setstate(). … (#1287)
(cherry picked from commit 9616a82e78)
2017-05-27 07:20:24 -07:00
Antoine Pitrou 2783cc4262 [3.6] bpo-30414: multiprocessing.Queue._feed do not break from main loop on exc (GH-1683) (#1815)
* bpo-30414: multiprocesing.Queue._feed do not break from main loop on exc

Queue background running thread was not handling exceptions correctly.
Any exception occurred inside thread (putting unpickable object) cause
feeder to finish running. After that every message put into queue is
silently ignored.

* bpo-30414: multiprocesing.Queue._feed do not break from main loop on exc

Queue background running thread was not handling exceptions correctly.
Any exception occurred inside thread (putting unpickable object) cause
feeder to finish running. After that every message put into queue is
silently ignored.
(cherry picked from commit bc50f03db4)
2017-05-25 16:57:46 +02:00
Serhiy Storchaka f43b293f2f [3.6] bpo-29943: Do not replace the function PySlice_GetIndicesEx() with a macro (GH-1049) (#1813)
if Py_LIMITED_API is not defined.
(cherry picked from commit 49a9059)
2017-05-25 15:32:08 +03:00
Serhiy Storchaka 89a3102640 [3.6] bpo-29104: Fixed parsing backslashes in f-strings. (GH-490) (#1812)
(cherry picked from commit 0cd7a3f)
2017-05-25 14:18:55 +03:00
T. Wouters 07a40a1623 [3.6] Add --with-assertions configure flag to enable C assertions(GH-1731) (#1739)
Defaults to 'no', but as before assertions are implied by --with-pydebug..
(cherry picked from commit ddbfa2c35b)
2017-05-22 22:25:17 -07:00
Łukasz Langa d29feccec3 [3.6] bpo-30395 _PyGILState_Reinit deadlock fix (GH-1734) (#1740)
head_lock could be held by another thread when fork happened. We should
reset it to avoid deadlock.
(cherry picked from commit f82c951d1c)
2017-05-22 22:23:05 -07:00
Xiang Zhang 54af41d42e bpo-30003: Fix handling escape characters in HZ codec (#1556) (#1719) 2017-05-23 01:03:00 +08:00
Serhiy Storchaka 564398af6c [3.6] bpo-27945: Fixed various segfaults with dict. (GH-1657) (#1677)
Based on patches by Duane Griffin and Tim Mitchell.
(cherry picked from commit 753bca3934)
2017-05-20 13:06:26 +03:00
Serhiy Storchaka 193f7e094f [3.6] bpo-25794: Fix `type.__setattr__()` for non-interned attribute names. (GH-1652) (#1673)
Based on patch by Eryk Sun.
(cherry picked from commit d896985bb2)
2017-05-20 10:23:31 +03:00
Xiang Zhang 43d4c0329e bpo-30301: Fix AttributeError when using SimpleQueue.empty() (#1601) (#1628)
Under *spawn* and *forkserver* start methods, SimpleQueue.empty() could
raise AttributeError due to not setting _poll in __setstate__.
2017-05-17 22:03:35 +08:00
Victor Stinner 6b5b85aecf bpo-30357: test_thread now uses threading_cleanup() (#1592) (#1622)
test_thread: setUp() now uses support.threading_setup() and
support.threading_cleanup() to wait until threads complete to avoid
random side effects on following tests.

Co-Authored-By:  Grzegorz Grzywacz <grzegorz.grzywacz@nazwa.pl>
(cherry picked from commit 79ef7f8e88)
2017-05-16 17:06:14 -07:00
Victor Stinner aaa0536525 bpo-30329: Catch Windows error 10022 on shutdown() (#1538) (#1620)
Catch the Windows socket WSAEINVAL error (code 10022) in imaplib and
poplib on shutdown(SHUT_RDWR): An invalid operation was attempted

This error occurs sometimes on SSL connections.
(cherry picked from commit 83a2c28798)
2017-05-16 15:29:41 -07:00
Serhiy Storchaka 8fc1ff51ad [3.6] bpo-30380: Fix Sphinx 1.6.1 warnings. (GH-1613) (#1614)
* Use explicit numbering for footnotes referred by explicit number.
* Restore missed footnote reference in stdtypes.rst.
* Fix literal strings formatting in howto/urllib2.rst.
* Update susp-ignored.csv for zipapp.rst.
* Fix suspicious mark up in Misc/NEWS..
(cherry picked from commit d97b7dc94b)
2017-05-17 00:09:23 +03:00
Serhiy Storchaka 73fb45df04 [3.6] bpo-30375: Correct the stacklevel of regex compiling warnings. (GH-1595) (#1604)
Warnings emitted when compile a regular expression now always point
to the line in the user code.  Previously they could point into inners
of the re module if emitted from inside of groups or conditionals..
(cherry picked from commit c7ac7280c3)
2017-05-16 18:16:15 +03:00
Victor Stinner d05f7fdf6c [3.6] bpo-30345: Update test_gdb.py and python-gdb.py from master (#1549)
* python-gdb.py supports method-wrapper

bpo-29367: python-gdb.py now supports also method-wrapper (wrapperobject)
objects.

(cherry picked from commit 611083331d)

* Update and enhance python-gdb.py

bpo-29259: Detect PyCFunction is the current frame, not only in the
older frame.
2017-05-13 00:21:50 +02:00
INADA Naoki 3dc7c52a9f bpo-30048: asyncio: fix Task.cancel() was ignored. (GH-1546)
when there are no more `await` or `yield (from)` before return in coroutine,
cancel was ignored.

example:

    async def coro():
        asyncio.Task.current_task().cancel()
        return 42
    ...
    res = await coro()  # should raise CancelledError

(cherry picked from commit 991adca012)
2017-05-11 21:56:42 +09:00
stratakis a4465a5bd0 [3.6] bpo-28787: Fix out of tree --with-dtrace builds (GH-135) (#1543)
* bpo-28787: Fix out of tree --with-dtrace builds

* Unsilence directory creation

* Add Misc/NEWS and Misc/ACKS entries.
(cherry picked from commit f6eae5bf1c)
2017-05-11 13:13:18 +02:00
Serhiy Storchaka 418d60a525 [3.6] bpo-30298: Weaken the condition of deprecation warnings for inline modifiers. (GH-1490) (#1525)
Now allowed several subsequential inline modifiers at the start of the
pattern (e.g. '(?i)(?s)...').  In verbose mode whitespaces and comments
now are allowed before and between inline modifiers (e.g.
'(?x) (?i) (?s)...')..
(cherry picked from commit 305ccbe27e)
2017-05-10 06:44:02 +03:00
torsava 03b8a378df [3.6] bpo-29243: Fix Makefile with respect to --enable-optimizations (GH-1478) (#1518)
* bpo-29243: Fix Makefile with respect to --enable-optimizations (#1478)

* bpo-29243: Fix Makefile with respect to --enable-optimizations

When using the Profile Guided Optimization (./configure --enable-optimizations)
Python is built not only during `make` but rebuilt again during `make test`,
`make install` and others. This patch fixes the issue.

Note that this fix produces no change at all in the Makefile if configure is
run witout --enable-optimizations.

* !squash

(cherry picked from commit a1054c3b00)

* [3.6] bpo-29243: Fix Makefile with respect to --enable-optimizations (GH-1478)

* bpo-29243: Fix Makefile with respect to --enable-optimizations

When using the Profile Guided Optimization (./configure --enable-optimizations)
Python is built not only during `make` but rebuilt again during `make test`,
`make install` and others. This patch fixes the issue.

Note that this fix produces no change at all in the Makefile if configure is
run witout --enable-optimizations.

* !squash.
(cherry picked from commit a1054c3b00)
2017-05-09 16:05:10 +02:00
Xiang Zhang 72e1b61da0 bpo-29990: Fix range checking in GB18030 decoder (#1495) (#1507)
When decoding a 4-byte GB18030 sequence, the first and third byte cannot exceed 0xFE.
2017-05-09 12:16:50 +08:00
Serhiy Storchaka 70dc6a7a0b [3.6] Revert bpo-26293 for zipfile breakage. See also bpo-29094. (GH-1484). (#1485)
(cherry picked from commit 3763ea865c)
2017-05-06 15:10:50 +03:00