Commit Graph

5719 Commits

Author SHA1 Message Date
Miss Islington (bot) 820219f786
bpo-33509: Fix _warnings for module_globals=None (GH-6833)
Don't crash on warnings.warn_explicit() if module_globals is not a dict.
(cherry picked from commit b056562860)

Co-authored-by: Victor Stinner <vstinner@redhat.com>
2018-05-15 13:56:28 -07:00
Eric Snow 6bd0c476c5
bpo-32604: Remove xid registry. (#6813)
Remove the interpreters testing helper (and xid registry).
2018-05-15 09:56:18 -04:00
Miss Islington (bot) 8ed545f6de
bpo-28055: Fix unaligned accesses in siphash24(). (GH-6123)
The hash implementation casts the input pointer to uint64_t* and directly reads
from this, which may cause unaligned accesses. Use memcpy() instead so this code
will not crash with SIGBUS on sparc.

https://bugs.gentoo.org/show_bug.cgi?id=636400
(cherry picked from commit 1e2ec8a996)

Co-authored-by: Rolf Eike Beer <eike@sf-mail.de>
2018-05-13 04:17:07 -07:00
Zsolt Dollenstein a93a663d6c [3.7] bpo-33363: raise SyntaxError for async for/with outside async functions (GH-6616). (GH-6619) 2018-04-27 18:33:37 -04:00
Miss Islington (bot) 291c9d4f74
bpo-33330: Improve error handling in PyImport_Cleanup(). (GH-6564)
(cherry picked from commit e9d9494d6b)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2018-04-25 11:32:52 -07:00
Miss Islington (bot) 52a5a17338
bpo-33128 Fix duplicated call to importlib._install_external_importers (GH-6273)
External importers were being added in both phases of the import
system initialisation.

They're only supposed to be added in the second phase, after the
import machinery has been appropriately configured.
(cherry picked from commit 0977091dca)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2018-04-24 19:48:04 -07:00
Miss Islington (bot) 32955299b4
Spelling fixes to docs, docstrings, and comments (GH-6374)
(cherry picked from commit 61f82e0e33)

Co-authored-by: Ville Skyttä <ville.skytta@iki.fi>
2018-04-20 14:00:41 -07:00
Brett Cannon a09bb87c1e
[3.7] bpo-33169: Remove values of `None` from sys.path_importer_cache when invalidating caches (GH-6402) (GH-6403)
An entry of None in sys.path_importer_cache represents a negative/missing finder for a path, so clearing it out makes sense.
(cherry picked from commit 9e2be60634)
2018-04-06 17:02:18 -07:00
Miss Islington (bot) 3c193cf8af [3.7] bpo-29673: fix gdb scripts pystack and pystackv (GH-6126) (GH-6399)
(cherry picked from commit 3a9ccee0e5)

Co-authored-by: Marcel Plch <gmarcel.plch@gmail.com>
2018-04-06 18:14:29 -04:00
Miss Islington (bot) 64421d9237
bpo-33231: Fix potential leak in normalizestring() (GH-6386)
(cherry picked from commit 0c1c4563a6)

Co-authored-by: INADA Naoki <methane@users.noreply.github.com>
2018-04-06 00:12:38 -07:00
Miss Islington (bot) fcd4e03e08
bpo-29922: Improve error messages in 'async with' (GH-6352)
when __aenter__() or __aexit__() return non-awaitable object.
(cherry picked from commit a68f2f0578)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2018-04-04 07:09:00 -07:00
Miss Islington (bot) 9e96e7b24e
bpo-33132: Fix more reference counting issues in the compiler. (GH-6323)
(cherry picked from commit aa8e51f5eb)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2018-03-31 16:41:28 -07:00
Nick Coghlan ee3784594b
bpo-33053: -m now adds *starting* directory to sys.path (GH-6231) (#6236)
Historically, -m added the empty string as sys.path
zero, meaning it resolved imports against the current
working directory, the same way -c and the interactive
prompt do.

This changes the sys.path initialisation to add the
*starting* working directory as sys.path[0] instead,
such that changes to the working directory while the
program is running will have no effect on imports
when using the -m switch.

(cherry picked from commit d5d9e02dd3)
2018-03-25 23:43:50 +10:00
Miss Islington (bot) c6d94c37f4
bpo-33042: Fix pre-initialization sys module configuration (GH-6157)
- new test case for pre-initialization of sys.warnoptions and sys._xoptions
- restored ability to call these APIs prior to Py_Initialize
- updated the docs for the affected APIs to make it clear they can be
  called before Py_Initialize
- also enhanced the existing embedding test cases
  to check for expected settings in the sys module
(cherry picked from commit bc77eff8b9)

Co-authored-by: Nick Coghlan <ncoghlan@gmail.com>
2018-03-25 04:27:57 -07:00
Miss Islington (bot) 471364b4d9
bpo-33132: Fix reference counting issues in the compiler. (GH-6209)
(cherry picked from commit a95d98607e)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2018-03-24 14:27:06 -07:00
Serhiy Storchaka b9744e924c
bpo-33041: Fixed jumping if the function contains an "async for" loop. (GH-6154) 2018-03-23 14:35:33 +02:00
Miss Islington (bot) 56cce1ca84 [3.7] bpo-33021: Release the GIL during fstat() calls (GH-6019) (GH-6159)
fstat may block for long time if the file descriptor is on a
non-responsive NFS server, hanging all threads. Most fstat() calls are
handled by _Py_fstat(), releasing the GIL internally, but but
_Py_fstat_noraise() does not release the GIL, and most calls release the
GIL explicitly around it.

This patch fixes last 2 calls to _Py_fstat_no_raise(), avoiding hangs
when calling:
- mmap.mmap()
- os.urandom()
- random.seed()
(cherry picked from commit 4484f9dca9)

Co-authored-by: Nir Soffer <nirsof@gmail.com>
2018-03-20 20:16:30 +01:00
Miss Islington (bot) 5506d60302
bpo-32836: Remove obsolete code from symtable pass (GH-5680)
When comprehensions switched to using a nested scope, the old
code for generating a temporary name to hold the accumulation
target became redundant, but was never actually removed.

Patch by Nitish Chandra.
(cherry picked from commit 3a087beddd)

Co-authored-by: Nitish Chandra <nitishchandrachinta@gmail.com>
2018-03-10 15:11:47 -08:00
Serhiy Storchaka 9e94c0d3c7
[3.7] bpo-33041: Add missed error checks when compile "async for" (GH-6053) (GH-6060)
and remove redundant code.
(cherry picked from commit 67ee07795b)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2018-03-10 20:45:05 +02:00
Miss Islington (bot) e724dd4e5e
Add two missing error checks in hamt.c (GH-5851)
(cherry picked from commit 3c7ac7ea20)

Co-authored-by: Xiang Zhang <angwerzx@126.com>
2018-03-07 22:21:34 -08:00
Miss Islington (bot) 31e2b76f7b
bpo-33005: Fix _PyGILState_Reinit() (GH-6001)
Fix a crash on fork when using a custom memory allocator (ex: using
PYTHONMALLOC env var).

_PyGILState_Reinit() and _PyInterpreterState_Enable() now use the
default RAW memory allocator to allocate a new interpreters mutex on
fork.
(cherry picked from commit 5d92647102)

Co-authored-by: Victor Stinner <victor.stinner@gmail.com>
2018-03-06 05:52:27 -08:00
Miss Islington (bot) 8d9d4b57ae
bpo-32838: Fix Python versions in the table of magic numbers. (GH-5658)
(cherry picked from commit 4af8fd5614)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2018-02-23 11:01:54 -08:00
Miss Islington (bot) eed3c7af4e bpo-32604: Swap threads only if the interpreter is different. (gh-5783)
The CPython runtime assumes that there is a one-to-one relationship (for a given interpreter) between PyThreadState and OS threads. Sending and receiving on a channel in the same interpreter was causing crashes because of this (specifically due to a check in PyThreadState_Swap()). The solution is to not switch threads if the interpreter is the same.
(cherry picked from commit f53d9f2778)

Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
2018-02-20 17:09:41 -07:00
Miss Islington (bot) ca82e3c0ec
bpo-32869: Fix incorrect dst buffer size for MultiByteToWideChar (GH-5739)
This function expects the destination buffer size to be given
in wide characters, not bytes.
(cherry picked from commit b3b4a9d300)

Co-authored-by: Alexey Izbyshev <izbyshev@users.noreply.github.com>
2018-02-18 10:40:07 -08:00
Miss Islington (bot) 3db05a3a9c bpo-32604: Clean up created subinterpreters before runtime finalization. (gh-5710)
(cherry picked from commit 4c6955e2b0)

Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
2018-02-16 19:15:24 -07:00
Miss Islington (bot) 7df80492fc Fix some warnings produced by different compilers. (GH-5593) (GH-5600)
(cherry picked from commit bfe4fd5f2e)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2018-02-09 17:56:34 +02:00
Miss Islington (bot) 2bb0bfafb0 bpo-32777: Fix _Py_set_inheritable async-safety in subprocess (GH-5560) (GH-5562)
Fix a rare but potential pre-exec child process deadlock in subprocess on POSIX systems when marking file descriptors inheritable on exec in the child process.  This bug appears to have been introduced in 3.4 with the inheritable file descriptors support.

This also changes Python/fileutils.c `set_inheritable` to use the "slow" two `fcntl` syscall path instead of the "fast" single `ioctl` syscall path when asked to be async signal safe (by way of being asked not to raise exceptions).  `ioctl` is not a POSIX async-signal-safe approved function.

ref: http://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html
(cherry picked from commit c1e46e94de)

Co-authored-by: Alexey Izbyshev <izbyshev@users.noreply.github.com>
2018-02-05 22:31:22 -08:00
Miss Islington (bot) e86db34dd3 [3.7] bpo-32746: Fix multiple typos (GH-5144) (GH-5520)
Fix typos found by codespell in docs, docstrings, and comments.
(cherry picked from commit c3d9508ff2)

Co-authored-by: Leo Arias <leo.arias@canonical.com>
2018-02-03 20:41:43 -05:00
Miss Islington (bot) f33ecedcad bpo-32604: Fix memory leaks in the new _xxsubinterpreters module. (GH-5507)
(cherry picked from commit 4e9da0d163)

Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
2018-02-02 22:38:57 -07:00
Barry Warsaw a23d30f64b
bpo-32303 - Consistency fixes for namespace loaders (GH-5481) (#5503)
* Make sure ``__spec__.loader`` matches ``__loader__`` for namespace packages.
* Make sure ``__spec__.origin` matches ``__file__`` for namespace packages.

https://bugs.python.org/issue32303
https://bugs.python.org/issue32305
(cherry picked from commit bbbcf8693b)

Co-authored-by: Barry Warsaw <barry@python.org>
2018-02-02 19:49:25 -05:00
Miss Islington (bot) e985afc38d bpo-32674: Improve the docstring for __import__ (GH-5339) (GH-5488)
Clarify that the level argument is used to determine whether to
perform absolute or relative imports: 0 is absolute, while a positive number
is the number of parent directories to search relative to the current module.
(cherry picked from commit 461d225b19)

Co-authored-by: oldk <oldk1331@users.noreply.github.com>
2018-02-01 23:51:36 -05:00
Miss Islington (bot) 78767786a8 bpo-32436: Fix compiler warning (GH-5483) (GH-5486)
(cherry picked from commit 55e0839f26)

Co-authored-by: Yury Selivanov <yury@magic.io>
2018-02-01 22:58:19 -05:00
Miss Islington (bot) 5d2dcd044d Fix typo in hamt.c comments (GH-5478) (#5484)
(cherry picked from commit 01a0cb8916)

Co-authored-by: Dmitry Alimov <dvalimov@gmail.com>
2018-02-01 22:30:34 -05:00
Miss Islington (bot) 78758f29b1 [3.7] bpo-32711: Fix warnings for Python/ast_unparse.c (GH-5426) (#5475)
* bpo-32711: Fix warnings for Python/ast_unparse.c
(cherry picked from commit 83ab995871)

Co-authored-by: Stéphane Wirtel <stephane@wirtel.be>
2018-02-01 18:41:24 +01:00
Eric Snow 7f8bfc9b9a
bpo-32604: Expose the subinterpreters C-API in a "private" stdlib module. (gh-1748)
The module is primarily intended for internal use in the test suite.  Building the module under Windows will come in a follow-up PR.
2018-01-29 18:23:44 -07:00
Mark Shannon 332cd5ee4f bpo-32550. Remove the STORE_ANNOTATION bytecode. (GH-5181) 2018-01-29 16:41:04 -08:00
Benjamin Peterson c65ef772c3
rename _imp initialization function to follow conventions (#5432)
When the C imp module became _imp in 6f44d66bc4, the initialization function should have been renamed from PyInit_imp to PyInit__imp.
2018-01-29 11:33:57 -08:00
Yury Selivanov b647d7039d
bpo-32707: Fix warnings in hamt.c (#5430) 2018-01-29 13:31:37 -05:00
Victor Stinner 2914bb32e2
bpo-20891: Py_Initialize() now creates the GIL (#4700)
The GIL is no longer created "on demand" to fix a race condition when
PyGILState_Ensure() is called in a non-Python thread.
2018-01-29 11:57:45 +01:00
Xiang Zhang 997478eb79
bpo-32688: Make why the line is commented out clear (GH-5405) 2018-01-29 11:32:12 +08:00
Yury Selivanov bc4123b0b3
bpo-32436: Use PyThreadState_GET() in all hot paths (GH-5363) 2018-01-27 13:24:20 -05:00
Yury Selivanov 226e50049d
bpo-32436: Make PyContextVar_Get a little bit faster (#5350)
Since context.c is compiled with Py_BUILD_CORE, using a macro
will result in a slightly more optimal code.
2018-01-26 17:24:52 -05:00
Yury Selivanov 43c47fe096
bpo-32670: Enforce PEP 479. (#5327) 2018-01-26 15:24:24 -05:00
Nathaniel J. Smith dba976b8a2 bpo-32591: fix abort in _PyErr_WarnUnawaitedCoroutine during shutdown (#5337)
When an unawaited coroutine is collected very late in shutdown --
like, during the final GC at the end of PyImport_Cleanup -- then it
was triggering an interpreter abort, because we'd try to look up the
"warnings" module and not only was it missing (we were prepared for
that), but the entire module system was missing (which we were not
prepared for).

I've tried to fix this at the source, by making the utility function
get_warnings_attr robust against this in general. Note that it already
has the convention that it can return NULL without setting an error,
which is how it signals that the attribute it was asked to fetch is
missing, and that all callers already check for NULL returns.

There's a similar check for being late in shutdown at the top of
warn_explicit, which might be unnecessary after this fix, but I'm not
sure so I'm going to leave it.
2018-01-26 14:28:31 -05:00
Guido van Rossum 95e4d58913 String annotations [PEP 563] (#4390)
* Document `from __future__ import annotations`
* Provide plumbing and tests for `from __future__ import annotations`
* Implement unparsing the AST back to string form

This is required for PEP 563 and as such only implements a part of the
unparsing process that covers expressions.
2018-01-26 08:20:18 -08:00
Yury Selivanov 6ab62920c8
bpo-32436: Fix a refleak; var GC tracking; a GCC warning (#5326)
The refleak in question wasn't really important, as context vars
are usually created at the toplevel and live as long as the interpreter
lives, so the context var name isn't ever GCed anyways.
2018-01-25 14:18:55 -05:00
Dmitry Alimov 93a6119f08 Fix 3rd level node's binary representation in HAMT algorithm description (#5319) 2018-01-25 12:54:41 -05:00
Serhiy Storchaka f320be77ff bpo-32571: Avoid raising unneeded AttributeError and silencing it in C code (GH-5222)
Add two new private APIs: _PyObject_LookupAttr() and _PyObject_LookupAttrId()
2018-01-25 17:49:40 +09:00
Yury Selivanov b7a80d543e
bpo-32436: Don't use native popcount() (also fixes bpo-32641) (#5292) 2018-01-23 22:17:04 -05:00
Yury Selivanov 0bad4d63c6
bpo-32436: Fix potential NULL dereference (#5286) 2018-01-23 16:26:07 -05:00