Commit Graph

5813 Commits

Author SHA1 Message Date
Miss Islington (bot) 192c54713b
bpo-26558: Fix Py_FatalError() with GIL released (GH-10267)
Don't call _Py_FatalError_PrintExc() nor flush_std_files() if the
current thread doesn't hold the GIL, or if the current thread
has no Python state thread.
(cherry picked from commit 3a228ab17c)

Co-authored-by: Victor Stinner <vstinner@redhat.com>
2018-10-31 16:45:42 -07:00
Victor Stinner 7d35f79012
bpo-34403: Always implement _Py_GetForceASCII() (GH-10235)
Compilation fails on macOS because _Py_GetForceASCII() wasn't define:
always implement implement (default implementation: just return 0).
2018-10-30 14:32:01 +01:00
Victor Stinner 21220bbe65
bpo-34403: Fix initfsencoding() for ASCII (GH-10233)
* Add _Py_GetForceASCII(): check if Python forces the usage of ASCII
  in Py_DecodeLocale() and Py_EncodeLocale().
* initfsencoding() now uses ASCII if _Py_GetForceASCII() is true.
2018-10-30 12:59:20 +01:00
Miss Islington (bot) 178d1c0777
bpo-24658: Fix read/write greater than 2 GiB on macOS (GH-1705)
On macOS, fix reading from and writing into a file with a size larger than 2 GiB.
(cherry picked from commit 74a8b6ea7e)

Co-authored-by: Stéphane Wirtel <stephane@wirtel.be>
2018-10-17 23:58:40 -07:00
Miss Islington (bot) 72927d0d17
Fix an incorrect check in compiler_try_except(). (GH-9810)
(cherry picked from commit 53ebf4b070)

Co-authored-by: Zackery Spytz <zspytz@gmail.com>
2018-10-12 00:19:20 -07:00
Miss Islington (bot) 1c2cb516e4
Fix a possible decref of a borrowed reference in symtable.c. (GH-9786)
(cherry picked from commit fc439d20de)

Co-authored-by: Zackery Spytz <zspytz@gmail.com>
2018-10-10 22:24:14 -07:00
Miss Islington (bot) cd4dd9374d
closes bpo-34868: Improve error message with '_' is combined with an invalid type specifier. (GH-9666)
(cherry picked from commit cbda8fc5d7)

Co-authored-by: Benjamin Peterson <benjamin@python.org>
2018-10-01 22:12:02 -07:00
Miss Islington (bot) 0f161b3079
bpo-34854: Fix compiling string annotations containing lambdas. (GH-9645)
* Compiling a string annotation containing a lambda with keyword-only
argument without default value caused a crash.

* Remove the final "*" (it is incorrect syntax) in the representation of
lambda without *args and keyword-only arguments when compile from AST.

* Improve the representation of lambda without arguments.
(cherry picked from commit 2a2940e5c3)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2018-09-30 11:19:15 -07:00
Victor Stinner b54fc15e32
Drop confusing commented out code in pystrtod.c (GH-6072) (GH-9586)
Fix the following warning:

Python/pystrtod.c: In function 'format_float_short':
Python/pystrtod.c:1007:13: warning: 'strncpy' output truncated before terminating nul copying 3 bytes from a string of the same length [-Wstringop-truncation]
             strncpy(p, "ERR", 3);

(cherry picked from commit 9fb8415759)
2018-09-26 07:45:19 -07:00
Miss Islington (bot) 187f2dd256
bpo-34762: Fix contextvars C API to use PyObject* pointer types. (GH-9473)
(cherry picked from commit 2ec872b31e)

Co-authored-by: Yury Selivanov <yury@magic.io>
2018-09-21 12:48:10 -07:00
Victor Stinner 95cc3ee00c
Revert "[3.7] bpo-34589: Add -X coerce_c_locale option; C locale coercion off by default (GH-9379)" (GH-9416)
This reverts commit 144f1e2c6f.
2018-09-19 12:01:52 -07:00
Victor Stinner 144f1e2c6f
[3.7] bpo-34589: Add -X coerce_c_locale option; C locale coercion off by default (GH-9379)
* bpo-34589: Make _PyCoreConfig.coerce_c_locale private (GH-9371)

_PyCoreConfig:

* Rename coerce_c_locale to _coerce_c_locale
* Rename coerce_c_locale_warn to _coerce_c_locale_warn

These fields are now private (name prefixed by "_").

(cherry picked from commit 188ebfa475)

* bpo-34589: C locale coercion off by default (GH-9073)

Py_Initialize() and Py_Main() cannot enable the C locale coercion
(PEP 538) anymore: it is always disabled. It can now only be enabled
by the Python program ("python3).

test_embed: get_filesystem_encoding() doesn't have to set PYTHONUTF8
nor PYTHONCOERCECLOCALE, these variables are already set in the
parent.

(cherry picked from commit 7a0791b699)

* bpo-34589: Add -X coerce_c_locale command line option (GH-9378)

Add a new -X coerce_c_locale command line option to control C locale
coercion (PEP 538).

(cherry picked from commit dbdee0073c)
2018-09-17 18:01:39 -07:00
Miss Islington (bot) 4007e4e207
Make sure the line comes from the same node as the col offset. (GH-9189)
Followup to 90fc8980bb.

<!--
Thanks for your contribution!
Please read this comment in its entirety. It's quite important.

GH- Pull Request title

It should be in the following format:

```
bpo-NNNN: Summary of the changes made
```

Where: bpo-NNNN refers to the issue number in the https://bugs.python.org.

Most PRs will require an issue number. Trivial changes, like fixing a typo, do not need an issue.

GH- Backport Pull Request title

If this is a backport PR (PR made against branches other than `master`),
please ensure that the PR title is in the following format:

```
[X.Y] <title from the original PR> (GH-NNNN)
```

Where: [X.Y] is the branch name, e.g. [3.6].

GH-NNNN refers to the PR number from `master`.

-->
(cherry picked from commit d13e59c1b5)

Co-authored-by: Benjamin Peterson <benjamin@python.org>
2018-09-11 16:32:52 -07:00
Miss Islington (bot) d8bc7a666b
closes bpo-31902: Fix the col_offset attribute for ast.Async* nodes to point to the "async" keyword. (GH-4175)
Previously, col_offset points to the keyword after "async".
(cherry picked from commit 90fc8980bb)

Co-authored-by: guoci <zguoci@gmail.com>
2018-09-11 15:21:11 -07:00
Miss Islington (bot) 4902017430
bpo-34588: Fix an off-by-one error in traceback formatting. (GH-9077)
The recursive frame pruning code always undercounted the number of elided frames
by one. That is, in the "[Previous line repeated N more times]" message, N would
always be one too few. Near the recursive pruning cutoff, one frame could be
silently dropped. That situation is demonstrated in the OP of the bug report.

The fix is to start the identical frame counter at 1.
(cherry picked from commit d545869d08)

Co-authored-by: Benjamin Peterson <benjamin@python.org>
2018-09-10 09:00:08 -07:00
Victor Stinner 84b0129b5e
_Py_CoerceLegacyLocale() restores LC_CTYPE on fail (GH-9044) (GH-9046)
bpo-34544: If _Py_CoerceLegacyLocale() fails to coerce the C locale,
restore the LC_CTYPE locale to the its previous value.

(cherry picked from commit 8ea09110d4)
2018-09-03 22:17:07 +02:00
Victor Stinner 0b9ea4b211
[3.7] bpo-34485: stdout uses surrogateescape on POSIX locale (GH-8986) (GH-8987)
* bpo-34485: stdout uses surrogateescape on POSIX locale (GH-8986)

Standard streams like sys.stdout now use the "surrogateescape" error
handler, instead of "strict", on the POSIX locale (when the C locale is not
coerced and the UTF-8 Mode is disabled).

Add tests on sys.stdout.errors with LC_ALL=POSIX.

Fix the error handler of standard streams like sys.stdout:
PYTHONIOENCODING=":" is now ignored instead of setting the error handler to
"strict".

(cherry picked from commit 315877dc36)
2018-08-29 11:01:33 +02:00
Victor Stinner 98c49c6ab2
bpo-34485: Fix _Py_InitializeCore() for C locale coercion (GH-8979) (GH-8981)
* _Py_InitializeCore() now sets the LC_CTYPE locale to the user
  preferred locale before checking if the C locale should be coerced
  or not in _PyCoreConfig_Read().
* Fix pymain_read_conf(): remember if the C locale has been coerced
  when the configuration should be read again if the encoding has
  changed.

(cherry picked from commit 2c8ddcf4f1)
2018-08-29 01:13:29 +02:00
Victor Stinner 65ef7425a3
bpo-34527: POSIX locale enables the UTF-8 Mode (GH-8972) (GH-8974)
* The UTF-8 Mode is now also enabled by the "POSIX" locale, not only
  by the "C" locale.
* On FreeBSD, Py_DecodeLocale() and Py_EncodeLocale() now also forces
  the ASCII encoding if the LC_CTYPE locale is "POSIX", not only if
  the LC_CTYPE locale is "C".
* test_utf8_mode.test_cmd_line() checks also that the command line
  arguments are decoded from UTF-8 when the the UTF-8 Mode is enabled
  with POSIX locale or C locale.

(cherry picked from commit 5cb258950c)
2018-08-28 13:51:20 +02:00
Miss Islington (bot) 2caf86ff6d
bpo-34503: Fix refleak in PyErr_SetObject() (GH-8934)
(cherry picked from commit a2eefa6754)

Co-authored-by: Alexey Izbyshev <izbyshev@ispras.ru>
2018-08-26 13:13:47 -04:00
Miss Islington (bot) a59036365b
closes bpo-34474: Python/bltinmodule.c: Add missing NULL check in builtin_sum_impl() (GH-8872)
Reported by Svace static analyzer.
(cherry picked from commit 2b824b2538)

Co-authored-by: Alexey Izbyshev <izbyshev@ispras.ru>
2018-08-24 00:59:16 -04:00
Miss Islington (bot) b8e7319d39
bpo-34457: Python/ast.c: Add missing NULL check to alias_for_import_name(). (GH-8852)
Reported by Svace static analyzer.
(cherry picked from commit 28853a249b)

Co-authored-by: Alexey Izbyshev <izbyshev@ispras.ru>
2018-08-22 01:54:46 -04:00
Victor Stinner 0c90d6f759
[3.7] bpo-34247: Fix Python 3.7 initialization (#8659)
* -X dev: it is now possible to override the memory allocator using
  PYTHONMALLOC even if the developer mode is enabled.
* Add _Py_InitializeFromConfig()
* Add _Py_Initialize_ReadEnvVars() to set global configuration
  variables from environment variables
* Fix the code to initialize Python: Py_Initialize() now also reads
  environment variables
* _Py_InitializeCore() can now be called twice: the second call
  only replaces the configuration.
* Write unit tests on Py_Initialize() and the different ways to
  configure Python
* The isolated mode now always sets Py_IgnoreEnvironmentFlag and
  Py_NoUserSiteDirectory to 1.
* pymain_read_conf() now saves/restores the configuration
  if the encoding changed
2018-08-05 12:31:59 +02:00
Miss Islington (bot) 220afffb68 bpo-34084: Fix setting an error message for the "Barry as BDFL" easter egg. (GH-8262) (GH-8423)
(cherry picked from commit aba24ff360)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2018-07-24 01:41:26 +03:00
Miss Islington (bot) 25b87d3567 bpo-34190: Fix reference leak in call_function() (GH-8413) (GH-8418)
(cherry picked from commit 147d95511f)

Co-authored-by: jdemeyer <jdemeyer@cage.ugent.be>
2018-07-23 23:45:26 +02:00
Miss Islington (bot) 7638eb892a bpo-34126: Fix crashes while profiling invalid calls. (GH-8300) (GH-8371)
(cherry picked from commit 56868f940e)

Co-authored-by: jdemeyer <jdemeyer@cage.ugent.be>
2018-07-21 19:58:35 +03:00
Miss Islington (bot) 03ec4df67d bpo-34008: Allow to call Py_Main() after Py_Initialize() (GH-8043) (GH-8352)
Py_Main() can again be called after Py_Initialize(), as in Python
3.6. The new configuration is ignored, except of
_PyMainInterpreterConfig.argv which is used to update sys.argv.
(cherry picked from commit fb47bca9ee)

Co-authored-by: Victor Stinner <vstinner@redhat.com>
2018-07-21 02:16:22 +02:00
Miss Islington (bot) c721472fb8
bpo-34087: Fix buffer overflow in int(s) and similar functions (GH-8274)
`_PyUnicode_TransformDecimalAndSpaceToASCII()` missed trailing NUL char.
It caused buffer overflow in `_Py_string_to_number_with_underscores()`.

This bug is introduced in 9b6c60cb.
(cherry picked from commit 16dfca4d82)

Co-authored-by: INADA Naoki <methane@users.noreply.github.com>
2018-07-13 20:58:12 -07:00
Miss Islington (bot) 9beed0c36f
bpo-34080: Fix a memory leak in the compiler. (GH-8222)
(cherry picked from commit 993030aac5)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2018-07-11 14:54:29 -07:00
Miss Islington (bot) 48d2aeb009
bpo-23927: Make getargs.c skipitem() skipping 'w*'. (GH-8192)
(cherry picked from commit 504373c59b)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2018-07-11 09:56:05 -07:00
Miss Islington (bot) 6f4fbf8ea4
bpo-22689: Copy the result of getenv() in sys_breakpointhook(). (GH-8194)
(cherry picked from commit f60bf0e168)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2018-07-09 12:06:02 -07:00
Miss Islington (bot) f5197ddfd0
bpo-34066: Disabled interruption before SETUP_WITH and BEFORE_ASYNC_WITH. (GH-8159)
This will prevent emitting a resource warning when the execution was
interrupted by Ctrl-C between calling open() and entering a 'with' block
in "with open()".
(cherry picked from commit 3f4d90d4d7)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2018-07-09 06:31:03 -07:00
Miss Islington (bot) 625755eb8c
Fix GCC warning in Python/hamt.c (GH-7618)
(cherry picked from commit d8c3e820b4)

Co-authored-by: Zackery Spytz <zspytz@gmail.com>
2018-07-06 02:43:05 -07:00
Miss Islington (bot) 20ae4c6025 bpo-24596: Decref module in PyRun_SimpleFileExFlags() on SystemExit (GH-7918) (GH-8070)
PyErr_Print() will not return when the exception is a SystemExit, so
decref the __main__ module object in that case.
(cherry picked from commit d8cba5d16f)

Co-authored-by: Zackery Spytz <zspytz@gmail.com>
2018-07-03 22:13:28 +02:00
Miss Islington (bot) 4c20d2bf5d
bpo-33985: Implement ContextVar.name attribute. (GH-7980)
(cherry picked from commit 41cb0baea9)

Co-authored-by: Yury Selivanov <yury@magic.io>
2018-06-28 10:39:54 -07:00
Miss Islington (bot) 56aaef0ddb
bpo-33451: Close pyc files before calling PyEval_EvalCode() (GH-7884)
Directly executed pyc files were being kept open longer than necessary.
(cherry picked from commit ea737751b1)

Co-authored-by: Zackery Spytz <zspytz@gmail.com>
2018-06-23 20:31:21 -07:00
Miss Islington (bot) 3747dd16d5
bpo-33932: Calling Py_Initialize() twice does nothing (GH-7845)
Calling Py_Initialize() twice does nothing, instead of failing with a
fatal error: restore the Python 3.6 behaviour.
(cherry picked from commit 209abf7469)

Co-authored-by: Victor Stinner <vstinner@redhat.com>
2018-06-22 10:33:48 -07:00
Miss Islington (bot) a971a6fdb1 bpo-33803: Fix a crash in hamt.c (GH-7504) (GH-7505)
(cherry picked from commit 378c53cc31)

Co-authored-by: Yury Selivanov <yury@magic.io>
2018-06-07 20:44:09 -04:00
Miss Islington (bot) 103058e19b
bpo-33720: Reduces maximum marshal recursion depth on release builds. (GH-7401)
(cherry picked from commit 2a4a62ba4a)

Co-authored-by: Steve Dower <steve.dower@microsoft.com>
2018-06-04 13:41:49 -07:00
Miss Islington (bot) 150033d159
bpo-31849: Fix warning in pyhash.c (GH-6799)
(cherry picked from commit a8eb58546b)

Co-authored-by: A. Jesse Jiryu Davis <jesse@emptysquare.net>
2018-06-04 10:31:07 -07:00
Miss Islington (bot) a4dd46a47f
bpo-33677: Fix signatures of tp_clear handlers for AST and deque. (GH-7196)
(cherry picked from commit a5c42284e6)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2018-05-30 22:31:21 -07:00
Serhiy Storchaka 2641ee5040
bpo-32911: Revert bpo-29463. (GH-7121)
Remove the docstring attribute of AST types and restore docstring
expression as a first stmt in their body.

Co-authored-by: INADA Naoki <methane@users.noreply.github.com>
2018-05-29 10:49:10 +03:00
Miss Islington (bot) 508d7693bc bpo-33612: Remove PyThreadState_Clear() assertion (GH-7069) (GH-7074)
bpo-25612, bpo-33612: Remove an assertion from PyThreadState_Clear()
which failed at Python shutdown or on fork if a thread was running a
generator.
(cherry picked from commit b6dccf54fd)

Co-authored-by: Victor Stinner <vstinner@redhat.com>
2018-05-23 15:58:21 -04:00
Serhiy Storchaka b32f8897ea
[3.7] bpo-33475: Fix and improve converting annotations to strings. (GH-6774). (GH-6927)
(cherry picked from commit 64fddc423f)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2018-05-20 18:06:08 +03:00
Miss Islington (bot) 820219f786
bpo-33509: Fix _warnings for module_globals=None (GH-6833)
Don't crash on warnings.warn_explicit() if module_globals is not a dict.
(cherry picked from commit b056562860)

Co-authored-by: Victor Stinner <vstinner@redhat.com>
2018-05-15 13:56:28 -07:00
Eric Snow 6bd0c476c5
bpo-32604: Remove xid registry. (#6813)
Remove the interpreters testing helper (and xid registry).
2018-05-15 09:56:18 -04:00
Miss Islington (bot) 8ed545f6de
bpo-28055: Fix unaligned accesses in siphash24(). (GH-6123)
The hash implementation casts the input pointer to uint64_t* and directly reads
from this, which may cause unaligned accesses. Use memcpy() instead so this code
will not crash with SIGBUS on sparc.

https://bugs.gentoo.org/show_bug.cgi?id=636400
(cherry picked from commit 1e2ec8a996)

Co-authored-by: Rolf Eike Beer <eike@sf-mail.de>
2018-05-13 04:17:07 -07:00
Zsolt Dollenstein a93a663d6c [3.7] bpo-33363: raise SyntaxError for async for/with outside async functions (GH-6616). (GH-6619) 2018-04-27 18:33:37 -04:00
Miss Islington (bot) 291c9d4f74
bpo-33330: Improve error handling in PyImport_Cleanup(). (GH-6564)
(cherry picked from commit e9d9494d6b)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2018-04-25 11:32:52 -07:00
Miss Islington (bot) 52a5a17338
bpo-33128 Fix duplicated call to importlib._install_external_importers (GH-6273)
External importers were being added in both phases of the import
system initialisation.

They're only supposed to be added in the second phase, after the
import machinery has been appropriately configured.
(cherry picked from commit 0977091dca)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2018-04-24 19:48:04 -07:00
Miss Islington (bot) 32955299b4
Spelling fixes to docs, docstrings, and comments (GH-6374)
(cherry picked from commit 61f82e0e33)

Co-authored-by: Ville Skyttä <ville.skytta@iki.fi>
2018-04-20 14:00:41 -07:00
Brett Cannon a09bb87c1e
[3.7] bpo-33169: Remove values of `None` from sys.path_importer_cache when invalidating caches (GH-6402) (GH-6403)
An entry of None in sys.path_importer_cache represents a negative/missing finder for a path, so clearing it out makes sense.
(cherry picked from commit 9e2be60634)
2018-04-06 17:02:18 -07:00
Miss Islington (bot) 3c193cf8af [3.7] bpo-29673: fix gdb scripts pystack and pystackv (GH-6126) (GH-6399)
(cherry picked from commit 3a9ccee0e5)

Co-authored-by: Marcel Plch <gmarcel.plch@gmail.com>
2018-04-06 18:14:29 -04:00
Miss Islington (bot) 64421d9237
bpo-33231: Fix potential leak in normalizestring() (GH-6386)
(cherry picked from commit 0c1c4563a6)

Co-authored-by: INADA Naoki <methane@users.noreply.github.com>
2018-04-06 00:12:38 -07:00
Miss Islington (bot) fcd4e03e08
bpo-29922: Improve error messages in 'async with' (GH-6352)
when __aenter__() or __aexit__() return non-awaitable object.
(cherry picked from commit a68f2f0578)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2018-04-04 07:09:00 -07:00
Miss Islington (bot) 9e96e7b24e
bpo-33132: Fix more reference counting issues in the compiler. (GH-6323)
(cherry picked from commit aa8e51f5eb)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2018-03-31 16:41:28 -07:00
Nick Coghlan ee3784594b
bpo-33053: -m now adds *starting* directory to sys.path (GH-6231) (#6236)
Historically, -m added the empty string as sys.path
zero, meaning it resolved imports against the current
working directory, the same way -c and the interactive
prompt do.

This changes the sys.path initialisation to add the
*starting* working directory as sys.path[0] instead,
such that changes to the working directory while the
program is running will have no effect on imports
when using the -m switch.

(cherry picked from commit d5d9e02dd3)
2018-03-25 23:43:50 +10:00
Miss Islington (bot) c6d94c37f4
bpo-33042: Fix pre-initialization sys module configuration (GH-6157)
- new test case for pre-initialization of sys.warnoptions and sys._xoptions
- restored ability to call these APIs prior to Py_Initialize
- updated the docs for the affected APIs to make it clear they can be
  called before Py_Initialize
- also enhanced the existing embedding test cases
  to check for expected settings in the sys module
(cherry picked from commit bc77eff8b9)

Co-authored-by: Nick Coghlan <ncoghlan@gmail.com>
2018-03-25 04:27:57 -07:00
Miss Islington (bot) 471364b4d9
bpo-33132: Fix reference counting issues in the compiler. (GH-6209)
(cherry picked from commit a95d98607e)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2018-03-24 14:27:06 -07:00
Serhiy Storchaka b9744e924c
bpo-33041: Fixed jumping if the function contains an "async for" loop. (GH-6154) 2018-03-23 14:35:33 +02:00
Miss Islington (bot) 56cce1ca84 [3.7] bpo-33021: Release the GIL during fstat() calls (GH-6019) (GH-6159)
fstat may block for long time if the file descriptor is on a
non-responsive NFS server, hanging all threads. Most fstat() calls are
handled by _Py_fstat(), releasing the GIL internally, but but
_Py_fstat_noraise() does not release the GIL, and most calls release the
GIL explicitly around it.

This patch fixes last 2 calls to _Py_fstat_no_raise(), avoiding hangs
when calling:
- mmap.mmap()
- os.urandom()
- random.seed()
(cherry picked from commit 4484f9dca9)

Co-authored-by: Nir Soffer <nirsof@gmail.com>
2018-03-20 20:16:30 +01:00
Miss Islington (bot) 5506d60302
bpo-32836: Remove obsolete code from symtable pass (GH-5680)
When comprehensions switched to using a nested scope, the old
code for generating a temporary name to hold the accumulation
target became redundant, but was never actually removed.

Patch by Nitish Chandra.
(cherry picked from commit 3a087beddd)

Co-authored-by: Nitish Chandra <nitishchandrachinta@gmail.com>
2018-03-10 15:11:47 -08:00
Serhiy Storchaka 9e94c0d3c7
[3.7] bpo-33041: Add missed error checks when compile "async for" (GH-6053) (GH-6060)
and remove redundant code.
(cherry picked from commit 67ee07795b)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2018-03-10 20:45:05 +02:00
Miss Islington (bot) e724dd4e5e
Add two missing error checks in hamt.c (GH-5851)
(cherry picked from commit 3c7ac7ea20)

Co-authored-by: Xiang Zhang <angwerzx@126.com>
2018-03-07 22:21:34 -08:00
Miss Islington (bot) 31e2b76f7b
bpo-33005: Fix _PyGILState_Reinit() (GH-6001)
Fix a crash on fork when using a custom memory allocator (ex: using
PYTHONMALLOC env var).

_PyGILState_Reinit() and _PyInterpreterState_Enable() now use the
default RAW memory allocator to allocate a new interpreters mutex on
fork.
(cherry picked from commit 5d92647102)

Co-authored-by: Victor Stinner <victor.stinner@gmail.com>
2018-03-06 05:52:27 -08:00
Miss Islington (bot) 8d9d4b57ae
bpo-32838: Fix Python versions in the table of magic numbers. (GH-5658)
(cherry picked from commit 4af8fd5614)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2018-02-23 11:01:54 -08:00
Miss Islington (bot) eed3c7af4e bpo-32604: Swap threads only if the interpreter is different. (gh-5783)
The CPython runtime assumes that there is a one-to-one relationship (for a given interpreter) between PyThreadState and OS threads. Sending and receiving on a channel in the same interpreter was causing crashes because of this (specifically due to a check in PyThreadState_Swap()). The solution is to not switch threads if the interpreter is the same.
(cherry picked from commit f53d9f2778)

Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
2018-02-20 17:09:41 -07:00
Miss Islington (bot) ca82e3c0ec
bpo-32869: Fix incorrect dst buffer size for MultiByteToWideChar (GH-5739)
This function expects the destination buffer size to be given
in wide characters, not bytes.
(cherry picked from commit b3b4a9d300)

Co-authored-by: Alexey Izbyshev <izbyshev@users.noreply.github.com>
2018-02-18 10:40:07 -08:00
Miss Islington (bot) 3db05a3a9c bpo-32604: Clean up created subinterpreters before runtime finalization. (gh-5710)
(cherry picked from commit 4c6955e2b0)

Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
2018-02-16 19:15:24 -07:00
Miss Islington (bot) 7df80492fc Fix some warnings produced by different compilers. (GH-5593) (GH-5600)
(cherry picked from commit bfe4fd5f2e)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2018-02-09 17:56:34 +02:00
Miss Islington (bot) 2bb0bfafb0 bpo-32777: Fix _Py_set_inheritable async-safety in subprocess (GH-5560) (GH-5562)
Fix a rare but potential pre-exec child process deadlock in subprocess on POSIX systems when marking file descriptors inheritable on exec in the child process.  This bug appears to have been introduced in 3.4 with the inheritable file descriptors support.

This also changes Python/fileutils.c `set_inheritable` to use the "slow" two `fcntl` syscall path instead of the "fast" single `ioctl` syscall path when asked to be async signal safe (by way of being asked not to raise exceptions).  `ioctl` is not a POSIX async-signal-safe approved function.

ref: http://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html
(cherry picked from commit c1e46e94de)

Co-authored-by: Alexey Izbyshev <izbyshev@users.noreply.github.com>
2018-02-05 22:31:22 -08:00
Miss Islington (bot) e86db34dd3 [3.7] bpo-32746: Fix multiple typos (GH-5144) (GH-5520)
Fix typos found by codespell in docs, docstrings, and comments.
(cherry picked from commit c3d9508ff2)

Co-authored-by: Leo Arias <leo.arias@canonical.com>
2018-02-03 20:41:43 -05:00
Miss Islington (bot) f33ecedcad bpo-32604: Fix memory leaks in the new _xxsubinterpreters module. (GH-5507)
(cherry picked from commit 4e9da0d163)

Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
2018-02-02 22:38:57 -07:00
Barry Warsaw a23d30f64b
bpo-32303 - Consistency fixes for namespace loaders (GH-5481) (#5503)
* Make sure ``__spec__.loader`` matches ``__loader__`` for namespace packages.
* Make sure ``__spec__.origin` matches ``__file__`` for namespace packages.

https://bugs.python.org/issue32303
https://bugs.python.org/issue32305
(cherry picked from commit bbbcf8693b)

Co-authored-by: Barry Warsaw <barry@python.org>
2018-02-02 19:49:25 -05:00
Miss Islington (bot) e985afc38d bpo-32674: Improve the docstring for __import__ (GH-5339) (GH-5488)
Clarify that the level argument is used to determine whether to
perform absolute or relative imports: 0 is absolute, while a positive number
is the number of parent directories to search relative to the current module.
(cherry picked from commit 461d225b19)

Co-authored-by: oldk <oldk1331@users.noreply.github.com>
2018-02-01 23:51:36 -05:00
Miss Islington (bot) 78767786a8 bpo-32436: Fix compiler warning (GH-5483) (GH-5486)
(cherry picked from commit 55e0839f26)

Co-authored-by: Yury Selivanov <yury@magic.io>
2018-02-01 22:58:19 -05:00
Miss Islington (bot) 5d2dcd044d Fix typo in hamt.c comments (GH-5478) (#5484)
(cherry picked from commit 01a0cb8916)

Co-authored-by: Dmitry Alimov <dvalimov@gmail.com>
2018-02-01 22:30:34 -05:00
Miss Islington (bot) 78758f29b1 [3.7] bpo-32711: Fix warnings for Python/ast_unparse.c (GH-5426) (#5475)
* bpo-32711: Fix warnings for Python/ast_unparse.c
(cherry picked from commit 83ab995871)

Co-authored-by: Stéphane Wirtel <stephane@wirtel.be>
2018-02-01 18:41:24 +01:00
Eric Snow 7f8bfc9b9a
bpo-32604: Expose the subinterpreters C-API in a "private" stdlib module. (gh-1748)
The module is primarily intended for internal use in the test suite.  Building the module under Windows will come in a follow-up PR.
2018-01-29 18:23:44 -07:00
Mark Shannon 332cd5ee4f bpo-32550. Remove the STORE_ANNOTATION bytecode. (GH-5181) 2018-01-29 16:41:04 -08:00
Benjamin Peterson c65ef772c3
rename _imp initialization function to follow conventions (#5432)
When the C imp module became _imp in 6f44d66bc4, the initialization function should have been renamed from PyInit_imp to PyInit__imp.
2018-01-29 11:33:57 -08:00
Yury Selivanov b647d7039d
bpo-32707: Fix warnings in hamt.c (#5430) 2018-01-29 13:31:37 -05:00
Victor Stinner 2914bb32e2
bpo-20891: Py_Initialize() now creates the GIL (#4700)
The GIL is no longer created "on demand" to fix a race condition when
PyGILState_Ensure() is called in a non-Python thread.
2018-01-29 11:57:45 +01:00
Xiang Zhang 997478eb79
bpo-32688: Make why the line is commented out clear (GH-5405) 2018-01-29 11:32:12 +08:00
Yury Selivanov bc4123b0b3
bpo-32436: Use PyThreadState_GET() in all hot paths (GH-5363) 2018-01-27 13:24:20 -05:00
Yury Selivanov 226e50049d
bpo-32436: Make PyContextVar_Get a little bit faster (#5350)
Since context.c is compiled with Py_BUILD_CORE, using a macro
will result in a slightly more optimal code.
2018-01-26 17:24:52 -05:00
Yury Selivanov 43c47fe096
bpo-32670: Enforce PEP 479. (#5327) 2018-01-26 15:24:24 -05:00
Nathaniel J. Smith dba976b8a2 bpo-32591: fix abort in _PyErr_WarnUnawaitedCoroutine during shutdown (#5337)
When an unawaited coroutine is collected very late in shutdown --
like, during the final GC at the end of PyImport_Cleanup -- then it
was triggering an interpreter abort, because we'd try to look up the
"warnings" module and not only was it missing (we were prepared for
that), but the entire module system was missing (which we were not
prepared for).

I've tried to fix this at the source, by making the utility function
get_warnings_attr robust against this in general. Note that it already
has the convention that it can return NULL without setting an error,
which is how it signals that the attribute it was asked to fetch is
missing, and that all callers already check for NULL returns.

There's a similar check for being late in shutdown at the top of
warn_explicit, which might be unnecessary after this fix, but I'm not
sure so I'm going to leave it.
2018-01-26 14:28:31 -05:00
Guido van Rossum 95e4d58913 String annotations [PEP 563] (#4390)
* Document `from __future__ import annotations`
* Provide plumbing and tests for `from __future__ import annotations`
* Implement unparsing the AST back to string form

This is required for PEP 563 and as such only implements a part of the
unparsing process that covers expressions.
2018-01-26 08:20:18 -08:00
Yury Selivanov 6ab62920c8
bpo-32436: Fix a refleak; var GC tracking; a GCC warning (#5326)
The refleak in question wasn't really important, as context vars
are usually created at the toplevel and live as long as the interpreter
lives, so the context var name isn't ever GCed anyways.
2018-01-25 14:18:55 -05:00
Dmitry Alimov 93a6119f08 Fix 3rd level node's binary representation in HAMT algorithm description (#5319) 2018-01-25 12:54:41 -05:00
Serhiy Storchaka f320be77ff bpo-32571: Avoid raising unneeded AttributeError and silencing it in C code (GH-5222)
Add two new private APIs: _PyObject_LookupAttr() and _PyObject_LookupAttrId()
2018-01-25 17:49:40 +09:00
Yury Selivanov b7a80d543e
bpo-32436: Don't use native popcount() (also fixes bpo-32641) (#5292) 2018-01-23 22:17:04 -05:00
Yury Selivanov 0bad4d63c6
bpo-32436: Fix potential NULL dereference (#5286) 2018-01-23 16:26:07 -05:00
Yury Selivanov 83c8675edb
bpo-32436: Remove a redundant assert (#5275) 2018-01-23 02:00:03 -05:00
Yury Selivanov f23746a934
bpo-32436: Implement PEP 567 (#5027) 2018-01-22 19:11:18 -05:00
Victor Stinner 9089a26591
bpo-29240: PyUnicode_DecodeLocale() uses UTF-8 on Android (#5272)
PyUnicode_DecodeLocaleAndSize(), PyUnicode_DecodeLocale() and
PyUnicode_EncodeLocale() now use always use the UTF-8 encoding on
Android, instead of the current locale encoding.

On Android API 19, mbstowcs() and wcstombs() are broken and cannot be
used.
2018-01-22 19:07:32 +01:00
Victor Stinner 13ff24582c
bpo-32593: Drop FreeBSD 9 and older support (#5232)
Drop support of FreeBSD 9 and older.
2018-01-22 18:32:50 +01:00
Yury Selivanov 3510334361
bpo-32591: Fix PyExc_WarnFormat call (follow-up commit) (#5263)
The previous version was correct in terms of behaviour, but
checking the return value of PyErr_WarnFormat allows to
avoid calling PyErr_Occurred and silences the coverity alarm.
2018-01-21 20:47:04 -05:00
Nathaniel J. Smith fc2f407829 bpo-32591: Add native coroutine origin tracking (#5250)
* Add coro.cr_origin and sys.set_coroutine_origin_tracking_depth
* Use coroutine origin information in the unawaited coroutine warning
* Stop using set_coroutine_wrapper in asyncio debug mode
* In BaseEventLoop.set_debug, enable debugging in the correct thread
2018-01-21 09:44:07 -05:00