Commit Graph

6473 Commits

Author SHA1 Message Date
Zackery Spytz 08050e959e
bpo-40147: Fix a compiler warning on Windows in Python/compile.c (GH-19389)
Change the type of nkeywords to Py_ssize_t.
2020-04-06 07:47:47 +01:00
Batuhan Taşkaya 1b21573a89
closes bpo-40184: Only define pysiphash if the hash algorithm is SIPHASH24. (GH-19369) 2020-04-04 16:25:12 -05:00
Pablo Galindo 40cf35c5b0
bpo-40141: Include the value in the column position for keyword AST nodes (GH-19348) 2020-04-03 21:02:26 +01:00
Pablo Galindo 254ec78341
bpo-40147: Move the check for duplicate keywords to the compiler (GH-19289) 2020-04-03 20:37:13 +01:00
Pablo Galindo 168660b547
bpo-40141: Add line and column information to ast.keyword nodes (GH-19283) 2020-04-02 00:47:39 +01:00
Serhiy Storchaka 2c003eff8f
bpo-39943: Clean up marshal.c. (GH-19236)
* Add consts.
* Remove redundant casts and checks.
* Use concrete C API macros.
* Avoid raising and silencing OverflowError for ints.
2020-03-31 23:23:21 +03:00
Victor Stinner 400e1dbcad
Document most common signals (GH-19245)
Document individual signals (only the most common signals):
description, default action, availability.
2020-03-31 19:13:10 +02:00
Victor Stinner 8d84adcd73
bpo-32591: _PyErr_WarnUnawaitedCoroutine() sets source (GH-19247)
The _PyErr_WarnUnawaitedCoroutine() fallback now also sets the
coroutine object as the source of the warning, as done by the Python
implementation warnings._warn_unawaited_coroutine().

Moreover, don't truncate the coroutine name: Python supports
arbitrary string length to format the message.
2020-03-31 17:25:12 +02:00
Andy Lester fc2d8d62af
bpo-39943: Remove unnecessary casts in import.c that remove constness (GH-19209) 2020-03-30 13:19:14 -07:00
Victor Stinner 1c1e68cf3e
bpo-38644: Use _PySys_Audit(): pass tstate explicitly (GH-19183)
Add the dependency to tstate more explicit.
2020-03-27 15:11:45 +01:00
Victor Stinner 71a3522ef8
bpo-38644: Make tstate more explicit inside pystate.c (GH-19182)
Fix PyInterpreterState_New(): Don't call PyErr_SetString() when there
is no current Python thread state (if tstate is NULL).
2020-03-26 22:46:14 +01:00
Victor Stinner 728189884e
bpo-38644: Pass tstate explicitly in signalmodule.c (GH-19184)
PyOS_InterruptOccurred() now checks _Py_ThreadCanHandleSignals()
before checking if SIGINT is tripped.
2020-03-26 22:28:11 +01:00
Victor Stinner 08faf0016e
bpo-38644: Add _PySys_Audit() which takes tstate (GH-19180)
Add _PySys_Audit() function to the internal C API: similar to
PySys_Audit(), but requires a mandatory tstate parameter.

Cleanup sys_audit_tstate() code: remove code path for NULL tstate,
since the function exits at entry if tstate is NULL. Remove also code
path for NULL tstate->interp: should_audit() now ensures that it is
not NULL (even if tstate->interp cannot be NULL in practice).

PySys_AddAuditHook() now checks if tstate is not NULL to decide if
tstate can be used or not, and tstate is set to NULL if the runtime
is not initialized yet.

Use _PySys_Audit() in sysmodule.c.
2020-03-26 18:57:32 +01:00
Zackery Spytz 79ceccd1ec
bpo-38410: Properly handle PySys_Audit() failures (GH-16657) 2020-03-26 12:11:13 +00:00
Furkan Önder cb6534e1a8
bpo-40067: Improve error messages for multiple star expressions in assignments (GH-19168)
Co-Authored-By: Batuhan Taşkaya <isidentical@gmail.com>
Co-Authored-By: Pablo Galindo <Pablogsal@gmail.com>
2020-03-26 01:54:31 +00:00
Victor Stinner 5c3cda0d1a
bpo-39947: Add PyThreadState_GetID() function (GH-19163)
Add PyThreadState_GetID() function: get the unique identifier of a
Python thread state.
2020-03-25 21:23:53 +01:00
Victor Stinner 0e427c6d15
bpo-39947: Add _PyThreadState_GetDict() function (GH-19160) 2020-03-25 21:22:55 +01:00
Victor Stinner 87d3b9db4a
bpo-39882: Add _Py_FatalErrorFormat() function (GH-19157) 2020-03-25 19:27:36 +01:00
Andy Lester 7668a8bc93
Use calloc-based functions, not malloc. (GH-19152) 2020-03-24 23:26:44 -05:00
Victor Stinner 83d46e0622
bpo-40050: Fix importlib._bootstrap_external (GH-19135)
Remove two unused imports: _thread and _weakref. Avoid creating a new
winreg builtin module if it's already available in sys.modules.

The winreg module is now stored as "winreg" rather than "_winreg".
2020-03-24 18:03:34 +01:00
Victor Stinner 5804f878e7
bpo-20526: Fix PyThreadState_Clear(): don't decref frame (GH-19120)
PyThreadState.frame is a borrowed reference, not a strong reference:
PyThreadState_Clear() must not call Py_CLEAR(tstate->frame).

Remove test_threading.test_warnings_at_exit(): we cannot warranty
that the Python thread state of daemon threads is cleared in a
reliable way during Python shutdown.
2020-03-24 16:32:26 +01:00
Serhiy Storchaka bace59d8b8
bpo-39999: Improve compatibility of the ast module. (GH-19056)
* Re-add removed classes Suite, slice, Param, AugLoad and AugStore.
* Add docstrings for dummy classes.
* Add docstrings for attribute aliases.
* Set __module__ to "ast" instead of "_ast".
2020-03-22 20:33:34 +02:00
Ronald Oussoren 044cf94f61
bpo-22490: Remove __PYVENV_LAUNCHER__ from environment during launch (GH-9516)
* bpo-22490: Remove "__PYVENV_LAUNCHER__" from the shell environment on macOS

This changeset removes the environment varialbe "__PYVENV_LAUNCHER__"
during interpreter launch as it is only needed to communicate between
the stub executable in framework installs and the actual interpreter.

Leaving the environment variable present may lead to misbehaviour when
launching other scripts.

* Actually commit the changes for issue 22490...

* Correct typo

Co-Authored-By: Nicola Soranzo <nicola.soranzo@gmail.com>

* Run make patchcheck

Co-authored-by: Jason R. Coombs <jaraco@jaraco.com>
Co-authored-by: Nicola Soranzo <nicola.soranzo@gmail.com>
2020-03-22 14:31:46 -04:00
Dong-hee Na 05e4a296ec
bpo-40024: Add PyModule_AddType() helper function (GH-19088) 2020-03-22 17:17:34 +01:00
Victor Stinner 6723e933c4
bpo-39946: Remove _PyThreadState_GetFrame (GH-19094)
Remove _PyRuntime.getframe hook and remove _PyThreadState_GetFrame
macro which was an alias to _PyRuntime.getframe. They were only
exposed by the internal C API. Remove also PyThreadFrameGetter type.
2020-03-20 17:46:56 +01:00
Victor Stinner fd1e1a18fa
bpo-39947: Add PyThreadState_GetFrame() function (GH-19092)
Add PyThreadState_GetFrame() function: get the current frame
of a Python thread state.
2020-03-20 15:51:45 +01:00
Victor Stinner d83168854e
bpo-40010: Optimize pending calls in multithreaded applications (GH-19091)
If a thread different than the main thread schedules a pending call
(Py_AddPendingCall()), the bytecode evaluation loop is no longer
interrupted at each bytecode instruction to check for pending calls
which cannot be executed. Only the main thread can execute pending
calls.

Previously, the bytecode evaluation loop was interrupted at each
instruction until the main thread executes pending calls.

* Add _Py_ThreadCanHandlePendingCalls() function.
* SIGNAL_PENDING_CALLS() now only sets eval_breaker to 1 if the
  current thread can execute pending calls. Only the main thread can
  execute pending calls.
2020-03-20 14:50:35 +01:00
Victor Stinner d2a8e5b42c
bpo-40010: COMPUTE_EVAL_BREAKER() checks for subinterpreter (GH-19087)
COMPUTE_EVAL_BREAKER() now also checks if the Python thread state
belongs to the main interpreter. Don't break the evaluation loop if
there are pending signals but the Python thread state it belongs to a
subinterpeter.

* Add _Py_IsMainThread() function.
* Add _Py_ThreadCanHandleSignals() function.
2020-03-20 13:38:58 +01:00
Victor Stinner da2914db4b
bpo-40010: Pass tstate to ceval GIL functions (GH-19077)
* Add eval_frame_handle_pending() helper function: cold code path.
* Fix PyEval_ReleaseLock(): don't dereference tstate if it's NULL.
2020-03-20 09:29:08 +01:00
Victor Stinner a36adfa6bb
bpo-39877: 4th take_gil() fix for daemon threads (GH-19080)
bpo-39877, bpo-40010: Add a third tstate_must_exit() check in
take_gil() to prevent using tstate which has been freed.
2020-03-19 19:48:25 +01:00
Victor Stinner 5a3a71dddb
bpo-40010: Optimize signal handling in multithreaded applications (GH-19067)
If a thread different than the main thread gets a signal, the
bytecode evaluation loop is no longer interrupted at each bytecode
instruction to check for pending signals which cannot be handled.
Only the main thread of the main interpreter can handle signals.

Previously, the bytecode evaluation loop was interrupted at each
instruction until the main thread handles signals.

Changes:

* COMPUTE_EVAL_BREAKER() and SIGNAL_PENDING_SIGNALS() no longer set
  eval_breaker to 1 if the current thread cannot handle signals.
* take_gil() now always recomputes eval_breaker.
2020-03-19 17:40:12 +01:00
Batuhan Taşkaya 9052f7a41b
bpo-39562: Allow executing asynchronous comprehensions in the asyncio REPL (GH-18968)
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-03-19 11:35:44 +00:00
Batuhan Taşkaya 0ac59f93c0
bpo-40000: Improve error messages when validating invalid ast.Constant nodes (GH-19055)
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-03-19 11:32:28 +00:00
Victor Stinner 50e6e99178
bpo-39984: Move pending calls to PyInterpreterState (GH-19066)
If Py_AddPendingCall() is called in a subinterpreter, the function is
now scheduled to be called from the subinterpreter, rather than being
called from the main interpreter.

Each subinterpreter now has its own list of scheduled calls.

* Move pending and eval_breaker fields from _PyRuntimeState.ceval
  to PyInterpreterState.ceval.
* new_interpreter() now calls _PyEval_InitThreads() to create
  pending calls lock.
* Fix Py_AddPendingCall() for subinterpreters. It now calls
  _PyThreadState_GET() which works in a subinterpreter if the
  caller holds the GIL, and only falls back on
  PyGILState_GetThisThreadState() if _PyThreadState_GET()
  returns NULL.
2020-03-19 02:41:21 +01:00
Pablo Galindo d112c600ab
bpo-39220: Do not optimise annotation if 'from __future__ import annotations' is used (GH-17866)
Do not apply AST-based optimizations if 'from __future__ import annotations' is used in order to
prevent information lost in the final version of the annotations.
2020-03-18 23:02:09 +00:00
Victor Stinner 8849e5962b
bpo-39984: trip_signal() uses PyGILState_GetThisThreadState() (GH-19061)
bpo-37127, bpo-39984:

* trip_signal() and Py_AddPendingCall() now get the current Python
  thread state using PyGILState_GetThisThreadState() rather than
  _PyRuntimeState_GetThreadState() to be able to get it even if the
  GIL is released.
* _PyEval_SignalReceived() now expects tstate rather than ceval.
* Remove ceval parameter of _PyEval_AddPendingCall(): ceval is now
  get from tstate parameter.
2020-03-18 19:28:53 +01:00
Victor Stinner 56bfdebfb1
bpo-39984: Pass tstate to _PyEval_SignalAsyncExc() (GH-19049)
_PyEval_SignalAsyncExc() and _PyEval_FiniThreads() now expect tstate,
instead of ceval.
2020-03-18 09:26:25 +01:00
Victor Stinner 29356e03d4
bpo-39877: Fix take_gil() for daemon threads (GH-19054)
bpo-39877, bpo-39984: If the thread must exit, don't access tstate to
prevent a potential crash: tstate memory has been freed.
2020-03-18 03:04:33 +01:00
Victor Stinner 23ef89db7a
bpo-39984: _PyThreadState_DeleteCurrent() takes tstate (GH-19051)
* _PyThreadState_DeleteCurrent() now takes tstate rather than
  runtime.
* Add ensure_tstate_not_null() helper to pystate.c.
* Add _PyEval_ReleaseLock() function.
* _PyThreadState_DeleteCurrent() now calls
  _PyEval_ReleaseLock(tstate) and frees PyThreadState memory after
  this call, not before.
* PyGILState_Release(): rename "tcur" variable to "tstate".
2020-03-18 02:26:04 +01:00
Victor Stinner d7fabc1162
bpo-39984: Pass tstate to handle_signals() (GH-19050)
handle_signals() and make_pending_calls() now expect tstate rather
than runtime.
2020-03-18 01:56:21 +01:00
Serhiy Storchaka 6b97598fb6
bpo-39988: Remove ast.AugLoad and ast.AugStore node classes. (GH-19038) 2020-03-17 23:41:08 +02:00
Victor Stinner dab8423d22
bpo-39984: Add PyInterpreterState.ceval (GH-19047)
subinterpreters: Move _PyRuntimeState.ceval.tracing_possible to
PyInterpreterState.ceval.tracing_possible: each interpreter now has
its own variable.

Changes:

* Add _ceval_state structure.
* Add PyInterpreterState.ceval field.
* _PyEval_EvalFrameDefault(): add ceval2 variable (struct _ceval_state*).
* Rename _PyEval_Initialize() to _PyEval_InitRuntimeState().
* Add _PyEval_InitState().
* Don't export internal _Py_FinishPendingCalls() and
  _PyEval_FiniThreads() functions anymore.
2020-03-17 18:56:44 +01:00
Serhiy Storchaka 61cb3d02b8
bpo-39987: Simplify setting lineno in the compiler. (GH-19037) 2020-03-17 18:07:30 +02:00
Victor Stinner f6a5850782
bpo-35370: PyEval_SetTrace() logs unraisable error (GH-18977)
If PySys_Audit() fails in PyEval_SetProfile() or PyEval_SetTrace(),
log the error as an unraisable exception.
2020-03-16 17:41:44 +01:00
Dong-hee Na 87ec86c425
bpo-37207: Add _PyArg_NoKwnames() helper function (GH-18980) 2020-03-16 15:06:20 +01:00
Batuhan Taşkaya 4ab362cec6
bpo-39638: Keep ASDL signatures in the AST nodes (GH-18515) 2020-03-16 10:12:53 +02:00
Batuhan Taşkaya 8689209e03
bpo-39969: Remove ast.Param node class as is no longer used (GH-19020) 2020-03-15 19:32:17 +00:00
Pablo Galindo 90235810ec
bpo-39965: Correctly raise SyntaxError if await is used outside async functions when PyCF_ALLOW_TOP_LEVEL_AWAIT is set (GH-19010) 2020-03-15 04:29:22 +00:00
Hai Shi 57c7810489
bpo-39585: Delete a pending comment in _warning.c (GH-18414) 2020-03-14 15:40:58 +02:00
Victor Stinner 8fb02b6e19
bpo-39947: Add PyThreadState_GetInterpreter() (GH-18981)
Add PyThreadState_GetInterpreter(tstate): get the interpreter of a
Python thread state.
2020-03-13 23:38:08 +01:00