Commit Graph

6433 Commits

Author SHA1 Message Date
Serhiy Storchaka 6b97598fb6
bpo-39988: Remove ast.AugLoad and ast.AugStore node classes. (GH-19038) 2020-03-17 23:41:08 +02:00
Victor Stinner dab8423d22
bpo-39984: Add PyInterpreterState.ceval (GH-19047)
subinterpreters: Move _PyRuntimeState.ceval.tracing_possible to
PyInterpreterState.ceval.tracing_possible: each interpreter now has
its own variable.

Changes:

* Add _ceval_state structure.
* Add PyInterpreterState.ceval field.
* _PyEval_EvalFrameDefault(): add ceval2 variable (struct _ceval_state*).
* Rename _PyEval_Initialize() to _PyEval_InitRuntimeState().
* Add _PyEval_InitState().
* Don't export internal _Py_FinishPendingCalls() and
  _PyEval_FiniThreads() functions anymore.
2020-03-17 18:56:44 +01:00
Serhiy Storchaka 61cb3d02b8
bpo-39987: Simplify setting lineno in the compiler. (GH-19037) 2020-03-17 18:07:30 +02:00
Victor Stinner f6a5850782
bpo-35370: PyEval_SetTrace() logs unraisable error (GH-18977)
If PySys_Audit() fails in PyEval_SetProfile() or PyEval_SetTrace(),
log the error as an unraisable exception.
2020-03-16 17:41:44 +01:00
Dong-hee Na 87ec86c425
bpo-37207: Add _PyArg_NoKwnames() helper function (GH-18980) 2020-03-16 15:06:20 +01:00
Batuhan Taşkaya 4ab362cec6
bpo-39638: Keep ASDL signatures in the AST nodes (GH-18515) 2020-03-16 10:12:53 +02:00
Batuhan Taşkaya 8689209e03
bpo-39969: Remove ast.Param node class as is no longer used (GH-19020) 2020-03-15 19:32:17 +00:00
Pablo Galindo 90235810ec
bpo-39965: Correctly raise SyntaxError if await is used outside async functions when PyCF_ALLOW_TOP_LEVEL_AWAIT is set (GH-19010) 2020-03-15 04:29:22 +00:00
Hai Shi 57c7810489
bpo-39585: Delete a pending comment in _warning.c (GH-18414) 2020-03-14 15:40:58 +02:00
Victor Stinner 8fb02b6e19
bpo-39947: Add PyThreadState_GetInterpreter() (GH-18981)
Add PyThreadState_GetInterpreter(tstate): get the interpreter of a
Python thread state.
2020-03-13 23:38:08 +01:00
Victor Stinner be79373a78
bpo-39947: Add PyInterpreterState_Get() function (GH-18979)
* Rename _PyInterpreterState_Get() to PyInterpreterState_Get() and
  move it the limited C API.
* Add _PyInterpreterState_Get() alias to PyInterpreterState_Get() for
  backward compatibility with Python 3.8.
2020-03-13 18:15:33 +01:00
Victor Stinner ff4584caca
bpo-39947: Use _PyInterpreterState_GET_UNSAFE() (GH-18978)
Replace _PyInterpreterState_Get() function call with
_PyInterpreterState_GET_UNSAFE() macro which is more efficient but
don't check if tstate or interp is NULL.

_Py_GetConfigsAsDict() now uses _PyThreadState_GET().
2020-03-13 18:03:56 +01:00
Victor Stinner 309d7cc5df
bpo-35370: Add _PyEval_SetTrace() function (GH-18975)
* sys.settrace(), sys.setprofile() and _lsprof.Profiler.enable() now
  properly report PySys_Audit() error if "sys.setprofile" or
  "sys.settrace" audit event is denied.
* Add _PyEval_SetProfile() and _PyEval_SetTrace() function: similar
  to PyEval_SetProfile() and PyEval_SetTrace() but take a tstate
  parameter and return -1 on error.
* Add _PyObject_FastCallTstate() function.
2020-03-13 16:39:12 +01:00
Victor Stinner 0b72b23fb0
bpo-38500: Add _PyInterpreterState_SetEvalFrameFunc() (GH-17340)
PyInterpreterState.eval_frame function now requires a tstate (Python
thread state) parameter.

Add private functions to the C API to get and set the frame
evaluation function:

* Add tstate parameter to _PyFrameEvalFunction function type.
* Add _PyInterpreterState_GetEvalFrameFunc() and
  _PyInterpreterState_SetEvalFrameFunc() functions.
* Add tstate parameter to _PyEval_EvalFrameDefault().
2020-03-12 23:18:39 +01:00
Andy Lester 75cd5bf428
Remove unused args from two pylifecycle.c functions (GH-18867)
* Remove unused tstate argument from init_set_builtins_open()
* Replace fd (int) with tstate (PyThreadState *) in in _Py_FatalError_PrintExc()
2020-03-12 08:49:05 +01:00
bobince 64838ce717
bpo-39847: EnterNonRecursiveMutex() uses GetTickCount64() (GH-18780)
The 32-bit (49-day) TickCount relied on in EnterNonRecursiveMutex can overflow
in the gap between the 'target' time and the 'now' time WaitForSingleObjectEx
returns, causing the loop to think it needs to wait another 49 days. This is
most likely to happen when the machine is hibernated during
WaitForSingleObjectEx.

This makes acquiring a lock/event/etc from the _thread or threading module
appear to never timeout.

Replace with GetTickCount64 - this is OK now Python no longer supports XP which
lacks it, and is in use for time.monotonic().

Co-authored-by: And Clover <and.clover@bromium.com>
2020-03-12 00:39:02 +01:00
Andy Lester 76d5877b72
closes bpo-39922: Remove unused args from four functions. (GH-18893) 2020-03-10 19:18:12 -07:00
Victor Stinner 4e53abb0f4
bpo-38631: _PyGILState_Init() returns PyStatus (GH-18908)
_PyGILState_Init() now returns PyStatus rather than calling
Py_FatalError() on failure.
2020-03-10 23:49:16 +01:00
Serhiy Storchaka 13d52c2686
bpo-34822: Simplify AST for subscription. (GH-9605)
* Remove the slice type.
* Make Slice a kind of the expr type instead of the slice type.
* Replace ExtSlice(slices) with Tuple(slices, Load()).
* Replace Index(value) with a value itself.

All non-terminal nodes in AST for expressions are now of the expr type.
2020-03-10 18:52:34 +02:00
Victor Stinner 8510f43078
bpo-1294959: Add sys.platlibdir attribute (GH-18381)
Add --with-platlibdir option to the configure script: name of the
platform-specific library directory, stored in the new sys.platlitdir
attribute. It is used to build the path of platform-specific dynamic
libraries and the path of the standard library.

It is equal to "lib" on most platforms. On Fedora and SuSE, it is
equal to "lib64" on 64-bit systems.

Co-Authored-By: Jan Matějek <jmatejek@suse.com>
Co-Authored-By: Matěj Cepl <mcepl@cepl.eu>
Co-Authored-By: Charalampos Stratakis <cstratak@redhat.com>
2020-03-10 09:53:09 +01:00
Victor Stinner b4698ecfdb
bpo-39877: Deprecate PyEval_InitThreads() (GH-18892)
Deprecated PyEval_InitThreads() and PyEval_ThreadsInitialized().
Calling PyEval_InitThreads() now does nothing.
2020-03-10 01:28:54 +01:00
Victor Stinner 175a704abf
bpo-39877: PyGILState_Ensure() don't call PyEval_InitThreads() (GH-18891)
PyGILState_Ensure() doesn't call PyEval_InitThreads() anymore when a
new Python thread state is created. The GIL is created by
Py_Initialize() since Python 3.7, it's not needed to call
PyEval_InitThreads() explicitly.

Add an assertion to ensure that the GIL is already created.
2020-03-10 00:37:48 +01:00
Victor Stinner 9ad58acbe8
bpo-19466: Py_Finalize() clears daemon threads earlier (GH-18848)
Clear the frames of daemon threads earlier during the Python shutdown to
call objects destructors. So "unclosed file" resource warnings are now
emitted for daemon threads in a more reliable way.

Cleanup _PyThreadState_DeleteExcept() code: rename "garbage" to
"list".
2020-03-09 23:37:49 +01:00
Victor Stinner 9229eeee10
bpo-39877: take_gil() checks tstate_must_exit() twice (GH-18890)
take_gil() now also checks tstate_must_exit() after acquiring
the GIL: exit the thread if Py_Finalize() has been called.
2020-03-09 23:10:53 +01:00
Serhiy Storchaka b7e9525f9c
bpo-36287: Make ast.dump() not output optional fields and attributes with default values. (GH-18843)
The default values for optional fields and attributes of AST nodes are now set
as class attributes (e.g. Constant.kind is set to None).
2020-03-10 00:07:47 +02:00
Victor Stinner 85f5a69ae1
bpo-39877: Refactor take_gil() function (GH-18885)
* Remove ceval parameter of take_gil(): get it from tstate.
* Move exit_thread_if_finalizing() call inside take_gil(). Replace
  exit_thread_if_finalizing() with tstate_must_exit(): the caller is
  now responsible to call PyThread_exit_thread().
* Move is_tstate_valid() assertion inside take_gil(). Remove
  is_tstate_valid(): inline code into take_gil().
* Move gil_created() assertion inside take_gil().
2020-03-09 22:12:04 +01:00
Victor Stinner 111e4ee52a
bpo-39877: Py_Initialize() pass tstate to PyEval_InitThreads() (GH-18884) 2020-03-09 21:24:14 +01:00
Victor Stinner 3225b9f973
bpo-39877: Remove useless PyEval_InitThreads() calls (GH-18883)
Py_Initialize() calls PyEval_InitThreads() since Python 3.7. It's no
longer needed to call it explicitly.
2020-03-09 20:56:57 +01:00
Serhiy Storchaka eebaa9bfc5
bpo-38249: Expand Py_UNREACHABLE() to __builtin_unreachable() in the release mode. (GH-16329)
Co-authored-by: Victor Stinner <vstinner@python.org>
2020-03-09 20:49:52 +02:00
idomic fc72ab6913
bpo-38691: importlib ignores PYTHONCASEOK if -E is used (GH-18627)
The importlib module now ignores the PYTHONCASEOK
environment variable when the -E or -I command line
options are being used.
2020-03-09 12:57:53 +01:00
Andy Lester 28ca43b7e3
closes bpo-39898: Remove unused arg from append_formattedvalue. (GH-18840) 2020-03-08 09:53:59 -07:00
Victor Stinner eb4e2ae2b8
bpo-39877: Fix PyEval_RestoreThread() for daemon threads (GH-18811)
* exit_thread_if_finalizing() does now access directly _PyRuntime
  variable, rather than using tstate->interp->runtime since tstate
  can be a dangling pointer after Py_Finalize() has been called.
* exit_thread_if_finalizing() is now called *before* calling
  take_gil(). _PyRuntime.finalizing is an atomic variable,
  we don't need to hold the GIL to access it.
* Add ensure_tstate_not_null() function to check that tstate is not
  NULL at runtime. Check tstate earlier. take_gil() does not longer
  check if tstate is NULL.

Cleanup:

* PyEval_RestoreThread() no longer saves/restores errno: it's already
  done inside take_gil().
* PyEval_AcquireLock(), PyEval_AcquireThread(),
  PyEval_RestoreThread() and _PyEval_EvalFrameDefault() now check if
  tstate is valid with the new is_tstate_valid() function which uses
  _PyMem_IsPtrFreed().
2020-03-08 11:57:45 +01:00
Brandt Bucher d5aa2e941c
bpo-39890: Don't mutate the AST when compiling starred assignments (GH-18833) 2020-03-08 03:44:18 +00:00
Andy Lester aa450a0364
closes bpo-39886: Remove unused arg from config_get_stdio_errors. (GH-18823) 2020-03-07 09:36:04 -08:00
Andy Lester ad0c775ea2
closes bpo-39878: Remove unused arguments from static functions. (GH-18822)
calc_number_widths -> PyObject *number

fill_number -> Py_ssize_t d_end
2020-03-07 09:29:10 -08:00
Victor Stinner 9e5d30cc99
bpo-39882: Py_FatalError() logs the function name (GH-18819)
The Py_FatalError() function is replaced with a macro which logs
automatically the name of the current function, unless the
Py_LIMITED_API macro is defined.

Changes:

* Add _Py_FatalErrorFunc() function.
* Remove the function name from the message of Py_FatalError() calls
  which included the function name.
* Update tests.
2020-03-07 00:54:20 +01:00
Victor Stinner 7b3c252dc7
bpo-39877: _PyRuntimeState.finalizing becomes atomic (GH-18816)
Convert _PyRuntimeState.finalizing field to an atomic variable:

* Rename it to _finalizing
* Change its type to _Py_atomic_address
* Add _PyRuntimeState_GetFinalizing() and _PyRuntimeState_SetFinalizing()
  functions
* Remove _Py_CURRENTLY_FINALIZING() function: replace it with testing
  directly _PyRuntimeState_GetFinalizing() value

Convert _PyRuntimeState_GetThreadState() to static inline function.
2020-03-07 00:24:23 +01:00
Andy Lester 557287075c
bpo-39573: Use Py_IS_TYPE() macro to check for types (GH-18809)
Co-authored-by: Victor Stinner <vstinner@python.org>
2020-03-06 23:53:17 +01:00
Andy Lester 9566842e89
closes bpo-39872: Remove unused args from symtable_exit_block and symtable_visit_annotations. (GH-18800) 2020-03-06 07:46:04 -08:00
Andy Lester da4d656e95
closes bpo-39870: Remove unused arg from sys_displayhook_unencodable. (GH-18796)
Also move int err to its innermost scope.
2020-03-05 20:34:36 -08:00
Victor Stinner efc28bbbee
Add a comment to _Py_RestoreSignals() (GH-18792)
subprocess _posix_spawn() should stay in sync with _Py_RestoreSignals().
2020-03-05 18:13:56 +01:00
Batuhan Taşkaya d82e469048
bpo-39639: Remove the AST "Suite" node and associated code (GH-18513)
The AST "Suite" node is no longer used and it can be removed from the ASDL definition and related structures (compiler, visitors, ...).

Co-Authored-By: Victor Stinner <vstinner@python.org>
Co-authored-by: Brett Cannon <54418+brettcannon@users.noreply.github.com>
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-03-04 16:16:46 +00:00
Andy Lester dffe4c0709
bpo-39573: Finish converting to new Py_IS_TYPE() macro (GH-18601) 2020-03-04 14:15:20 +01:00
Brandt Bucher be501ca241
bpo-39702: Relax grammar restrictions on decorators (PEP 614) (GH-18570) 2020-03-03 14:25:44 -08:00
Serhiy Storchaka ae75a29435
bpo-39831: Remove outdated comment. (GH-18764) 2020-03-03 19:43:29 +02:00
Armin Rigo 6daa37fd42
bpo-38091: Import deadlock detection causes deadlock (GH-17518)
Automerge-Triggered-By: @brettcannon
2020-03-02 17:37:25 -08:00
Stefan Krah b3b9ade4a3
bpo-39776: Lock ++interp->tstate_next_unique_id. (GH-18746) (#18746)
- Threads created by PyGILState_Ensure() could have a duplicate tstate->id.
2020-03-02 21:22:36 +01:00
Serhiy Storchaka 2d2f85517f
bpo-39831: Fix a reference leak in PyErr_WarnEx(). (GH-18750) 2020-03-02 22:05:08 +02:00
Victor Stinner 66b7973c1b
bpo-39796: Fix _warnings module initialization (GH-18739)
* Add _PyWarnings_InitState() which only initializes the _warnings
  module state (tstate->interp->warnings) without creating a module
  object
* Py_InitializeFromConfig() now calls _PyWarnings_InitState() instead
  of _PyWarnings_Init()
* Rename also private functions of _warnings.c to avoid confusion
  between the public C API and the private C API.
2020-03-02 15:02:18 +01:00
Serhiy Storchaka 28d0bcac8b
bpo-38913: Fix segfault in Py_BuildValue("(s#O)", ...) if entered with exception raised. (GH-18656) 2020-03-02 08:42:39 +02:00