Commit Graph

94 Commits

Author SHA1 Message Date
Mark Shannon ae0a2b7562
bpo-44590: Lazily allocate frame objects (GH-27077)
* Convert "specials" array to InterpreterFrame struct, adding f_lasti, f_state and other non-debug FrameObject fields to it.

* Refactor, calls pushing the call to the interpreter upward toward _PyEval_Vector.

* Compute f_back when on thread stack, only filling in value when frame object outlives stack invocation.

* Move ownership of InterpreterFrame in generator from frame object to generator object.

* Do not create frame objects for Python calls.

* Do not create frame objects for generators.
2021-07-26 11:22:16 +01:00
Victor Stinner 3bb09947ec
bpo-43916: Add Py_TPFLAGS_DISALLOW_INSTANTIATION type flag (GH-25721)
Add a new Py_TPFLAGS_DISALLOW_INSTANTIATION type flag to disallow
creating type instances: set tp_new to NULL and don't create the
"__new__" key in the type dictionary.

The flag is set automatically on static types if tp_base is NULL or
&PyBaseObject_Type and tp_new is NULL.

Use the flag on the following types:

* _curses.ncurses_version type
* _curses_panel.panel
* _tkinter.Tcl_Obj
* _tkinter.tkapp
* _tkinter.tktimertoken
* _xxsubinterpretersmodule.ChannelID
* sys.flags type
* sys.getwindowsversion() type
* sys.version_info type

Update MyStr example in the C API documentation to use
Py_TPFLAGS_DISALLOW_INSTANTIATION.

Add _PyStructSequence_InitType() function to create a structseq type
with the Py_TPFLAGS_DISALLOW_INSTANTIATION flag set.

type_new() calls _PyType_CheckConsistency() at exit.
2021-04-30 12:46:15 +02:00
Steve Dower 7b86e47617
bpo-43472: Ensure PyInterpreterState_New audit events are raised when called through _xxsubinterpreters module (GH-25506) 2021-04-21 23:34:29 +01:00
Mark Shannon cb9879b948
bpo-40941: Unify implicit and explicit state in the frame and generator objects into a single value. (GH-20803)
* Merge gen and frame state variables into one.

* Replace stack pointer with depth in PyFrameObject. Makes code easier to read and saves a word of memory.
2020-07-17 11:44:23 +01:00
Victor Stinner f2c3b6823b
Revert "bpo-32604: [_xxsubinterpreters] Propagate exceptions. (GH-19768)" (GH-20089)
* Revert "bpo-40613: Remove compiler warning from _xxsubinterpretersmodule (GH-20069)"

This reverts commit fa0a66e62d.

* Revert "bpo-32604: [_xxsubinterpreters] Propagate exceptions. (GH-19768)"

This reverts commit a1d9e0accd.
2020-05-14 18:46:24 +02:00
Dong-hee Na fa0a66e62d
bpo-40613: Remove compiler warning from _xxsubinterpretersmodule (GH-20069) 2020-05-13 22:38:27 +09:00
Eric Snow a1d9e0accd
bpo-32604: [_xxsubinterpreters] Propagate exceptions. (GH-19768)
(Note: PEP 554 is not accepted and the implementation in the code base is a private one for use in the test suite.)

If code running in a subinterpreter raises an uncaught exception then the "run" call in the calling interpreter fails. A RunFailedError is raised there that summarizes the original exception as a string. The actual exception type, __cause__, __context__, state, etc. are all discarded. This turned out to be functionally insufficient in practice. There is a more helpful solution (and PEP 554 has been updated appropriately).

This change adds the exception propagation behavior described in PEP 554 to the _xxsubinterpreters module. With this change a copy of the original exception is set to __cause__ on the RunFailedError. For now we are using "pickle", which preserves the exception's state. We also preserve the original __cause__, __context__, and __traceback__ (since "pickle" does not preserve those).

https://bugs.python.org/issue32604
2020-05-07 08:56:01 -06:00
Victor Stinner fb2c7c4afb
bpo-40513: _xxsubinterpreters.run_string() releases the GIL (GH-19944)
In the experimental isolated subinterpreters build mode,
_xxsubinterpreters.run_string() now releases the GIL.
2020-05-05 20:33:06 +02:00
Victor Stinner 252346acd9
bpo-40453: Add PyConfig._isolated_subinterpreter (GH-19820)
An isolated subinterpreter cannot spawn threads, spawn a child
process or call os.fork().

* Add private _Py_NewInterpreter(isolated_subinterpreter) function.
* Add isolated=True keyword-only parameter to
  _xxsubinterpreters.create().
* Allow again os.fork() in "non-isolated" subinterpreters.
2020-05-01 11:33:44 +02:00
Victor Stinner 4386b9045e
bpo-40429: PyThreadState_GetFrame() returns a strong ref (GH-19781)
The PyThreadState_GetFrame() function now returns a strong reference
to the frame.
2020-04-29 03:01:43 +02:00
Lewis Gaul f7bbf58aa9
bpo-38880: List interpreters associated with a channel end (GH-17323)
This PR adds the functionality requested by https://github.com/ericsnowcurrently/multi-core-python/issues/52.

Automerge-Triggered-By: @ericsnowcurrently
2020-04-28 17:18:42 -07:00
Eric Snow 5e8c691594
bpo-32604: Add support for a "default" arg in channel_recv(). (GH-19770)
This allows the caller to avoid creation of an exception when the channel is empty (just like `dict.get()` works).  `ChannelEmptyError` is still raised if no default is provided.

Automerge-Triggered-By: @ericsnowcurrently
2020-04-28 16:11:32 -07:00
Victor Stinner 1a1bd2e238
bpo-40302: Replace PY_INT64_T with int64_t (GH-19573)
* Replace PY_INT64_T with int64_t
* Replace PY_UINT32_T with uint32_t
* Replace PY_UINT64_T with uint64_t

sha3module.c no longer checks if PY_UINT64_T is defined since it's
always defined and uint64_t is always available on platforms
supported by Python.
2020-04-17 19:13:06 +02:00
Victor Stinner 3072338642
bpo-39947: Use PyThreadState_GetFrame() (GH-19159)
_tracemalloc.c and _xxsubinterpretersmodule.c use
PyThreadState_GetFrame() and PyThreadState_GetInterpreter() to no
longer depend on the PyThreadState structure.
2020-03-25 19:52:02 +01:00
Victor Stinner be79373a78
bpo-39947: Add PyInterpreterState_Get() function (GH-18979)
* Rename _PyInterpreterState_Get() to PyInterpreterState_Get() and
  move it the limited C API.
* Add _PyInterpreterState_Get() alias to PyInterpreterState_Get() for
  backward compatibility with Python 3.8.
2020-03-13 18:15:33 +01:00
Victor Stinner daa9756cb6
bpo-39573: Use Py_TYPE() macro in Modules directory (GH-18393)
Replace direct access to PyObject.ob_type with Py_TYPE().
2020-02-07 03:37:06 +01:00
Alex Henrie 5cae042f68 closes bpo-39262: Use specific out-of-memory message in _sharedexception_bind. (GH-17908) 2020-01-08 18:48:52 -08:00
Victor Stinner 9da7430675
bpo-36854: Clear the current thread later (GH-17279)
Clear the current thread later in the Python finalization.

* The PyInterpreterState_Delete() function is now responsible
  to call PyThreadState_Swap(NULL).
* The tstate_delete_common() function is now responsible to clear the
  "autoTSSKey" thread local storage and it only clears it once the
  thread state is fully cleared. It allows to still get the current
  thread from TSS in tstate_delete_common().
2019-11-20 11:17:17 +01:00
Serhiy Storchaka bf169915ec
bpo-38005: Fixed comparing and creating of InterpreterID and ChannelID. (GH-15652)
* Fix a crash in comparing with float (and maybe other crashes).
* They are now never equal to strings and non-integer numbers.
* Comparison with a large number no longer raises OverflowError.
* Arbitrary exceptions no longer silenced in constructors and comparisons.
* TypeError raised in the constructor contains now the name of the type.
* Accept only ChannelID and int-like objects in channel functions.
* Accept only InterpreterId, int-like objects and str in the InterpreterId constructor.
* Accept int-like objects, not just int in interpreter related functions.
2019-09-13 22:50:27 +03:00
Jeroen Demeyer 530f506ac9 bpo-36974: tp_print -> tp_vectorcall_offset and tp_reserved -> tp_as_async (GH-13464)
Automatically replace
tp_print -> tp_vectorcall_offset
tp_compare -> tp_as_async
tp_reserved -> tp_as_async
2019-05-30 19:13:39 -07:00
Zackery Spytz 1a2252ed39 bpo-36594: Fix incorrect use of %p in format strings (GH-12769)
In addition, fix some other minor violations of C99.
2019-05-06 12:56:50 -04:00
Eric Snow c11183cdcf
bpo-36097: Use only public C-API in the_xxsubinterpreters module (adding as necessary). (gh-12359) 2019-03-15 16:35:46 -06:00
Victor Stinner 4d61e6e3b8
Revert: bpo-33608: Factor out a private, per-interpreter _Py_AddPendingCall(). (GH-11617) (GH-12159)
* Revert "bpo-36097: Use only public C-API in the_xxsubinterpreters module (adding as necessary). (#12003)"

This reverts commit bcfa450f21.

* Revert "bpo-33608: Simplify ceval's DISPATCH by hoisting eval_breaker ahead of time. (gh-12062)"

This reverts commit bda918bf65.

* Revert "bpo-33608: Use _Py_AddPendingCall() in _PyCrossInterpreterData_Release(). (gh-12024)"

This reverts commit b05b711a2c.

* Revert "bpo-33608: Factor out a private, per-interpreter _Py_AddPendingCall(). (GH-11617)"

This reverts commit ef4ac967e2.
2019-03-04 14:21:28 +01:00
Eric Snow bcfa450f21
bpo-36097: Use only public C-API in the_xxsubinterpreters module (adding as necessary). (#12003) 2019-03-01 16:50:31 -07:00
Alexey Izbyshev 36433221f0 bpo-35984: _xxsubinterpreters: Fix memory leak in _channel_send() (GH-11845)
https://bugs.python.org/issue35984
2019-02-15 14:28:58 -08:00
Serhiy Storchaka 62be74290a
bpo-33012: Fix invalid function cast warnings with gcc 8. (GH-6749)
Fix invalid function cast warnings with gcc 8
for method conventions different from METH_NOARGS, METH_O and
METH_VARARGS excluding Argument Clinic generated code.
2018-11-27 13:27:31 +02:00
Victor Stinner 621cebe81b
bpo-35081: Rename internal headers (GH-10275)
Rename Include/internal/ headers:

* pycore_hash.h -> pycore_pyhash.h
* pycore_lifecycle.h -> pycore_pylifecycle.h
* pycore_mem.h -> pycore_pymem.h
* pycore_state.h -> pycore_pystate.h

Add missing headers to Makefile.pre.in and PCbuild:

* pycore_condvar.h.
* pycore_hamt.h
* pycore_pyhash.h
2018-11-12 16:53:38 +01:00
Victor Stinner 27e2d1f219
bpo-35081: Add pycore_ prefix to internal header files (GH-10263)
* Rename Include/internal/ header files:

  * pyatomic.h -> pycore_atomic.h
  * ceval.h -> pycore_ceval.h
  * condvar.h -> pycore_condvar.h
  * context.h -> pycore_context.h
  * pygetopt.h -> pycore_getopt.h
  * gil.h -> pycore_gil.h
  * hamt.h -> pycore_hamt.h
  * hash.h -> pycore_hash.h
  * mem.h -> pycore_mem.h
  * pystate.h -> pycore_state.h
  * warnings.h -> pycore_warnings.h

* PCbuild project, Makefile.pre.in, Modules/Setup: add the
  Include/internal/ directory to the search paths of header files.
* Update includes. For example, replace #include "internal/mem.h"
  with #include "pycore_mem.h".
2018-11-01 00:52:28 +01:00
Peter Eisentraut 0e0bc4e221 Fix misleading mentions of tp_size in comments (GH-9093)
Many type object initializations labeled a field "tp_size" in the
comment, but the name of that field is tp_basicsize.
2018-09-10 09:46:08 -07:00
Alexey Izbyshev c583919ffc bpo-34462: Add missing NULL check to _copy_raw_string() (GH-8863)
Reported by Svace static analyzer.
2018-08-22 21:27:32 +03:00
Victor Stinner caba55b3b7
bpo-34301: Add _PyInterpreterState_Get() helper function (GH-8592)
sys_setcheckinterval() now uses a local variable to parse arguments,
before writing into interp->check_interval.
2018-08-03 15:33:52 +02:00
Benjamin Peterson 4629c0d531
Hide some symbols from _xxsubinterpreters. (GH-8151) 2018-07-06 23:28:35 -07:00
Eric Snow ab4a1988fd
bpo-33615: Re-enable subinterpreter tests. (#7552)
All the subinterpreter tests were disabled in gh-7513. This commit re-enables them, but leaves one bad test disabled. The test is partly causing problems because it makes assumptions about the availability of a high-level interpreters module (see PEP 554). So I'm disabling the test until such a high-level module is available.
2018-06-13 08:02:39 -06:00
Eric Snow 63799136e6
bpo-33615: Re-enable a subinterpreter test. (gh-7251)
For bpo-32604 I added extra subinterpreter-related tests (see #6914), which caused a few buildbots to crash. This patch fixes the crash by ensuring that refcounts in channels are handled properly.
2018-06-01 18:45:20 -06:00
Eric Snow 6854e803b7
bpo-33724: Use the right format code for int64_t in subinterpreters code. (gh-7330) 2018-06-01 16:26:01 -06:00
Eric Snow 3ab0136ac5
bpo-32604: Implement force-closing channels. (gh-6937)
This will make it easier to clean up channels (e.g. when used in tests).
2018-05-17 10:27:09 -04:00
Eric Snow 6d2cd9036c
bpo-32604: Improve subinterpreter tests. (#6914)
Add more tests for subinterpreters. This patch also fixes a few small defects in the channel implementation.
2018-05-16 15:04:57 -04:00
Serhiy Storchaka d8dcd57edb
bpo-33454: Fix arguments parsing in _xxsubinterpreters.channel_close(). (GH-6747) 2018-05-16 01:08:09 +03:00
Siddhesh Poyarekar 55edd0c185 bpo-33012: Fix invalid function cast warnings with gcc 8 for METH_NOARGS. (GH-6030)
METH_NOARGS functions need only a single argument but they are cast
into a PyCFunction, which takes two arguments.  This triggers an
invalid function cast warning in gcc8 due to the argument mismatch.
Fix this by adding a dummy unused argument.
2018-04-29 21:59:33 +03:00
Eric Snow f53d9f2778
bpo-32604: Swap threads only if the interpreter is different. (gh-5778)
The CPython runtime assumes that there is a one-to-one relationship (for a given interpreter) between PyThreadState and OS threads. Sending and receiving on a channel in the same interpreter was causing crashes because of this (specifically due to a check in PyThreadState_Swap()). The solution is to not switch threads if the interpreter is the same.
2018-02-20 16:30:17 -07:00
Eric Snow 4c6955e2b0
bpo-32604: Clean up created subinterpreters before runtime finalization. (gh-5709) 2018-02-16 18:53:40 -07:00
Eric Snow 4e9da0d163
bpo-32604: Fix memory leaks in the new _xxsubinterpreters module. (#5507) 2018-02-02 21:49:49 -07:00
Eric Snow 83e64c8a54
bpo-32604: NULL-terminate kwlist in channel_drop_interpreter(). (gh-5437) 2018-01-29 21:04:15 -07:00
Eric Snow 7f8bfc9b9a
bpo-32604: Expose the subinterpreters C-API in a "private" stdlib module. (gh-1748)
The module is primarily intended for internal use in the test suite.  Building the module under Windows will come in a follow-up PR.
2018-01-29 18:23:44 -07:00