Commit Graph

39 Commits

Author SHA1 Message Date
Radislav Chugunov 69a39bd9ad
gh-105873: Make `_xxsubinterpreters` use exception type name in shared exception (#105874) 2023-07-08 08:44:50 +00:00
Rodolfo M. Pereira 8291ae31dd
GH-103805: Lib test f541 linting issue fix (#103812)
This PR makes some minor linting adjustments to the Lib/test module
caught by [ruff](https://github.com/charliermarsh/ruff). The adjustments
are all related to the `F541 f-string without any placeholders` issue.

Issue: https://github.com/python/cpython/issues/103805

<!-- gh-issue-number: gh-103805 -->
* Issue: gh-103805
<!-- /gh-issue-number -->

---------

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
2023-04-24 23:24:49 +00:00
Alex Waygood 401d7a7f00
gh-102515: Remove unused imports in the `Lib/` directory (#102516) 2023-03-08 11:45:38 +00:00
Eric Snow c67b00534a
gh-101524: Split Up the _xxsubinterpreters Module (gh-101526)
This is step 1 in potentially dropping all the "channel"-related code. Channels have already been removed from PEP 554.

https://github.com/python/cpython/issues/101524
2023-02-03 18:14:43 -07:00
Eric Snow 530cc9dbb6
gh-99741: Implement Multi-Phase Init for the _xxsubinterpreters Module (gh-99742)
_xxsubinterpreters is an internal module used for testing.

https://github.com/python/cpython/issues/99741
2022-12-05 13:40:20 -07:00
Eric Snow 0547a981ae
gh-99741: Clean Up the _xxsubinterpreters Module (gh-99940)
This cleanup up resolves a few subtle bugs and makes the implementation for multi-phase init much cleaner.

https://github.com/python/cpython/issues/99741
2022-12-02 11:36:57 -07:00
Eric Snow 4702552885
gh-98610: Adjust the Optional Restrictions on Subinterpreters (GH-98618)
Previously, the optional restrictions on subinterpreters were: disallow fork, subprocess, and threads.  By default, we were disallowing all three for "isolated" interpreters.  We always allowed all three for the main interpreter and those created through the legacy `Py_NewInterpreter()` API.

Those settings were a bit conservative, so here we've adjusted the optional restrictions to: fork, exec, threads, and daemon threads.  The default for "isolated" interpreters disables fork, exec, and daemon threads.  Regular threads are allowed by default.  We continue always allowing everything For the main interpreter and the legacy API.

In the code, we add `_PyInterpreterConfig.allow_exec` and  `_PyInterpreterConfig.allow_daemon_threads`.  We also add `Py_RTFLAGS_DAEMON_THREADS` and `Py_RTFLAGS_EXEC`.
2022-10-31 12:35:54 -07:00
Victor Stinner 7e9eaad864
Add test.support.busy_retry() (#93770)
Add busy_retry() and sleeping_retry() functions to test.support.
2022-06-15 11:42:10 +02:00
Christian Heimes ef1327e3b6
bpo-40280: Skip more tests on Emscripten (GH-31947)
- lchmod, lchown are not fully implemented
- skip umask tests
- cannot fstat unlinked or renamed files yet
- ignore musl libc issues that affect Emscripten
2022-03-17 12:09:57 +01:00
Christian Clauss 745c9d9dfc
Fix typos in the Lib directory (GH-28775)
Fix typos in the Lib directory as identified by codespell.

Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
2021-10-06 16:13:48 -07:00
Hai Shi 9101b39e67
To fix the random failed test cases of test___xxsubinterpreters in multiprocess. (GH-27240) 2021-07-29 19:05:49 +02:00
Inada Naoki 8bbfeb3330
bpo-43651: PEP 597: Fix EncodingWarning in some tests (GH-25142)
* test__xxsubinterpreters
* test_builtin
* test_doctest
* test_exceptions
* test_opcodes
* test_support
* test_argparse
* test_baseexception
* test_bdb
* test_bool
* test_asdl_parser
2021-04-02 12:53:46 +09:00
Hai Shi f7ba40b505
bpo-40275: Use new test.support helper submodules in tests (GH-20849) 2020-06-25 12:38:51 +02:00
Joannah Nanjekye 6f79838fc1
Skip tests to fix bot (GH-20777)
Co-authored-by: nanjekyejoannah <joannah.nanjekye@ibm.com>
2020-06-21 20:59:43 -03:00
Eric Snow 818f5b597a
bpo-32604: Clean up test.support.interpreters. (gh-20926)
There were some minor adjustments needed and a few tests were missing.

https://bugs.python.org/issue32604
2020-06-16 18:24:40 -06:00
An Long 29c117202e
bpo-40834: Fix truncate when sending str object with channel (GH-20555) 2020-06-13 05:26:01 -07:00
Victor Stinner f2c3b6823b
Revert "bpo-32604: [_xxsubinterpreters] Propagate exceptions. (GH-19768)" (GH-20089)
* Revert "bpo-40613: Remove compiler warning from _xxsubinterpretersmodule (GH-20069)"

This reverts commit fa0a66e62d.

* Revert "bpo-32604: [_xxsubinterpreters] Propagate exceptions. (GH-19768)"

This reverts commit a1d9e0accd.
2020-05-14 18:46:24 +02:00
Eric Snow a1d9e0accd
bpo-32604: [_xxsubinterpreters] Propagate exceptions. (GH-19768)
(Note: PEP 554 is not accepted and the implementation in the code base is a private one for use in the test suite.)

If code running in a subinterpreter raises an uncaught exception then the "run" call in the calling interpreter fails. A RunFailedError is raised there that summarizes the original exception as a string. The actual exception type, __cause__, __context__, state, etc. are all discarded. This turned out to be functionally insufficient in practice. There is a more helpful solution (and PEP 554 has been updated appropriately).

This change adds the exception propagation behavior described in PEP 554 to the _xxsubinterpreters module. With this change a copy of the original exception is set to __cause__ on the RunFailedError. For now we are using "pickle", which preserves the exception's state. We also preserve the original __cause__, __context__, and __traceback__ (since "pickle" does not preserve those).

https://bugs.python.org/issue32604
2020-05-07 08:56:01 -06:00
Victor Stinner 252346acd9
bpo-40453: Add PyConfig._isolated_subinterpreter (GH-19820)
An isolated subinterpreter cannot spawn threads, spawn a child
process or call os.fork().

* Add private _Py_NewInterpreter(isolated_subinterpreter) function.
* Add isolated=True keyword-only parameter to
  _xxsubinterpreters.create().
* Allow again os.fork() in "non-isolated" subinterpreters.
2020-05-01 11:33:44 +02:00
Victor Stinner 17014e4586
Remove dead code in test__xxsubinterpreters (GH-19826) 2020-05-01 02:34:22 +02:00
Lewis Gaul f7bbf58aa9
bpo-38880: List interpreters associated with a channel end (GH-17323)
This PR adds the functionality requested by https://github.com/ericsnowcurrently/multi-core-python/issues/52.

Automerge-Triggered-By: @ericsnowcurrently
2020-04-28 17:18:42 -07:00
Eric Snow 5e8c691594
bpo-32604: Add support for a "default" arg in channel_recv(). (GH-19770)
This allows the caller to avoid creation of an exception when the channel is empty (just like `dict.get()` works).  `ChannelEmptyError` is still raised if no default is provided.

Automerge-Triggered-By: @ericsnowcurrently
2020-04-28 16:11:32 -07:00
Kyle Stanley f03a8f8d50
bpo-37224: Improve test__xxsubinterpreters.DestroyTests (GH-18058)
Adds an additional assertion check based on a race condition for `test__xxsubinterpreters.DestroyTests.test_still_running` discovered in the bpo issue.


https://bugs.python.org/issue37224
2020-01-31 12:07:09 -08:00
Serhiy Storchaka 543a3951a1
bpo-38005: Remove support of string argument in InterpreterID(). (GH-16227)
Make negative interpreter id to raise ValueError instead of RuntimeError.
2019-09-25 18:35:57 +03:00
Serhiy Storchaka bf169915ec
bpo-38005: Fixed comparing and creating of InterpreterID and ChannelID. (GH-15652)
* Fix a crash in comparing with float (and maybe other crashes).
* They are now never equal to strings and non-integer numbers.
* Comparison with a large number no longer raises OverflowError.
* Arbitrary exceptions no longer silenced in constructors and comparisons.
* TypeError raised in the constructor contains now the name of the type.
* Accept only ChannelID and int-like objects in channel functions.
* Accept only InterpreterId, int-like objects and str in the InterpreterId constructor.
* Accept int-like objects, not just int in interpreter related functions.
2019-09-13 22:50:27 +03:00
Victor Stinner 8f4ef3b019
Remove unused imports in tests (GH-14518) 2019-07-01 18:28:25 +02:00
Alexey Izbyshev 16f842da3c bpo-35972: _xxsubinterpreters: Fix potential integer truncation on 32-bit in channel_send() (gh-11822) 2019-02-12 09:06:43 -07:00
Eric Snow 5903296045
bpo-34651: Only allow the main interpreter to fork. (gh-9279)
When os.fork() is called (on platforms that support it) all threads but the current one are destroyed in the child process. Consequently we must ensure that all but the associated interpreter are likewise destroyed. The main interpreter is critical for runtime operation, so we must ensure that fork only happens in the main interpreter.

https://bugs.python.org/issue34651
2018-09-14 14:17:20 -07:00
Eric Snow ab4a1988fd
bpo-33615: Re-enable subinterpreter tests. (#7552)
All the subinterpreter tests were disabled in gh-7513. This commit re-enables them, but leaves one bad test disabled. The test is partly causing problems because it makes assumptions about the availability of a high-level interpreters module (see PEP 554). So I'm disabling the test until such a high-level module is available.
2018-06-13 08:02:39 -06:00
Victor Stinner c4f3cb772b
bpo-33615: Skip test__xxsubinterpreters (GH-7503)
The test does crash on multiple CIs causing many troubles. For
example, the test prevents to get results of the two Refleak 3.x
buildbots.
2018-06-08 02:28:28 +02:00
Eric Snow 63799136e6
bpo-33615: Re-enable a subinterpreter test. (gh-7251)
For bpo-32604 I added extra subinterpreter-related tests (see #6914), which caused a few buildbots to crash. This patch fixes the crash by ensuring that refcounts in channels are handled properly.
2018-06-01 18:45:20 -06:00
Eric Snow 110bc01407
bpo-33615: Temporarily disable a test that is triggering crashes on a few buildbots. (gh-7288)
For bpo-32604 I added some subinterpreter-related tests (see #6914) that are causing crashes on a few buildbots. I'm working on fixing the crashes (see #7251).  This change temporarily disables the triggering test.
2018-05-31 10:17:24 -06:00
Eric Snow 3ab0136ac5
bpo-32604: Implement force-closing channels. (gh-6937)
This will make it easier to clean up channels (e.g. when used in tests).
2018-05-17 10:27:09 -04:00
Eric Snow 6d2cd9036c
bpo-32604: Improve subinterpreter tests. (#6914)
Add more tests for subinterpreters. This patch also fixes a few small defects in the channel implementation.
2018-05-16 15:04:57 -04:00
Ville Skyttä 61f82e0e33 Spelling fixes to docs, docstrings, and comments (GH-6374) 2018-04-20 16:08:45 -04:00
Eric Snow f53d9f2778
bpo-32604: Swap threads only if the interpreter is different. (gh-5778)
The CPython runtime assumes that there is a one-to-one relationship (for a given interpreter) between PyThreadState and OS threads. Sending and receiving on a channel in the same interpreter was causing crashes because of this (specifically due to a check in PyThreadState_Swap()). The solution is to not switch threads if the interpreter is the same.
2018-02-20 16:30:17 -07:00
Eric Snow 4c6955e2b0
bpo-32604: Clean up created subinterpreters before runtime finalization. (gh-5709) 2018-02-16 18:53:40 -07:00
Eric Snow 4e9da0d163
bpo-32604: Fix memory leaks in the new _xxsubinterpreters module. (#5507) 2018-02-02 21:49:49 -07:00
Eric Snow 7f8bfc9b9a
bpo-32604: Expose the subinterpreters C-API in a "private" stdlib module. (gh-1748)
The module is primarily intended for internal use in the test suite.  Building the module under Windows will come in a follow-up PR.
2018-01-29 18:23:44 -07:00