Commit Graph

1250 Commits

Author SHA1 Message Date
Eric Snow a9c6e0618f
gh-99113: Add Py_MOD_PER_INTERPRETER_GIL_SUPPORTED (gh-104205)
Here we are doing no more than adding the value for Py_mod_multiple_interpreters and using it for stdlib modules.  We will start checking for it in gh-104206 (once PyInterpreterState.ceval.own_gil is added in gh-104204).
2023-05-05 21:11:27 +00:00
Dong-hee Na e5b8b19d99
gh-104106: Add gcc fallback of mkfifoat/mknodat for macOS (gh-104129) 2023-05-05 12:27:25 +09:00
Finn Womack b701dce340
gh-102765: Update ntpath.isdir/isfile/islink/exists to use GetFileInformationByName when available (GH-103485) 2023-04-27 14:23:26 +00:00
Nikita Sobolev 119f67de08
gh-103167: Fix `-Wstrict-prototypes` warnings by using `(void)` for functions with no args (GH-103168) 2023-04-05 09:22:33 +02:00
Steve Dower 0f175766e2
gh-99726: Improves correctness of stat results for Windows, and uses faster API when available (GH-102149)
This deprecates `st_ctime` fields on Windows, with the intent to change them to contain the correct value in 3.14. For now, they should keep returning the creation time as they always have.
2023-03-16 17:27:21 +00:00
Steve Dower cb35882773
gh-102519: Add os.listdrives, os.listvolumes and os.listmounts on Windows (GH-102544) 2023-03-10 12:21:37 +00:00
Max Bachmann c6858d1e7f
gh-102255: Improve build support for Windows API partitions (GH-102256)
Add `MS_WINDOWS_DESKTOP`, `MS_WINDOWS_APPS`, `MS_WINDOWS_SYSTEM` and `MS_WINDOWS_GAMES` preprocessor definitions to allow switching off functionality missing from particular API partitions ("partitions" are used in Windows to identify overlapping subsets of APIs).
CPython only officially supports `MS_WINDOWS_DESKTOP` and `MS_WINDOWS_SYSTEM` (APPS is included by normal desktop builds, but APPS without DESKTOP is not covered). Other configurations are a convenience for people building their own runtimes.
`MS_WINDOWS_GAMES` is for the Xbox subset of the Windows API, which is also available on client OS, but is restricted compared to `MS_WINDOWS_DESKTOP`. These restrictions may change over time, as they relate to the build headers rather than the OS support, and so we assume that Xbox builds will use the latest available version of the GDK.
2023-03-09 21:09:12 +00:00
Eric Snow cf6e7c5e55
gh-100227: Isolate the Import State to Each Interpreter (gh-101941)
Specific changes:

* move the import lock to PyInterpreterState
* move the "find_and_load" diagnostic state to PyInterpreterState

Note that the import lock exists to keep multiple imports of the same module in the same interpreter (but in different threads) from stomping on each other.  Independently, we use a distinct global lock to protect globally shared import state, especially related to loaded extension modules.  For now we can rely on the GIL as that lock but with a per-interpreter GIL we'll need a new global lock.

The remaining state in _PyRuntimeState.imports will (probably) continue being global.

https://github.com/python/cpython/issues/100227
2023-03-09 09:46:21 -07:00
Alexey Izbyshev c2bd55d26f
gh-102179: Fix `os.dup2` error reporting for negative fds (#102180) 2023-03-04 19:54:08 +05:30
Max Bachmann 938e36f824
gh-102336: Remove code specifically for handling Windows 7 (GH-102337) 2023-03-01 00:31:21 +00:00
Irit Katriel 2db23d10bf
gh-102192: Replace PyErr_Fetch/Restore etc by more efficient alternatives (in Modules/) (#102196) 2023-02-24 21:43:03 +00:00
Max Bachmann 1fa38906f0
gh-102141: replace use of getpid on Windows with GetCurrentProcessId (GH-102142) 2023-02-24 12:38:21 +00:00
Rayyan Ansari 739c026f44
gh-101881: Support (non-)blocking read/write functions on Windows pipes (GH-101882)
* fileutils: handle non-blocking pipe IO on Windows

Handle erroring operations on non-blocking pipes by reading the _doserrno code.
Limit writes on non-blocking pipes that are too large.

* Support blocking functions on Windows

Use the GetNamedPipeHandleState and SetNamedPipeHandleState Win32 API functions to add support for os.get_blocking and os.set_blocking.
2023-02-16 14:52:24 +00:00
Sam James 8be8101bca
gh-101857: Allow xattr detection on musl libc (#101858)
Previously, we checked exclusively for `__GLIBC__` (AND'd with some other
conditions). Checking for `__linux__` instead should be fine.

This fixes using e.g. `os.listxattr()` on systems using musl libc.

Bug: https://bugs.gentoo.org/894130

Co-authored-by: Gregory P. Smith <greg@krypto.org>
2023-02-13 23:21:58 -08:00
Michael Droettboom 86ebd5c3fa
gh-101196: Make isdir/isfile/exists faster on Windows (GH-101324)
Co-authored-by: Eryk Sun <eryksun@gmail.com>
2023-02-08 14:34:24 +00:00
Eric Snow 6036c3e856
gh-59956: Clarify GILState-related Code (gh-101161)
The objective of this change is to help make the GILState-related code easier to understand.  This mostly involves moving code around and some semantically equivalent refactors.  However, there are a also a small number of slight changes in structure and behavior:

* tstate_current is moved out of _PyRuntimeState.gilstate
* autoTSSkey is moved out of _PyRuntimeState.gilstate
* autoTSSkey is initialized earlier
* autoTSSkey is re-initialized (after fork) earlier

https://github.com/python/cpython/issues/59956
2023-01-19 16:04:14 -07:00
Gabriele N. Tornetta d52d4942cf
gh-100649: Update native_thread_id after fork (gh-100650)
Update native_thread_id after fork
2023-01-01 23:20:38 +09:00
Gregory P. Smith 894f2c3c16
gh-100228: Warn from os.fork() if other threads exist. (#100229)
Not comprehensive, best effort warning. There are cases when threads exist on some platforms that this code cannot detect. macOS when API permissions allow and Linux with a readable /proc procfs present are the currently supported cases where a warning should show up reliably.

Starting with a DeprecationWarning for now, it is less disruptive than something like RuntimeWarning and most likely to only be seen in people's CI tests - a good place to start with this messaging.
2022-12-29 14:41:39 -08:00
amaajemyfren bbf4a66a18
GH-69564: Clarify use of octal format of mode argument in help(os.chmod) (#20621)
Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
2022-12-21 23:18:25 +05:30
Eric Snow 7a0f3c1d92
gh-81057: Fix a Reference Leak in the posix Module (gh-100140)
The leak was introduced in gh-100082.

https://github.com/python/cpython/issues/81057
2022-12-09 10:18:29 -07:00
Eric Snow 8a3f06c54b
gh-81057: Move time Globals to _PyRuntimeState (gh-100122)
https://github.com/python/cpython/issues/81057
2022-12-08 16:46:09 -07:00
Eric Snow cda9f0236f
gh-81057: Move OS-Related Globals to _PyRuntimeState (gh-100082)
https://github.com/python/cpython/issues/81057
2022-12-08 15:38:06 -07:00
Serhiy Storchaka a87c46eab3
bpo-15999: Accept arbitrary values for boolean parameters. (#15609)
builtins and extension module functions and methods that expect boolean values for parameters now accept any Python object rather than just a bool or int type. This is more consistent with how native Python code itself behaves.
2022-12-03 11:52:21 -08:00
TheShermanTanker ec2b76aa8b
GH-95896: posixmodule.c: fix osdefs.h inclusion to not depend on compiler (#95897)
Co-authored-by: Steve Dower <steve.dower@python.org>
2022-11-26 15:01:42 +05:30
Victor Stinner 81f7359f67
gh-99537: Use Py_SETREF(var, NULL) in C code (#99687)
Replace "Py_DECREF(var); var = NULL;" with "Py_SETREF(var, NULL);".
2022-11-23 14:57:50 +01:00
Charles Machalow 1b2de89bce
gh-99547: Add isjunction methods for checking if a path is a junction (GH-99548) 2022-11-22 17:19:34 +00:00
Victor Stinner 7e3f09cad9
gh-99537: Use Py_SETREF() function in C code (#99656)
Fix potential race condition in code patterns:

* Replace "Py_DECREF(var); var = new;" with "Py_SETREF(var, new);"
* Replace "Py_XDECREF(var); var = new;" with "Py_XSETREF(var, new);"
* Replace "Py_CLEAR(var); var = new;" with "Py_XSETREF(var, new);"

Other changes:

* Replace "old = var; var = new; Py_DECREF(var)"
  with "Py_SETREF(var, new);"
* Replace "old = var; var = new; Py_XDECREF(var)"
  with "Py_XSETREF(var, new);"
* And remove the "old" variable.
2022-11-22 14:22:22 +01:00
Ronald Oussoren cdde29dde9
gh-99337: Fix compile errors with gcc 12 on macOS (#99470)
Fix a number of compile errors with GCC-12 on macOS:

1. In pylifecycle.c the compile rejects _Pragma within a declaration
2. posixmodule.c was missing a number of ..._RUNTIME macros for non-clang on macOS
3. _ctypes assumed that __builtin_available is always present on macOS
2022-11-21 10:50:20 +01:00
Victor Stinner 3e2f7135e6
gh-99300: Use Py_NewRef() in Modules/ directory (#99469)
Replace Py_INCREF() and Py_XINCREF() with Py_NewRef() and
Py_XNewRef() in test C files of the Modules/ directory.
2022-11-14 16:21:23 +01:00
Eric Snow 4702552885
gh-98610: Adjust the Optional Restrictions on Subinterpreters (GH-98618)
Previously, the optional restrictions on subinterpreters were: disallow fork, subprocess, and threads.  By default, we were disallowing all three for "isolated" interpreters.  We always allowed all three for the main interpreter and those created through the legacy `Py_NewInterpreter()` API.

Those settings were a bit conservative, so here we've adjusted the optional restrictions to: fork, exec, threads, and daemon threads.  The default for "isolated" interpreters disables fork, exec, and daemon threads.  Regular threads are allowed by default.  We continue always allowing everything For the main interpreter and the legacy API.

In the code, we add `_PyInterpreterConfig.allow_exec` and  `_PyInterpreterConfig.allow_daemon_threads`.  We also add `Py_RTFLAGS_DAEMON_THREADS` and `Py_RTFLAGS_EXEC`.
2022-10-31 12:35:54 -07:00
Eric Snow f32369480d
gh-98608: Change _Py_NewInterpreter() to _Py_NewInterpreterFromConfig() (gh-98609)
(see https://github.com/python/cpython/issues/98608)

This change does the following:

1. change the argument to a new `_PyInterpreterConfig` struct
2. rename the function to `_Py_NewInterpreterFromConfig()`, inspired by `Py_InitializeFromConfig()` (takes a `_PyInterpreterConfig`  instead of `isolated_subinterpreter`)
3. split up the boolean `isolated_subinterpreter` into the corresponding multiple granular settings
   * allow_fork
   * allow_subprocess
   * allow_threads
4. add `PyInterpreterState.feature_flags` to store those settings
5. add a function for checking if a feature is enabled on an opaque `PyInterpreterState *`
6. drop `PyConfig._isolated_interpreter`

The existing default (see `Py_NewInterpeter()` and `Py_Initialize*()`) allows fork, subprocess, and threads and the optional "isolated" interpreter (see the `_xxsubinterpreters` module) disables all three.  None of that changes here; the defaults are preserved.

Note that the given `_PyInterpreterConfig` will not be used outside `_Py_NewInterpreterFromConfig()`, nor preserved.  This contrasts with how `PyConfig` is currently preserved, used, and even modified outside `Py_InitializeFromConfig()`.  I'd rather just avoid that mess from the start for `_PyInterpreterConfig`.  We can preserve it later if we find an actual need.

This change allows us to follow up with a number of improvements (e.g. stop disallowing subprocess and support disallowing exec instead).

(Note that this PR adds "private" symbols.  We'll probably make them public, and add docs, in a separate change.)
2022-10-26 11:16:30 -06:00
Noam Cohen a371a7e03e
gh-95023: Added os.setns and os.unshare functions (#95046)
Added os.setns and os.unshare to easily switch between namespaces
on Linux.

Co-authored-by: Christian Heimes <christian@python.org>
Co-authored-by: CAM Gerlach <CAM.Gerlach@Gerlach.CAM>
Co-authored-by: Victor Stinner <vstinner@python.org>
2022-10-20 11:08:54 +02:00
Victor Stinner db03c8066a
gh-98393: os module reject bytes-like, only accept bytes (#98394)
The os module and the PyUnicode_FSDecoder() function no longer accept
bytes-like paths, like bytearray and memoryview types: only the exact
bytes type is accepted for bytes strings.
2022-10-18 17:52:31 +02:00
Hagai Helman Tov 1523c9e9d4
gh-96288: Add a sentence to `os.mkdir`'s docstring. (#96271) 2022-10-07 15:06:16 -07:00
Dong-hee Na b9d2e81716
fixes gh-96078: os.sched_yield release the GIL while calling sched_yield(2). (gh-97965) 2022-10-06 15:57:37 -07:00
Ned Deily 6d0a0191a4
gh-97897: Prevent os.mkfifo and os.mknod segfaults with macOS 13 SDK (GH-97944)
The macOS 13 SDK includes support for the `mkfifoat` and `mknodat` system calls.
 Using the `dir_fd` option with either `os.mkfifo` or `os.mknod` could result in a
 segfault if cpython is built with the macOS 13 SDK but run on an earlier
 version of macOS. Prevent this by adding runtime support for detection of
 these system calls ("weaklinking") as is done for other newer syscalls on
 macOS.
2022-10-06 09:11:47 -07:00
Pablo Galindo Salgado 6d791a9736
gh-96143: Allow Linux perf profiler to see Python calls (GH-96123)
⚠️  ⚠️ Note for reviewers, hackers and fellow systems/low-level/compiler engineers ⚠️ ⚠️ 

If you have a lot of experience with this kind of shenanigans and want to improve the **first** version, **please make a PR against my branch** or **reach out by email** or **suggest code changes directly on GitHub**. 

If you have any **refinements or optimizations** please, wait until the first version is merged before starting hacking or proposing those so we can keep this PR productive.
2022-08-30 10:11:18 -07:00
Raghunandan Bhat edb72047f3
gh-43414: os.get_terminal_size() now uses the actual file descriptor on Windows instead of mapping to standard handles (#93203) 2022-07-28 23:48:33 +01:00
Kumar Aditya 4dd099baff
GH-93899: fix checks for eventfd flags (GH-95170) 2022-07-27 06:05:29 -07:00
Christian Heimes 0d35a59ce3
gh-95174: Handle missing dup() and constants in WASI (GH-95229)
- check for ``dup()`` libc function
- handle missing ``F_DUPFD`` in ``dup2()`` replacement function
- add workaround for WASI libc bug in MSG_TRUNC
- ESHUTDOWN is missing, use EPIPE instead
- POLLPRI is missing, define as 0 (no-op)
2022-07-26 11:16:51 +02:00
Serhiy Storchaka 6fd4c8ec77
gh-93741: Add private C API _PyImport_GetModuleAttrString() (GH-93742)
It combines PyImport_ImportModule() and PyObject_GetAttrString()
and saves 4-6 lines of code on every use.

Add also _PyImport_GetModuleAttr() which takes Python strings as arguments.
2022-06-14 07:15:26 +03:00
Christian Heimes 22fed605e0
gh-90473: Make chmod a dummy on WASI, skip chmod tests (GH-93534)
WASI does not have the ``chmod(2)`` syscall yet.
2022-06-06 19:24:11 +02:00
Kumar Aditya a565ab0fd5
GH-93312: Add os.PIDFD_NONBLOCK flag (#93313) 2022-05-31 12:51:29 +02:00
Inada Naoki f9c9354a7a
gh-92536: PEP 623: Remove wstr and legacy APIs from Unicode (GH-92537) 2022-05-12 14:48:38 +09:00
Soumendra Ganguly ae553b3561
bpo-41818: Add os.login_tty() for *nix. (#29658)
* Add `os.login_tty(fd)` for Unix.

Reviewed-by: Christian Heimes <christian@python.org>
Signed-off-by: Gregory P. Smith [Google LLC] <greg@krypto.org>
2022-05-05 09:04:44 -07:00
David CARLIER 04dc4b06a3
gh-90887: posix module: Add more flags for fcopy_file (#31300)
Closes #90887

Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2022-05-03 08:53:51 -06:00
Itai Steinherz 39e6b8ae6a
bpo-46785: Fix race condition between os.stat() and unlink on Windows (GH-31858) 2022-05-03 00:19:13 +01:00
Victor Stinner 20cc695286
gh-64783: Fix signal.NSIG value on FreeBSD (#91929)
Fix signal.NSIG value on FreeBSD to accept signal numbers greater
than 32, like signal.SIGRTMIN and signal.SIGRTMAX.

* Add Py_NSIG constant.
* Add pycore_signal.h internal header file.
* _Py_Sigset_Converter() now includes the range of valid signals in
  the error message.
2022-04-26 00:13:31 +02:00
Victor Stinner 031f1e6040
gh-90623: signal.raise_signal() calls PyErr_CheckSignals() (#91756)
signal.raise_signal() and os.kill() now call PyErr_CheckSignals() to
check immediately for pending signals.
2022-04-21 03:14:57 +02:00
Victor Stinner 7cdaf87ec5
gh-91731: Replace Py_BUILD_ASSERT() with static_assert() (#91730)
Python 3.11 now uses C11 standard which adds static_assert()
to <assert.h>.

* In pytime.c, replace Py_BUILD_ASSERT() with preprocessor checks on
  SIZEOF_TIME_T with #error.
* On macOS, py_mach_timebase_info() now accepts timebase members with
  the same size than _PyTime_t.
* py_get_monotonic_clock() now saturates GetTickCount64() to
  _PyTime_MAX: GetTickCount64() is unsigned, whereas _PyTime_t is
  signed.
2022-04-20 19:26:40 +02:00