Commit Graph

176 Commits

Author SHA1 Message Date
Sam Gross 460cc9e14e
gh-117657: Fix TSan reported data race on ioctl_works (#120175) 2024-06-17 13:23:40 -04:00
Victor Stinner e93c39b47e
gh-118422: Fix run_fileexflags() test (#118429)
Don't test the undefined behavior of fileno()
on a closed file, but use fstat() as a reliable
test if the file was closed or not.
2024-04-30 22:32:55 +02:00
Nice Zombies 10bb90ed49
gh-102511: Speed up os.path.splitroot() with native helpers (GH-118089) 2024-04-25 10:07:38 +01:00
Sam James c454e934d3
gh-112970: Detect and use closefrom() when available (#112969)
glibc-2.34 implements closefrom(3) using the same semantics as on BSD.
Check for closefrom() in configure and use the check result in
fileutils.c, rather than hardcoding a FreeBSD check.

Some implementations of closefrom() return an int. Explicitly discard 
the return value by casting it to void, to avoid future compiler
warnings.

Signed-off-by: Sam James <sam@gentoo.org>
2023-12-12 11:25:27 +01:00
Victor Stinner 05a370abd6
gh-112567: Add _Py_GetTicksPerSecond() function (#112587)
* Move _PyRuntimeState.time to _posixstate.ticks_per_second and
  time_module_state.ticks_per_second.
* Add time_module_state.clocks_per_second.
* Rename _PyTime_GetClockWithInfo() to py_clock().
* Rename _PyTime_GetProcessTimeWithInfo() to py_process_time().
* Add process_time_times() helper function, called by
  py_process_time().
* os.times() is now always built: no longer rely on HAVE_TIMES.
2023-12-01 17:05:56 +01:00
AN Long 29af7369db
gh-111856: Fix os.fstat on windows with FAT32 and exFAT filesystem (GH-112038) 2023-11-13 16:10:06 +00:00
Victor Stinner 7513994c92
gh-110014: Include explicitly <unistd.h> header (#110155)
* Remove unused <locale.h> includes.
* Remove unused <fcntl.h> include in traceback.h.
* Remove redundant <assert.h> and <stddef.h> includes. They  are already
  included by "Python.h".
* Remove <object.h> include in faulthandler.c. Python.h already includes it.
* Add missing <stdbool.h> in pycore_pythread.h if HAVE_PTHREAD_STUBS
  is defined.
* Fix also warnings in pthread_stubs.h: don't redefine macros if they
  are already defined, like the __NEED_pthread_t macro.
2023-09-30 20:06:45 +00:00
Serhiy Storchaka 2b15536fa9
gh-107913: Fix possible losses of OSError error codes (GH-107930)
Functions like PyErr_SetFromErrno() and SetFromWindowsErr() should be
called immediately after using the C API which sets errno or the Windows
error code.
2023-08-27 00:35:06 +03:00
Steve Dower 34e1917912
gh-106242: Minor fixup to avoid compiler warnings (GH-107983) 2023-08-15 17:02:32 +00:00
Finn Womack 0932272431
gh-106242: Fix path truncation in os.path.normpath (GH-106816) 2023-08-15 16:33:00 +01:00
Brandt Bucher 05a824f294
GH-84436: Skip refcounting for known immortals (GH-107605) 2023-08-04 16:24:50 -07:00
Victor Stinner 4b65d5638c
gh-80064: Fix is_valid_wide_char() return type (#105099)
Return a classical int, rather than size_t. The size_t type was
kept from copied/pasted code related to mbstowcs().
2023-05-30 18:36:20 +02:00
Steve Dower 6031727a37
gh-104820: Fixes os.stat on Windows to better handle file systems that do not support FileIdInformation (GH-104892) 2023-05-25 00:16:13 +01:00
Steve Dower 0f175766e2
gh-99726: Improves correctness of stat results for Windows, and uses faster API when available (GH-102149)
This deprecates `st_ctime` fields on Windows, with the intent to change them to contain the correct value in 3.14. For now, they should keep returning the creation time as they always have.
2023-03-16 17:27:21 +00:00
Max Bachmann afa6092ee4
gh-102281: Fix potential nullptr dereference + use of uninitialized memory (gh-102282) 2023-03-15 21:58:43 +09:00
Max Bachmann c6858d1e7f
gh-102255: Improve build support for Windows API partitions (GH-102256)
Add `MS_WINDOWS_DESKTOP`, `MS_WINDOWS_APPS`, `MS_WINDOWS_SYSTEM` and `MS_WINDOWS_GAMES` preprocessor definitions to allow switching off functionality missing from particular API partitions ("partitions" are used in Windows to identify overlapping subsets of APIs).
CPython only officially supports `MS_WINDOWS_DESKTOP` and `MS_WINDOWS_SYSTEM` (APPS is included by normal desktop builds, but APPS without DESKTOP is not covered). Other configurations are a convenience for people building their own runtimes.
`MS_WINDOWS_GAMES` is for the Xbox subset of the Windows API, which is also available on client OS, but is restricted compared to `MS_WINDOWS_DESKTOP`. These restrictions may change over time, as they relate to the build headers rather than the OS support, and so we assume that Xbox builds will use the latest available version of the GDK.
2023-03-09 21:09:12 +00:00
Max Bachmann 938e36f824
gh-102336: Remove code specifically for handling Windows 7 (GH-102337) 2023-03-01 00:31:21 +00:00
Rayyan Ansari 739c026f44
gh-101881: Support (non-)blocking read/write functions on Windows pipes (GH-101882)
* fileutils: handle non-blocking pipe IO on Windows

Handle erroring operations on non-blocking pipes by reading the _doserrno code.
Limit writes on non-blocking pipes that are too large.

* Support blocking functions on Windows

Use the GetNamedPipeHandleState and SetNamedPipeHandleState Win32 API functions to add support for os.get_blocking and os.set_blocking.
2023-02-16 14:52:24 +00:00
James Lee 95cbb3d908
gh-101810: Remove duplicated st_ino calculation (GH-101811) 2023-02-13 13:49:44 +00:00
Eric Snow 5f55067e23
gh-81057: Move More Globals in Core Code to _PyRuntimeState (gh-99516)
https://github.com/python/cpython/issues/81057
2022-11-16 09:37:14 -07:00
Christian Heimes 0d35a59ce3
gh-95174: Handle missing dup() and constants in WASI (GH-95229)
- check for ``dup()`` libc function
- handle missing ``F_DUPFD`` in ``dup2()`` replacement function
- add workaround for WASI libc bug in MSG_TRUNC
- ESHUTDOWN is missing, use EPIPE instead
- POLLPRI is missing, define as 0 (no-op)
2022-07-26 11:16:51 +02:00
Victor Stinner 32b7bcffba
gh-93103: Py_DecodeLocale() uses _PyRuntime.preconfig (#93187)
The Py_DecodeLocale() and Py_EncodeLocale() now use
_PyRuntime.preconfig, rather than Py_UTF8Mode and
Py_LegacyWindowsFSEncodingFlag global configuration varibles, to
decide if the UTF-8 encoding is used or not.

As documented, these functions must not be called before Python is
preinitialized. The new PyConfig API should now be used, rather than
using deprecated functions like Py_SetPath() or PySys_SetArgv().
2022-05-25 00:09:48 +02:00
Inada Naoki f9c9354a7a
gh-92536: PEP 623: Remove wstr and legacy APIs from Unicode (GH-92537) 2022-05-12 14:48:38 +09:00
Victor Stinner 7cdaf87ec5
gh-91731: Replace Py_BUILD_ASSERT() with static_assert() (#91730)
Python 3.11 now uses C11 standard which adds static_assert()
to <assert.h>.

* In pytime.c, replace Py_BUILD_ASSERT() with preprocessor checks on
  SIZEOF_TIME_T with #error.
* On macOS, py_mach_timebase_info() now accepts timebase members with
  the same size than _PyTime_t.
* py_get_monotonic_clock() now saturates GetTickCount64() to
  _PyTime_MAX: GetTickCount64() is unsigned, whereas _PyTime_t is
  signed.
2022-04-20 19:26:40 +02:00
Inada Naoki 6773203487
bpo-47000: Add `locale.getencoding()` (GH-32068) 2022-04-09 09:54:54 +09:00
Alexey Izbyshev 1c8b3b5d66
bpo-47260: Fix os.closerange() potentially being a no-op in a seccomp sandbox (GH-32418)
_Py_closerange() currently assumes that close_range() closes
all file descriptors even if it returns an error (other than ENOSYS).
This assumption can be wrong on Linux if a seccomp sandbox denies
the underlying syscall, pretending that it returns EPERM or EACCES.
In this case _Py_closerange() won't close any descriptors at all,
which in the worst case can be a security issue.

Fix this by falling back to other methods in case of any close_range()
error. Note that fallbacks will not be triggered on any problems with
closing individual file descriptors because close_range() is documented
to ignore such errors on both Linux[1] and FreeBSD[2].

[1] https://man7.org/linux/man-pages/man2/close_range.2.html
[2] https://www.freebsd.org/cgi/man.cgi?query=close_range&sektion=2
2022-04-08 10:40:39 -07:00
Serhiy Storchaka 6927632492
Remove trailing spaces (GH-31695) 2022-03-05 17:47:00 +02:00
neonene d4e64cd4b0
bpo-46362: Ensure ntpath.abspath() uses the Windows API correctly (GH-30571)
This makes ntpath.abspath()/getpath_abspath() follow normpath(), since some WinAPIs such as PathCchSkipRoot() require backslashed paths.
2022-01-13 23:35:42 +00:00
Steve Dower d81182b8ec
bpo-46217: Revert use of Windows constant that is newer than what we support (GH-30473) 2022-01-08 00:06:53 +00:00
neonene 9c5fa9c97c
bpo-46208: Fix normalization of relative paths in _Py_normpath()/os.path.normpath (GH-30362) 2022-01-06 19:13:10 +00:00
Steve Dower 99fcf15052
bpo-45582: Port getpath[p].c to Python (GH-29041)
The getpath.py file is frozen at build time and executed as code over a namespace. It is never imported, nor is it meant to be importable or reusable. However, it should be easier to read, modify, and patch than the previous code.

This commit attempts to preserve every previously tested quirk, but these may be changed in the future to better align platforms.
2021-12-03 00:08:42 +00:00
Eric Snow 17c61045c5
bpo-45506: Normalize _PyPathConfig.stdlib_dir when calculated. (#29040)
The recently added PyConfig.stdlib_dir was being set with ".." entries. When __file__ was added for from modules this caused a problem on out-of-tree builds. This PR fixes that by normalizing "stdlib_dir" when it is calculated in getpath.c.

https://bugs.python.org/issue45506
2021-10-22 17:20:03 -06:00
Eric Snow b9cdd0fb9c
bpo-45020: Default to using frozen modules unless running from source tree. (gh-28940)
The default was "off".  Switching it to "on" means users get the benefit of frozen stdlib modules without having to do anything.  There's a special-case for running-in-source-tree, so contributors don't get surprised when their stdlib changes don't get used.

https://bugs.python.org/issue45020
2021-10-16 13:16:08 -06:00
Victor Stinner aac29af678
bpo-45434: pyport.h no longer includes <stdlib.h> (GH-28914)
Include <stdlib.h> explicitly in C files.

Python.h includes <wchar.h>.
2021-10-13 19:25:53 +02:00
Christian Clauss db693df3e1
Fix typos in the Python directory (GH-28767) 2021-10-06 15:55:27 -07:00
Eric Snow 16b5bc6896
Do not check isabs() on Windows. (gh-28584)
I missed this in gh-28550.

https://bugs.python.org/issue45211
2021-09-27 10:52:19 -06:00
Eric Snow ae7839bbe8
bpo-45211: Move helpers from getpath.c to internal API. (gh-28550)
This accomplishes 2 things:

* consolidates some common code between getpath.c and getpathp.c
* makes the helpers available to code in other files

FWIW, the signature of the join_relfile() function (in fileutils.c) intentionally mirrors that of Windows' PathCchCombineEx().

Note that this change is mostly moving code around. No behavior is meant to change.

https://bugs.python.org/issue45211
2021-09-27 10:00:32 -06:00
Vincent Michel 06148b1870
bpo-44219: Release the GIL during isatty syscalls (GH-28250)
Release the GIL while performing isatty() system calls on arbitrary
file descriptors. In particular, this affects os.isatty(),
os.device_encoding() and io.TextIOWrapper. By extension,
io.open() in text mode is also affected.
2021-09-09 15:12:03 +02:00
Victor Stinner c24896c0e3
bpo-44849: Fix os.set_inheritable() on FreeBSD 14 with O_PATH (GH-27623)
Fix the os.set_inheritable() function on FreeBSD 14 for file
descriptor opened with the O_PATH flag: ignore the EBADF error on
ioctl(), fallback on the fcntl() implementation.
2021-08-06 15:15:10 +02:00
Jakub Kulík 9032cf5cb1
bpo-43667: Fix broken Unicode encoding in non-UTF locales on Solaris (GH-25096) 2021-04-30 15:21:42 +02:00
Segev Finer 5e437fb872
bpo-30555: Fix WindowsConsoleIO fails in the presence of fd redirection (GH-1927)
This works by not caching the handle and instead getting the handle from
the file descriptor each time, so that if the actual handle changes by
fd redirection closing/opening the console handle beneath our feet, we
will keep working correctly.
2021-04-23 23:00:27 +01:00
Victor Stinner 9976834f80
bpo-35883: Py_DecodeLocale() escapes invalid Unicode characters (GH-24843)
Python no longer fails at startup with a fatal error if a command
line argument contains an invalid Unicode character.

The Py_DecodeLocale() function now escapes byte sequences which would
be decoded as Unicode characters outside the [U+0000; U+10ffff]
range.

Use MAX_UNICODE constant in unicodeobject.c.
2021-03-17 21:46:53 +01:00
cptpcrd 7dc71c425c
bpo-42780: Fix set_inheritable() for O_PATH file descriptors on Linux (GH-24172) 2021-01-20 15:05:51 +01:00
Victor Stinner ca06440207
bpo-32381: Remove unused _Py_fopen() function (GH-23711)
Remove the private _Py_fopen() function which is no longer needed.
Use _Py_wfopen() or _Py_fopen_obj() instead.
2020-12-09 20:54:31 +01:00
pxinwr 06afac6c57
bpo-41462: Add os.set_blocking() support for VxWorks RTOS (GH-21713) 2020-12-07 21:41:12 +01:00
Victor Stinner 3529718925
bpo-42236: os.device_encoding() respects UTF-8 Mode (GH-23119)
On Unix, the os.device_encoding() function now returns 'UTF-8' rather
than the device encoding if the Python UTF-8 Mode is enabled.
2020-11-04 11:20:10 +01:00
Victor Stinner e662c398d8
bpo-42236: Use UTF-8 encoding if nl_langinfo(CODESET) fails (GH-23086)
If the nl_langinfo(CODESET) function returns an empty string, Python
now uses UTF-8 as the filesystem encoding.

In May 2010 (commit b744ba1d14), I
modified Python to log a warning and use UTF-8 as the filesystem
encoding (instead of None) if nl_langinfo(CODESET) returns an empty
string.

In August 2020 (commit 94908bbc15), I
modified Python startup to fail with a fatal error and a specific
error message if nl_langinfo(CODESET) returns an empty string. The
intent was to prevent guessing the encoding and also investigate user
configuration where this case happens.

In 10 years (2010 to 2020), I saw zero user report about the error
message related to nl_langinfo(CODESET) returning an empty string.

Today, UTF-8 became the defacto standard and it's safe to make the
assumption that the user expects UTF-8. For example,
nl_langinfo(CODESET) can return an empty string on macOS if the
LC_CTYPE locale is not supported, and UTF-8 is the default encoding
on macOS.

While this change is likely to not affect anyone in practice, it
should make UTF-8 lover happy ;-)

Rewrite also the documentation explaining how Python selects the
filesystem encoding and error handler.
2020-11-01 23:07:23 +01:00
Victor Stinner 82458b6cdb
bpo-42236: Enhance _locale._get_locale_encoding() (GH-23083)
* Rename _Py_GetLocaleEncoding() to _Py_GetLocaleEncodingObject()
* Add _Py_GetLocaleEncoding() which returns a wchar_t* string to
  share code between _Py_GetLocaleEncodingObject()
  and config_get_locale_encoding().
* _Py_GetLocaleEncodingObject() now decodes nl_langinfo(CODESET)
  from the current locale encoding with surrogateescape,
  rather than using UTF-8.
2020-11-01 20:59:35 +01:00
Victor Stinner 710e826307
bpo-42208: Add _Py_GetLocaleEncoding() (GH-23050)
_io.TextIOWrapper no longer calls getpreferredencoding(False) of
_bootlocale to get the locale encoding, but calls
_Py_GetLocaleEncoding() instead.

Add config_get_fs_encoding() sub-function. Reorganize also
config_get_locale_encoding() code.
2020-10-31 01:02:09 +01:00
TIGirardi f2312037e3
bpo-38324: Fix test__locale.py Windows failures (GH-20529)
Use wide-char _W_* fields of lconv structure on Windows
Remove "ps_AF" from test__locale.known_numerics on Windows
2020-10-20 12:39:52 +01:00