Commit Graph

1160 Commits

Author SHA1 Message Date
Victor Stinner 5776663675
bpo-42029: Remove IRIX code (GH-23023)
IRIX code was slowy removed in Python 2.4 (--with-sgi-dl), Python 3.3
(Irix threads), and Python 3.7.
2020-10-29 15:16:23 +01:00
Serhiy Storchaka b510e101f8
bpo-42152: Use PyDict_Contains and PyDict_SetDefault if appropriate. (GH-22986)
If PyDict_GetItemWithError is only used to check whether the key is in dict,
it is better to use PyDict_Contains instead.

And if it is used in combination with PyDict_SetItem, PyDict_SetDefault can
replace the combination.
2020-10-26 12:47:57 +02:00
Kyle Evans 7992579cd2
bpo-40422: Move _Py_closerange to fileutils.c (GH-22680)
This API is relatively lightweight and organizationally, given that it's
used by multiple modules, it makes sense to move it to fileutils.

Requires making sure that _posixsubprocess is compiled with the appropriate
Py_BUIILD_CORE_BUILTIN macro.
2020-10-13 22:04:44 +02:00
Kyle Evans 64eb259cc1
bpo-40422: Move _Py_*_SUPPRESS_IPH bits into _Py_closerange (GH-22672)
This suppression is no longer needed in os_closerange_impl, as it just
invokes the internal _Py_closerange implementation. On the other hand,
consumers of _Py_closerange may not have any other reason to suppress
invalid parameter issues, so narrow the scope to here.
2020-10-12 16:53:16 -07:00
Kyle Evans 1800c60080
bpo-40423: Optimization: use close_range(2) if available (GH-22651)
close_range(2) should be preferred at all times if it's available, otherwise we'll use closefrom(2) if available with a fallback to fdwalk(3) or plain old loop over fd range in order of most efficient to least.

[note that this version does check for ENOSYS, but currently ignores all other errors]

Automerge-Triggered-By: @pablogsal
2020-10-11 13:18:53 -07:00
Kyle Evans c230fde847
bpo-40422: create a common _Py_closerange API (GH-19754)
Such an API can be used both for os.closerange and subprocess. For the latter, this yields potential improvement for platforms that have fdwalk but wouldn't have used it there. This will prove even more beneficial later for platforms that have close_range(2), as the new API will prefer that over all else if it's available.

The new API is structured to look more like close_range(2), closing from [start, end] rather than the [low, high) of os.closerange().

Automerge-Triggered-By: @gpshead
2020-10-11 11:54:11 -07:00
Serhiy Storchaka 9975cc5008
bpo-41985: Add _PyLong_FileDescriptor_Converter and AC converter for "fildes". (GH-22620) 2020-10-09 23:00:45 +03:00
Jakub Kulík fa8c9e7010
bpo-41687: Fix error handling in Solaris sendfile implementation (GH-22128)
I just realized that my recent PR with sendfile on Solaris ([PR 22040](https://github.com/python/cpython/pull/22040)) has broken error handling.

Sorry for that, this simple followup fixes that.

Automerge-Triggered-By: @1st1
2020-09-09 12:29:42 -07:00
Jakub Kulík 8c0be6fd91
bpo-41687: Fix sendfile implementation to work with Solaris (#22040) 2020-09-05 12:10:01 -07:00
pxinwr 3405e05428
bpo-41440: add os.cpu_count() support for VxWorks RTOS (GH-21685) 2020-08-07 14:21:52 +09:00
Serhiy Storchaka 4c8f09d7ce
bpo-36346: Make using the legacy Unicode C API optional (GH-21437)
Add compile time option USE_UNICODE_WCHAR_CACHE. Setting it to 0
makes the interpreter not using the wchar_t cache and the legacy Unicode C API.
2020-07-10 23:26:06 +03:00
Gregory P. Smith 3ccb96c978
bpo-41056: Use the fildes converter for fd to please Coverity. (GH-21011)
There are a bunch of other fd: int uses in this file, I expect many if not
all of them would be better off using the fildes converter.  This particular
one was flagged by Coverity as it presumably flags fpathconf as not accepting
negative fds.  I'd expect the other fd's to have been flagged as well
otherwise.

I'm marking this one as skip news as it really is a no-op.
2020-06-20 15:06:48 -07:00
Victor Stinner 317bab0bf6
PyOS_AfterFork_Child() pass tstate to _PyEval_ReInitThreads() (GH-20598) 2020-06-02 18:44:54 +02:00
Victor Stinner 26881c8fae
PyOS_AfterFork_Child() uses PyStatus (GH-20596)
PyOS_AfterFork_Child() helper functions now return a PyStatus:
PyOS_AfterFork_Child() is now responsible to handle errors.

* Move _PySignal_AfterFork() to the internal C API
* Add #ifdef HAVE_FORK on _PyGILState_Reinit(), _PySignal_AfterFork()
  and _PyInterpreterState_DeleteExceptMain().
2020-06-02 15:51:37 +02:00
Serhiy Storchaka 5f4b229df7
bpo-40792: Make the result of PyNumber_Index() always having exact type int. (GH-20443)
Previously, the result could have been an instance of a subclass of int.

Also revert bpo-26202 and make attributes start, stop and step of the range
object having exact type int.

Add private function _PyNumber_Index() which preserves the old behavior
of PyNumber_Index() for performance to use it in the conversion functions
like PyLong_AsLong().
2020-05-28 10:33:45 +03:00
YoSTEALTH 76ef255bde
bpo-37129: Add os.RWF_APPEND flag for os.pwritev() (GH-20336) 2020-05-27 23:32:22 +02:00
Serhiy Storchaka 578c3955e0
bpo-37999: No longer use __int__ in implicit integer conversions. (GH-15636)
Only __index__ should be used to make integer conversions lossless.
2020-05-26 18:43:38 +03:00
Minmin Gong 7f21c9ac87
bpo-40653: Move _dirnameW out of #ifdef HAVE_SYMLINK/#endif (GH-20144) 2020-05-18 17:17:19 +01:00
Victor Stinner 97f33c3544
bpo-40549: posixmodule.c uses defining_class (GH-20075)
Pass PEP 573 defining_class to os.DirEntry methods. The module state
is now retrieve from defining_class rather than Py_TYPE(self), to
support subclasses (even if DirEntry doesn't support subclasses yet).

* Pass the module rather than defining_class to DirEntry_fetch_stat().
* Only get the module state once in _posix_clear(),
  _posix_traverse() and _posixmodule_exec().
2020-05-14 18:05:58 +02:00
Victor Stinner 1c2fa78156
bpo-40549: Convert posixmodule.c to multiphase init (GH-19982)
Convert posixmodule.c ("posix" or "nt" module) to the multiphase
initialization (PEP 489).

* Create the module using PyModuleDef_Init().
* Create ScandirIteratorType and DirEntryType with the new
  PyType_FromModuleAndSpec() (PEP 573)
* Get the module state from ScandirIteratorType and DirEntryType with
  the new PyType_GetModule() (PEP 573)
* Pass module to functions which access the module state.
* convert_sched_param() gets a new module parameter. It is now called
  directly since Argument Clinic doesn't support passing the module
  to an argument converter callback.
* Remove _posixstate_global macro.
2020-05-10 11:05:29 +02:00
Victor Stinner 252346acd9
bpo-40453: Add PyConfig._isolated_subinterpreter (GH-19820)
An isolated subinterpreter cannot spawn threads, spawn a child
process or call os.fork().

* Add private _Py_NewInterpreter(isolated_subinterpreter) function.
* Add isolated=True keyword-only parameter to
  _xxsubinterpreters.create().
* Allow again os.fork() in "non-isolated" subinterpreters.
2020-05-01 11:33:44 +02:00
Victor Stinner 162c567d16
bpo-38061: os.closerange() uses closefrom() on FreeBSD (GH-19696)
On FreeBSD, os.closerange(fd_low, fd_high) now calls
closefrom(fd_low) if fd_high is greater than or equal to
sysconf(_SC_OPEN_MAX).

Initial patch by Ed Maste (emaste), Conrad Meyer (cem), Kyle Evans
(kevans) and Kubilay Kocak (koobs):
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=242274
2020-04-24 12:00:51 +02:00
Victor Stinner 9bee32b34e
bpo-40138: Fix Windows os.waitpid() for large exit code (GH-19637)
Fix the Windows implementation of os.waitpid() for exit code
larger than "INT_MAX >> 8". The exit status is now interpreted as an
unsigned number.

os.waitstatus_to_exitcode() now accepts wait status larger than
INT_MAX.
2020-04-22 16:30:35 +02:00
Serhiy Storchaka 2b5603140c
bpo-40178: Convert the remaining os functions to Argument Clinic. (GH-19360)
Convert os.getgrouplist(), os.initgroups(), os.sendfile() and
os.get_terminal_size().
2020-04-18 19:14:10 +03:00
Victor Stinner 5eca75df03
bpo-40268: Reformat posixmodule.c includes (GH-19536) 2020-04-15 15:07:31 +02:00
Victor Stinner 361dcdcefc
bpo-40268: Remove unused osdefs.h includes (GH-19532)
When the include is needed, add required symbol in a comment.
2020-04-15 03:24:57 +02:00
Victor Stinner 4a21e57fe5
bpo-40268: Remove unused structmember.h includes (GH-19530)
If only offsetof() is needed: include stddef.h instead.

When structmember.h is used, add a comment explaining that
PyMemberDef is used.
2020-04-15 02:35:41 +02:00
Victor Stinner 62183b8d6d
bpo-40268: Remove explicit pythread.h includes (#19529)
Remove explicit pythread.h includes: it is always included
by Python.h.
2020-04-15 02:04:42 +02:00
Victor Stinner e5014be049
bpo-40268: Remove a few pycore_pystate.h includes (GH-19510) 2020-04-14 17:52:15 +02:00
Victor Stinner 81a7be3fa2
bpo-40268: Rename _PyInterpreterState_GET_UNSAFE() (GH-19509)
Rename _PyInterpreterState_GET_UNSAFE() to _PyInterpreterState_GET()
for consistency with _PyThreadState_GET() and to have a shorter name
(help to fit into 80 columns).

Add also "assert(tstate != NULL);" to the function.
2020-04-14 15:14:01 +02:00
Batuhan Taşkaya f9dd51e7db
bpo-39481: Make os.DirEntry generic (GH-19415) 2020-04-07 14:37:19 -07:00
Victor Stinner 87255be696
bpo-40089: Add _at_fork_reinit() method to locks (GH-19195)
Add a private _at_fork_reinit() method to _thread.Lock,
_thread.RLock, threading.RLock and threading.Condition classes:
reinitialize the lock after fork in the child process; reset the lock
to the unlocked state.

Rename also the private _reset_internal_locks() method of
threading.Event to _at_fork_reinit().

* Add _PyThread_at_fork_reinit() private function. It is excluded
  from the limited C API.
* threading.Thread._reset_internal_locks() now calls
  _at_fork_reinit() on self._tstate_lock rather than creating a new
  Python lock object.
2020-04-07 23:11:49 +02:00
Batuhan Taşkaya 909f4a3009
bpo-40190: Add support for _SC_AIX_REALMEM in sysconf (GH-19380) 2020-04-05 02:40:49 +02:00
Victor Stinner 65a796e527
bpo-40094: Add os.waitstatus_to_exitcode() (GH-19201)
Add os.waitstatus_to_exitcode() function to convert a wait status to an
exitcode.

Suggest waitstatus_to_exitcode() usage in the documentation when
appropriate.

Use waitstatus_to_exitcode() in:

* multiprocessing, os, subprocess and _bootsubprocess modules;
* test.support.wait_process();
* setup.py: run_command();
* and many tests.
2020-04-01 18:49:29 +02:00
Victor Stinner f5c7cabb2b
bpo-40014: Fix os.getgrouplist() (GH-19126)
Fix os.getgrouplist(): if getgrouplist() function fails because the
group list is too small, retry with a larger group list.

On failure, the glibc implementation of getgrouplist() sets ngroups
to the total number of groups. For other implementations, double the
group list size.
2020-03-24 18:22:10 +01:00
Victor Stinner 8ec7370c89
bpo-40014: Fix os.getgrouplist() on macOS (GH-19118)
On macOS, getgrouplist() returns a non-zero value without setting
errno if the group list is too small. Double the list size and call
it again in this case.
2020-03-23 20:00:57 +01:00
Hai Shi f707d94af6
bpo-39968: Convert extension modules' macros of get_module_state() to inline functions (GH-19017) 2020-03-16 14:15:01 +01:00
Victor Stinner ff4584caca
bpo-39947: Use _PyInterpreterState_GET_UNSAFE() (GH-18978)
Replace _PyInterpreterState_Get() function call with
_PyInterpreterState_GET_UNSAFE() macro which is more efficient but
don't check if tstate or interp is NULL.

_Py_GetConfigsAsDict() now uses _PyThreadState_GET().
2020-03-13 18:03:56 +01:00
Andy Lester 557287075c
bpo-39573: Use Py_IS_TYPE() macro to check for types (GH-18809)
Co-authored-by: Victor Stinner <vstinner@python.org>
2020-03-06 23:53:17 +01:00
Saiyang Gou 7514f4f625
bpo-39184: Add audit events to functions in `fcntl`, `msvcrt`, `os`, `resource`, `shutil`, `signal`, `syslog` (GH-18407) 2020-02-13 07:47:42 +00:00
Saiyang Gou 95f6001021
bpo-39184: Add audit events to command execution functions in os and pty modules (GH-17824) 2020-02-05 11:15:00 +11:00
Victor Stinner b8d1262e8a
bpo-39395: putenv() and unsetenv() always available (GH-18135)
The os.putenv() and os.unsetenv() functions are now always available.

On non-Windows platforms, Python now requires setenv() and unsetenv()
functions to build.

Remove putenv_dict from posixmodule.c: it's not longer needed.
2020-01-24 14:05:48 +01:00
Victor Stinner 161e7b36b1
bpo-39413: Implement os.unsetenv() on Windows (GH-18163)
The os.unsetenv() function is now also available on Windows.
2020-01-24 11:53:44 +01:00
Victor Stinner b477d19a6b
bpo-39406: Implement os.putenv() with setenv() if available (GH-18128)
If setenv() C function is available, os.putenv() is now implemented
with setenv() instead of putenv(), so Python doesn't have to handle
the environment variable memory.
2020-01-22 22:48:16 +01:00
Victor Stinner 0852c7dd52
bpo-39406: os.putenv() avoids putenv_dict on Windows (GH-18126)
Windows: _wputenv(env) copies the *env* string and doesn't require
the caller to manage the variable memory.
2020-01-22 21:53:26 +01:00
Victor Stinner b73dd02ea7
Revert "bpo-39413: Implement os.unsetenv() on Windows (GH-18104)" (GH-18124)
This reverts commit 56cd3710a1.
2020-01-22 21:11:17 +01:00
Victor Stinner 623ed6171e
bpo-39406: Add PY_PUTENV_DICT macro to posixmodule.c (GH-18106)
Rename posix_putenv_garbage to putenv_dict.
2020-01-21 19:25:32 +01:00
Victor Stinner 56cd3710a1
bpo-39413: Implement os.unsetenv() on Windows (GH-18104)
The os.unsetenv() function is now also available on Windows.

It is implemented with SetEnvironmentVariableW(name, NULL).
2020-01-21 16:13:09 +01:00
Benoit Hudson 723f71abf7 bpo-37931: Fix crash on OSX re-initializing os.environ (GH-15428)
On most platforms, the `environ` symbol is accessible everywhere.

In a dylib on OSX, it's not easily accessible, you need to find it with
_NSGetEnviron.

The code was caching the *value* of environ. But a setenv() can change the value,
leaving garbage at the old value. Fix: don't cache the value of environ, just
read it every time.
2019-12-06 20:15:03 +01:00
Eddie Elizondo e4db1f05e9 closes bpo-38803: Fix leak in posixmodule. (GH-17373) 2019-11-25 19:07:37 -08:00