bpo-38061: subprocess uses closefrom() on FreeBSD (GH-19697)

Optimize the subprocess module on FreeBSD using closefrom().
A single close(fd) syscall is cheap, but when sysconf(_SC_OPEN_MAX)
is high, the loop calling close(fd) on each file descriptor can take
several milliseconds.

The workaround on FreeBSD to improve performance was to load and
mount the fdescfs kernel module, but this is not enabled by default.

Initial patch by Ed Maste (emaste), Conrad Meyer (cem), Kyle Evans
(kevans) and Kubilay Kocak (koobs):
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=242274
This commit is contained in:
Victor Stinner 2020-04-24 12:06:58 +02:00 committed by GitHub
parent 162c567d16
commit e6f8abd500
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
3 changed files with 22 additions and 1 deletions

View File

@ -471,6 +471,10 @@ Optimizations
until the main thread handles signals.
(Contributed by Victor Stinner in :issue:`40010`.)
* Optimize the :mod:`subprocess` module on FreeBSD using ``closefrom()``.
(Contributed by Ed Maste, Conrad Meyer, Kyle Evans, Kubilay Kocak and Victor
Stinner in :issue:`38061`.)
Build and C API Changes
=======================

View File

@ -0,0 +1,11 @@
Optimize the :mod:`subprocess` module on FreeBSD using ``closefrom()``.
A single ``close(fd)`` syscall is cheap, but when ``sysconf(_SC_OPEN_MAX)`` is
high, the loop calling ``close(fd)`` on each file descriptor can take several
milliseconds.
The workaround on FreeBSD to improve performance was to load and mount the
fdescfs kernel module, but this is not enabled by default.
Initial patch by Ed Maste (emaste), Conrad Meyer (cem), Kyle Evans (kevans) and
Kubilay Kocak (koobs):
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=242274

View File

@ -264,9 +264,15 @@ _close_fds_by_brute_force(long start_fd, PyObject *py_fds_to_keep)
start_fd = keep_fd + 1;
}
if (start_fd <= end_fd) {
#if defined(__FreeBSD__)
/* Any errors encountered while closing file descriptors are ignored */
closefrom(start_fd);
#else
for (fd_num = start_fd; fd_num < end_fd; ++fd_num) {
close(fd_num);
/* Ignore errors */
(void)close(fd_num);
}
#endif
}
}