Issue #18571: Implementation of the PEP 446: file descriptors and file handles

are now created non-inheritable; add functions os.get/set_inheritable(),
os.get/set_handle_inheritable() and socket.socket.get/set_inheritable().
This commit is contained in:
Victor Stinner 2013-08-28 00:53:59 +02:00
parent 46e1ce214b
commit daf455554b
51 changed files with 1448 additions and 317 deletions

View File

@ -979,6 +979,8 @@ are always available. They are listed here in alphabetical order.
:mod:`os.open` as *opener* results in functionality similar to passing
``None``).
The newly created file is :ref:`non-inheritable <fd_inheritance>`.
The following example uses the :ref:`dir_fd <dir_fd>` parameter of the
:func:`os.open` function to open a file relative to a given directory::
@ -992,10 +994,6 @@ are always available. They are listed here in alphabetical order.
...
>>> os.close(dir_fd) # don't leak a file descriptor
.. versionchanged:: 3.3
The *opener* parameter was added.
The ``'x'`` mode was added.
The type of :term:`file object` returned by the :func:`open` function
depends on the mode. When :func:`open` is used to open a file in a text
mode (``'w'``, ``'r'``, ``'wt'``, ``'rt'``, etc.), it returns a subclass of
@ -1022,10 +1020,15 @@ are always available. They are listed here in alphabetical order.
and :mod:`shutil`.
.. versionchanged:: 3.3
The *opener* parameter was added.
The ``'x'`` mode was added.
:exc:`IOError` used to be raised, it is now an alias of :exc:`OSError`.
:exc:`FileExistsError` is now raised if the file opened in exclusive
creation mode (``'x'``) already exists.
.. versionchanged:: 3.4
The file is now non-inheritable.
.. XXX works for bytes too, but should it?
.. function:: ord(c)

View File

@ -522,6 +522,8 @@ Raw File I/O
:mod:`os.open` as *opener* results in functionality similar to passing
``None``).
The newly created file is :ref:`non-inheritable <fd_inheritance>`.
See the :func:`open` built-in function for examples on using the *opener*
parameter.
@ -529,6 +531,9 @@ Raw File I/O
The *opener* parameter was added.
The ``'x'`` mode was added.
.. versionchanged:: 3.4
The file is now non-inheritable.
In addition to the attributes and methods from :class:`IOBase` and
:class:`RawIOBase`, :class:`FileIO` provides the following data
attributes:

View File

@ -685,17 +685,30 @@ as internal buffering of data.
.. function:: dup(fd)
Return a duplicate of file descriptor *fd*.
Return a duplicate of file descriptor *fd*. The new file descriptor is
:ref:`non-inheritable <fd_inheritance>`.
On Windows, when duplicating a standard stream (0: stdin, 1: stdout,
2: stderr), the new file descriptor is :ref:`inheritable
<fd_inheritance>`.
Availability: Unix, Windows.
.. versionchanged:: 3.4
The new file descriptor is now non-inheritable.
.. function:: dup2(fd, fd2)
.. function:: dup2(fd, fd2, inheritable=True)
Duplicate file descriptor *fd* to *fd2*, closing the latter first if necessary.
The file descriptor *fd2* is :ref:`inheritable <fd_inheritance>` by default,
or non-inheritable if *inheritable* is ``False``.
Availability: Unix, Windows.
.. versionchanged:: 3.4
Add the optional *inheritable* parameter.
.. function:: fchmod(fd, mode)
@ -848,6 +861,7 @@ as internal buffering of data.
Open the file *file* and set various flags according to *flags* and possibly
its mode according to *mode*. When computing *mode*, the current umask value
is first masked out. Return the file descriptor for the newly opened file.
The new file descriptor is :ref:`non-inheritable <fd_inheritance>`.
For a description of the flag and mode values, see the C run-time documentation;
flag constants (like :const:`O_RDONLY` and :const:`O_WRONLY`) are defined in
@ -859,6 +873,9 @@ as internal buffering of data.
Availability: Unix, Windows.
.. versionchanged:: 3.4
The new file descriptor is now non-inheritable.
.. note::
This function is intended for low-level I/O. For normal usage, use the
@ -933,20 +950,28 @@ or `the MSDN <http://msdn.microsoft.com/en-us/library/z0kc8e3z.aspx>`_ on Window
.. index:: module: pty
Open a new pseudo-terminal pair. Return a pair of file descriptors ``(master,
slave)`` for the pty and the tty, respectively. For a (slightly) more portable
approach, use the :mod:`pty` module.
Open a new pseudo-terminal pair. Return a pair of file descriptors
``(master, slave)`` for the pty and the tty, respectively. The new file
descriptors are :ref:`non-inheritable <fd_inheritance>`. For a (slightly) more
portable approach, use the :mod:`pty` module.
Availability: some flavors of Unix.
.. versionchanged:: 3.4
The new file descriptors are now non-inheritable.
.. function:: pipe()
Create a pipe. Return a pair of file descriptors ``(r, w)`` usable for reading
and writing, respectively.
Create a pipe. Return a pair of file descriptors ``(r, w)`` usable for
reading and writing, respectively. The new file descriptor are
:ref:`non-inheritable <fd_inheritance>`.
Availability: Unix, Windows.
.. versionchanged:: 3.4
The new file descriptors are now non-inheritable.
.. function:: pipe2(flags)
@ -1178,6 +1203,50 @@ Querying the size of a terminal
Height of the terminal window in characters.
.. _fd_inheritance:
Inheritance of File Descriptors
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
A file descriptor has a inheritable flag which indicates if the file descriptor
can be inherited or not in child processes. Since Python 3.4, file descriptors
created by Python are non-inheritable by default.
On UNIX, non-inheritable file descriptors are closed in child processes at the
execution of a new program, other file descriptors are inherited.
On Windows, non-inheritable handles and file descriptors are closed in child
processes, except standard streams (file descriptors 0, 1 and 2: stdin, stdout
and stderr) which are always inherited. Using :func:`os.spawn*` functions,
all inheritable handles and all inheritable file descriptors are inherited.
Using the :mod:`subprocess` module, all file descriptors except standard
streams are closed, inheritable handles are only inherited if the *close_fds*
parameter is ``False``.
.. versionadded:: 3.4
.. function:: get_inheritable(fd)
Get the `inheritable flag <fd_inheritance>`_ of the specified file
descriptor. Return a :class:`bool`.
.. function:: set_inheritable(fd, inheritable)
Set the `inheritable flag <fd_inheritance>`_ of the specified file descriptor.
.. function:: get_handle_inheritable(handle)
Get the `inheritable flag <fd_inheritance>`_ of the specified handle. Return a :class:`bool`.
Availability: Windows.
.. function:: set_handle_inheritable(handle, inheritable)
Set the `inheritable flag <fd_inheritance>`_ of the specified handle.
Availability: Windows.
.. _os-file-dir:
Files and Directories

View File

@ -37,8 +37,13 @@ The module defines the following:
increases this value, :c:func:`devpoll` may return an
incomplete list of active file descriptors.
The new file descriptor is :ref:`non-inheritable <fd_inheritance>`.
.. versionadded:: 3.3
.. versionchanged:: 3.4
The new file descriptor is now non-inheritable.
.. function:: epoll(sizehint=-1, flags=0)
(Only supported on Linux 2.5.44 and newer.) Return an edge polling object,
@ -49,11 +54,14 @@ The module defines the following:
:ref:`epoll-objects` below for the methods supported by epolling objects.
They also support the :keyword:`with` statement.
The new file descriptor is :ref:`non-inheritable <fd_inheritance>`.
.. versionchanged:: 3.3
Added the *flags* parameter.
.. versionchanged:: 3.4
Support for the :keyword:`with` statement was added.
The new file descriptor is now non-inheritable.
.. function:: poll()
@ -69,6 +77,11 @@ The module defines the following:
(Only supported on BSD.) Returns a kernel queue object; see section
:ref:`kqueue-objects` below for the methods supported by kqueue objects.
The new file descriptor is :ref:`non-inheritable <fd_inheritance>`.
.. versionchanged:: 3.4
The new file descriptor is now non-inheritable.
.. function:: kevent(ident, filter=KQ_FILTER_READ, flags=KQ_EV_ADD, fflags=0, data=0, udata=0)

View File

@ -460,7 +460,7 @@ The module :mod:`socket` exports the following constants and functions:
``'udp'``, otherwise any protocol will match.
.. function:: socket([family[, type[, proto]]])
.. function:: socket(family=AF_INET, type=SOCK_STREAM, proto=0, fileno=None)
Create a new socket using the given address family, socket type and protocol
number. The address family should be :const:`AF_INET` (the default),
@ -471,6 +471,8 @@ The module :mod:`socket` exports the following constants and functions:
case where the address family is :const:`AF_CAN` the protocol should be one
of :const:`CAN_RAW` or :const:`CAN_BCM`.
The newly created socket is :ref:`non-inheritable <fd_inheritance>`.
.. versionchanged:: 3.3
The AF_CAN family was added.
The AF_RDS family was added.
@ -478,6 +480,10 @@ The module :mod:`socket` exports the following constants and functions:
.. versionchanged:: 3.4
The CAN_BCM protocol was added.
.. versionchanged:: 3.4
The socket is now non-inheritable.
.. function:: socketpair([family[, type[, proto]]])
Build a pair of connected socket objects using the given address family, socket
@ -486,12 +492,17 @@ The module :mod:`socket` exports the following constants and functions:
if defined on the platform; otherwise, the default is :const:`AF_INET`.
Availability: Unix.
The newly created sockets are :ref:`non-inheritable <fd_inheritance>`.
.. versionchanged:: 3.2
The returned socket objects now support the whole socket API, rather
than a subset.
.. versionchanged:: 3.4
The sockets are now non-inheritable.
.. function:: fromfd(fd, family, type[, proto])
.. function:: fromfd(fd, family, type, proto=0)
Duplicate the file descriptor *fd* (an integer as returned by a file object's
:meth:`fileno` method) and build a socket object from the result. Address
@ -502,6 +513,11 @@ The module :mod:`socket` exports the following constants and functions:
a socket passed to a program as standard input or output (such as a server
started by the Unix inet daemon). The socket is assumed to be in blocking mode.
The newly created socket is :ref:`non-inheritable <fd_inheritance>`.
.. versionchanged:: 3.4
The socket is now non-inheritable.
.. function:: ntohl(x)
@ -730,6 +746,11 @@ correspond to Unix system calls applicable to sockets.
*new* socket object usable to send and receive data on the connection, and
*address* is the address bound to the socket on the other end of the connection.
The newly created socket is :ref:`non-inheritable <fd_inheritance>`.
.. versionchanged:: 3.4
The socket is now non-inheritable.
.. method:: socket.bind(address)
@ -775,6 +796,16 @@ correspond to Unix system calls applicable to sockets.
.. versionadded:: 3.2
.. method:: socket.dup()
Duplicate the socket.
The newly created socket is :ref:`non-inheritable <fd_inheritance>`.
.. versionchanged:: 3.4
The socket is now non-inheritable.
.. method:: socket.fileno()
Return the socket's file descriptor (a small integer). This is useful with
@ -785,6 +816,15 @@ correspond to Unix system calls applicable to sockets.
this limitation.
.. method:: socket.get_inheritable()
Get the :ref:`inheritable flag <fd_inheritance>` of the socket's file
descriptor or socket's handle: ``True`` if the socket can be inherited in
child processes, ``False`` if it cannot.
.. versionadded:: 3.4
.. method:: socket.getpeername()
Return the remote address to which the socket is connected. This is useful to
@ -1068,6 +1108,14 @@ correspond to Unix system calls applicable to sockets.
.. versionadded:: 3.3
.. method:: socket.set_inheritable(inheritable)
Set the :ref:`inheritable flag <fd_inheritance>` of the socket's file
descriptor or socket's handle.
.. versionadded:: 3.4
.. method:: socket.setblocking(flag)
Set blocking or non-blocking mode of the socket: if *flag* is false, the

View File

@ -96,6 +96,7 @@ New built-in features:
* :ref:`PEP 442: Safe object finalization <pep-442>`.
* :ref:`PEP 445: Configurable memory allocators <pep-445>`.
* :ref:`PEP 446: Make newly created file descriptors non-inheritable <pep-446>`.
Implementation improvements:
@ -118,6 +119,19 @@ Security improvements:
Please read on for a comprehensive list of user-facing changes.
.. _pep-446:
PEP 446: Make newly created file descriptors non-inheritable
============================================================
The :pep:`446` makes newly created file descriptors `non-inheritable
<fd_inheritance>`_. New functions and methods:
* :func:`os.get_inheritable`, :func:`os.set_inheritable`
* :func:`os.get_handle_inheritable`, :func:`os.set_handle_inheritable`
* :meth:`socket.socket.get_inheritable`, :meth:`socket.socket.set_inheritable`
.. _pep-445:
PEP 445: Add new APIs to customize Python memory allocators
@ -267,6 +281,16 @@ Also, except when using the old *fork* start method, child processes
will no longer inherit unneeded handles/file descriptors from their parents.
os
--
New functions to get and set the `inheritable flag <fd_inheritance>`_ of a file
descriptors or a Windows handle:
* :func:`os.get_inheritable`, :func:`os.set_inheritable`
* :func:`os.get_handle_inheritable`, :func:`os.set_handle_inheritable`
poplib
------
@ -288,6 +312,15 @@ try/except statement by code that only cares whether or not an error occurred.
(:issue:`2118`).
socket
------
Socket objects have new methods to get or set their `inheritable flag
<fd_inheritance>`_:
* :meth:`socket.socket.get_inheritable`, :meth:`socket.socket.set_inheritable`
ssl
---

View File

@ -27,11 +27,19 @@ PyAPI_FUNC(int) _Py_stat(
struct stat *statbuf);
#endif
PyAPI_FUNC(int) _Py_open(
const char *pathname,
int flags);
PyAPI_FUNC(FILE *) _Py_wfopen(
const wchar_t *path,
const wchar_t *mode);
PyAPI_FUNC(FILE*) _Py_fopen(
const char *pathname,
const char *mode);
PyAPI_FUNC(FILE*) _Py_fopen_obj(
PyObject *path,
const char *mode);
@ -53,6 +61,13 @@ PyAPI_FUNC(wchar_t*) _Py_wgetcwd(
wchar_t *buf,
size_t size);
PyAPI_FUNC(int) _Py_get_inheritable(int fd);
PyAPI_FUNC(int) _Py_set_inheritable(int fd, int inheritable,
int *atomic_flag_works);
PyAPI_FUNC(int) _Py_dup(int fd);
#ifdef __cplusplus
}
#endif

View File

@ -129,6 +129,8 @@ def open(file, mode="r", buffering=-1, encoding=None, errors=None,
be kept open when the file is closed. This does not work when a file name is
given and must be True in that case.
The newly created file is non-inheritable.
A custom opener can be used by passing a callable as *opener*. The
underlying file descriptor for the file object is then obtained by calling
*opener* with (*file*, *flags*). *opener* must return an open file

View File

@ -509,7 +509,7 @@ if sys.platform != 'win32':
c1 = Connection(s1.detach())
c2 = Connection(s2.detach())
else:
fd1, fd2 = util.pipe()
fd1, fd2 = os.pipe()
c1 = Connection(fd1, writable=False)
c2 = Connection(fd2, readable=False)
@ -536,7 +536,9 @@ else:
_winapi.FILE_FLAG_FIRST_PIPE_INSTANCE,
_winapi.PIPE_TYPE_MESSAGE | _winapi.PIPE_READMODE_MESSAGE |
_winapi.PIPE_WAIT,
1, obsize, ibsize, _winapi.NMPWAIT_WAIT_FOREVER, _winapi.NULL
1, obsize, ibsize, _winapi.NMPWAIT_WAIT_FOREVER,
# default security descriptor: the handle cannot be inherited
_winapi.NULL
)
h2 = _winapi.CreateFile(
address, access, 0, _winapi.NULL, _winapi.OPEN_EXISTING,

View File

@ -60,8 +60,8 @@ def connect_to_new_process(fds):
raise ValueError('too many fds')
with socket.socket(socket.AF_UNIX) as client:
client.connect(_forkserver_address)
parent_r, child_w = util.pipe()
child_r, parent_w = util.pipe()
parent_r, child_w = os.pipe()
child_r, parent_w = os.pipe()
allfds = [child_r, child_w, _forkserver_alive_fd,
semaphore_tracker._semaphore_tracker_fd]
allfds += fds

View File

@ -66,7 +66,7 @@ class Popen(object):
def _launch(self, process_obj):
code = 1
parent_r, child_w = util.pipe()
parent_r, child_w = os.pipe()
self.pid = os.fork()
if self.pid == 0:
try:

View File

@ -54,8 +54,8 @@ class Popen(popen_fork.Popen):
parent_r = child_w = child_r = parent_w = None
try:
parent_r, child_w = util.pipe()
child_r, parent_w = util.pipe()
parent_r, child_w = os.pipe()
child_r, parent_w = os.pipe()
cmd = spawn.get_command_line(tracker_fd=tracker_fd,
pipe_handle=child_r)
self._fds.extend([child_r, child_w])

View File

@ -45,7 +45,7 @@ def ensure_running():
except Exception:
pass
cmd = 'from multiprocessing.semaphore_tracker import main; main(%d)'
r, w = util.pipe()
r, w = os.pipe()
try:
fds_to_pass.append(r)
# process will out live us, so no need to wait on pid

View File

@ -365,7 +365,7 @@ def spawnv_passfds(path, args, passfds):
if flag & fcntl.FD_CLOEXEC:
fcntl.fcntl(fd, fcntl.F_SETFD, flag & ~fcntl.FD_CLOEXEC)
tmp.append((fd, flag))
errpipe_read, errpipe_write = _posixsubprocess.cloexec_pipe()
errpipe_read, errpipe_write = os.pipe()
try:
return _posixsubprocess.fork_exec(
args, [os.fsencode(path)], True, passfds, None, None,
@ -381,7 +381,9 @@ def spawnv_passfds(path, args, passfds):
#
# Return pipe with CLOEXEC set on fds
#
# Deprecated: os.pipe() creates non-inheritable file descriptors
# since Python 3.4
#
def pipe():
import _posixsubprocess
return _posixsubprocess.cloexec_pipe()
return os.pipe()

View File

@ -137,7 +137,8 @@ class socket(_socket.socket):
def dup(self):
"""dup() -> socket object
Return a new socket object connected to the same system resource.
Duplicate the socket. Return a new socket object connected to the same
system resource. The new socket is non-inheritable.
"""
fd = dup(self.fileno())
sock = self.__class__(self.family, self.type, self.proto, fileno=fd)
@ -229,6 +230,20 @@ class socket(_socket.socket):
self._closed = True
return super().detach()
if os.name == 'nt':
def get_inheritable(self):
return os.get_handle_inheritable(self.fileno())
def set_inheritable(self, inheritable):
os.set_handle_inheritable(self.fileno(), inheritable)
else:
def get_inheritable(self):
return os.get_inheritable(self.fileno())
def set_inheritable(self, inheritable):
os.set_inheritable(self.fileno(), inheritable)
get_inheritable.__doc__ = "Get the inheritable flag of the socket"
set_inheritable.__doc__ = "Set the inheritable flag of the socket"
def fromfd(fd, family, type, proto=0):
""" fromfd(fd, family, type[, proto]) -> socket object

View File

@ -405,7 +405,6 @@ else:
import select
_has_poll = hasattr(select, 'poll')
import _posixsubprocess
_create_pipe = _posixsubprocess.cloexec_pipe
# When select or poll has indicated that the file is writable,
# we can write up to _PIPE_BUF bytes without risk of blocking.
@ -1258,7 +1257,7 @@ class Popen(object):
if stdin is None:
pass
elif stdin == PIPE:
p2cread, p2cwrite = _create_pipe()
p2cread, p2cwrite = os.pipe()
elif stdin == DEVNULL:
p2cread = self._get_devnull()
elif isinstance(stdin, int):
@ -1270,7 +1269,7 @@ class Popen(object):
if stdout is None:
pass
elif stdout == PIPE:
c2pread, c2pwrite = _create_pipe()
c2pread, c2pwrite = os.pipe()
elif stdout == DEVNULL:
c2pwrite = self._get_devnull()
elif isinstance(stdout, int):
@ -1282,7 +1281,7 @@ class Popen(object):
if stderr is None:
pass
elif stderr == PIPE:
errread, errwrite = _create_pipe()
errread, errwrite = os.pipe()
elif stderr == STDOUT:
errwrite = c2pwrite
elif stderr == DEVNULL:
@ -1334,7 +1333,7 @@ class Popen(object):
# For transferring possible exec failure from child to parent.
# Data format: "exception name:hex errno:description"
# Pickle is not used; it is complex and involves memory allocation.
errpipe_read, errpipe_write = _create_pipe()
errpipe_read, errpipe_write = os.pipe()
try:
try:
# We must avoid complex work that could involve

View File

@ -34,23 +34,6 @@ import os as _os
import errno as _errno
from random import Random as _Random
try:
import fcntl as _fcntl
except ImportError:
def _set_cloexec(fd):
pass
else:
def _set_cloexec(fd):
try:
flags = _fcntl.fcntl(fd, _fcntl.F_GETFD, 0)
except OSError:
pass
else:
# flags read successfully, modify
flags |= _fcntl.FD_CLOEXEC
_fcntl.fcntl(fd, _fcntl.F_SETFD, flags)
try:
import _thread
except ImportError:
@ -58,10 +41,6 @@ except ImportError:
_allocate_lock = _thread.allocate_lock
_text_openflags = _os.O_RDWR | _os.O_CREAT | _os.O_EXCL
if hasattr(_os, 'O_CLOEXEC'):
_text_openflags |= _os.O_CLOEXEC
if hasattr(_os, 'O_NOINHERIT'):
_text_openflags |= _os.O_NOINHERIT
if hasattr(_os, 'O_NOFOLLOW'):
_text_openflags |= _os.O_NOFOLLOW
@ -90,8 +69,8 @@ else:
# Fallback. All we need is something that raises OSError if the
# file doesn't exist.
def _stat(fn):
f = open(fn)
f.close()
fd = _os.open(fn, _os.O_RDONLY)
os.close(fd)
def _exists(fn):
try:
@ -217,7 +196,6 @@ def _mkstemp_inner(dir, pre, suf, flags):
file = _os.path.join(dir, pre + name + suf)
try:
fd = _os.open(file, flags, 0o600)
_set_cloexec(fd)
return (fd, _os.path.abspath(file))
except FileExistsError:
continue # try again

View File

@ -0,0 +1,22 @@
"""Similar to fd_status.py, but only checks file descriptors passed on the
command line."""
import errno
import os
import sys
import stat
if __name__ == "__main__":
fds = map(int, sys.argv[1:])
inherited = []
for fd in fds:
try:
st = os.fstat(fd)
except OSError as e:
if e.errno == errno.EBADF:
continue
raise
# Ignore Solaris door files
if not stat.S_ISDOOR(st.st_mode):
inherited.append(fd)
print(','.join(map(str, inherited)))

View File

@ -744,7 +744,12 @@ class BaseTestAPI:
s.create_socket(self.family)
self.assertEqual(s.socket.family, self.family)
SOCK_NONBLOCK = getattr(socket, 'SOCK_NONBLOCK', 0)
self.assertEqual(s.socket.type, socket.SOCK_STREAM | SOCK_NONBLOCK)
sock_type = socket.SOCK_STREAM | SOCK_NONBLOCK
if hasattr(socket, 'SOCK_CLOEXEC'):
self.assertIn(s.socket.type,
(sock_type | socket.SOCK_CLOEXEC, sock_type))
else:
self.assertEqual(s.socket.type, sock_type)
def test_bind(self):
if HAS_UNIX_SOCKETS and self.family == socket.AF_UNIX:

View File

@ -1015,6 +1015,11 @@ class BuiltinTest(unittest.TestCase):
os.environ.clear()
os.environ.update(old_environ)
def test_open_non_inheritable(self):
fileobj = open(__file__)
with fileobj:
self.assertFalse(os.get_inheritable(fileobj.fileno()))
def test_ord(self):
self.assertEqual(ord(' '), 32)
self.assertEqual(ord('A'), 65)

View File

@ -2,7 +2,11 @@
# Initial tests are copied as is from "test_poll.py"
import os, select, random, unittest, sys
import os
import random
import select
import sys
import unittest
from test.support import TESTFN, run_unittest
try:
@ -111,6 +115,11 @@ class DevPollTests(unittest.TestCase):
self.assertRaises(ValueError, devpoll.register, fd, fd, select.POLLIN)
self.assertRaises(ValueError, devpoll.unregister, fd)
def test_fd_non_inheritable(self):
devpoll = select.devpoll()
self.addCleanup(devpoll.close)
self.assertEqual(os.get_inheritable(devpoll.fileno()), False)
def test_main():
run_unittest(DevPollTests)

View File

@ -21,10 +21,11 @@
"""
Tests for epoll wrapper.
"""
import socket
import errno
import time
import os
import select
import socket
import time
import unittest
from test import support
@ -249,6 +250,11 @@ class TestEPoll(unittest.TestCase):
self.assertRaises(ValueError, epoll.register, fd, select.EPOLLIN)
self.assertRaises(ValueError, epoll.unregister, fd)
def test_fd_non_inheritable(self):
epoll = select.epoll()
self.addCleanup(epoll.close)
self.assertEqual(os.get_inheritable(epoll.fileno()), False)
def test_main():
support.run_unittest(TestEPoll)

View File

@ -206,6 +206,11 @@ class TestKQueue(unittest.TestCase):
# operations must fail with ValueError("I/O operation on closed ...")
self.assertRaises(ValueError, kqueue.control, None, 4)
def test_fd_non_inheritable(self):
kqueue = select.kqueue()
self.addCleanup(kqueue.close)
self.assertEqual(os.get_inheritable(kqueue.fileno()), False)
def test_main():
support.run_unittest(TestKQueue)

View File

@ -2298,6 +2298,72 @@ class CPUCountTests(unittest.TestCase):
else:
self.skipTest("Could not determine the number of CPUs")
class FDInheritanceTests(unittest.TestCase):
def test_get_inheritable(self):
fd = os.open(__file__, os.O_RDONLY)
self.addCleanup(os.close, fd)
for inheritable in (False, True):
os.set_inheritable(fd, inheritable)
self.assertEqual(os.get_inheritable(fd), inheritable)
def test_set_inheritable(self):
fd = os.open(__file__, os.O_RDONLY)
self.addCleanup(os.close, fd)
os.set_inheritable(fd, True)
self.assertEqual(os.get_inheritable(fd), True)
def test_open(self):
fd = os.open(__file__, os.O_RDONLY)
self.addCleanup(os.close, fd)
self.assertEqual(os.get_inheritable(fd), False)
@unittest.skipUnless(hasattr(os, 'pipe'), "need os.pipe()")
def test_pipe(self):
rfd, wfd = os.pipe()
self.addCleanup(os.close, rfd)
self.addCleanup(os.close, wfd)
self.assertEqual(os.get_inheritable(rfd), False)
self.assertEqual(os.get_inheritable(wfd), False)
def test_dup(self):
fd1 = os.open(__file__, os.O_RDONLY)
self.addCleanup(os.close, fd1)
fd2 = os.dup(fd1)
self.addCleanup(os.close, fd2)
self.assertEqual(os.get_inheritable(fd2), False)
@unittest.skipUnless(hasattr(os, 'dup2'), "need os.dup2()")
def test_dup2(self):
fd = os.open(__file__, os.O_RDONLY)
self.addCleanup(os.close, fd)
# inheritable by default
fd2 = os.open(__file__, os.O_RDONLY)
try:
os.dup2(fd, fd2)
self.assertEqual(os.get_inheritable(fd2), True)
finally:
os.close(fd2)
# force non-inheritable
fd3 = os.open(__file__, os.O_RDONLY)
try:
os.dup2(fd, fd3, inheritable=False)
self.assertEqual(os.get_inheritable(fd3), False)
finally:
os.close(fd3)
@unittest.skipUnless(hasattr(os, 'openpty'), "need os.openpty()")
def test_openpty(self):
master_fd, slave_fd = os.openpty()
self.addCleanup(os.close, master_fd)
self.addCleanup(os.close, slave_fd)
self.assertEqual(os.get_inheritable(master_fd), False)
self.assertEqual(os.get_inheritable(slave_fd), False)
@support.reap_threads
def test_main():
support.run_unittest(
@ -2330,6 +2396,7 @@ def test_main():
OSErrorTests,
RemoveDirsTests,
CPUCountTests,
FDInheritanceTests,
)
if __name__ == "__main__":

View File

@ -22,10 +22,6 @@ import signal
import math
import pickle
import struct
try:
import fcntl
except ImportError:
fcntl = False
try:
import multiprocessing
except ImportError:
@ -1108,8 +1104,14 @@ class GeneralModuleTests(unittest.TestCase):
def testNewAttributes(self):
# testing .family, .type and .protocol
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.assertEqual(sock.family, socket.AF_INET)
if hasattr(socket, 'SOCK_CLOEXEC'):
self.assertIn(sock.type,
(socket.SOCK_STREAM | socket.SOCK_CLOEXEC,
socket.SOCK_STREAM))
else:
self.assertEqual(sock.type, socket.SOCK_STREAM)
self.assertEqual(sock.proto, 0)
sock.close()
@ -4749,16 +4751,46 @@ class ContextManagersTest(ThreadedTCPSocketTest):
self.assertRaises(OSError, sock.sendall, b'foo')
class InheritanceTest(unittest.TestCase):
@unittest.skipUnless(hasattr(socket, "SOCK_CLOEXEC"),
"SOCK_CLOEXEC not defined")
@unittest.skipUnless(fcntl, "module fcntl not available")
class CloexecConstantTest(unittest.TestCase):
@support.requires_linux_version(2, 6, 28)
def test_SOCK_CLOEXEC(self):
with socket.socket(socket.AF_INET,
socket.SOCK_STREAM | socket.SOCK_CLOEXEC) as s:
self.assertTrue(s.type & socket.SOCK_CLOEXEC)
self.assertTrue(fcntl.fcntl(s, fcntl.F_GETFD) & fcntl.FD_CLOEXEC)
self.assertTrue(sock.get_inheritable())
def test_default_inheritable(self):
sock = socket.socket()
with sock:
self.assertEqual(sock.get_inheritable(), False)
def test_dup(self):
sock = socket.socket()
with sock:
newsock = sock.dup()
sock.close()
with newsock:
self.assertEqual(newsock.get_inheritable(), False)
def test_set_inheritable(self):
sock = socket.socket()
with sock:
sock.set_inheritable(True)
self.assertEqual(sock.get_inheritable(), True)
sock.set_inheritable(False)
self.assertEqual(sock.get_inheritable(), False)
@unittest.skipUnless(hasattr(socket, "socketpair"),
"need socket.socketpair()")
def test_socketpair(self):
s1, s2 = socket.socketpair()
self.addCleanup(s1.close)
self.addCleanup(s2.close)
self.assertEqual(s1.get_inheritable(), False)
self.assertEqual(s2.get_inheritable(), False)
@unittest.skipUnless(hasattr(socket, "SOCK_NONBLOCK"),
@ -4927,7 +4959,7 @@ def test_main():
NetworkConnectionAttributesTest,
NetworkConnectionBehaviourTest,
ContextManagersTest,
CloexecConstantTest,
InheritanceTest,
NonblockConstantTest
])
if hasattr(socket, "socketpair"):

View File

@ -1501,16 +1501,28 @@ class POSIXProcessTestCase(BaseTestCase):
# Terminating a dead process
self._kill_dead_process('terminate')
def _save_fds(self, save_fds):
fds = []
for fd in save_fds:
inheritable = os.get_inheritable(fd)
saved = os.dup(fd)
fds.append((fd, saved, inheritable))
return fds
def _restore_fds(self, fds):
for fd, saved, inheritable in fds:
os.dup2(saved, fd, inheritable=inheritable)
os.close(saved)
def check_close_std_fds(self, fds):
# Issue #9905: test that subprocess pipes still work properly with
# some standard fds closed
stdin = 0
newfds = []
for a in fds:
b = os.dup(a)
newfds.append(b)
if a == 0:
stdin = b
saved_fds = self._save_fds(fds)
for fd, saved, inheritable in saved_fds:
if fd == 0:
stdin = saved
break
try:
for fd in fds:
os.close(fd)
@ -1525,10 +1537,7 @@ class POSIXProcessTestCase(BaseTestCase):
err = support.strip_python_stderr(err)
self.assertEqual((out, err), (b'apple', b'orange'))
finally:
for b, a in zip(newfds, fds):
os.dup2(b, a)
for b in newfds:
os.close(b)
self._restore_fds(saved_fds)
def test_close_fd_0(self):
self.check_close_std_fds([0])
@ -1568,7 +1577,7 @@ class POSIXProcessTestCase(BaseTestCase):
os.lseek(temp_fds[1], 0, 0)
# move the standard file descriptors out of the way
saved_fds = [os.dup(fd) for fd in range(3)]
saved_fds = self._save_fds(range(3))
try:
# duplicate the file objects over the standard fd's
for fd, temp_fd in enumerate(temp_fds):
@ -1584,10 +1593,7 @@ class POSIXProcessTestCase(BaseTestCase):
stderr=temp_fds[0])
p.wait()
finally:
# restore the original fd's underneath sys.stdin, etc.
for std, saved in enumerate(saved_fds):
os.dup2(saved, std)
os.close(saved)
self._restore_fds(saved_fds)
for fd in temp_fds:
os.lseek(fd, 0, 0)
@ -1611,7 +1617,7 @@ class POSIXProcessTestCase(BaseTestCase):
os.unlink(fname)
# save a copy of the standard file descriptors
saved_fds = [os.dup(fd) for fd in range(3)]
saved_fds = self._save_fds(range(3))
try:
# duplicate the temp files over the standard fd's 0, 1, 2
for fd, temp_fd in enumerate(temp_fds):
@ -1637,9 +1643,7 @@ class POSIXProcessTestCase(BaseTestCase):
out = os.read(stdout_no, 1024)
err = support.strip_python_stderr(os.read(stderr_no, 1024))
finally:
for std, saved in enumerate(saved_fds):
os.dup2(saved, std)
os.close(saved)
self._restore_fds(saved_fds)
self.assertEqual(out, b"got STDIN")
self.assertEqual(err, b"err")
@ -1810,6 +1814,9 @@ class POSIXProcessTestCase(BaseTestCase):
self.addCleanup(os.close, fd)
open_fds.add(fd)
for fd in open_fds:
os.set_inheritable(fd, True)
p = subprocess.Popen([sys.executable, fd_status],
stdout=subprocess.PIPE, close_fds=False)
output, ignored = p.communicate()
@ -1854,6 +1861,8 @@ class POSIXProcessTestCase(BaseTestCase):
fds = os.pipe()
self.addCleanup(os.close, fds[0])
self.addCleanup(os.close, fds[1])
os.set_inheritable(fds[0], True)
os.set_inheritable(fds[1], True)
open_fds.update(fds)
for fd in open_fds:
@ -1876,6 +1885,32 @@ class POSIXProcessTestCase(BaseTestCase):
close_fds=False, pass_fds=(fd, )))
self.assertIn('overriding close_fds', str(context.warning))
def test_pass_fds_inheritable(self):
script = support.findfile("inherited.py", subdir="subprocessdata")
inheritable, non_inheritable = os.pipe()
self.addCleanup(os.close, inheritable)
self.addCleanup(os.close, non_inheritable)
os.set_inheritable(inheritable, True)
os.set_inheritable(non_inheritable, False)
pass_fds = (inheritable, non_inheritable)
args = [sys.executable, script]
args += list(map(str, pass_fds))
p = subprocess.Popen(args,
stdout=subprocess.PIPE, close_fds=True,
pass_fds=pass_fds)
output, ignored = p.communicate()
fds = set(map(int, output.split(b',')))
# the inheritable file descriptor must be inherited, so its inheritable
# flag must be set in the child process after fork() and before exec()
self.assertEqual(fds, set(pass_fds))
# inheritable flag must not be changed in the parent process
self.assertEqual(os.get_inheritable(inheritable), True)
self.assertEqual(os.get_inheritable(non_inheritable), False)
def test_stdout_stdin_are_single_inout_fd(self):
with io.open(os.devnull, "r+") as inout:
p = subprocess.Popen([sys.executable, "-c", "import sys; sys.exit(0)"],

View File

@ -333,6 +333,7 @@ class TestMkstempInner(BaseTestCase):
v="q"
file = self.do_create()
self.assertEqual(os.get_inheritable(file.fd), False)
fd = "%d" % file.fd
try:

View File

@ -584,13 +584,6 @@ class SimpleXMLRPCServer(socketserver.TCPServer,
SimpleXMLRPCDispatcher.__init__(self, allow_none, encoding, use_builtin_types)
socketserver.TCPServer.__init__(self, addr, requestHandler, bind_and_activate)
# [Bug #1222790] If possible, set close-on-exec flag; if a
# method spawns a subprocess, the subprocess shouldn't have
# the listening socket open.
if fcntl is not None and hasattr(fcntl, 'FD_CLOEXEC'):
flags = fcntl.fcntl(self.fileno(), fcntl.F_GETFD)
flags |= fcntl.FD_CLOEXEC
fcntl.fcntl(self.fileno(), fcntl.F_SETFD, flags)
class MultiPathXMLRPCServer(SimpleXMLRPCServer):
"""Multipath XML-RPC Server

View File

@ -10,6 +10,11 @@ Projected Release date: 2013-09-08
Core and Builtins
-----------------
- Issue #18571: Implementation of the PEP 446: file descriptors and file
handles are now created non-inheritable; add functions
os.get/set_inheritable(), os.get/set_handle_inheritable() and
socket.socket.get/set_inheritable().
- Issue #11619: The parser and the import machinery do not encode Unicode
filenames anymore on Windows.

View File

@ -1694,26 +1694,24 @@ PyCursesWindow_PutWin(PyCursesWindowObject *self, PyObject *stream)
/* We have to simulate this by writing to a temporary FILE*,
then reading back, then writing to the argument stream. */
char fn[100];
int fd;
FILE *fp;
PyObject *res;
int fd = -1;
FILE *fp = NULL;
PyObject *res = NULL;
strcpy(fn, "/tmp/py.curses.putwin.XXXXXX");
fd = mkstemp(fn);
if (fd < 0)
return PyErr_SetFromErrnoWithFilename(PyExc_IOError, fn);
if (_Py_set_inheritable(fd, 0, NULL) < 0)
goto exit;
fp = fdopen(fd, "wb+");
if (fp == NULL) {
close(fd);
remove(fn);
return PyErr_SetFromErrnoWithFilename(PyExc_IOError, fn);
PyErr_SetFromErrnoWithFilename(PyExc_IOError, fn);
goto exit;
}
res = PyCursesCheckERR(putwin(self->win, fp), "putwin");
if (res == NULL) {
fclose(fp);
remove(fn);
return res;
}
if (res == NULL)
goto exit;
fseek(fp, 0, 0);
while (1) {
char buf[BUFSIZ];
@ -1727,7 +1725,12 @@ PyCursesWindow_PutWin(PyCursesWindowObject *self, PyObject *stream)
if (res == NULL)
break;
}
exit:
if (fp != NULL)
fclose(fp);
else if (fd != -1)
close(fd);
remove(fn);
return res;
}
@ -2252,12 +2255,13 @@ static PyObject *
PyCurses_GetWin(PyCursesWindowObject *self, PyObject *stream)
{
char fn[100];
int fd;
FILE *fp;
int fd = -1;
FILE *fp = NULL;
PyObject *data;
size_t datalen;
WINDOW *win;
_Py_IDENTIFIER(read);
PyObject *res = NULL;
PyCursesInitialised;
@ -2265,44 +2269,47 @@ PyCurses_GetWin(PyCursesWindowObject *self, PyObject *stream)
fd = mkstemp(fn);
if (fd < 0)
return PyErr_SetFromErrnoWithFilename(PyExc_IOError, fn);
if (_Py_set_inheritable(fd, 0, NULL) < 0)
goto error;
fp = fdopen(fd, "wb+");
if (fp == NULL) {
close(fd);
remove(fn);
return PyErr_SetFromErrnoWithFilename(PyExc_IOError, fn);
PyErr_SetFromErrnoWithFilename(PyExc_IOError, fn);
goto error;
}
data = _PyObject_CallMethodId(stream, &PyId_read, "");
if (data == NULL) {
fclose(fp);
remove(fn);
return NULL;
}
if (data == NULL)
goto error;
if (!PyBytes_Check(data)) {
PyErr_Format(PyExc_TypeError,
"f.read() returned %.100s instead of bytes",
data->ob_type->tp_name);
Py_DECREF(data);
fclose(fp);
remove(fn);
return NULL;
goto error;
}
datalen = PyBytes_GET_SIZE(data);
if (fwrite(PyBytes_AS_STRING(data), 1, datalen, fp) != datalen) {
Py_DECREF(data);
fclose(fp);
remove(fn);
return PyErr_SetFromErrnoWithFilename(PyExc_IOError, fn);
PyErr_SetFromErrnoWithFilename(PyExc_IOError, fn);
goto error;
}
Py_DECREF(data);
fseek(fp, 0, 0);
win = getwin(fp);
fclose(fp);
remove(fn);
if (win == NULL) {
PyErr_SetString(PyCursesError, catchall_NULL);
return NULL;
goto error;
}
return PyCursesWindow_New(win, NULL);
res = PyCursesWindow_New(win, NULL);
error:
if (fp != NULL)
fclose(fp);
else if (fd != -1)
close(fd);
remove(fn);
return res;
}
static PyObject *

View File

@ -202,6 +202,9 @@ check_fd(int fd)
return 0;
}
#ifdef O_CLOEXEC
extern int _Py_open_cloexec_works;
#endif
static int
fileio_init(PyObject *oself, PyObject *args, PyObject *kwds)
@ -221,6 +224,11 @@ fileio_init(PyObject *oself, PyObject *args, PyObject *kwds)
int fd = -1;
int closefd = 1;
int fd_is_own = 0;
#ifdef O_CLOEXEC
int *atomic_flag_works = &_Py_open_cloexec_works;
#elif !defined(MS_WINDOWS)
int *atomic_flag_works = NULL;
#endif
assert(PyFileIO_Check(oself));
if (self->fd >= 0) {
@ -345,6 +353,11 @@ fileio_init(PyObject *oself, PyObject *args, PyObject *kwds)
if (append)
flags |= O_APPEND;
#endif
#ifdef MS_WINDOWS
flags |= O_NOINHERIT;
#elif defined(O_CLOEXEC)
flags |= O_CLOEXEC;
#endif
if (fd >= 0) {
if (check_fd(fd))
@ -369,10 +382,18 @@ fileio_init(PyObject *oself, PyObject *args, PyObject *kwds)
else
#endif
self->fd = open(name, flags, 0666);
Py_END_ALLOW_THREADS
} else {
PyObject *fdobj = PyObject_CallFunction(
opener, "Oi", nameobj, flags);
}
else {
PyObject *fdobj;
#ifndef MS_WINDOWS
/* the opener may clear the atomic flag */
atomic_flag_works = NULL;
#endif
fdobj = PyObject_CallFunction(opener, "Oi", nameobj, flags);
if (fdobj == NULL)
goto error;
if (!PyLong_Check(fdobj)) {
@ -394,6 +415,11 @@ fileio_init(PyObject *oself, PyObject *args, PyObject *kwds)
PyErr_SetFromErrnoWithFilenameObject(PyExc_OSError, nameobj);
goto error;
}
#ifndef MS_WINDOWS
if (_Py_set_inheritable(self->fd, 0, atomic_flag_works) < 0)
goto error;
#endif
}
if (dircheck(self, nameobj) < 0)
goto error;

View File

@ -35,7 +35,7 @@
# define FD_DIR "/proc/self/fd"
#endif
#define POSIX_CALL(call) if ((call) == -1) goto error
#define POSIX_CALL(call) do { if ((call) == -1) goto error; } while (0)
/* Maximum file descriptor, initialized on module load. */
@ -136,6 +136,29 @@ _is_fd_in_sorted_fd_sequence(int fd, PyObject *fd_sequence)
return 0;
}
static int
make_inheritable(PyObject *py_fds_to_keep, int errpipe_write)
{
Py_ssize_t i, len;
len = PySequence_Length(py_fds_to_keep);
for (i = 0; i < len; ++i) {
PyObject* fdobj = PySequence_Fast_GET_ITEM(py_fds_to_keep, i);
long fd = PyLong_AsLong(fdobj);
assert(!PyErr_Occurred());
assert(0 <= fd && fd <= INT_MAX);
if (fd == errpipe_write) {
/* errpipe_write is part of py_fds_to_keep. It must be closed at
exec(), but kept open in the child process until exec() is
called. */
continue;
}
if (_Py_set_inheritable((int)fd, 1, NULL) < 0)
return -1;
}
return 0;
}
/* Close all file descriptors in the range start_fd inclusive to
* end_fd exclusive except for those in py_fds_to_keep. If the
@ -205,18 +228,8 @@ _close_open_fd_range_safe(int start_fd, int end_fd, PyObject* py_fds_to_keep)
int fd_dir_fd;
if (start_fd >= end_fd)
return;
#ifdef O_CLOEXEC
fd_dir_fd = open(FD_DIR, O_RDONLY | O_CLOEXEC, 0);
#else
fd_dir_fd = open(FD_DIR, O_RDONLY, 0);
#ifdef FD_CLOEXEC
{
int old = fcntl(fd_dir_fd, F_GETFD);
if (old != -1)
fcntl(fd_dir_fd, F_SETFD, old | FD_CLOEXEC);
}
#endif
#endif
fd_dir_fd = _Py_open(FD_DIR, O_RDONLY);
if (fd_dir_fd == -1) {
/* No way to get a list of open fds. */
_close_fds_by_brute_force(start_fd, end_fd, py_fds_to_keep);
@ -356,16 +369,16 @@ child_exec(char *const exec_array[],
/* Buffer large enough to hold a hex integer. We can't malloc. */
char hex_errno[sizeof(saved_errno)*2+1];
if (make_inheritable(py_fds_to_keep, errpipe_write) < 0)
goto error;
/* Close parent's pipe ends. */
if (p2cwrite != -1) {
if (p2cwrite != -1)
POSIX_CALL(close(p2cwrite));
}
if (c2pread != -1) {
if (c2pread != -1)
POSIX_CALL(close(c2pread));
}
if (errread != -1) {
if (errread != -1)
POSIX_CALL(close(errread));
}
POSIX_CALL(close(errpipe_read));
/* When duping fds, if there arises a situation where one of the fds is
@ -379,38 +392,34 @@ child_exec(char *const exec_array[],
dup2() removes the CLOEXEC flag but we must do it ourselves if dup2()
would be a no-op (issue #10806). */
if (p2cread == 0) {
int old = fcntl(p2cread, F_GETFD);
if (old != -1)
fcntl(p2cread, F_SETFD, old & ~FD_CLOEXEC);
} else if (p2cread != -1) {
if (_Py_set_inheritable(p2cread, 1, NULL) < 0)
goto error;
}
else if (p2cread != -1)
POSIX_CALL(dup2(p2cread, 0)); /* stdin */
}
if (c2pwrite == 1) {
int old = fcntl(c2pwrite, F_GETFD);
if (old != -1)
fcntl(c2pwrite, F_SETFD, old & ~FD_CLOEXEC);
} else if (c2pwrite != -1) {
if (_Py_set_inheritable(c2pwrite, 1, NULL) < 0)
goto error;
}
else if (c2pwrite != -1)
POSIX_CALL(dup2(c2pwrite, 1)); /* stdout */
}
if (errwrite == 2) {
int old = fcntl(errwrite, F_GETFD);
if (old != -1)
fcntl(errwrite, F_SETFD, old & ~FD_CLOEXEC);
} else if (errwrite != -1) {
POSIX_CALL(dup2(errwrite, 2)); /* stderr */
if (_Py_set_inheritable(errwrite, 1, NULL) < 0)
goto error;
}
else if (errwrite != -1)
POSIX_CALL(dup2(errwrite, 2)); /* stderr */
/* Close pipe fds. Make sure we don't close the same fd more than */
/* once, or standard fds. */
if (p2cread > 2) {
if (p2cread > 2)
POSIX_CALL(close(p2cread));
}
if (c2pwrite > 2 && c2pwrite != p2cread) {
if (c2pwrite > 2 && c2pwrite != p2cread)
POSIX_CALL(close(c2pwrite));
}
if (errwrite != c2pwrite && errwrite != p2cread && errwrite > 2) {
if (errwrite != c2pwrite && errwrite != p2cread && errwrite > 2)
POSIX_CALL(close(errwrite));
}
if (cwd)
POSIX_CALL(chdir(cwd));
@ -721,52 +730,6 @@ Returns: the child process's PID.\n\
Raises: Only on an error in the parent process.\n\
");
PyDoc_STRVAR(subprocess_cloexec_pipe_doc,
"cloexec_pipe() -> (read_end, write_end)\n\n\
Create a pipe whose ends have the cloexec flag set.");
static PyObject *
subprocess_cloexec_pipe(PyObject *self, PyObject *noargs)
{
int fds[2];
int res;
#ifdef HAVE_PIPE2
Py_BEGIN_ALLOW_THREADS
res = pipe2(fds, O_CLOEXEC);
Py_END_ALLOW_THREADS
if (res != 0 && errno == ENOSYS)
{
{
#endif
/* We hold the GIL which offers some protection from other code calling
* fork() before the CLOEXEC flags have been set but we can't guarantee
* anything without pipe2(). */
long oldflags;
res = pipe(fds);
if (res == 0) {
oldflags = fcntl(fds[0], F_GETFD, 0);
if (oldflags < 0) res = oldflags;
}
if (res == 0)
res = fcntl(fds[0], F_SETFD, oldflags | FD_CLOEXEC);
if (res == 0) {
oldflags = fcntl(fds[1], F_GETFD, 0);
if (oldflags < 0) res = oldflags;
}
if (res == 0)
res = fcntl(fds[1], F_SETFD, oldflags | FD_CLOEXEC);
#ifdef HAVE_PIPE2
}
}
#endif
if (res != 0)
return PyErr_SetFromErrno(PyExc_OSError);
return Py_BuildValue("(ii)", fds[0], fds[1]);
}
/* module level code ********************************************************/
PyDoc_STRVAR(module_doc,
@ -775,7 +738,6 @@ PyDoc_STRVAR(module_doc,
static PyMethodDef module_methods[] = {
{"fork_exec", subprocess_fork_exec, METH_VARARGS, subprocess_fork_exec_doc},
{"cloexec_pipe", subprocess_cloexec_pipe, METH_NOARGS, subprocess_cloexec_pipe_doc},
{NULL, NULL} /* sentinel */
};

View File

@ -2350,7 +2350,7 @@ load_dh_params(PySSLContext *self, PyObject *filepath)
FILE *f;
DH *dh;
f = _Py_fopen(filepath, "rb");
f = _Py_fopen_obj(filepath, "rb");
if (f == NULL) {
if (!PyErr_Occurred())
PyErr_SetFromErrnoWithFilenameObject(PyExc_OSError, filepath);

View File

@ -143,7 +143,7 @@ static void RunStartupFile(PyCompilerFlags *cf)
{
char *startup = Py_GETENV("PYTHONSTARTUP");
if (startup != NULL && startup[0] != '\0') {
FILE *fp = fopen(startup, "r");
FILE *fp = _Py_fopen(startup, "r");
if (fp != NULL) {
(void) PyRun_SimpleFileExFlags(fp, startup, 0, cf);
PyErr_Clear();

View File

@ -1208,18 +1208,18 @@ new_mmap_object(PyTypeObject *type, PyObject *args, PyObject *kwdict)
flags |= MAP_ANONYMOUS;
#else
/* SVR4 method to map anonymous memory is to open /dev/zero */
fd = devzero = open("/dev/zero", O_RDWR);
fd = devzero = _Py_open("/dev/zero", O_RDWR);
if (devzero == -1) {
Py_DECREF(m_obj);
PyErr_SetFromErrno(PyExc_OSError);
return NULL;
}
#endif
} else {
m_obj->fd = dup(fd);
}
else {
m_obj->fd = _Py_dup(fd);
if (m_obj->fd == -1) {
Py_DECREF(m_obj);
PyErr_SetFromErrno(PyExc_OSError);
return NULL;
}
}

View File

@ -115,7 +115,9 @@ newossobject(PyObject *arg)
one open at a time. This does *not* affect later I/O; OSS
provides a special ioctl() for non-blocking read/write, which is
exposed via oss_nonblock() below. */
if ((fd = open(devicename, imode|O_NONBLOCK)) == -1) {
fd = _Py_open(devicename, imode|O_NONBLOCK);
if (fd == -1) {
PyErr_SetFromErrnoWithFilename(PyExc_IOError, devicename);
return NULL;
}
@ -177,7 +179,8 @@ newossmixerobject(PyObject *arg)
devicename = "/dev/mixer";
}
if ((fd = open(devicename, O_RDWR)) == -1) {
fd = _Py_open(devicename, O_RDWR);
if (fd == -1) {
PyErr_SetFromErrnoWithFilename(PyExc_IOError, devicename);
return NULL;
}

View File

@ -3535,10 +3535,7 @@ _posix_listdir(path_t *path, PyObject *list)
#ifdef HAVE_FDOPENDIR
if (path->fd != -1) {
/* closedir() closes the FD, so we duplicate it */
Py_BEGIN_ALLOW_THREADS
fd = dup(path->fd);
Py_END_ALLOW_THREADS
fd = _Py_dup(path->fd);
if (fd == -1) {
list = posix_error();
goto exit;
@ -5768,7 +5765,7 @@ Open a pseudo-terminal, returning open fd's for both master and slave end.\n");
static PyObject *
posix_openpty(PyObject *self, PyObject *noargs)
{
int master_fd, slave_fd;
int master_fd = -1, slave_fd = -1;
#ifndef HAVE_OPENPTY
char * slave_name;
#endif
@ -5781,37 +5778,52 @@ posix_openpty(PyObject *self, PyObject *noargs)
#ifdef HAVE_OPENPTY
if (openpty(&master_fd, &slave_fd, NULL, NULL, NULL) != 0)
return posix_error();
goto posix_error;
if (_Py_set_inheritable(master_fd, 0, NULL) < 0)
goto error;
if (_Py_set_inheritable(slave_fd, 0, NULL) < 0)
goto error;
#elif defined(HAVE__GETPTY)
slave_name = _getpty(&master_fd, O_RDWR, 0666, 0);
if (slave_name == NULL)
return posix_error();
goto posix_error;
if (_Py_set_inheritable(master_fd, 0, NULL) < 0)
goto error;
slave_fd = open(slave_name, O_RDWR);
slave_fd = _Py_open(slave_name, O_RDWR);
if (slave_fd < 0)
return posix_error();
goto posix_error;
#else
master_fd = open(DEV_PTY_FILE, O_RDWR | O_NOCTTY); /* open master */
master_fd = _Py_open(DEV_PTY_FILE, O_RDWR | O_NOCTTY); /* open master */
if (master_fd < 0)
return posix_error();
goto posix_error;
sig_saved = PyOS_setsig(SIGCHLD, SIG_DFL);
/* change permission of slave */
if (grantpt(master_fd) < 0) {
PyOS_setsig(SIGCHLD, sig_saved);
return posix_error();
goto posix_error;
}
/* unlock slave */
if (unlockpt(master_fd) < 0) {
PyOS_setsig(SIGCHLD, sig_saved);
return posix_error();
goto posix_error;
}
PyOS_setsig(SIGCHLD, sig_saved);
slave_name = ptsname(master_fd); /* get name of slave */
if (slave_name == NULL)
return posix_error();
slave_fd = open(slave_name, O_RDWR | O_NOCTTY); /* open slave */
goto posix_error;
slave_fd = _Py_open(slave_name, O_RDWR | O_NOCTTY); /* open slave */
if (slave_fd < 0)
return posix_error();
goto posix_error;
#if !defined(__CYGWIN__) && !defined(HAVE_DEV_PTC)
ioctl(slave_fd, I_PUSH, "ptem"); /* push ptem */
ioctl(slave_fd, I_PUSH, "ldterm"); /* push ldterm */
@ -5823,6 +5835,14 @@ posix_openpty(PyObject *self, PyObject *noargs)
return Py_BuildValue("(ii)", master_fd, slave_fd);
posix_error:
posix_error();
error:
if (master_fd != -1)
close(master_fd);
if (slave_fd != -1)
close(slave_fd);
return NULL;
}
#endif /* defined(HAVE_OPENPTY) || defined(HAVE__GETPTY) || defined(HAVE_DEV_PTMX) */
@ -7432,6 +7452,10 @@ posix_tcsetpgrp(PyObject *self, PyObject *args)
/* Functions acting on file descriptors */
#ifdef O_CLOEXEC
extern int _Py_open_cloexec_works;
#endif
PyDoc_STRVAR(posix_open__doc__,
"open(path, flags, mode=0o777, *, dir_fd=None)\n\n\
Open a file for low level IO. Returns a file handle (integer).\n\
@ -7451,6 +7475,11 @@ posix_open(PyObject *self, PyObject *args, PyObject *kwargs)
int fd;
PyObject *return_value = NULL;
static char *keywords[] = {"path", "flags", "mode", "dir_fd", NULL};
#ifdef O_CLOEXEC
int *atomic_flag_works = &_Py_open_cloexec_works;
#elif !defined(MS_WINDOWS)
int *atomic_flag_works = NULL;
#endif
memset(&path, 0, sizeof(path));
path.function_name = "open";
@ -7465,6 +7494,12 @@ posix_open(PyObject *self, PyObject *args, PyObject *kwargs)
))
return NULL;
#ifdef MS_WINDOWS
flags |= O_NOINHERIT;
#elif defined(O_CLOEXEC)
flags |= O_CLOEXEC;
#endif
Py_BEGIN_ALLOW_THREADS
#ifdef MS_WINDOWS
if (path.wide)
@ -7484,6 +7519,13 @@ posix_open(PyObject *self, PyObject *args, PyObject *kwargs)
goto exit;
}
#ifndef MS_WINDOWS
if (_Py_set_inheritable(fd, 0, atomic_flag_works) < 0) {
close(fd);
goto exit;
}
#endif
return_value = PyLong_FromLong((long)fd);
exit:
@ -7540,13 +7582,14 @@ static PyObject *
posix_dup(PyObject *self, PyObject *args)
{
int fd;
if (!PyArg_ParseTuple(args, "i:dup", &fd))
return NULL;
if (!_PyVerify_fd(fd))
return posix_error();
fd = dup(fd);
if (fd < 0)
return posix_error();
fd = _Py_dup(fd);
if (fd == -1)
return NULL;
return PyLong_FromLong((long)fd);
}
@ -7556,16 +7599,82 @@ PyDoc_STRVAR(posix_dup2__doc__,
Duplicate file descriptor.");
static PyObject *
posix_dup2(PyObject *self, PyObject *args)
posix_dup2(PyObject *self, PyObject *args, PyObject *kwargs)
{
int fd, fd2, res;
if (!PyArg_ParseTuple(args, "ii:dup2", &fd, &fd2))
static char *keywords[] = {"fd", "fd2", "inheritable", NULL};
int fd, fd2;
int inheritable = 1;
int res;
#if defined(HAVE_DUP3) && \
!(defined(HAVE_FCNTL_H) && defined(F_DUP2FD_CLOEXEC))
/* dup3() is available on Linux 2.6.27+ and glibc 2.9 */
int dup3_works = -1;
#endif
if (!PyArg_ParseTupleAndKeywords(args, kwargs, "ii|i:dup2", keywords,
&fd, &fd2, &inheritable))
return NULL;
if (!_PyVerify_fd_dup2(fd, fd2))
return posix_error();
#ifdef MS_WINDOWS
Py_BEGIN_ALLOW_THREADS
res = dup2(fd, fd2);
Py_END_ALLOW_THREADS
if (res < 0)
return posix_error();
/* Character files like console cannot be make non-inheritable */
if (!inheritable && _Py_set_inheritable(fd2, 0, NULL) < 0) {
close(fd2);
return NULL;
}
#elif defined(HAVE_FCNTL_H) && defined(F_DUP2FD_CLOEXEC)
Py_BEGIN_ALLOW_THREADS
if (!inheritable)
res = fcntl(fd, F_DUP2FD_CLOEXEC, fd2);
else
res = dup2(fd, fd2);
Py_END_ALLOW_THREADS
if (res < 0)
return posix_error();
#else
#ifdef HAVE_DUP3
if (!inheritable && dup3_works != 0) {
Py_BEGIN_ALLOW_THREADS
res = dup3(fd, fd2, O_CLOEXEC);
Py_END_ALLOW_THREADS
if (res < 0) {
if (dup3_works == -1)
dup3_works = (errno != ENOSYS);
if (dup3_works)
return posix_error();
}
}
if (inheritable || dup3_works == 0)
{
#endif
Py_BEGIN_ALLOW_THREADS
res = dup2(fd, fd2);
Py_END_ALLOW_THREADS
if (res < 0)
return posix_error();
if (!inheritable && _Py_set_inheritable(fd2, 0, NULL) < 0) {
close(fd2);
return NULL;
}
#ifdef HAVE_DUP3
}
#endif
#endif
Py_INCREF(Py_None);
return Py_None;
}
@ -8052,24 +8161,69 @@ Create a pipe.");
static PyObject *
posix_pipe(PyObject *self, PyObject *noargs)
{
#if !defined(MS_WINDOWS)
int fds[2];
int res;
res = pipe(fds);
if (res != 0)
return posix_error();
return Py_BuildValue("(ii)", fds[0], fds[1]);
#else /* MS_WINDOWS */
#ifdef MS_WINDOWS
HANDLE read, write;
int read_fd, write_fd;
SECURITY_ATTRIBUTES attr;
BOOL ok;
ok = CreatePipe(&read, &write, NULL, 0);
#else
int res;
#endif
#ifdef MS_WINDOWS
attr.nLength = sizeof(attr);
attr.lpSecurityDescriptor = NULL;
attr.bInheritHandle = FALSE;
Py_BEGIN_ALLOW_THREADS
ok = CreatePipe(&read, &write, &attr, 0);
if (ok) {
fds[0] = _open_osfhandle((Py_intptr_t)read, _O_RDONLY);
fds[1] = _open_osfhandle((Py_intptr_t)write, _O_WRONLY);
if (fds[0] == -1 || fds[1] == -1) {
CloseHandle(read);
CloseHandle(write);
ok = 0;
}
}
Py_END_ALLOW_THREADS
if (!ok)
return PyErr_SetFromWindowsErr(0);
read_fd = _open_osfhandle((Py_intptr_t)read, 0);
write_fd = _open_osfhandle((Py_intptr_t)write, 1);
return Py_BuildValue("(ii)", read_fd, write_fd);
#endif /* MS_WINDOWS */
#else
#ifdef HAVE_PIPE2
Py_BEGIN_ALLOW_THREADS
res = pipe2(fds, O_CLOEXEC);
Py_END_ALLOW_THREADS
if (res != 0 && errno == ENOSYS)
{
#endif
Py_BEGIN_ALLOW_THREADS
res = pipe(fds);
Py_END_ALLOW_THREADS
if (res == 0) {
if (_Py_set_inheritable(fds[0], 0, NULL) < 0) {
close(fds[0]);
close(fds[1]);
return NULL;
}
if (_Py_set_inheritable(fds[1], 0, NULL) < 0) {
close(fds[0]);
close(fds[1]);
return NULL;
}
}
#ifdef HAVE_PIPE2
}
#endif
if (res != 0)
return PyErr_SetFromErrno(PyExc_OSError);
#endif /* !MS_WINDOWS */
return Py_BuildValue("(ii)", fds[0], fds[1]);
}
#endif /* HAVE_PIPE */
@ -10659,6 +10813,102 @@ posix_cpu_count(PyObject *self)
Py_RETURN_NONE;
}
PyDoc_STRVAR(get_inheritable__doc__,
"get_inheritable(fd) -> bool\n" \
"\n" \
"Get the close-on-exe flag of the specified file descriptor.");
static PyObject*
posix_get_inheritable(PyObject *self, PyObject *args)
{
int fd;
int inheritable;
if (!PyArg_ParseTuple(args, "i:get_inheritable", &fd))
return NULL;
if (!_PyVerify_fd(fd))
return posix_error();
inheritable = _Py_get_inheritable(fd);
if (inheritable < 0)
return NULL;
return PyBool_FromLong(inheritable);
}
PyDoc_STRVAR(set_inheritable__doc__,
"set_inheritable(fd, inheritable)\n" \
"\n" \
"Set the inheritable flag of the specified file descriptor.");
static PyObject*
posix_set_inheritable(PyObject *self, PyObject *args)
{
int fd, inheritable;
if (!PyArg_ParseTuple(args, "ii:set_inheritable", &fd, &inheritable))
return NULL;
if (!_PyVerify_fd(fd))
return posix_error();
if (_Py_set_inheritable(fd, inheritable, NULL) < 0)
return NULL;
Py_RETURN_NONE;
}
#ifdef MS_WINDOWS
PyDoc_STRVAR(get_handle_inheritable__doc__,
"get_handle_inheritable(fd) -> bool\n" \
"\n" \
"Get the close-on-exe flag of the specified file descriptor.");
static PyObject*
posix_get_handle_inheritable(PyObject *self, PyObject *args)
{
Py_intptr_t handle;
DWORD flags;
if (!PyArg_ParseTuple(args, _Py_PARSE_INTPTR ":get_handle_inheritable", &handle))
return NULL;
if (!GetHandleInformation((HANDLE)handle, &flags)) {
PyErr_SetFromWindowsErr(0);
return NULL;
}
return PyBool_FromLong(flags & HANDLE_FLAG_INHERIT);
}
PyDoc_STRVAR(set_handle_inheritable__doc__,
"set_handle_inheritable(fd, inheritable)\n" \
"\n" \
"Set the inheritable flag of the specified handle.");
static PyObject*
posix_set_handle_inheritable(PyObject *self, PyObject *args)
{
int inheritable = 1;
Py_intptr_t handle;
DWORD flags;
if (!PyArg_ParseTuple(args, _Py_PARSE_INTPTR "i:set_handle_inheritable",
&handle, &inheritable))
return NULL;
if (inheritable)
flags = HANDLE_FLAG_INHERIT;
else
flags = 0;
if (!SetHandleInformation((HANDLE)handle, HANDLE_FLAG_INHERIT, flags)) {
PyErr_SetFromWindowsErr(0);
return NULL;
}
Py_RETURN_NONE;
}
#endif /* MS_WINDOWS */
static PyMethodDef posix_methods[] = {
{"access", (PyCFunction)posix_access,
@ -10934,7 +11184,8 @@ static PyMethodDef posix_methods[] = {
{"closerange", posix_closerange, METH_VARARGS, posix_closerange__doc__},
{"device_encoding", device_encoding, METH_VARARGS, device_encoding__doc__},
{"dup", posix_dup, METH_VARARGS, posix_dup__doc__},
{"dup2", posix_dup2, METH_VARARGS, posix_dup2__doc__},
{"dup2", (PyCFunction)posix_dup2,
METH_VARARGS | METH_KEYWORDS, posix_dup2__doc__},
#ifdef HAVE_LOCKF
{"lockf", posix_lockf, METH_VARARGS, posix_lockf__doc__},
#endif
@ -11105,6 +11356,14 @@ static PyMethodDef posix_methods[] = {
#endif
{"cpu_count", (PyCFunction)posix_cpu_count,
METH_NOARGS, posix_cpu_count__doc__},
{"get_inheritable", posix_get_inheritable, METH_VARARGS, get_inheritable__doc__},
{"set_inheritable", posix_set_inheritable, METH_VARARGS, set_inheritable__doc__},
#ifdef MS_WINDOWS
{"get_handle_inheritable", posix_get_handle_inheritable,
METH_VARARGS, get_handle_inheritable__doc__},
{"set_handle_inheritable", posix_set_handle_inheritable,
METH_VARARGS, set_handle_inheritable__doc__},
#endif
{NULL, NULL} /* Sentinel */
};

View File

@ -1004,7 +1004,7 @@ newDevPollObject(void)
*/
limit_result = getrlimit(RLIMIT_NOFILE, &limit);
if (limit_result != -1)
fd_devpoll = open("/dev/poll", O_RDWR);
fd_devpoll = _Py_open("/dev/poll", O_RDWR);
Py_END_ALLOW_THREADS
if (limit_result == -1) {
@ -1194,6 +1194,7 @@ newPyEpoll_Object(PyTypeObject *type, int sizehint, int flags, SOCKET fd)
if (fd == -1) {
Py_BEGIN_ALLOW_THREADS
#ifdef HAVE_EPOLL_CREATE1
flags |= EPOLL_CLOEXEC;
if (flags)
self->epfd = epoll_create1(flags);
else
@ -1209,6 +1210,14 @@ newPyEpoll_Object(PyTypeObject *type, int sizehint, int flags, SOCKET fd)
PyErr_SetFromErrno(PyExc_OSError);
return NULL;
}
#ifndef HAVE_EPOLL_CREATE1
if (_Py_set_inheritable(self->epfd, 0, NULL) < 0) {
Py_DECREF(self);
return NULL;
}
#endif
return (PyObject *)self;
}
@ -1896,13 +1905,19 @@ newKqueue_Object(PyTypeObject *type, SOCKET fd)
PyErr_SetFromErrno(PyExc_OSError);
return NULL;
}
if (fd == -1) {
if (_Py_set_inheritable(self->kqfd, 0, NULL) < 0) {
Py_DECREF(self);
return NULL;
}
}
return (PyObject *)self;
}
static PyObject *
kqueue_queue_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
{
if ((args != NULL && PyObject_Size(args)) ||
(kwds != NULL && PyObject_Size(kwds))) {
PyErr_SetString(PyExc_ValueError,

View File

@ -97,13 +97,14 @@ Local naming conventions:
/* Socket object documentation */
PyDoc_STRVAR(sock_doc,
"socket([family[, type[, proto]]]) -> socket object\n\
"socket(family=AF_INET, type=SOCK_STREAM, proto=0, fileno=None) -> socket object\n\
\n\
Open a socket of the given type. The family argument specifies the\n\
address family; it defaults to AF_INET. The type argument specifies\n\
whether this is a stream (SOCK_STREAM, this is the default)\n\
or datagram (SOCK_DGRAM) socket. The protocol argument defaults to 0,\n\
specifying the default protocol. Keyword arguments are accepted.\n\
The socket is created as non-inheritable.\n\
\n\
A socket object represents one endpoint of a network connection.\n\
\n\
@ -114,7 +115,7 @@ bind(addr) -- bind the socket to a local address\n\
close() -- close the socket\n\
connect(addr) -- connect the socket to a remote address\n\
connect_ex(addr) -- connect, return an error code instead of an exception\n\
_dup() -- return a new socket fd duplicated from fileno()\n\
dup() -- return a new socket fd duplicated from fileno()\n\
fileno() -- return underlying file descriptor\n\
getpeername() -- return remote address [*]\n\
getsockname() -- return local address\n\
@ -356,22 +357,7 @@ const char *inet_ntop(int af, const void *src, char *dst, socklen_t size);
#endif
#ifdef MS_WINDOWS
/* On Windows a socket is really a handle not an fd */
static SOCKET
dup_socket(SOCKET handle)
{
WSAPROTOCOL_INFO info;
if (WSADuplicateSocket(handle, GetCurrentProcessId(), &info))
return INVALID_SOCKET;
return WSASocket(FROM_PROTOCOL_INFO, FROM_PROTOCOL_INFO,
FROM_PROTOCOL_INFO, &info, 0, WSA_FLAG_OVERLAPPED);
}
#define SOCKETCLOSE closesocket
#else
/* On Unix we can use dup to duplicate the file descriptor of a socket*/
#define dup_socket(fd) dup(fd)
#endif
#ifdef MS_WIN32
@ -499,6 +485,11 @@ select_error(void)
(errno == expected)
#endif
#ifdef MS_WINDOWS
/* Does WSASocket() support the WSA_FLAG_NO_HANDLE_INHERIT flag? */
static int support_wsa_no_inherit = -1;
#endif
/* Convenience function to raise an error according to errno
and return a NULL pointer from a function. */
@ -1955,6 +1946,11 @@ sock_accept(PySocketSockObject *s)
PyObject *addr = NULL;
PyObject *res = NULL;
int timeout;
#if defined(HAVE_ACCEPT4) && defined(SOCK_CLOEXEC)
/* accept4() is available on Linux 2.6.28+ and glibc 2.10 */
static int accept4_works = -1;
#endif
if (!getsockaddrlen(s, &addrlen))
return NULL;
memset(&addrbuf, 0, addrlen);
@ -1963,10 +1959,24 @@ sock_accept(PySocketSockObject *s)
return select_error();
BEGIN_SELECT_LOOP(s)
Py_BEGIN_ALLOW_THREADS
timeout = internal_select_ex(s, 0, interval);
if (!timeout) {
#if defined(HAVE_ACCEPT4) && defined(SOCK_CLOEXEC)
if (accept4_works != 0) {
newfd = accept4(s->sock_fd, SAS2SA(&addrbuf), &addrlen,
SOCK_CLOEXEC);
if (newfd == INVALID_SOCKET && accept4_works == -1) {
/* On Linux older than 2.6.28, accept4() fails with ENOSYS */
accept4_works = (errno != ENOSYS);
}
}
if (accept4_works == 0)
newfd = accept(s->sock_fd, SAS2SA(&addrbuf), &addrlen);
#else
newfd = accept(s->sock_fd, SAS2SA(&addrbuf), &addrlen);
#endif
}
Py_END_ALLOW_THREADS
@ -1979,6 +1989,25 @@ sock_accept(PySocketSockObject *s)
if (newfd == INVALID_SOCKET)
return s->errorhandler();
#ifdef MS_WINDOWS
if (!SetHandleInformation((HANDLE)newfd, HANDLE_FLAG_INHERIT, 0)) {
PyErr_SetFromWindowsErr(0);
SOCKETCLOSE(newfd);
goto finally;
}
#else
#if defined(HAVE_ACCEPT4) && defined(SOCK_CLOEXEC)
if (!accept4_works)
#endif
{
if (_Py_set_inheritable(newfd, 0, NULL) < 0) {
SOCKETCLOSE(newfd);
goto finally;
}
}
#endif
sock = PyLong_FromSocket_t(newfd);
if (sock == NULL) {
SOCKETCLOSE(newfd);
@ -3909,6 +3938,12 @@ sock_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
/* Initialize a new socket object. */
#ifdef SOCK_CLOEXEC
/* socket() and socketpair() fail with EINVAL on Linux kernel older
* than 2.6.27 if SOCK_CLOEXEC flag is set in the socket type. */
static int sock_cloexec_works = -1;
#endif
/*ARGSUSED*/
static int
sock_initobj(PyObject *self, PyObject *args, PyObject *kwds)
@ -3918,6 +3953,13 @@ sock_initobj(PyObject *self, PyObject *args, PyObject *kwds)
SOCKET_T fd = INVALID_SOCKET;
int family = AF_INET, type = SOCK_STREAM, proto = 0;
static char *keywords[] = {"family", "type", "proto", "fileno", 0};
#ifndef MS_WINDOWS
#ifdef SOCK_CLOEXEC
int *atomic_flag_works = &sock_cloexec_works;
#else
int *atomic_flag_works = NULL;
#endif
#endif
if (!PyArg_ParseTupleAndKeywords(args, kwds,
"|iiiO:socket", keywords,
@ -3962,14 +4004,74 @@ sock_initobj(PyObject *self, PyObject *args, PyObject *kwds)
}
}
else {
#ifdef MS_WINDOWS
/* Windows implementation */
#ifndef WSA_FLAG_NO_HANDLE_INHERIT
#define WSA_FLAG_NO_HANDLE_INHERIT 0x80
#endif
Py_BEGIN_ALLOW_THREADS
if (support_wsa_no_inherit) {
fd = WSASocket(family, type, proto,
NULL, 0,
WSA_FLAG_OVERLAPPED | WSA_FLAG_NO_HANDLE_INHERIT);
if (fd == INVALID_SOCKET) {
/* Windows 7 or Windows 2008 R2 without SP1 or the hotfix */
support_wsa_no_inherit = 0;
fd = socket(family, type, proto);
}
}
else {
fd = socket(family, type, proto);
}
Py_END_ALLOW_THREADS
if (fd == INVALID_SOCKET) {
set_error();
return -1;
}
if (!support_wsa_no_inherit) {
if (!SetHandleInformation((HANDLE)fd, HANDLE_FLAG_INHERIT, 0)) {
closesocket(fd);
PyErr_SetFromWindowsErr(0);
return -1;
}
}
#else
/* UNIX */
Py_BEGIN_ALLOW_THREADS
#ifdef SOCK_CLOEXEC
if (sock_cloexec_works != 0) {
fd = socket(family, type | SOCK_CLOEXEC, proto);
if (sock_cloexec_works == -1) {
if (fd >= 0) {
sock_cloexec_works = 1;
}
else if (errno == EINVAL) {
/* Linux older than 2.6.27 does not support SOCK_CLOEXEC */
sock_cloexec_works = 0;
fd = socket(family, type, proto);
}
}
}
else
#endif
{
fd = socket(family, type, proto);
}
Py_END_ALLOW_THREADS
if (fd == INVALID_SOCKET) {
set_error();
return -1;
}
if (_Py_set_inheritable(fd, 0, atomic_flag_works) < 0) {
SOCKETCLOSE(fd);
return -1;
}
#endif
}
init_sockobject(s, fd, family, type, proto);
@ -4535,16 +4637,36 @@ socket_dup(PyObject *self, PyObject *fdobj)
{
SOCKET_T fd, newfd;
PyObject *newfdobj;
#ifdef MS_WINDOWS
WSAPROTOCOL_INFO info;
#endif
fd = PyLong_AsSocket_t(fdobj);
if (fd == (SOCKET_T)(-1) && PyErr_Occurred())
return NULL;
newfd = dup_socket(fd);
#ifdef MS_WINDOWS
if (WSADuplicateSocket(fd, GetCurrentProcessId(), &info))
return set_error();
newfd = WSASocket(FROM_PROTOCOL_INFO, FROM_PROTOCOL_INFO,
FROM_PROTOCOL_INFO,
&info, 0, WSA_FLAG_OVERLAPPED);
if (newfd == INVALID_SOCKET)
return set_error();
if (!SetHandleInformation((HANDLE)newfd, HANDLE_FLAG_INHERIT, 0)) {
closesocket(newfd);
PyErr_SetFromWindowsErr(0);
return NULL;
}
#else
/* On UNIX, dup can be used to duplicate the file descriptor of a socket */
newfd = _Py_dup(fd);
if (newfd == INVALID_SOCKET)
return NULL;
#endif
newfdobj = PyLong_FromSocket_t(newfd);
if (newfdobj == NULL)
SOCKETCLOSE(newfd);
@ -4572,6 +4694,12 @@ socket_socketpair(PyObject *self, PyObject *args)
SOCKET_T sv[2];
int family, type = SOCK_STREAM, proto = 0;
PyObject *res = NULL;
#ifdef SOCK_CLOEXEC
int *atomic_flag_works = &sock_cloexec_works;
#else
int *atomic_flag_works = NULL;
#endif
int ret;
#if defined(AF_UNIX)
family = AF_UNIX;
@ -4581,9 +4709,38 @@ socket_socketpair(PyObject *self, PyObject *args)
if (!PyArg_ParseTuple(args, "|iii:socketpair",
&family, &type, &proto))
return NULL;
/* Create a pair of socket fds */
if (socketpair(family, type, proto, sv) < 0)
Py_BEGIN_ALLOW_THREADS
#ifdef SOCK_CLOEXEC
if (sock_cloexec_works != 0) {
ret = socketpair(family, type | SOCK_CLOEXEC, proto, sv);
if (sock_cloexec_works == -1) {
if (ret >= 0) {
sock_cloexec_works = 1;
}
else if (errno == EINVAL) {
/* Linux older than 2.6.27 does not support SOCK_CLOEXEC */
sock_cloexec_works = 0;
ret = socketpair(family, type, proto, sv);
}
}
}
else
#endif
{
ret = socketpair(family, type, proto, sv);
}
Py_END_ALLOW_THREADS
if (ret < 0)
return set_error();
if (_Py_set_inheritable(sv[0], 0, atomic_flag_works) < 0)
goto finally;
if (_Py_set_inheritable(sv[1], 0, atomic_flag_works) < 0)
goto finally;
s0 = new_sockobject(sv[0], family, type, proto);
if (s0 == NULL)
goto finally;
@ -5539,6 +5696,16 @@ PyInit__socket(void)
if (!os_init())
return NULL;
#ifdef MS_WINDOWS
if (support_wsa_no_inherit == -1) {
DWORD version = GetVersion();
DWORD major = (DWORD)LOBYTE(LOWORD(version));
DWORD minor = (DWORD)HIBYTE(LOWORD(version));
/* need Windows 7 SP1, 2008 R2 SP1 or later */
support_wsa_no_inherit = (major >= 6 && minor >= 1);
}
#endif
Py_TYPE(&sock_type) = &PyType_Type;
m = PyModule_Create(&socketmodule);
if (m == NULL)

View File

@ -870,7 +870,7 @@ read_directory(PyObject *archive)
const char *charset;
int bootstrap;
fp = _Py_fopen(archive, "rb");
fp = _Py_fopen_obj(archive, "rb");
if (fp == NULL) {
if (!PyErr_Occurred())
PyErr_Format(ZipImportError, "can't open Zip file: %R", archive);
@ -1064,7 +1064,7 @@ get_data(PyObject *archive, PyObject *toc_entry)
return NULL;
}
fp = _Py_fopen(archive, "rb");
fp = _Py_fopen_obj(archive, "rb");
if (!fp) {
if (!PyErr_Occurred())
PyErr_Format(PyExc_IOError,

View File

@ -55,7 +55,7 @@ static FNFCIFREE(cb_free)
static FNFCIOPEN(cb_open)
{
int result = _open(pszFile, oflag, pmode);
int result = _open(pszFile, oflag | O_NOINHERIT, pmode);
if (result == -1)
*err = errno;
return result;
@ -179,7 +179,7 @@ static FNFCIGETOPENINFO(cb_getopeninfo)
CloseHandle(handle);
return _open(pszName, _O_RDONLY | _O_BINARY);
return _open(pszName, _O_RDONLY | _O_BINARY | O_NOINHERIT);
}
static PyObject* fcicreate(PyObject* obj, PyObject* args)

View File

@ -743,7 +743,7 @@ do_run_installscript(HINSTANCE hPython, char *pathname, int argc, char **argv)
if (pathname == NULL || pathname[0] == '\0')
return 2;
fh = open(pathname, _O_RDONLY);
fh = open(pathname, _O_RDONLY | O_NOINHERIT);
if (-1 == fh) {
fprintf(stderr, "Could not open postinstall-script %s\n",
pathname);

View File

@ -1730,10 +1730,15 @@ PyTokenizer_FindEncodingFilename(int fd, PyObject *filename)
FILE *fp;
char *p_start =NULL , *p_end =NULL , *encoding = NULL;
#ifndef PGEN
fd = _Py_dup(fd);
#else
fd = dup(fd);
#endif
if (fd < 0) {
return NULL;
}
fp = fdopen(fd, "r");
if (fp == NULL) {
return NULL;

View File

@ -1042,7 +1042,7 @@ PyErr_ProgramText(const char *filename, int lineno)
FILE *fp;
if (filename == NULL || *filename == '\0' || lineno <= 0)
return NULL;
fp = fopen(filename, "r" PY_STDIOTEXTMODE);
fp = _Py_fopen(filename, "r" PY_STDIOTEXTMODE);
return err_programtext(fp, lineno);
}
@ -1052,7 +1052,7 @@ PyErr_ProgramTextObject(PyObject *filename, int lineno)
FILE *fp;
if (filename == NULL || lineno <= 0)
return NULL;
fp = _Py_fopen(filename, "r" PY_STDIOTEXTMODE);
fp = _Py_fopen_obj(filename, "r" PY_STDIOTEXTMODE);
return err_programtext(fp, lineno);
}

View File

@ -9,10 +9,29 @@
#include <langinfo.h>
#endif
#ifdef HAVE_SYS_IOCTL_H
#include <sys/ioctl.h>
#endif
#ifdef HAVE_FCNTL_H
#include <fcntl.h>
#endif /* HAVE_FCNTL_H */
#ifdef __APPLE__
extern wchar_t* _Py_DecodeUTF8_surrogateescape(const char *s, Py_ssize_t size);
#endif
#ifdef O_CLOEXEC
/* Does open() supports the O_CLOEXEC flag? Possible values:
-1: unknown
0: open() ignores O_CLOEXEC flag, ex: Linux kernel older than 2.6.23
1: open() supports O_CLOEXEC flag, close-on-exec is set
The flag is used by _Py_open(), io.FileIO and os.open() */
int _Py_open_cloexec_works = -1;
#endif
PyObject *
_Py_device_encoding(int fd)
{
@ -547,14 +566,215 @@ _Py_stat(PyObject *path, struct stat *statbuf)
#endif
/* Open a file. Use _wfopen() on Windows, encode the path to the locale
encoding and use fopen() otherwise. */
int
get_inheritable(int fd, int raise)
{
#ifdef MS_WINDOWS
HANDLE handle;
DWORD flags;
if (!_PyVerify_fd(fd)) {
if (raise)
PyErr_SetFromErrno(PyExc_OSError);
return -1;
}
handle = (HANDLE)_get_osfhandle(fd);
if (handle == INVALID_HANDLE_VALUE) {
if (raise)
PyErr_SetFromWindowsErr(0);
return -1;
}
if (!GetHandleInformation(handle, &flags)) {
if (raise)
PyErr_SetFromWindowsErr(0);
return -1;
}
return (flags & HANDLE_FLAG_INHERIT);
#else
int flags;
flags = fcntl(fd, F_GETFD, 0);
if (flags == -1) {
if (raise)
PyErr_SetFromErrno(PyExc_OSError);
return -1;
}
return !(flags & FD_CLOEXEC);
#endif
}
/* Get the inheritable flag of the specified file descriptor.
Return 1 if it the file descriptor can be inherited, 0 if it cannot,
raise an exception and return -1 on error. */
int
_Py_get_inheritable(int fd)
{
return get_inheritable(fd, 1);
}
static int
set_inheritable(int fd, int inheritable, int raise, int *atomic_flag_works)
{
#ifdef MS_WINDOWS
HANDLE handle;
DWORD flags;
#elif defined(HAVE_SYS_IOCTL_H) && defined(FIOCLEX) && defined(FIONCLEX)
int request;
int err;
#elif defined(HAVE_FCNTL_H)
int flags;
int res;
#endif
/* atomic_flag_works can only be used to make the file descriptor
non-inheritable */
assert(!(atomic_flag_works != NULL && inheritable));
if (atomic_flag_works != NULL && !inheritable) {
if (*atomic_flag_works == -1) {
int inheritable = get_inheritable(fd, raise);
if (inheritable == -1)
return -1;
*atomic_flag_works = !inheritable;
}
if (*atomic_flag_works)
return 0;
}
#ifdef MS_WINDOWS
if (!_PyVerify_fd(fd)) {
if (raise)
PyErr_SetFromErrno(PyExc_OSError);
return -1;
}
handle = (HANDLE)_get_osfhandle(fd);
if (handle == INVALID_HANDLE_VALUE) {
if (raise)
PyErr_SetFromWindowsErr(0);
return -1;
}
if (inheritable)
flags = HANDLE_FLAG_INHERIT;
else
flags = 0;
if (!SetHandleInformation(handle, HANDLE_FLAG_INHERIT, flags)) {
if (raise)
PyErr_SetFromWindowsErr(0);
return -1;
}
return 0;
#elif defined(HAVE_SYS_IOCTL_H) && defined(FIOCLEX) && defined(FIONCLEX)
if (inheritable)
request = FIONCLEX;
else
request = FIOCLEX;
err = ioctl(fd, request);
if (err) {
if (raise)
PyErr_SetFromErrno(PyExc_OSError);
return -1;
}
return 0;
#else
flags = fcntl(fd, F_GETFD);
if (flags < 0) {
if (raise)
PyErr_SetFromErrno(PyExc_OSError);
return -1;
}
if (inheritable)
flags &= ~FD_CLOEXEC;
else
flags |= FD_CLOEXEC;
res = fcntl(fd, F_SETFD, flags);
if (res < 0) {
if (raise)
PyErr_SetFromErrno(PyExc_OSError);
return -1;
}
return 0;
#endif
}
/* Make the file descriptor non-inheritable.
Return 0 success, set errno and return -1 on error. */
static int
make_non_inheritable(int fd)
{
return set_inheritable(fd, 0, 0, NULL);
}
/* Set the inheritable flag of the specified file descriptor.
On success: return 0, on error: raise an exception if raise is nonzero
and return -1.
If atomic_flag_works is not NULL:
* if *atomic_flag_works==-1, check if the inheritable is set on the file
descriptor: if yes, set *atomic_flag_works to 1, otherwise set to 0 and
set the inheritable flag
* if *atomic_flag_works==1: do nothing
* if *atomic_flag_works==0: set inheritable flag to False
Set atomic_flag_works to NULL if no atomic flag was used to create the
file descriptor.
atomic_flag_works can only be used to make a file descriptor
non-inheritable: atomic_flag_works must be NULL if inheritable=1. */
int
_Py_set_inheritable(int fd, int inheritable, int *atomic_flag_works)
{
return set_inheritable(fd, inheritable, 1, atomic_flag_works);
}
/* Open a file with the specified flags (wrapper to open() function).
The file descriptor is created non-inheritable. */
int
_Py_open(const char *pathname, int flags)
{
int fd;
#ifdef MS_WINDOWS
fd = open(pathname, flags | O_NOINHERIT);
if (fd < 0)
return fd;
#else
int *atomic_flag_works;
#ifdef O_CLOEXEC
atomic_flag_works = &_Py_open_cloexec_works;
flags |= O_CLOEXEC;
#else
atomic_flag_works = NULL;
#endif
fd = open(pathname, flags);
if (fd < 0)
return fd;
if (set_inheritable(fd, 0, 0, atomic_flag_works) < 0) {
close(fd);
return -1;
}
#endif /* !MS_WINDOWS */
return fd;
}
/* Open a file. Use _wfopen() on Windows, encode the path to the locale
encoding and use fopen() otherwise. The file descriptor is created
non-inheritable. */
FILE *
_Py_wfopen(const wchar_t *path, const wchar_t *mode)
{
#ifndef MS_WINDOWS
FILE *f;
#ifndef MS_WINDOWS
char *cpath;
char cmode[10];
size_t r;
@ -568,21 +788,42 @@ _Py_wfopen(const wchar_t *path, const wchar_t *mode)
return NULL;
f = fopen(cpath, cmode);
PyMem_Free(cpath);
return f;
#else
return _wfopen(path, mode);
f = _wfopen(path, mode);
#endif
if (f == NULL)
return NULL;
if (make_non_inheritable(fileno(f)) < 0) {
fclose(f);
return NULL;
}
return f;
}
/* Call _wfopen() on Windows, or encode the path to the filesystem encoding and
call fopen() otherwise.
/* Wrapper to fopen(). The file descriptor is created non-inheritable. */
FILE*
_Py_fopen(const char *pathname, const char *mode)
{
FILE *f = fopen(pathname, mode);
if (f == NULL)
return NULL;
if (make_non_inheritable(fileno(f)) < 0) {
fclose(f);
return NULL;
}
return f;
}
/* Open a file. Call _wfopen() on Windows, or encode the path to the filesystem
encoding and call fopen() otherwise. The file descriptor is created
non-inheritable.
Return the new file object on success, or NULL if the file cannot be open or
(if PyErr_Occurred()) on unicode error */
(if PyErr_Occurred()) on unicode error. */
FILE*
_Py_fopen(PyObject *path, const char *mode)
_Py_fopen_obj(PyObject *path, const char *mode)
{
FILE *f;
#ifdef MS_WINDOWS
wchar_t *wpath;
wchar_t wmode[10];
@ -602,16 +843,21 @@ _Py_fopen(PyObject *path, const char *mode)
if (usize == 0)
return NULL;
return _wfopen(wpath, wmode);
f = _wfopen(wpath, wmode);
#else
FILE *f;
PyObject *bytes;
if (!PyUnicode_FSConverter(path, &bytes))
return NULL;
f = fopen(PyBytes_AS_STRING(bytes), mode);
Py_DECREF(bytes);
return f;
#endif
if (f == NULL)
return NULL;
if (make_non_inheritable(fileno(f)) < 0) {
fclose(f);
return NULL;
}
return f;
}
#ifdef HAVE_READLINK
@ -729,3 +975,72 @@ _Py_wgetcwd(wchar_t *buf, size_t size)
#endif
}
/* Duplicate a file descriptor. The new file descriptor is created as
non-inheritable. Return a new file descriptor on success, raise an OSError
exception and return -1 on error.
The GIL is released to call dup(). The caller must hold the GIL. */
int
_Py_dup(int fd)
{
#ifdef MS_WINDOWS
HANDLE handle;
DWORD ftype;
#endif
if (!_PyVerify_fd(fd)) {
PyErr_SetFromErrno(PyExc_OSError);
return -1;
}
#ifdef MS_WINDOWS
handle = (HANDLE)_get_osfhandle(fd);
if (handle == INVALID_HANDLE_VALUE) {
PyErr_SetFromWindowsErr(0);
return -1;
}
/* get the file type, ignore the error if it failed */
ftype = GetFileType(handle);
Py_BEGIN_ALLOW_THREADS
fd = dup(fd);
Py_END_ALLOW_THREADS
if (fd < 0) {
PyErr_SetFromErrno(PyExc_OSError);
return -1;
}
/* Character files like console cannot be make non-inheritable */
if (ftype != FILE_TYPE_CHAR) {
if (_Py_set_inheritable(fd, 0, NULL) < 0) {
close(fd);
return -1;
}
}
#elif defined(HAVE_FCNTL_H) && defined(F_DUPFD_CLOEXEC)
Py_BEGIN_ALLOW_THREADS
fd = fcntl(fd, F_DUPFD_CLOEXEC, 0);
Py_END_ALLOW_THREADS
if (fd < 0) {
PyErr_SetFromErrno(PyExc_OSError);
return -1;
}
#else
Py_BEGIN_ALLOW_THREADS
fd = dup(fd);
Py_END_ALLOW_THREADS
if (fd < 0) {
PyErr_SetFromErrno(PyExc_OSError);
return -1;
}
if (_Py_set_inheritable(fd, 0, NULL) < 0) {
close(fd);
return -1;
}
#endif
return fd;
}

View File

@ -1797,7 +1797,7 @@ imp_load_dynamic(PyObject *self, PyObject *args)
&name, PyUnicode_FSDecoder, &pathname, &fob))
return NULL;
if (fob != NULL) {
fp = _Py_fopen(pathname, "r");
fp = _Py_fopen_obj(pathname, "r");
if (fp == NULL) {
Py_DECREF(pathname);
if (!PyErr_Occurred())

View File

@ -1454,7 +1454,7 @@ PyRun_SimpleFileExFlags(FILE *fp, const char *filename, int closeit,
/* Try to run a pyc file. First, re-open in binary */
if (closeit)
fclose(fp);
if ((pyc_fp = fopen(filename, "rb")) == NULL) {
if ((pyc_fp = _Py_fopen(filename, "rb")) == NULL) {
fprintf(stderr, "python: Can't reopen .pyc file\n");
goto done;
}

View File

@ -101,7 +101,7 @@ dev_urandom_noraise(char *buffer, Py_ssize_t size)
assert (0 < size);
fd = open("/dev/urandom", O_RDONLY);
fd = _Py_open("/dev/urandom", O_RDONLY);
if (fd < 0)
Py_FatalError("Failed to open /dev/urandom");
@ -134,7 +134,7 @@ dev_urandom_python(char *buffer, Py_ssize_t size)
return 0;
Py_BEGIN_ALLOW_THREADS
fd = open("/dev/urandom", O_RDONLY);
fd = _Py_open("/dev/urandom", O_RDONLY);
Py_END_ALLOW_THREADS
if (fd < 0)
{

2
configure vendored
View File

@ -10275,7 +10275,7 @@ fi
# checks for library functions
for ac_func in alarm accept4 setitimer getitimer bind_textdomain_codeset chown \
clock confstr ctermid execv faccessat fchmod fchmodat fchown fchownat \
clock confstr ctermid dup3 execv faccessat fchmod fchmodat fchown fchownat \
fexecve fdopendir fork fpathconf fstatat ftime ftruncate futimesat \
futimens futimes gai_strerror \
getgrouplist getgroups getlogin getloadavg getpeername getpgid getpid \

View File

@ -2807,7 +2807,7 @@ fi
# checks for library functions
AC_CHECK_FUNCS(alarm accept4 setitimer getitimer bind_textdomain_codeset chown \
clock confstr ctermid execv faccessat fchmod fchmodat fchown fchownat \
clock confstr ctermid dup3 execv faccessat fchmod fchmodat fchown fchownat \
fexecve fdopendir fork fpathconf fstatat ftime ftruncate futimesat \
futimens futimes gai_strerror \
getgrouplist getgroups getlogin getloadavg getpeername getpgid getpid \

View File

@ -193,6 +193,9 @@
/* Define to 1 if you have the `dup2' function. */
#undef HAVE_DUP2
/* Define to 1 if you have the `dup3' function. */
#undef HAVE_DUP3
/* Defined when any dynamic module loading is enabled. */
#undef HAVE_DYNAMIC_LOADING