Mirror
/
cpython
mirror of https://github.com/python/cpython
4
1
Fork 0
Code Issues Releases Wiki Activity

Issue #6397: Support '/dev/poll' polling objects in select module, under Solaris & derivatives.

This commit is contained in:
Jesus Cea 2011-11-14 19:07:41 +01:00
parent d5d4406c8e
commit d8b9ae6e8f
7 changed files with 561 additions and 7 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

84
Doc/library/select.rst
View File

@ -6,7 +6,8 @@
This module provides access to the :c:func:`select` and :c:func:`poll` functions
available in most operating systems, :c:func:`epoll` available on Linux 2.5+ and
available in most operating systems, :c:func:`devpoll` available on
Solaris and derivatives, :c:func:`epoll` available on Linux 2.5+ and
:c:func:`kqueue` available on most BSD.
Note that on Windows, it only works for sockets; on other operating systems,
it also works for other file types (in particular, on Unix, it works on pipes).
@ -24,6 +25,19 @@ The module defines the following:
Following :pep:`3151`, this class was made an alias of :exc:`OSError`.
.. function:: devpoll()
(Only supported on Solaris and derivatives.) Returns a ``/dev/poll``
polling object; see section :ref:`devpoll-objects` below for the
methods supported by devpoll objects.
:c:func:`devpoll` objects are linked to the number of file
descriptors allowed at the time of instantiation. If your program
reduces this value, :c:func:`devpoll` will fail. If your program
increases this value, c:func:`devpoll` may return an
incomplete list of active file descriptors.
.. versionadded:: 3.3
.. function:: epoll(sizehint=-1)
(Only supported on Linux 2.5.44 and newer.) Returns an edge polling object,
@ -107,6 +121,74 @@ The module defines the following:
.. versionadded:: 3.2
.. _devpoll-objects:
``/dev/poll`` Polling Objects
----------------------------------------------
http://developers.sun.com/solaris/articles/using_devpoll.html
http://developers.sun.com/solaris/articles/polling_efficient.html
Solaris and derivatives have ``/dev/poll``. While :c:func:`select` is
O(highest file descriptor) and :c:func:`poll` is O(number of file
descriptors), ``/dev/poll`` is O(active file descriptors).
``/dev/poll`` behaviour is very close to the standard :c:func:`poll`
object.
.. method:: devpoll.register(fd[, eventmask])
Register a file descriptor with the polling object. Future calls to the
:meth:`poll` method will then check whether the file descriptor has any pending
I/O events. *fd* can be either an integer, or an object with a :meth:`fileno`
method that returns an integer. File objects implement :meth:`fileno`, so they
can also be used as the argument.
*eventmask* is an optional bitmask describing the type of events you want to
check for. The constants are the same that with :c:func:`poll`
object. The default value is a combination of the constants :const:`POLLIN`,
:const:`POLLPRI`, and :const:`POLLOUT`.
.. warning::
Registering a file descriptor that's already registered is not an
error, but the result is undefined. The appropiate action is to
unregister or modify it first. This is an important difference
compared with :c:func:`poll`.
.. method:: devpoll.modify(fd[, eventmask])
This method does an :meth:`unregister` followed by a
:meth:`register`. It is (a bit) more efficient that doing the same
explicitly.
.. method:: devpoll.unregister(fd)
Remove a file descriptor being tracked by a polling object. Just like the
:meth:`register` method, *fd* can be an integer or an object with a
:meth:`fileno` method that returns an integer.
Attempting to remove a file descriptor that was never registered is
safely ignored.
.. method:: devpoll.poll([timeout])
Polls the set of registered file descriptors, and returns a possibly-empty list
containing ``(fd, event)`` 2-tuples for the descriptors that have events or
errors to report. *fd* is the file descriptor, and *event* is a bitmask with
bits set for the reported events for that descriptor --- :const:`POLLIN` for
waiting input, :const:`POLLOUT` to indicate that the descriptor can be written
to, and so forth. An empty list indicates that the call timed out and no file
descriptors had any events to report. If *timeout* is given, it specifies the
length of time in milliseconds which the system will wait for events before
returning. If *timeout* is omitted, -1, or :const:`None`, the call will
block until there is an event for this poll object.
.. _epoll-objects:
Edge and Level Trigger Polling (epoll) Objects

94
Lib/test/test_devpoll.py Normal file
View File

@ -0,0 +1,94 @@
# Test case for the select.devpoll() function
# Initial tests are copied as is from "test_poll.py"
import os, select, random, unittest, sys
from test.support import TESTFN, run_unittest
try:
select.devpoll
except AttributeError:
raise unittest.SkipTest("select.devpoll not defined -- skipping test_devpoll")
def find_ready_matching(ready, flag):
match = []
for fd, mode in ready:
if mode & flag:
match.append(fd)
return match
class DevPollTests(unittest.TestCase):
def test_devpoll1(self):
# Basic functional test of poll object
# Create a bunch of pipe and test that poll works with them.
p = select.devpoll()
NUM_PIPES = 12
MSG = b" This is a test."
MSG_LEN = len(MSG)
readers = []
writers = []
r2w = {}
w2r = {}
for i in range(NUM_PIPES):
rd, wr = os.pipe()
p.register(rd)
p.modify(rd, select.POLLIN)
p.register(wr, select.POLLOUT)
readers.append(rd)
writers.append(wr)
r2w[rd] = wr
w2r[wr] = rd
bufs = []
while writers:
ready = p.poll()
ready_writers = find_ready_matching(ready, select.POLLOUT)
if not ready_writers:
self.fail("no pipes ready for writing")
wr = random.choice(ready_writers)
os.write(wr, MSG)
ready = p.poll()
ready_readers = find_ready_matching(ready, select.POLLIN)
if not ready_readers:
self.fail("no pipes ready for reading")
self.assertEqual([w2r[wr]], ready_readers)
rd = ready_readers[0]
buf = os.read(rd, MSG_LEN)
self.assertEqual(len(buf), MSG_LEN)
bufs.append(buf)
os.close(r2w[rd]) ; os.close(rd)
p.unregister(r2w[rd])
p.unregister(rd)
writers.remove(r2w[rd])
self.assertEqual(bufs, [MSG] * NUM_PIPES)
def test_timeout_overflow(self):
pollster = select.devpoll()
w, r = os.pipe()
pollster.register(w)
pollster.poll(-1)
self.assertRaises(OverflowError, pollster.poll, -2)
self.assertRaises(OverflowError, pollster.poll, -1 << 31)
self.assertRaises(OverflowError, pollster.poll, -1 << 64)
pollster.poll(0)
pollster.poll(1)
pollster.poll(1 << 30)
self.assertRaises(OverflowError, pollster.poll, 1 << 31)
self.assertRaises(OverflowError, pollster.poll, 1 << 63)
self.assertRaises(OverflowError, pollster.poll, 1 << 64)
def test_main():
run_unittest(DevPollTests)
if __name__ == '__main__':
test_main()

3
Misc/NEWS
View File

@ -365,6 +365,9 @@ Core and Builtins
Library
-------
- Issue #6397: Support "/dev/poll" polling objects in select module,
under Solaris & derivatives.
- Issues #1745761, #755670, #13357, #12629, #1200313: HTMLParser now correctly
handles non-valid attributes, including adjacent and unquoted attributes.

370
Modules/selectmodule.c
View File

@ -7,6 +7,14 @@
#include "Python.h"
#include <structmember.h>
#ifdef HAVE_SYS_DEVPOLL_H
#include <sys/resource.h>
#include <sys/devpoll.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#endif
#ifdef __APPLE__
/* Perform runtime testing for a broken poll on OSX to make it easier
* to use the same binary on multiple releases of the OS.
@ -648,6 +656,339 @@ static PyTypeObject poll_Type = {
poll_methods, /*tp_methods*/
};
#ifdef HAVE_SYS_DEVPOLL_H
typedef struct {
PyObject_HEAD
int fd_devpoll;
int max_n_fds;
int n_fds;
struct pollfd *fds;
} devpollObject;
static PyTypeObject devpoll_Type;
static int devpoll_flush(devpollObject *self)
{
int size, n;
if (!self->n_fds) return 0;
size = sizeof(struct pollfd)*self->n_fds;
self->n_fds = 0;
Py_BEGIN_ALLOW_THREADS
n = write(self->fd_devpoll, self->fds, size);
Py_END_ALLOW_THREADS
if (n == -1 ) {
PyErr_SetFromErrno(PyExc_IOError);
return -1;
}
if (n < size) {
/*
** Data writed to /dev/poll is a binary data structure. It is not
** clear what to do if a partial write occurred. For now, raise
** an exception and see if we actually found this problem in
** the wild.
** See http://bugs.python.org/issue6397.
*/
PyErr_Format(PyExc_IOError, "failed to write all pollfds. "
"Please, report at http://bugs.python.org/. "
"Data to report: Size tried: %d, actual size written: %d.",
size, n);
return -1;
}
return 0;
}
static PyObject *
internal_devpoll_register(devpollObject *self, PyObject *args, int remove)
{
PyObject *o;
int fd, events = POLLIN | POLLPRI | POLLOUT;
if (!PyArg_ParseTuple(args, "O|i:register", &o, &events)) {
return NULL;
}
fd = PyObject_AsFileDescriptor(o);
if (fd == -1) return NULL;
if (remove) {
self->fds[self->n_fds].fd = fd;
self->fds[self->n_fds].events = POLLREMOVE;
if (++self->n_fds == self->max_n_fds) {
if (devpoll_flush(self))
return NULL;
}
}
self->fds[self->n_fds].fd = fd;
self->fds[self->n_fds].events = events;
if (++self->n_fds == self->max_n_fds) {
if (devpoll_flush(self))
return NULL;
}
Py_RETURN_NONE;
}
PyDoc_STRVAR(devpoll_register_doc,
"register(fd [, eventmask] ) -> None\n\n\
Register a file descriptor with the polling object.\n\
fd -- either an integer, or an object with a fileno() method returning an\n\
int.\n\
events -- an optional bitmask describing the type of events to check for");
static PyObject *
devpoll_register(devpollObject *self, PyObject *args)
{
return internal_devpoll_register(self, args, 0);
}
PyDoc_STRVAR(devpoll_modify_doc,
"modify(fd[, eventmask]) -> None\n\n\
Modify a possible already registered file descriptor.\n\
fd -- either an integer, or an object with a fileno() method returning an\n\
int.\n\
events -- an optional bitmask describing the type of events to check for");
static PyObject *
devpoll_modify(devpollObject *self, PyObject *args)
{
return internal_devpoll_register(self, args, 1);
}
PyDoc_STRVAR(devpoll_unregister_doc,
"unregister(fd) -> None\n\n\
Remove a file descriptor being tracked by the polling object.");
static PyObject *
devpoll_unregister(devpollObject *self, PyObject *o)
{
int fd;
fd = PyObject_AsFileDescriptor( o );
if (fd == -1)
return NULL;
self->fds[self->n_fds].fd = fd;
self->fds[self->n_fds].events = POLLREMOVE;
if (++self->n_fds == self->max_n_fds) {
if (devpoll_flush(self))
return NULL;
}
Py_RETURN_NONE;
}
PyDoc_STRVAR(devpoll_poll_doc,
"poll( [timeout] ) -> list of (fd, event) 2-tuples\n\n\
Polls the set of registered file descriptors, returning a list containing \n\
any descriptors that have events or errors to report.");
static PyObject *
devpoll_poll(devpollObject *self, PyObject *args)
{
struct dvpoll dvp;
PyObject *result_list = NULL, *tout = NULL;
int poll_result, i;
long timeout;
PyObject *value, *num1, *num2;
if (!PyArg_UnpackTuple(args, "poll", 0, 1, &tout)) {
return NULL;
}
/* Check values for timeout */
if (tout == NULL || tout == Py_None)
timeout = -1;
else if (!PyNumber_Check(tout)) {
PyErr_SetString(PyExc_TypeError,
"timeout must be an integer or None");
return NULL;
}
else {
tout = PyNumber_Long(tout);
if (!tout)
return NULL;
timeout = PyLong_AsLong(tout);
Py_DECREF(tout);
if (timeout == -1 && PyErr_Occurred())
return NULL;
}
if ((timeout < -1) || (timeout > INT_MAX)) {
PyErr_SetString(PyExc_OverflowError,
"timeout is out of range");
return NULL;
}
if (devpoll_flush(self))
return NULL;
dvp.dp_fds = self->fds;
dvp.dp_nfds = self->max_n_fds;
dvp.dp_timeout = timeout;
/* call devpoll() */
Py_BEGIN_ALLOW_THREADS
poll_result = ioctl(self->fd_devpoll, DP_POLL, &dvp);
Py_END_ALLOW_THREADS
if (poll_result < 0) {
PyErr_SetFromErrno(PyExc_IOError);
return NULL;
}
/* build the result list */
result_list = PyList_New(poll_result);
if (!result_list)
return NULL;
else {
for (i = 0; i < poll_result; i++) {
num1 = PyLong_FromLong(self->fds[i].fd);
num2 = PyLong_FromLong(self->fds[i].revents);
if ((num1 == NULL) || (num2 == NULL)) {
Py_XDECREF(num1);
Py_XDECREF(num2);
goto error;
}
value = PyTuple_Pack(2, num1, num2);
Py_DECREF(num1);
Py_DECREF(num2);
if (value == NULL)
goto error;
if ((PyList_SetItem(result_list, i, value)) == -1) {
Py_DECREF(value);
goto error;
}
}
}
return result_list;
error:
Py_DECREF(result_list);
return NULL;
}
static PyMethodDef devpoll_methods[] = {
{"register", (PyCFunction)devpoll_register,
METH_VARARGS, devpoll_register_doc},
{"modify", (PyCFunction)devpoll_modify,
METH_VARARGS, devpoll_modify_doc},
{"unregister", (PyCFunction)devpoll_unregister,
METH_O, devpoll_unregister_doc},
{"poll", (PyCFunction)devpoll_poll,
METH_VARARGS, devpoll_poll_doc},
{NULL, NULL} /* sentinel */
};
static devpollObject *
newDevPollObject(void)
{
devpollObject *self;
int fd_devpoll, limit_result;
struct pollfd *fds;
struct rlimit limit;
Py_BEGIN_ALLOW_THREADS
/*
** If we try to process more that getrlimit()
** fds, the kernel will give an error, so
** we set the limit here. It is a dynamic
** value, because we can change rlimit() anytime.
*/
limit_result = getrlimit(RLIMIT_NOFILE, &limit);
if (limit_result != -1)
fd_devpoll = open("/dev/poll", O_RDWR);
Py_END_ALLOW_THREADS
if (limit_result == -1) {
PyErr_SetFromErrno(PyExc_OSError);
return NULL;
}
if (fd_devpoll == -1) {
PyErr_SetFromErrnoWithFilename(PyExc_IOError, "/dev/poll");
return NULL;
}
fds = PyMem_NEW(struct pollfd, limit.rlim_cur);
if (fds == NULL) {
close(fd_devpoll);
PyErr_NoMemory();
return NULL;
}
self = PyObject_New(devpollObject, &devpoll_Type);
if (self == NULL) {
close(fd_devpoll);
PyMem_DEL(fds);
return NULL;
}
self->fd_devpoll = fd_devpoll;
self->max_n_fds = limit.rlim_cur;
self->n_fds = 0;
self->fds = fds;
return self;
}
static void
devpoll_dealloc(devpollObject *self)
{
Py_BEGIN_ALLOW_THREADS
close(self->fd_devpoll);
Py_END_ALLOW_THREADS
PyMem_DEL(self->fds);
PyObject_Del(self);
}
static PyTypeObject devpoll_Type = {
/* The ob_type field must be initialized in the module init function
* to be portable to Windows without using C++. */
PyVarObject_HEAD_INIT(NULL, 0)
"select.devpoll", /*tp_name*/
sizeof(devpollObject), /*tp_basicsize*/
0, /*tp_itemsize*/
/* methods */
(destructor)devpoll_dealloc, /*tp_dealloc*/
0, /*tp_print*/
0, /*tp_getattr*/
0, /*tp_setattr*/
0, /*tp_reserved*/
0, /*tp_repr*/
0, /*tp_as_number*/
0, /*tp_as_sequence*/
0, /*tp_as_mapping*/
0, /*tp_hash*/
0, /*tp_call*/
0, /*tp_str*/
0, /*tp_getattro*/
0, /*tp_setattro*/
0, /*tp_as_buffer*/
Py_TPFLAGS_DEFAULT, /*tp_flags*/
0, /*tp_doc*/
0, /*tp_traverse*/
0, /*tp_clear*/
0, /*tp_richcompare*/
0, /*tp_weaklistoffset*/
0, /*tp_iter*/
0, /*tp_iternext*/
devpoll_methods, /*tp_methods*/
};
#endif /* HAVE_SYS_DEVPOLL_H */
PyDoc_STRVAR(poll_doc,
"Returns a polling object, which supports registering and\n\
unregistering file descriptors, and then polling them for I/O events.");
@ -658,6 +999,19 @@ select_poll(PyObject *self, PyObject *unused)
return (PyObject *)newPollObject();
}
#ifdef HAVE_SYS_DEVPOLL_H
PyDoc_STRVAR(devpoll_doc,
"Returns a polling object, which supports registering and\n\
unregistering file descriptors, and then polling them for I/O events.");
static PyObject *
select_devpoll(PyObject *self, PyObject *unused)
{
return (PyObject *)newDevPollObject();
}
#endif
#ifdef __APPLE__
/*
* On some systems poll() sets errno on invalid file descriptors. We test
@ -1715,6 +2069,11 @@ static PyTypeObject kqueue_queue_Type = {
};
#endif /* HAVE_KQUEUE */
/* ************************************************************************ */
PyDoc_STRVAR(select_doc,
@ -1746,6 +2105,9 @@ static PyMethodDef select_methods[] = {
#ifdef HAVE_POLL
{"poll", select_poll, METH_NOARGS, poll_doc},
#endif /* HAVE_POLL */
#ifdef HAVE_SYS_DEVPOLL_H
{"devpoll", select_devpoll, METH_NOARGS, devpoll_doc},
#endif
{0, 0}, /* sentinel */
};
@ -1768,6 +2130,9 @@ static struct PyModuleDef selectmodule = {
NULL
};
PyMODINIT_FUNC
PyInit_select(void)
{
@ -1824,6 +2189,11 @@ PyInit_select(void)
}
#endif /* HAVE_POLL */
#ifdef HAVE_SYS_DEVPOLL_H
if (PyType_Ready(&devpoll_Type) < 0)
return NULL;
#endif
#ifdef HAVE_EPOLL
Py_TYPE(&pyEpoll_Type) = &PyType_Type;
if (PyType_Ready(&pyEpoll_Type) < 0)

7
configure vendored
View File

@ -6139,12 +6139,13 @@ fi
for ac_header in asm/types.h conio.h curses.h direct.h dlfcn.h errno.h \
fcntl.h grp.h \
ieeefp.h io.h langinfo.h libintl.h ncurses.h poll.h process.h pthread.h \
ieeefp.h io.h langinfo.h libintl.h ncurses.h process.h pthread.h \
sched.h shadow.h signal.h stdint.h stropts.h termios.h \
unistd.h utime.h \
sys/audioio.h sys/xattr.h sys/bsdtty.h sys/epoll.h sys/event.h sys/file.h sys/loadavg.h \
poll.h sys/devpoll.h sys/epoll.h sys/poll.h \
sys/audioio.h sys/xattr.h sys/bsdtty.h sys/event.h sys/file.h sys/loadavg.h \
sys/lock.h sys/mkdev.h sys/modem.h \
sys/param.h sys/poll.h sys/select.h sys/sendfile.h sys/socket.h sys/statvfs.h \
sys/param.h sys/select.h sys/sendfile.h sys/socket.h sys/statvfs.h \
sys/stat.h sys/termio.h sys/time.h \
sys/times.h sys/types.h sys/uio.h sys/un.h sys/utsname.h sys/wait.h pty.h \
libutil.h sys/resource.h netpacket/packet.h sysexits.h bluetooth.h \

7
configure.in
View File

@ -1329,12 +1329,13 @@ dnl AC_MSG_RESULT($cpp_type)
AC_HEADER_STDC
AC_CHECK_HEADERS(asm/types.h conio.h curses.h direct.h dlfcn.h errno.h \
fcntl.h grp.h \
ieeefp.h io.h langinfo.h libintl.h ncurses.h poll.h process.h pthread.h \
ieeefp.h io.h langinfo.h libintl.h ncurses.h process.h pthread.h \
sched.h shadow.h signal.h stdint.h stropts.h termios.h \
unistd.h utime.h \
sys/audioio.h sys/xattr.h sys/bsdtty.h sys/epoll.h sys/event.h sys/file.h sys/loadavg.h \
poll.h sys/devpoll.h sys/epoll.h sys/poll.h \
sys/audioio.h sys/xattr.h sys/bsdtty.h sys/event.h sys/file.h sys/loadavg.h \
sys/lock.h sys/mkdev.h sys/modem.h \
sys/param.h sys/poll.h sys/select.h sys/sendfile.h sys/socket.h sys/statvfs.h \
sys/param.h sys/select.h sys/sendfile.h sys/socket.h sys/statvfs.h \
sys/stat.h sys/termio.h sys/time.h \
sys/times.h sys/types.h sys/uio.h sys/un.h sys/utsname.h sys/wait.h pty.h \
libutil.h sys/resource.h netpacket/packet.h sysexits.h bluetooth.h \

3
pyconfig.h.in
View File

@ -883,6 +883,9 @@
/* Define to 1 if you have the <sys/bsdtty.h> header file. */
#undef HAVE_SYS_BSDTTY_H
/* Define to 1 if you have the <sys/devpoll.h> header file. */
#undef HAVE_SYS_DEVPOLL_H
/* Define to 1 if you have the <sys/dir.h> header file, and it defines `DIR'.
*/
#undef HAVE_SYS_DIR_H