The attached patch enables shared extension
modules to build cleanly under Cygwin without
moving the static initialization of certain function
pointers (i.e., ones exported from the Python
DLL core) to a module initialization function.
Additionally, this patch fixes the modules that
have been changed in the past to accommodate
Cygwin.
On HPUX, Solaris, Tru64 (Dec UNIX), and IRIX (I think),
O_NONBLOCK is the POSIX version of non-blocking I/O
which is what we want.
On Linux and FreeBSD (at least), O_NONBLOCK and O_NDELAY are the same.
So this change should have no negative effect on those platforms.
Tested on Linux, Solaris, HPUX.
Thanks to Anders Qvist for diagnosing this problem.
value; others were inconsistent in what to name the argument or return
value; a few module-global functions had "socket." in front of their
name, against convention.
WSAEWOULDBLOCK, the second connect() attempt appears to yield WSAEISCONN
on Win98 but WSAEINVAL on Win2K. So accept either as meaning "yawn,
fine". This allows test_socket to succeed on my Win2K box (which it
already did on my Win98SE box).
The staticforward define was needed to support certain broken C
compilers (notably SCO ODT 3.0, perhaps early AIX as well) botched the
static keyword when it was used with a forward declaration of a static
initialized structure. Standard C allows the forward declaration with
static, and we've decided to stop catering to broken C compilers. (In
fact, we expect that the compilers are all fixed eight years later.)
I'm leaving staticforward and statichere defined in object.h as
static. This is only for backwards compatibility with C extensions
that might still use it.
XXX I haven't updated the documentation.
that retries the connect() call in timeout mode so it can be shared
between connect() and connect_ex(), and needs only a single #ifdef.
The test for this was doing funky stuff I don't approve of,
so I removed it in favor of a simpler test. This allowed me
to implement a simpler, "purer" form of the timeout retry code.
Hopefully that's enough (if you want to be fancy, use non-blocking
mode and decode the errors yourself, like before).
- setblocking(0) and settimeout(0) are now equivalent, and ditto for
setblocking(1) and settimeout(None).
- Don't raise an exception from internal_select(); let the final call
report the error (this means you will get an EAGAIN error instead of
an ETIMEDOUT error -- I don't care).
- Move the select to inside the Py_{BEGIN,END}_ALLOW_THREADS brackets,
so other theads can run (this was a bug in the original code).
- Redid the retry logic in connect() and connect_ex() to avoid masking
errors. This probably doesn't work for Windows yet; I'll fix that
next. It may also fail on other platforms, depending on what
retrying a connect does; I need help with this.
- Get rid of the retry logic in accept(). I don't think it was needed
at all. But I may be wrong.
settimeout(). Already, settimeout() canceled non-blocking mode; now,
setblocking() also cancels the timeout. This is easier to document.
(XXX should settimeout(0) be an alias for setblocking(0)? They seem
to have roughly the same effect. Also, I'm not sure that the code in
connect() and accept() is correct in all cases. We'll sort this out
soon enough.)
not testing it -- apparently test_timeout.py doesn't test anything
useful):
In internal_select():
- The tv_usec part of the timeout for select() was calculated wrong.
- The first argument to select() was one too low.
- The sense of the direction argument to internal_select() was
inverted.
In PySocketSock_settimeout():
- The calls to internal_setblocking() were swapped.
Also, repaired some comments and fixed the test for the return value
of internal_select() in sendall -- this was in the original patch.
I've made considerable changes to Michael's code, specifically to use
the select() system call directly and to store the timeout as a C
double instead of a Python object; internally, -1.0 (or anything
negative) represents the None from the API.
I'm not 100% sure that all corner cases are covered correctly, so
please keep an eye on this. Next I'm going to try it Windows before
Tim complains.
No way is this a bugfix candidate. :-)
don't understand how this function works, also beefed up the docs. The
most common usage error is of this form (often spread out across gotos):
if (_PyString_Resize(&s, n) < 0) {
Py_DECREF(s);
s = NULL;
goto outtahere;
}
The error is that if _PyString_Resize runs out of memory, it automatically
decrefs the input string object s (which also deallocates it, since its
refcount must be 1 upon entry), and sets s to NULL. So if the "if"
branch ever triggers, it's an error to call Py_DECREF(s): s is already
NULL! A correct way to write the above is the simpler (and intended)
if (_PyString_Resize(&s, n) < 0)
goto outtahere;
Bugfix candidate.
socketmodule.c. No code outside of the .c file references it, so it
doesn't belong the .h file (at least not yet ...), and declaring it
an imported symbol in the .h file can't be made to work on Windows (it's
a cross-DLL symbol then) without substantial code rewriting. Also
repaired the comment that goes along with the decl, to stop referring
to names and functions that haven't existed for 7 years <wink>.
socketmodule.c compiles cleanly on Windows again. The test_socket dies
at once, though (later).
helper module _ssl.
The support for the RAND_* APIs in _ssl is now only enabled
for OpenSSL 0.9.5 and up since they were added in that
release.
Note that socketmodule.* should really be renamed to _socket.* --
unfortunately, this seems to lose the CVS history of the file.
Please review and test... I was only able to test the header file
chaos in socketmodule.c/h on Linux. The test run through fine
and compiles don't give errors or warnings.
WARNING: This patch does *not* include changes to the various
non-Unix build process files.
"socket.socket" -- on Windows, "socket.socket" is the wrapper class.
Also added the module name to the SSL type (which is not a new-style
class -- I don't want to mess with it yet).
constructor acts just like socket() before. All three arguments have
a sensible default now; socket() is equivalent to
socket(AF_INET, SOCK_STREAM).
One minor issue: the socket() function and the SocketType had
different doc strings; socket.__doc__ gave the signature,
SocketType.__doc__ gave the methods. I've merged these for now, but
maybe the list of methods is no longer necessary since it can easily
be recovered through socket.__dict__.keys(). The problem with keeping
it is that the total doc string is a bit long (34 lines -- it scrolls
of a standard tty screen).
Another general issue with the socket module is that it's a big mess.
There's pages and pages of random platform #ifdefs, and the naming
conventions are totally wrong: it uses Py prefixes and CapWords for
static functions. That's a cleanup for another day... (Also I think
the big starting comment that summarizes the API can go -- it's a
repeat of the docstring.)
error occurs, and doesn't return a count. (This is my second patch
from SF patch #474307, with small change to the docstring for send().)
2.1.2 "bugfix" candidate.
Quoth the OpenSSL RAND_add man page:
OpenSSL makes sure that the PRNG state is unique for each
thread. On systems that provide /dev/urandom, the
randomness device is used to seed the PRNG transparently.
However, on all other systems, the application is
responsible for seeding the PRNG by calling RAND_add(),
RAND_egd(3) or RAND_load_file(3).
I decided to expose RAND_add() because it's general and RAND_egd()
because it's a useful special case. RAND_load_file() didn't seem to
offer much over RAND_add(), so I skipped it. Also supplied
RAND_status() which returns true if the PRNG is seeded and false if
not.
To whoever who changed a bunch of (PyCFunction) casts to
(PyNoArgsFunction) in PyMethodDef initializers: don't do that. The
cast is to shut the compiler up. The compiler wants the function
pointer initializer to be a PyCFunction.
a misunderstanding of the refcont behavior of the 'O' format code in
PyArg_ParseTuple() and Py_BuildValue(), respectively.
- pobj is only a borrowed reference, so should *not* be DECREF'ed at
the end. This was the cause of SF bug #470635.
- The Py_BuildValue() call would leak the object produced by
makesockaddr(). (I found this by eyeballing the code.)
Use #define X509_NAME_MAXLEN for server/issuer length on an SSL
object.
Update doc strings for socket.ssl() and ssl methods read() and
write().
PySSL_SSLwrite(): Check return value and raise exception on error.
Use int for len instead of size_t. (All the function the size_t obj
was passed to our from expected an int!)
PySSL_SSLread(): Check return value of PyArg_ParseTuple()! More
robust checks of return values from SSL_read().
Change all the local names that start with SSL to start with PySSL.
The OpenSSL library defines lots of calls that start with "SSL_". The
calls for Python's SSL objects also started with "SSL_". This choice
made it really confusing to figure out which calls were to the library
and which calls were local to the file.
Add PySSL_SetError() that sets an exception based on the information
from SSL_get_error(). This function will eventually replace all the
calls that set it with an error message that is based on the name of
the call that failed rather than the reason it failed. (Example: If
SSL_connect() failed it used to report "SSL_connect error" now it will
offer a specific message about why SSL_connect failed.)
XXX It might be helpful to augment the error message generated
below with the name of the SSL function that generated the error.
I expect it's obvious most of the time.
Remove several unnecessary INCREFs in the module's constructor call.
PyDict_SetItem() and friends do the INCREF for you.
In SSL_dealloc(), free/dealloc them only if they're non-NULL.
Fixes some obvious core dumps, but not sure yet if there are more
semantics to the SSL calls that would affect the dealloc.
XXX [1] These changes aren't tested very thoroughly, because regrtest
doesn't do any SSL tests. I've done some trivial tests on my own, but
don't really know how to use the key and cert files. In one case, an
SSL-level error causes Python to dump core. I'll get the fixed in the
next round of changes.
XXX [2] The checkin removes the x_attr member of the SSLObject struct.
I'm not sure if this is kosher for backwards compatibility at the
binary level. Perhaps its safer to keep the member but keep it
assigned to NULL.
And the leaks?
newSSLObject() called PyDict_New(), stored the result in x_attr
without checking it, and later stored NULL in x_attr without doing
anything to the dict. So the dict always leaks. There is no further
reference to x_attr, so I just removed it completely.
The error cases in newSSLObject() passed the return value of
PyString_FromString() directly to PyErr_SetObject().
PyErr_SetObject() expects a borrowed reference, so the string leaked.
pyport.h: typedef a new Py_intptr_t type.
DELICATE ASSUMPTION: That HAVE_UINTPTR_T implies intptr_t is
available as well as uintptr_t. If that turns out not to be
true, things must get uglier (C99 wants both, so I think it's
an assumption we're *likely* to get away with).
thread_nt.h, PyThread_start_new_thread: MS _beginthread is documented
as returning unsigned long; no idea why uintptr_t was being used.
Others: Always use Py_[u]intptr_t, never [u]intptr_t directly.
New functions getnameinfo, getaddrinfo. New exceptions socket.gaierror,
socket.herror. Various new constants, in particular AF_INET6 and error
codes and parameters for getaddrinfo.
AF_INET6 support in setipaddr, makesockaddr, getsockaddr, getsockaddrlen,
gethost_common, PySocket_gethostbyaddr.
When getting a string buffer for a string we just created, use
PyString_AS_STRING() instead of PyString_AsString() to avoid the
call overhead and extra type check.
this could cause invalid paths to be returned for AF_UNIX sockets on some
platforms (including FreeBSD 4.2-RELEASE), appearantly because there is
no assurance that the address will be nul-terminated when filled in by the
kernel.
PySocketSock_recvfrom(): Use PyString_AS_STRING() to get the data pointer
of a string we create ourselves; there is no need for the extra type
check from PyString_AsString().
This closes SF bug #416573.
OpenSSL versions beore 0.9.5. This just is too experimental to be
worth it, especially since the user would have to do some severe
hacking of the Modules/Setup file to even enable the EGD code, and
without the EGD code it would always spit out a warning on some
systems -- even when socket.ssl() is not used. Fixing that properly
is not my job; the EGD patch is clearly not so important that it
should hold up the 2.1 release.
problem reported by Neil Schemenauer on python-dev on 4/12/01, wth
subject "Problem with SSL and socketmodule on Debian Potato?".
It's tentative because Moshe objected, but Martin rebutted, and Moshe
seems unavailable for comments.
(Note that with OpenSSL 0.9.6a, I get a lot of compilation warnings
for socketmodule.c -- I'm assuming I can safely ignore these until 2.1
is released.)
Adds support for raw packets (AF_PACKET) under Linux. I haven't
tested this code thoroughly; it compiles and the basic calls all work
without crashing. Not sure what to actually do with raw sockets though.
Not sure what other platforms this might be useful for.
This doesn't change the copyright status for these files -- just the
markings! Doing it on the main branch for these three files for which
the HEAD revision was pushed back into 1.6.
for systems that are missing those declarations from system include files.
Start by moving a pointy-haired ones from their previous locations to the
new section.
(The gethostname() one, for instance, breaks on several systems, because
some define it as (char *, size_t) and some as (char *, int).)
I purposely decided not to include the summary of used #defines like Tim did
in the first section of pyport.h. In my opinion, the number of #defines
likedly to be used by this section would make such an overview unwieldy. I
would suggest documenting the non-obvious ones, though.
and a couple of functions that were missed in the previous batches. Not
terribly tested, but very carefully scrutinized, three times.
All these were found by the little findkrc.py that I posted to python-dev,
which means there might be more lurking. Cases such as this:
long
func(a, b)
long a;
long b; /* flagword */
{
and other cases where the last ; in the argument list isn't followed by a
newline and an opening curly bracket. Regexps to catch all are welcome, of
course ;)
This patch fixes possible overflows in the socket module for 64-bit
platforms (mainly Win64). The changes are:
- abstract the socket type to SOCKET_T (this is SOCKET on Windows, int
on Un*x), this is necessary because sizeof(SOCKET) > sizeof(int) on
Win64
- use INVALID_SOCKET on Win32/64 for an error return value for
accept()
- ensure no overflow of the socket variable for: (1) a PyObject return
value (use PyLong_FromLongLong if necessary); and (2) printf
formatting in repr().
Closes SourceForge patch #100516.
For more comments, read the patches@python.org archives.
For documentation read the comments in mymalloc.h and objimpl.h.
(This is not exactly what Vladimir posted to the patches list; I've
made a few changes, and Vladimir sent me a fix in private email for a
problem that only occurs in debug mode. I'm also holding back on his
change to main.c, which seems unnecessary to me.)
socklen_t (unsigned int) for most size parameters. Apparently this is
part of the UNIX 98 standard.
[GvR: the changes to configure.in etc. that I just checked in make
sure that socklen_t is defined everywhere, so I deleted the little
part of Jack's mod to define socklen_t if not in GUSI2. I suppose I
will have to add it to the Windows config.h in a minute.]
Windows), soclose (on OS2), or to close (everywhere else).
Hopefully this fixes a new compilation error that I suddenly get on
Windows because the macro definition for close -> closesocket
apparently was done before including io.h, which contains a prototype
for close. (No idea why this wasn't an error before.)
Brian E Gallew, which were improved and adapted to OpenSSL 0.9.4 by
Laszlo Kovacs of HP. Both have kindly given permission to include
the patches in the Python distribution. Final formatting by GvR.
the right variant of gethostbyname_r for us, since not all Linuxes are
equal in this respect. Reported by Laurent Pointal.
(2) On BeOS, Chris Herborth reports that instead of arpa/inet.h you
must include net/netdb.h to get the inet_ntoa() and inet_addr()
prototypes.
data struct before calling gethostby{name,addr}_r(); (2) ignore the
3/5/6 args determinations made by the configure script and switch on
platform identifiers instead:
AIX, OSF have 3 args
Sun, SGI have 5 args
Linux has 6 args
On all other platforms, undef HAVE_GETHOSTBYNAME_R altogether.
- Use HAVE_GETHOSTBYNAME_R_6_ARG instead of testing for Linux and
glibc2.
- If gethostbyname takes 3 args, undefine HAVE_GETHOSTBYNAME_R --
don't know what code should be used.
- New symbol USE_GETHOSTBYNAME_LOCK defined iff the lock should be used.
- Modify the gethostbyaddr() code to also hold on to the lock until
after it is safe to release, overlapping with the Python lock.
(Note: I think that it could in theory be possible that Python code
executed while gethostbyname_lock is held could attempt to reacquire
the lock -- e.g. in a signal handler or destructor. I will simply say
"don't do that then.")
Here's a patch to fix the race condition, which wasn't fixed by Rob's
patch. It holds the gethostbyname lock until the results are copied out,
which means that this lock and the Python global lock are held at the same
time. This shouldn't be a problem as long as the gethostbyname lock is
always acquired when the global lock is not held.
gethostbyaddr(). (Plain gethostbyname() returns only the IP address.)
This moves the code shared by gethostbyaddr() and gethostbyname_ex()
to a subroutine.
Original patch by Dan Stromberg; some tweaks by GvR.
so that our #ifdef test has the wrong effect. Substitute hardcoded
values for some important symbols (but not for the whole range -- some
are pretty obscure so it's not worth it).
(1) Use PyErr_NewException("module.class", NULL, NULL) to create the
exception object.
(2) Remove all calls to Py_FatalError(); instead, return or
ignore the errors -- the import code now checks PyErr_Occurred()
after calling a module's init function, so it's no longer a
fatal error for the initialization to fail.
Also did some small cleanups, e.g. removed unnecessary test for
"already initialized" from initfpectl(), and unified
initposix()/initnt().
I haven't checked this very thoroughly, so while the changes are
pretty trivial -- beware of untested code!
Guido van Rossum
Jason Tishler
Andrew MacIntyre
Martin v. Löwis
Neal Norwitz
Tim Peters
Mark Hammond
Jeremy Hylton
Jack Jansen
Neil Schemenauer
Fred Drake
Marc-André Lemburg
Andrew M. Kuchling
Michael W. Hudson
Barry Warsaw
Thomas Wouters
Moshe Zadka
Peter Schneider-Kamp
Fredrik Lundh