catch_warnings(), and clean up the API.
While expanding the test suite, a bug was found where a warning about the
'line' argument to showwarning() was not letting functions with '*args' go
without a warning.
Closes issue 3602.
Code review by Benjamin Peterson.
triggered warnings should be captured. This allows the context manager to be
used to just prevent the internal state of the 'warnings' framework and thus
allow triggered warnings to be displayed.
'warnings' code in places where it was previously not possible (e.g., the
parser). It could also potentially lead to a speed-up in interpreter start-up
if the C version of the code (_warnings) is imported over the use of the
Python version in key places.
Closes issue #1631171.
to listen on in network-oriented tests has been refined in an effort to
facilitate running multiple instances of the entire regression test suite
in parallel without issue. test_support.bind_port() has been fixed such
that it will always return a unique port -- which wasn't always the case
with the previous implementation, especially if socket options had been
set that affected address reuse (i.e. SO_REUSEADDR, SO_REUSEPORT). The
new implementation of bind_port() will actually raise an exception if it
is passed an AF_INET/SOCK_STREAM socket with either the SO_REUSEADDR or
SO_REUSEPORT socket option set. Furthermore, if available, bind_port()
will set the SO_EXCLUSIVEADDRUSE option on the socket it's been passed.
This currently only applies to Windows. This option prevents any other
sockets from binding to the host/port we've bound to, thus removing the
possibility of the 'non-deterministic' behaviour, as Microsoft puts it,
that occurs when a second SOCK_STREAM socket binds and accepts to a
host/port that's already been bound by another socket. The optional
preferred port parameter to bind_port() has been removed. Under no
circumstances should tests be hard coding ports!
test_support.find_unused_port() has also been introduced, which will pass
a temporary socket object to bind_port() in order to obtain an unused port.
The temporary socket object is then closed and deleted, and the port is
returned. This method should only be used for obtaining an unused port
in order to pass to an external program (i.e. the -accept [port] argument
to openssl's s_server mode) or as a parameter to a server-oriented class
that doesn't give you direct access to the underlying socket used.
Finally, test_support.HOST has been introduced, which should be used for
the host argument of any relevant socket calls (i.e. bind and connect).
The following tests were updated to following the new conventions:
test_socket, test_smtplib, test_asyncore, test_ssl, test_httplib,
test_poplib, test_ftplib, test_telnetlib, test_socketserver,
test_asynchat and test_socket_ssl.
It is now possible for multiple instances of the regression test suite to
run in parallel without issue.
run the test simultaneously. The simplest thing I found that worked
on both Windows and Unix was to use the PID. It's unique so should be
sufficient. This should prevent many of the spurious failures of
the automated tests since they run as different users.
Also cleanup the directory consistenly in the tearDown methods.
It would be nice if someone ensured that the directories are always
created with a consistent name.
This should make the tests more robust at the expense of allowing
tests to be sloppier by not requiring them to cleanup after themselves.
(It will legitamitely help when running two test suites simultaneously
or if another process is already using one of the predefined ports.)
Also simplifies (slightLy) the exception handling elsewhere.
test.test_support.catch_warning is more full-featured and provides the same
functionality.
Since guard_warnings_filter was added in 2.6 there is no
backwards-compatibility issues.
If the call to requires() doesn't precede the filesystem check, we get the following situation:
1. ./python Lib/test/regrtest.py test_foo # test needs urlfetch, not enabled, so skipped
2. ./python Lib/test/regrtest.py -u urlfetch test_foo # test runs
3. ./python Lib/test/regrtest.py test_foo # test runs (!)
By moving the call to requires() *before* the filesystem check, the fact that fetched files are cached on the local disk becomes an implementation detail, rather than a semantics-changing point of note.
nests test.test_support.TransientResource context managers that capture
exceptions raised when the Internet connection is flaky.
Initially using in test_socket_ssl but should probably be expanded to cover any
test that should not raise the captured exceptions if the Internet connection
works.
surround calls to resources that may or may not be available. Specifying the
expected exception and attributes to be raised if the resource is not available
prevents overly broad catching of exceptions.
This is meant to help suppress spurious failures by raising
test.test_support.ResourceDenied if the exception matches. It would probably
be good to go through the various network tests and surround the calls to catch
connection timeouts (as done with test_socket_ssl in this commit).
be called at the end of each test that spawns children (perhaps it
should be called from regrtest instead?). This will hopefully prevent
some of the unexplained failures in the buildbots (hppa and alpha)
during tests that spawn children. The problems were not reproducible.
There were many zombies that remained at the end of several tests.
In the worst case, this shouldn't cause any more problems,
though it may not help either. Time will tell.
issues that crop up from time to time, but this change seems to have been
pretty stable (no spurious warnings) for about a week.
Other modules which use threads may require similar use of
threading_setup/threading_cleanup from test_support.
tuples. Lots to be added, still, but this will give big-memory people
something to play with in 2.5 alpha 2, and hopefully get more people to
write these tests.
Nick Coghlan
Brett Cannon
Neal Norwitz
Benjamin Peterson
Georg Brandl
Alexandre Vassalotti
Skip Montanaro
Trent Nelson
Christian Heimes
Collin Winter
Walter Dörwald
Tim Peters
Armin Rigo
Thomas Wouters