* No longer attempts to close already freed socket file descriptor
* Use socket object to be compatible with Windows
* Do not use a timeout to avoid complication with non-blocking mode
* Use internal localhost server rather than depending on a third party
* Avoid trouble with buffered HTTP data by testing tunnelled CONNECT data
This has no semantic impact as the class is guarded with a hasattr()
check; this is being done to keep typing.py in sync between Python 3.5
and 3.6 as requested by Guido.
ByteArrayAsStringTest.fixtype() was converting test data to bytes, not byte-
array, therefore many of the test cases inherited in this class were not
actually being run on the bytearray type.
The tests in buffer_tests.py were redundant with methods in string_tests
.MixinStrUnicodeUserStringTest and string_tests.CommonTest. These methods are
now moved into string_tests.BaseTest, where they will also get run for bytes
and bytearray.
This change also moves test_additional_split(), test_additional_rsplit(), and
test_strip() from CommonTest to BaseTest, meaning these tests are now run for
bytes and bytearray. I plan to eliminate redundancies with existing tests in
test_bytes.py soon.
This syncs to rev 7b43ada77821d23e55e3a4b35f6055a59b9e1ad7 there.
Summary:
- Add typing.DefaultDict (as a generic variant of collections.defaultdict).
- Use collections.Reversible if it exists (only relevant for Python 3.6).
- Revamped generic class behavior to conform to updated PEP 484.
- Improve speed of Generic.__new__.
- Make sure __init__ is called for new Generic instances. Fix issue #26391.
- Refactor async support to be compatible with 3.2, 3.3, 3.4.
- Remove 'io' and 're' from __all__ (they still exist, just not
included by "import *"). Fix issue #26234.
- Change @overload -- you can now use it outside stubs (you still
cannot call the decorated function though).
test_forever() stores its state into the builtins module since the test module
is reloaded at each run.
Remove also warning to detect leaked tests of a previous run.
* Fix test_os.BytesWalkTests on Windows
* Mimick better the reference os.DirEntry on Windows
* _DummyDirEntry now caches os.stat() result
* _DummyDirEntry constructor now tries to get os.stat()
* Fix os._DummyDirEntry.is_symlink(), don't follow symbolic links:
use os.stat(path, follow_symlinks=False).
Move many tests from NetworkedTests and NetworkedBIOTests to a new Simple-
BackgroundTests class, using the existing ThreadedEchoServer and SIGNED_
CERTFILE infrastructure.
For tests that cause the server to crash by rejecting its certificate,
separate them into independent test methods.
Added custom root certificate to capath with the following commands:
cp Lib/test/{pycacert.pem,capath/}
# Edit copy to remove part before certificate
c_rehash -v Lib/test/capath/
c_rehash -v -old Lib/test/capath/
# Note the generated file names
cp Lib/test/capath/{pycacert.pem,b1930218.0}
mv Lib/test/capath/{pycacert.pem,ceff1710.0}
Change to pure PEM version of SIGNING_CA because PEM_cert_to_DER_cert() does
not like the extra text at the start.
Moved test_connect_ex_error() into BasicSocketTests and rewrote it to connect
to a reserved localhost port.
NetworkedTests.test_get_server_certificate_ipv6() split out because it needs
to connect to an IPv6 DNS address.
The only reference left to self-signed.pythontest.net is test_timeout_
connect_ex(), which needs a remote server to reliably time out the
connection, but does not rely on the server running SSL.
Made ThreadedEchoServer call unwrap() by default when it sees the client has
shut the connection down, so that the client can cleanly call unwrap().
* Replace get/restore methods with a Resource class and Resource subclasses
* Create ModuleAttr, ModuleAttrList and ModuleAttrDict helper classes
* Use __subclasses__() to get resource classes instead of using an hardcoded
list (2 shutil resources were missinged in the list!)
* Don't define MultiprocessingProcessDangling resource if the multiprocessing
module is missing
* Nicer diff for dictionaries. Useful for the big os.environ dict
* Reorder code to group resources
tag: tip
parent: 100742:ebae81b31cf6
user: Victor Stinner <victor.stinner@gmail.com>
date: Fri Mar 25 15:03:34 2016 +0100
files: Lib/test/test_os.py
description:
test_os: Win32ErrorTests checks if file exists
Don't use os.path.exists() since it ignores *any* OSError.
* multiprocessing: open file with closefd=False to avoid ResourceWarning
* _test_multiprocessing: open file with O_EXCL to detect bugs in tests (if a
previous test forgot to remove TESTFN)
* test_sys_exit(): remove TESTFN after each loop iteration
Initial patch written by Serhiy Storchaka.
* multiprocessing: open file with closefd=False to avoid ResourceWarning
* _test_multiprocessing: open file with O_EXCL to detect bugs in tests (if a
previous test forgot to remove TESTFN)
* test_sys_exit(): remove TESTFN after each loop iteration
Initial patch written by Serhiy Storchaka.
Issue #26637: The importlib module now emits an ImportError rather than a
TypeError if __import__() is tried during the Python shutdown process but
sys.path is already cleared (set to None).
Issue #21925: warnings.formatwarning() now catches exceptions when calling
linecache.getline() and tracemalloc.get_object_traceback() to be able to log
ResourceWarning emitted late during the Python shutdown process.
Issue #21925: warnings.formatwarning() now catches exceptions on
linecache.getline(...) to be able to log ResourceWarning emitted late during
the Python shutdown process.
* Rename libregrtest.main_in_temp_cwd() to libregrtest.main()
* Add regrtest.main_in_temp_cwd() alias to libregrtest.main()
* Move old main_in_temp_cwd() code into libregrtest.Regrtest.main()
* Update multiple scripts to call libregrtest.main()
* add create_file() helper function
* create files using "x" mode instead of "w" to detect when a previous test
forget to remove a file
* open file for writing in unbuferred mode (buffering=0)
* replace "try/finally: unlink" with self.addCleanup(support.unlink)
* register unlink cleanup function *before* creating new files
* Use context manager on urllib objects to ensure that they are closed on error
* Use self.addCleanup() to cleanup resources even if a test is interrupted
with CTRL+c
* Replace "try/finally: os.remove()" with self.addCleanup(support.unlink) or
self.addCleanup(support.rmdir): the support function handles the case when
the file doesn't exist
* Replace "try/finally: f.close()" with "with open(...) as f:"
* test_getsize: add a second test with a different size
* Create file using "x" mode to ensure that the file didn't exist before, to
detect bugs in tests
* Open files in unbuffered mode (buferring=0) to write immediatly data on disk
* Replace map() with simpler code
* Split isdir() unit test into two units tests to make them less dependant,
same change for isfile() test
* test_samefile(): test also two different files
Issue #25911:
* Try to fix test_os.BytesWalkTests on Windows
* Try to mimick better the reference os.DirEntry on Windows
* _DummyDirEntry now caches os.stat() result
* _DummyDirEntry constructor now tries to get os.stat()
Issue #23848, #26622:
* faulthandler now only logs fatal Windows exceptions.
* write error code as decimal, not as hexadecimal
* replace "Windows exception" with "Windows fatal exception"
Issue #23848: On Windows, faulthandler.enable() now also installs an exception
handler to dump the traceback of all Python threads on any Windows exception,
not only on UNIX signals (SIGSEGV, SIGFPE, SIGABRT).
Issue #26604:
* Add a new optional source parameter to _warnings.warn() and warnings.warn()
* Modify asyncore, asyncio and _pyio modules to set the source parameter when
logging a ResourceWarning warning
Issue #26530:
* Add C functions _PyTraceMalloc_Track() and _PyTraceMalloc_Untrack() to track
memory blocks using the tracemalloc module.
* Add _PyTraceMalloc_GetTraceback() to get the traceback of an object.
Issue #26588:
* The _tracemalloc now supports tracing memory allocations of multiple address
spaces (domains).
* Add domain parameter to tracemalloc_add_trace() and
tracemalloc_remove_trace().
* tracemalloc_add_trace() now starts by removing the previous trace, if any.
* _tracemalloc._get_traces() now returns a list of (domain, size,
traceback_frames): the domain is new.
* Add tracemalloc.DomainFilter
* tracemalloc.Filter: add an optional domain parameter to the constructor and a
domain attribute
* Sublte change: use Py_uintptr_t rather than void* in the traces key.
* Add tracemalloc_config.use_domain, currently hardcoded to 1
Issue #26590: support.check_warnings() stores warnins, but ResourceWarning now
comes with a reference to the socket object which indirectly keeps the socket
alive.
Issue #26567:
* Add a new function PyErr_ResourceWarning() function to pass the destroyed
object
* Add a source attribute to warnings.WarningMessage
* Add warnings._showwarnmsg() which uses tracemalloc to get the traceback where
source object was allocated.
Issue #26568: add new _showwarnmsg() and _formatwarnmsg() functions to the
warnings module.
The C function warn_explicit() now calls warnings._showwarnmsg() with a
warnings.WarningMessage as parameter, instead of calling warnings.showwarning()
with multiple parameters.
_showwarnmsg() calls warnings.showwarning() if warnings.showwarning() was
replaced. Same for _formatwarnmsg(): call warnings.formatwarning() if it was
replaced.
Issue #26563:
* Add _PyGILState_GetInterpreterStateUnsafe() function: the single
PyInterpreterState used by this process' GILState implementation.
* Enhance _Py_DumpTracebackThreads() to retrieve the interpreter state from
autoInterpreterState in last resort. The function now accepts NULL for interp
and current_tstate parameters.
* test_faulthandler: fix a ResourceWarning when test is interrupted by CTRL+c
Issue #26563: Debug hooks on Python memory allocators now raise a fatal error
if functions of the PyMem_Malloc() family are called without holding the GIL.
This reverts most of revision 3092cf163eb4. The change worked on x86
architectures, but did not work on ARM, probably due to extra ABI flags in
the ldconfig output.
Issue #26538: libregrtest: Fix setup_tests() to keep module.__path__ type
(_NamespacePath), don't convert to a list.
Add _NamespacePath.__setitem__() method to importlib._bootstrap_external.
* Add Lib/test/signalinterproctester.py
* Don't disable the garbage collector anymore
* Don't use os.fork() with a subprocess to not inherit existing signal handlers
or threads: start from a fresh process
* Don't use UNIX kill command to send a signal but Python os.kill()
* Use a timeout of 10 seconds to wait for the signal instead of 1 second
* Always use signal.pause(), instead of time.wait(1), to wait for a signal
* Use context manager on subprocess.Popen
* remove code to retry on EINTR: it's no more needed since the PEP 475
* remove unused function exit_subprocess()
* Cleanup the code
Issue #26516:
* Add PYTHONMALLOC environment variable to set the Python memory
allocators and/or install debug hooks.
* PyMem_SetupDebugHooks() can now also be used on Python compiled in release
mode.
* The PYTHONMALLOCSTATS environment variable can now also be used on Python
compiled in release mode. It now has no effect if set to an empty string.
* In debug mode, debug hooks are now also installed on Python memory allocators
when Python is configured without pymalloc.
Revert:
"Always test datetime.strftime("%4Y")
Issue #13305: Always test datetime.datetime.strftime("%4Y") for years < 1900."
In fact, strftime("%4Y") fails on most platforms.
Added an optional argument timespec to the datetime isoformat() method
to choose the precision of the time component.
Original patch by Alessandro Cucci.
Output of func(*args) stored in the next line:
item_help = join([func(*args) for func, args in self.items])
_Section.items only used by HelpFormatter._add_item() and it looks
like it doesn't have any side effects.
Patch by Yogesh Chaudhari.
Issue #26464: Fix str.translate() when string is ASCII and first replacements
removes character, but next replacement uses a non-ASCII character or a string
longer than 1 character. Regression introduced in Python 3.5.0.
Doc/build/html/library/idle.html, which is built from Doc/library/idle.rst.
Help.html is an rstrip()ed copy of idle.html that can be pushed.
The displayed content is unchanged because a) the version number is not
displayed, being inherent in the version displaying the file and b) the tt
tag was ignored by the custom parser in idlelib/help.py. The wrapped
<span class="pre"> tags remain as they were.
This will be merged in 3.6 (without conflict) because the 3.6 help.html is
a copy of the 3.5 file (which was a copy of the 3.4 file). The two files will
remain the same until either a) there is a content change that only applies
to one of them or b) 3.5 maintenance ends and the 3.6 file is itself rebased.
As of Xcode 7, SDKs for Apple platforms now include textual-format stub
libraries whose file names have a .tbd extension rather than the
standard OS X .dylib extension. The Apple compiler tool chain handles
these stub libraries transparently and the installed system shared libraries
are still .dylibs. However, the new stub libraries cause problems for
third-party programs that support building with Apple SDKs and make
build-time decisions based on the presence or paths of system-supplied
shared libraries in the SDK. In particular, building Python itself with
an SDK fails to find system-supplied libraries during setup.py's build of
standard library extension modules. The solution is to have
find_library_file() in Distutils search for .tbd files, along with
the existing types (.a, .so, and .dylib). Patch by Tim Smith.
* There are only two base-64 alphabets defined by the RFCs, not three
* Due to the internal translation, plus (+) and slash (/) are never discarded
* standard_ and urlsafe_b64decode() discard characters as well
Also update the doc strings to clarify data types, based on revision
92760d2edc9e, correct the exception raised by b16decode(), and correct the
parameter name for the base-85 functions.
importlib.util.LazyLoader.
The class was checking its argument as to whether its implementation
of create_module() came directly from importlib.abc.Loader. The
problem is that the classes coming from imoprtlib.machinery do not
directly inherit from the ABC as they come from _frozen_importlib.
Because the documentation has always said that create_module() was
ignored, the check has simply been removed.
* The Windows-specific binary notice was probably a Python 2 thing
* Make it more obvious gettarinfo() is based on stat(), and that non-ordinary
files may need special care
* The file name must be text; suggest dummy arcname as a workaround
* Indicate TarInfo may be used directly, not just via gettarinfo()
Microbenchmarks show 2-2.5x improvement. Built-in 'divmod' function
is now also ~10% faster.
-m timeit -s "x=22331" "x//2;x//-3;x//4;x//5;x//-6;x//7;x//8;x//-99;x//100;"
with patch: 0.321 without patch: 0.633
-m timeit -s "x=22331" "x%2;x%3;x%-4;x%5;x%6;x%-7;x%8;x%99;x%-100;"
with patch: 0.224 without patch: 0.66
Big thanks to Serhiy Storchaka, Mark Dickinson and Victor Stinner for
thorow code reviews and algorithms improvements.
The compile ignores constant statements and emit a SyntaxWarning warning.
Don't emit the warning for string statement because triple quoted string is a
common syntax for multiline comments.
Don't emit the warning on ellipis neither: 'def f(): ...' is a legit syntax for
abstract functions.
Changes:
* test_ast: ignore SyntaxWarning when compiling test statements. Modify
test_load_const() to use assignment expressions rather than constant
expression.
* test_code: add more kinds of constant statements, ignore SyntaxWarning when
testing that the compiler removes constant statements.
* test_grammar: ignore SyntaxWarning on the statement "1"
* Constant statements will be ignored and the compiler will emit a
SyntaxWarning.
* Replace constant statement (ex: "1") with an expression statement
(ex: "x=1").
* test_traceback: use context manager on the file.
Issue #26204.
Steven D'Aprano
Ethan Furman
Serhiy Storchaka
Senthil Kumaran
Martin Panter
Victor Stinner
Berker Peksag
Brett Cannon
Guido van Rossum
Guido van Rossum
Ned Deily
Alexander Belopolsky
Stefan Krah
Steve Dower
Donald Stufft
Antoine Pitrou
Robert Collins
Terry Jan Reedy
Raymond Hettinger
Benjamin Peterson
Yury Selivanov
Georg Brandl
Jason R. Coombs
Vinay Sajip
Charles-François Natali