* bpo-37399: Correctly attach tail text to the last element/comment/pi, even when comments or pis are discarded.
Also fixes the insertion of PIs when "insert_pis=True" is configured for a TreeBuilder.
Verify that it appears to find roughly the right number of tests in the stdlib's testsuite.
(cherry picked from commit e95ac20103)
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Reduce the number of unit tests run for the PGO generation task. This
speeds up the task by a factor of about 15x. Running the full unit test
suite is slow. This change may result in a slightly less optimized build
since not as many code branches will be executed. If you are willing to
wait for the much slower build, the old behavior can be restored using
'./configure [..] PROFILE_TASK="-m test --pgo-extended"'. We make no
guarantees as to which PGO task set produces a faster build. Users who
care should run their own relevant benchmarks as results can depend on
the environment, workload, and compiler tool chain.
(cherry picked from commit 4e16a4a311)
Co-authored-by: Neil Schemenauer <nas-github@arctrix.com>
A bug in MSVC UCRT version 17763.615 (which has been fixed in newer versions) is causing test failures in some strptime/strftime tests when the default code page is c65001. This change selectively skips the tests affected by this.
(cherry picked from commit 9cd39b16e2)
Co-authored-by: Paul Monson <paulmon@users.noreply.github.com>
This exception was caused because the input ended unexpectedly with only one
single quote instead of a pair with some value inside it.
(cherry picked from commit 719a062bcb)
Co-authored-by: Abhilash Raj <maxking@users.noreply.github.com>
* bpo-37461: Fix infinite loop in parsing of specially crafted email headers.
Some crafted email header would cause the get_parameter method to run in an
infinite loop causing a DoS attack surface when parsing those headers. This
patch fixes that by making sure the DQUOTE character is handled to prevent
going into an infinite loop.
(cherry picked from commit a4a994bd3e)
Co-authored-by: Abhilash Raj <maxking@users.noreply.github.com>
As far as I can tell, this infinite loop would be triggered if:
1. The value being folded contains a single word (no spaces) longer than
max_line_length
2. The max_line_length is shorter than the encoding's name + 9
characters.
bpo-36564: https://bugs.python.org/issue36564
(cherry picked from commit f69d5c6198)
Co-authored-by: Paul Ganssle <pganssle@users.noreply.github.com>
The `allow_abbrev` option for ArgumentParser is documented and intended to disable support for unique prefixes of --options, which may sometimes be ambiguous due to deferred parsing.
However, the initial implementation also broke parsing of grouped short flags, such as `-ab` meaning `-a -b` (or `-a=b`). Checking the argument for a leading `--` before rejecting it fixes this.
This was prompted by pytest-dev/pytestGH-5469, so a backport to at least 3.8 would be great 😄
And this is my first PR to CPython, so please let me know if I've missed anything!
https://bugs.python.org/issue26967
(cherry picked from commit dffca9e925)
Co-authored-by: Zac Hatfield-Dodds <Zac-HD@users.noreply.github.com>
Returns NotImplemented for timedelta and time in __eq__ for different types in Python implementation, which matches the C implementation.
This also adds tests to enforce that these objects will fall back to the right hand side's __eq__ and/or __ne__ implementation.
bpo-37579
(cherry picked from commit e6b46aafad)
Co-authored-by: Xtreak <tir.karthi@gmail.com>
With the addition of shared memory into Python 3.8, we now have three tests failing on Solaris, namely `test_multiprocessing_fork`, `test_multiprocessing_forkserver` and `test_multiprocessing_spawn`. The reason seems to be incorrect name handling which results in two slashes being prepended.
https://bugs.python.org/issue37558
(cherry picked from commit 4737265622)
Co-authored-by: Jakub Kulík <Kulikjak@gmail.com>
test_ssl.test_pha_required_nocert() now uses
support.catch_threading_exception() to ignore the expected SSLError
in ConnectionHandler of ThreadedEchoServer (it is only raised
sometimes on Windows).
(cherry picked from commit 73ea54620a)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
Nested BinOp instances (e.g. a+b+c) had a wrong col_offset for the
second BinOp (e.g. 2 instead of 0 in the example). Fix it by using the
correct st node to copy the line and col_offset from in ast.c.
(cherry picked from commit 110a47c4f4)
Co-authored-by: Carl Friedrich Bolz-Tereick <cfbolz@gmx.de>
test_concurrent_futures now explicitly stops the ForkServer instance
if it's running.
(cherry picked from commit e676244235)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
* bpo-37520: Correct behavior for zipfile.Path.parent
* 📜🤖 Added by blurb_it.
(cherry picked from commit 38f44b4a4a)
Co-authored-by: Jason R. Coombs <jaraco@jaraco.com>
multiprocessing tests now stop the ForkServer instance if it's
running: close the "alive" file descriptor to ask the server to stop
and then remove its UNIX address.
(cherry picked from commit 8fbeb14312)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
test_concurrent_futures now cleans up multiprocessing to remove
immediately temporary directories created by
multiprocessing.util.get_temp_dir().
The test now uses setUpModule() and tearDownModule().
(cherry picked from commit 684cb47fff)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
test_winconsoleio doesn't leak a temporary file anymore: use
tempfile.TemporaryFile() to remove it when the test completes.
(cherry picked from commit b71d8d6795)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
ssl.match_hostname() no longer accepts IPv4 addresses with additional text
after the address and only quad-dotted notation without trailing
whitespaces. Some inet_aton() implementations ignore whitespace and all data
after whitespace, e.g. '127.0.0.1 whatever'.
Short notations like '127.1' for '127.0.0.1' were already filtered out.
The bug was initially found by Dominik Czarnota and reported by Paul Kehrer.
Signed-off-by: Christian Heimes <christian@python.org>
https://bugs.python.org/issue37463
(cherry picked from commit 477b1b2576)
Co-authored-by: Christian Heimes <christian@python.org>
multiprocessing tests now call explicitly _run_finalizers() to remove
immediately temporary directories created by
multiprocessing.util.get_temp_dir().
(cherry picked from commit 039fb49c18)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
Python initialization now ensures that sys stream encoding
names are always normalized by codecs.lookup(encoding).name.
Simplify test_c_locale_coercion: it doesn't have to normalize
encoding names anymore.
(cherry picked from commit 61bf97e916)
Co-authored-by: Jakub Kulík <Kulikjak@gmail.com>
Under some conditions the earlier fix for bpo-18075, "Infinite recursion
tests triggering a segfault on Mac OS X", now causes failures on macOS
when attempting to change stack limit with resource.setrlimit
resource.RLIMIT_STACK, like regrtest does when running the test suite.
The reverted change had specified a non-default stack size when linking
the python executable on macOS. As of macOS 10.14.4, the previous
code causes a hard failure when running tests, although similar
failures had been seen under some conditions under some earlier
systems. Reverting the change to the interpreter stack size at link
time helped for release builds but caused some tests to fail when
built --with-pydebug. Try the opposite approach: continue to build
the interpreter with an increased stack size on macOS and remove
the failing setrlimit call in regrtest initialization. This will
definitely avoid the resource.RLIMIT_STACK error and should have
no, or fewer, side effects.
(cherry picked from commit 5bbbc733e6)
Co-authored-by: Ned Deily <nad@python.org>
Stop using "static PyConfig", PyConfig must now always use
dynamically allocated strings: use PyConfig_SetString(),
PyConfig_SetArgv() and PyConfig_Clear().
(cherry picked from commit 67310023f2)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
Fix sys.excepthook() and PyErr_Display() if a filename is a bytes
string. For example, for a SyntaxError exception where the filename
attribute is a bytes string.
Cleanup also test_sys:
* Sort imports.
* Rename numruns global var to INTERN_NUMRUNS.
* Add DisplayHookTest and ExceptHookTest test case classes.
* Don't save/restore sys.stdout and sys.displayhook using
setUp()/tearDown(): do it in each test method.
* Test error case (call hook with no argument) after the success case.
(cherry picked from commit f9b7457bd7)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
SSLContext.post_handshake_auth = True no longer sets
SSL_VERIFY_POST_HANDSHAKE verify flag for client connections. Although the
option is documented as ignored for clients, OpenSSL implicitly enables cert
chain validation when the flag is set.
Signed-off-by: Christian Heimes <christian@python.org>
https://bugs.python.org/issue37428
(cherry picked from commit f0f5930ac8)
sys._base_executable is now always defined on all platforms, and can be overridden through configuration.
Also adds test.support.PythonSymlink to encapsulate platform-specific logic for symlinking sys.executable
* Fix typo in supports_file2file_sendfile(); ensure that dst is
removed
* Fix test_copytree_custom_copy_function(): remove dst tree.
Use support.rmtree() rather than shutil.rmtree() to remove
temporary directories: support tries harder.
(cherry picked from commit 4c26abd14f)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
* patched string index out of range error in get_word function of _header_value_parser.py and created tests in test__header_value_parser.py for CFWS.
* Raise HeaderParseError instead of continuing when parsing a word.
(cherry picked from commit 7213df7bbf)
Co-authored-by: Abhilash Raj <maxking@users.noreply.github.com>
Fix test_wsgiref.testEnviron() to no longer depend on the environment
variables (don't fail if "X" variable is set).
testEnviron() now overrides os.environ to get a deterministic
environment. Test full TestHandler.environ content: not only a few
selected variables.
(cherry picked from commit 5150d32792)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
The os.getcwdb() function now uses the UTF-8 encoding on Windows,
rather than the ANSI code page: see PEP 529 for the rationale. The
function is no longer deprecated on Windows.
os.getcwd() and os.getcwdb() now detect integer overflow on memory
allocations. On Unix, these functions properly report MemoryError on
memory allocation failure.
(cherry picked from commit 689830ee62)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
Increase robustness of test_resource_tracker(): retry for 60 seconds.
(cherry picked from commit e1a63c4f21)
Co-authored-by: Pierre Glaser <pierreglaser@msn.com>
When certain malformed messages have content-type set to 'mulitpart/*' but
still have a single part body, iter_attachments can raise AttributeError. This
patch fixes it by returning a None value instead when the body is single part.
(cherry picked from commit 02257012f6)
Co-authored-by: Abhilash Raj <maxking@users.noreply.github.com>
https://bugs.python.org/issue33972
Use os.getgroups() rather than grp.getgrall() to get groups.
Rename also the test to test_chown_gid().
(cherry picked from commit d7c87d982d)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
* The UTF-8 incremental decoders fails now fast if encounter
a sequence that can't be handled by the error handler.
* The UTF-16 incremental decoders with the surrogatepass error
handler decodes now a lone low surrogate with final=False.
(cherry picked from commit 894263ba80)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
eval() was being called an extra time without a filter for
deprecation warnings.
(cherry picked from commit 9fe42b49c7)
Co-authored-by: Zackery Spytz <zspytz@gmail.com>
* regrtest: Add --cleanup option to remove "test_python_*" directories
of previous failed test jobs.
* Add "make cleantest" to run "python3 -m test --cleanup".
(cherry picked from commit 47fbc4e45b)
… as proposed in PEP 572; key is now evaluated before value.
https://bugs.python.org/issue35224
(cherry picked from commit c8a35417db)
Co-authored-by: Jörn Heissler <joernheissler@users.noreply.github.com>
When the test is ran with `PYTHONWARNINGS=error` the environment variable is passed to the python interpreter used in `assert_python_ok` where `DeprecationWarning` from `@asyncio.coroutine` is converted into an error. Ignore the `DeprecationWarning` in `assert_python_ok`.
https://bugs.python.org/issue37323
(cherry picked from commit 186f70905d)
Co-authored-by: Xtreak <tir.karthi@gmail.com>
test_gdb no longer fails if it gets an "unexpected" message on
stderr: it now ignores stderr. The purpose of test_gdb is to test
that python-gdb.py commands work as expected, not to test gdb.
(cherry picked from commit e56a123fd0)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
For datetime.datetime.strptime(), the leading zero for some two-digit formats is optional.
This adds a footnote to the strftime/strptime documentation to reflect this fact, and adds some tests to ensure that it is true.
bpo-34903
(cherry picked from commit 6b9c204ee7)
Co-authored-by: Mike Gleen <mike.gleen@gmail.com>
test_pre_initialization_sys_options() of test_embed now removes
PYTHON* environment variables like PYTHONWARNINGS.
(cherry picked from commit dbdc991a62)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
bpo-35031, bpo-35998: Reintroduce workaround on Windows and FreeBSD
in test_start_tls_server_1() of test_asyncio: disable TLS v1.3 on the
client context.
(cherry picked from commit c034b7824f)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
On Windows, os.dup() no longer creates an inheritable fd when handling a
character file.
(cherry picked from commit 28fca0c422)
Co-authored-by: Zackery Spytz <zspytz@gmail.com>
uuid could try fallback methods that had no chance of working on a particular
platform, and this could cause spurious test failures, as well as degraded
performance as fallback options were tried and failed.
This fixes both the uuid module and its test's SkipUnless logic to use a
prefiltered list of techniques that may at least potentially work on that platform.
Patch by Michael Felt (aixtools).
(cherry picked from commit 3a1d50e7e5)
Co-authored-by: Michael Felt <aixtools@users.noreply.github.com>
bpo-35537, bpo-35876: Fix also test_start_new_session() of
test_subprocess: use os.getsid() rather than os.getpgid().
(cherry picked from commit 5884043252)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
Document reference cycle and resurrected objects issues in
sys.unraisablehook() and threading.excepthook() documentation.
Fix test.support.catch_unraisable_exception(): __exit__() no longer
ignores unraisable exceptions.
Fix test_io test_writer_close_error_on_close(): use a second
catch_unraisable_exception() to catch the BufferedWriter unraisable
exception.
(cherry picked from commit 212646cae6)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
Join the thread to prevent leaking a running thread and leaking a
reference.
Cleanup also the test:
* asyncioWindowsProactorEventLoopPolicy became the default policy,
there is no need to set it manually.
* Only start the thread once the loop is running.
* Use a shorter sleep in the thread (100 ms rather than 1 sec).
* Use close_loop(loop) rather than loop.close().
* Use longer variable names.
(cherry picked from commit 07559450b2)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
Fix a regression introduced by af8646c805 that was causing code of the form:
if True and False:
do_something()
to be optimized incorrectly, eliminating the block.
(cherry picked from commit 05f8318655)
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
The peephole optimizer was not optimizing correctly bytecode after negative deltas were introduced. This is due to the fact that some special values (255) were being searched for in both instruction pointer delta and line number deltas.
(cherry picked from commit 3498c642f4)
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
The __exit__() method of test.support.catch_unraisable_exception
context manager now ignores unraisable exception raised when clearing
self.unraisable attribute.
(cherry picked from commit 6d22cc8e90)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
Allow pure Python implementation of pickle to work
even when the C _pickle module is unavailable.
Fix test_pickle when _pickle is missing: declare PyPicklerHookTests
outside "if has_c_implementation:" block.
(cherry picked from commit 63ab4ba07b)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
* bpo-36402: Fix threading._shutdown() race condition (GH-13948)
Fix a race condition at Python shutdown when waiting for threads.
Wait until the Python thread state of all non-daemon threads get
deleted (join all non-daemon threads), rather than just wait until
Python threads complete.
* Add threading._shutdown_locks: set of Thread._tstate_lock locks
of non-daemon threads used by _shutdown() to wait until all Python
thread states get deleted. See Thread._set_tstate_lock().
* Add also threading._shutdown_locks_lock to protect access to
threading._shutdown_locks.
* Add test_finalization_shutdown() test.
(cherry picked from commit 468e5fec8a)
* bpo-36402: Fix threading.Thread._stop() (GH-14047)
Remove the _tstate_lock from _shutdown_locks, don't remove None.
(cherry picked from commit 6f75c87375)
* bpo-18748: Fix _pyio.IOBase destructor (closed case) (GH-13952)
_pyio.IOBase destructor now does nothing if getting the closed
attribute fails to better mimick _io.IOBase finalizer.
(cherry picked from commit 4f6f7c5a61)
* bpo-37223: test_io: silence destructor errors (GH-13954)
Implement also MockNonBlockWriterIO.seek() method.
(cherry picked from commit b589cef9c4)
* bpo-37223, test_io: silence last 'Exception ignored in:' (GH-14029)
Use catch_unraisable_exception() to ignore 'Exception ignored in:'
error when the internal BufferedWriter of the BufferedRWPair is
destroyed. The C implementation doesn't give access to the
internal BufferedWriter, so just ignore the warning instead.
(cherry picked from commit 913fa1c824)
Rename compile() feature_version parameter to _feature_version and
convert it to a keyword-only parameter.
Update also test_type_comments to pass feature_version as a tuple.
(cherry picked from commit efdf6ca90f)
because "getaddrinfo()" behaves different on AIX
https://bugs.python.org/issue35545
(cherry picked from commit 32dda263e4)
Co-authored-by: Michael Felt <aixtools@users.noreply.github.com>
Mock the HTTPConnection.close() method in a few unit tests to avoid
logging "Exception ignored in: ..." messages.
(cherry picked from commit eb976e47e2)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
* [bpo-36520](https://bugs.python.org/issue36520): reset the encoded word offset when starting a new
line during an email header folding operation
* 📜🤖 Added by blurb_it.
* [bpo-36520](https://bugs.python.org/issue36520): add an additional test case, and provide descriptive
comments for the test_folding_with_utf8_encoding_* tests
* [bpo-36520](https://bugs.python.org/issue36520): fix whitespace issue
* [bpo-36520](https://bugs.python.org/issue36520): changes per reviewer request -- remove extraneous
backslashes; add whitespace between terminating quotes and
line-continuation backslashes; use "bpo-" instead of
"issue GH-" in comments
(cherry picked from commit f6713e84af)
Co-authored-by: websurfer5 <49998481+websurfer5@users.noreply.github.com>
https://bugs.python.org/issue36520
If preadv2 is not available, preadv will raise NotImplementedError.
(cherry picked from commit 44867bb937)
Co-authored-by: Benjamin Peterson <benjamin@python.org>
This test "works" if things are run in the right order, so it's better to use @skip than @expectedFailure here.
(cherry picked from commit 910b3fcb01)
Co-authored-by: Benjamin Peterson <benjamin@python.org>
Currently, inspect.getfile(str) will report nonsense:
```pytb
>>> inspect.getfile(str)
TypeError: <module 'builtins' (built-in)> is a built-in class
```
This fixes that
https://bugs.python.org/issue37173
(cherry picked from commit d407d2a726)
Co-authored-by: Philipp A <flying-sheep@web.de>
There is a possibility that someone (like me) accidentally will omit parentheses with `FileType` arguments after `FileType`, and parser will contain wrong file until someone will try to use it.
Example:
```python
parser = argparse.ArgumentParser()
parser.add_argument('-x', type=argparse.FileType)
```
https://bugs.python.org/issue37150
(cherry picked from commit 03d5831a2d)
Co-authored-by: zygocephalus <grrrr@protonmail.com>
Replace two Python function calls with a single one to ensure that no
memory allocation is done between the invalid object is created and
when _PyObject_IsFreed() is called.
(cherry picked from commit 3bf0f3ad20)
* bpo-30835: email: Fix AttributeError when parsing invalid Content-Transfer-Encoding
Parsing an email containing a multipart Content-Type, along with a
Content-Transfer-Encoding containing an invalid (non-ASCII-decodable) byte
will fail. email.feedparser.FeedParser._parsegen() gets the header and
attempts to convert it to lowercase before comparing it with the accepted
encodings, but as the header contains an invalid byte, it's returned as a
Header object rather than a str.
Cast the Content-Transfer-Encoding header to a str to avoid this.
Found using the AFL fuzzer.
Reported-by: Daniel Axtens <dja@axtens.net>
Signed-off-by: Andrew Donnellan <andrew@donnellan.id.au>
* Add email and NEWS entry for the bugfix.
* bpo-35805: Add parser for Message-ID header.
This parser is based on the definition of Identification Fields from RFC 5322
Sec 3.6.4.
This should also prevent folding of Message-ID header using RFC 2047 encoded
words and hence fix bpo-35805.
* Prevent folding of non-ascii message-id headers.
* Add fold method to MsgID token to prevent folding.
Replace asyncio.set_event_loop() with TestCase.set_event_loop() of
test_asyncio.utils: this method calls TestCase.close_loop() which
waits until the executor completes, to avoid leaking dangling
threads.
Inherit from test_asyncio.utils.TestCase rather than
unittest.TestCase.
Modify test_coroutines, test_cprofile, test_generators, test_raise,
test_ssl and test_yield_from to use
support.catch_unraisable_exception() rather than
support.captured_stderr().
test_thread: remove test_save_exception_state_on_error() which is now
updated. test_unraisable_exception() checks that sys.unraisablehook()
is called to handle _thread.start_new_thread() exception.
test_cprofile now rely on unittest for test discovery: replace
support.run_unittest() with unittest.main().
When inheriting a heap subclass from a vectorcall class that sets
`.tp_call=PyVectorcall_Call` (as recommended in PEP 590), the subclass does
not inherit `_Py_TPFLAGS_HAVE_VECTORCALL`, and thus `PyVectorcall_Call` does
not work for it.
This attempts to solve the issue by:
* always inheriting `tp_vectorcall_offset` unless `tp_call` is overridden
in the subclass
* inheriting _Py_TPFLAGS_HAVE_VECTORCALL for static types, unless `tp_call`
is overridden
* making `PyVectorcall_Call` ignore `_Py_TPFLAGS_HAVE_VECTORCALL`
This means it'll be ever more important to only call `PyVectorcall_Call`
on classes that support vectorcall. In `PyVectorcall_Call`'s intended role
as `tp_call` filler, that's not a problem.
Fixed QueueListener in order to avoid random deadlocks.
Unable to add regression tests atm due to time constraints, will add it in a bit.
Regarding implementation, although it's nested, it does not cause performance issues whatsoever, and does not call task_done() in case of an exception (which is the right thing to do IMHO).
https://bugs.python.org/issue36813
Adds a new option in trace that allows tracing runnable modules. It is
exposed as `--module module_name` as `-m` is already in use for another
argument.
The ssl module now can dump key material to a keylog file and trace TLS
protocol messages with a tracing callback. The default and stdlib
contexts also support SSLKEYLOGFILE env var.
The msg_callback and related enums are private members. The feature
is designed for internal debugging and not for end users.
Signed-off-by: Christian Heimes <christian@python.org>
This is an old feature request that appears from time to time. After a year of experimenting with various introspection capabilities in `typing_inspect` on PyPI, I propose to add these two most commonly used functions: `get_origin()` and `get_args()`. These are essentially thin public wrappers around private APIs: `__origin__` and `__args__`.
As discussed in the issue and on the typing tracker, exposing some public helpers instead of `__origin__` and `__args__` directly will give us more flexibility if we will decide to update the internal representation, while still maintaining backwards compatibility.
The implementation is very simple an is essentially a copy from `typing_inspect` with one exception: `ClassVar` was special-cased in `typing_inspect`, but I think this special-casing doesn't really help and only makes things more complicated.
From 3.8 async functions used with mock.patch return an `AsyncMock`. `_accept_connection2` is an async function where create_task is also mocked. Don't mock `create_task` so that tasks are created out of coroutine returned by `AsyncMock` and the tasks are completed.
https://bugs.python.org/issue37015
As it changes the way functions are called, the PEP 590 implementation
skipped the functions that the GDB integration is looking for
(by name) to find function calls.
Looking for the new helper `cfunction_call_varargs` hopefully fixes the
tests, and thus buildbots.
The changed frame nuber in test_gdb is due to there being fewer
C calls when calling a built-in method.
* bpo-26836: Add os.memfd_create()
* Use the glibc wrapper for memfd_create()
Co-Authored-By: Christian Heimes <christian@python.org>
* Fix deletions caused by autoreconf.
* Use MFD_CLOEXEC as the default value for *flags*.
* Add memset_s to configure.ac.
* Revert memset_s changes.
* Apply the requested changes.
* Tweak the docs.
* bpo-22385: Support output separators in hex methods.
Also in binascii.hexlify aka b2a_hex.
The underlying implementation behind all hex generation in CPython uses the
same pystrhex.c implementation. This adds support to bytes, bytearray,
and memoryview objects.
The binascii module functions exist rather than being slated for deprecation
because they return bytes rather than requiring an intermediate step through a
str object.
This change was inspired by MicroPython which supports sep in its binascii
implementation (and does not yet support the .hex methods).
https://bugs.python.org/issue22385
Add explicit `asyncSetUp` and `asyncTearDown` methods.
The rest is the same as for #13228
`AsyncTestCase` create a loop instance for every test for the sake of test isolation.
Sometimes a loop shared between all tests can speed up tests execution time a lot but it requires control of closed resources after every test finish. Basically, it requires nested supervisors support that was discussed with @1st1 many times. Sorry, asyncio supervisors have no chance to land on Python 3.8.
The PR intentionally does not provide API for changing the used event loop or getting the test loop: use `asyncio.set_event_loop_policy()` and `asyncio.get_event_loop()` instead.
The PR adds four overridable methods to base `unittest.TestCase` class:
```
def _callSetUp(self):
self.setUp()
def _callTestMethod(self, method):
method()
def _callTearDown(self):
self.tearDown()
def _callCleanup(self, function, /, *args, **kwargs):
function(*args, **kwargs)
```
It allows using asyncio facilities with minimal influence on the unittest code.
The last but not least: the PR respects contextvars. The context variable installed by `asyncSetUp` is available on test, `tearDown` and a coroutine scheduled by `addCleanup`.
https://bugs.python.org/issue32972
_thread.start_new_thread() now logs uncaught exception raised by the
function using sys.unraisablehook(), rather than sys.excepthook(), so
the hook gets access to the function which raised the exception.
I tried to get rid of the `_ProtocolMeta`, but unfortunately it didn'y work. My idea to return a generic alias from `@runtime_checkable` made runtime protocols unpickleable. I am not sure what is worse (a custom metaclass or having some classes unpickleable), so I decided to stick with the status quo (since there were no complains so far). So essentially this is a copy of the implementation in `typing_extensions` with two modifications:
* Rename `@runtime` to `@runtime_checkable` (plus corresponding updates).
* Allow protocols that extend `collections.abc.Iterable` etc.
It has been documented as deprecated and to be removed in 3.8;
From a comment on another thread – which I can't find ; leave get_coro_wrapper() for now, but always return `None`.
https://bugs.python.org/issue36933
Add a new threading.excepthook() function which handles uncaught
Thread.run() exception. It can be overridden to control how uncaught
exceptions are handled.
threading.ExceptHookArgs is not documented on purpose: it should not
be used directly.
* threading.excepthook() and threading.ExceptHookArgs.
* Add _PyErr_Display(): similar to PyErr_Display(), but accept a
'file' parameter.
* Add _thread._excepthook(): C implementation of the exception hook
calling _PyErr_Display().
* Add _thread._ExceptHookArgs: structseq type.
* Add threading._invoke_excepthook_wrapper() which handles the gory
details to ensure that everything remains alive during Python
shutdown.
* Add unit tests.
When using the "=" debug functionality of f-strings, use another Constant node (or a merged constant node) instead of adding expr_text to the FormattedValue node.
This will address the common mistake many asyncio users make:
an "except Exception" clause breaking Tasks cancellation.
In addition to this change, we stop inheriting asyncio.TimeoutError
and asyncio.InvalidStateError from their concurrent.futures.*
counterparts. There's no point for these exceptions to share the
inheritance chain.
In 3.9 we'll focus on implementing supervisors and cancel scopes,
which should allow better handling of all exceptions, including
SystemExit and KeyboardInterrupt
* sys.unraisablehook: add 'err_msg' field to UnraisableHookArgs.
* Use _PyErr_WriteUnraisableMsg() in _ctypes _DictRemover_call()
and gc delete_garbage().
The implementation is straightforward, it just mimics `ClassVar` (since the latter is also a name/access qualifier, not really a type). Also it is essentially copied from `typing_extensions`.
Previously, it was hard to tell whether a function should be awaited. It was also incorrect (per PEP 484) to put this in the type hint for coroutine functions. Added this info to the output of builtins.help and pydoc.
https://bugs.python.org/issue36045
* _PyPreConfig_InitCompatConfig() sets utf8_mode to 0.
* Change Py_UTF8Mode default value to 0.
* Fix _PyPreConfig_Copy(): copy also _config_init attrbibute.
* _PyPreConfig_AsDict() exports _config_init
* Fix _PyPreConfig_GetGlobalConfig(): use Py_UTF8Mode if it's greater
than 0, even if utf8_mode >= 0.
* Add unit tests on environment variables using Python API.
In development (-X dev) mode and in a debug build, IOBase finalizer
of the _pyio module now logs the exception if the close() method
fails. The exception is ignored silently by default in release build.
test_io: test_error_through_destructor() now uses
support.catch_unraisable_exception() rather than capturing stderr.
PyErr_WriteUnraisable() now creates a traceback object if there is no
current traceback. Moreover, call PyErr_NormalizeException() and
PyException_SetTraceback() to normalize the exception value. Ignore
silently any error.
* _PyPreConfig_GetGlobalConfig() and _PyCoreConfig_GetGlobalConfig()
now do nothing if the configuration was not initialized with
_PyPreConfig_InitCompatConfig() and _PyCoreConfig_InitCompatConfig()
* Remove utf8_mode=-2 special case: use utf8_mode=-1 instead.
* Fix _PyPreConfig_InitPythonConfig():
* isolated = 0 instead of -1
* use_environment = 1 instead of -1
* Rename _PyConfig_INIT to _PyConfig_INIT_COMPAT
* Rename _PyPreConfig_Init() to _PyPreConfig_InitCompatConfig()
* Rename _PyCoreConfig_Init() to _PyCoreConfig_InitCompatConfig()
* PyInterpreterState_New() now uses _PyCoreConfig_InitPythonConfig()
as default configuration, but it's very quickly overriden anyway.
* _freeze_importlib.c uses _PyCoreConfig_SetString() to set
program_name.
* Cleanup preconfig_init_utf8_mode(): cmdline is always non-NULL.
* Copy test_exceptions.test_unraisable() to
test_sys.UnraisableHookTest().
* Use catch_unraisable_exception() in test_coroutines,
test_exceptions, test_generators.
* Fixes issue 24882
* Add news file entry for change.
* Change test_concurrent_futures.ThreadPoolShutdownTest
Adjust the shutdown test so that, after submitting three jobs
to the executor, the test checks for less than three threads,
instead of looking for exactly three threads.
If idle threads are being recycled properly, then we should have
less than three threads.
* Switched idle count to semaphor, Updated tests
As suggested by reviewer tomMoral, swapped lock-protected counter
with a semaphore to track the number of unused threads.
Adjusted test_threads_terminate to wait for completiton of the
previous future before submitting a new one (and checking the
number of threads used).
Also added a new test to confirm the thread pool can be saturated.
* Updates tests as requested by pitrou.
* Correct minor whitespace error.
* Make test_saturation faster
Wrap the callback call within the `add_done_callback` function within concurrent.futures, in order to behave in an identical manner to callbacks added to a running future are triggered once it has completed.
This disallows things like `# type: ignoreé`, which seems wrong.
Also switch to using Py_ISALNUM for the alnum check, for consistency
with other code (and maybe correctness re: locale issues?).
https://bugs.python.org/issue36878
CVE-2019-9948: Avoid file reading as disallowing the unnecessary URL
scheme in URLopener().open() and URLopener().retrieve()
of urllib.request.
Co-Authored-By: SH <push0ebp@gmail.com>
GH-13238 made extra text after a # type: ignore accepted by the parser.
This finishes the job and actually plumbs the extra text through the
parser and makes it available in the AST.
In order to support typing checks calling hex(), oct() and bin() on user-defined classes, a SupportIndex protocol is required. The ability to check these at runtime would be good to add for completeness sake. This is pretty much just a copy of SupportsInt with the names tweaked.
It adds a missing testcase for bpo-34125. This is testing code which is
affected by PEP 590, so missing this test might accidentally break
CPython if we screw up with implementing PEP 590.
Add new sys.unraisablehook() function which can be overridden to
control how "unraisable exceptions" are handled. It is called when an
exception has occurred but there is no way for Python to handle it.
For example, when a destructor raises an exception or during garbage
collection (gc.collect()).
Changes:
* Add an internal UnraisableHookArgs type used to pass arguments to
sys.unraisablehook.
* Add _PyErr_WriteUnraisableDefaultHook().
* The default hook now ignores exception on writing the traceback.
* test_sys now uses unittest.main() to automatically discover tests:
remove test_main().
* Add _PyErr_Init().
* Fix PyErr_WriteUnraisable(): hold a strong reference to sys.stderr
while using it
* bpo-36929: Modify io/re tests to allow for missing mod name
For a vanishingly small number of internal types, CPython sets the
tp_name slot to mod_name.type_name, either in the PyTypeObject or the
PyType_Spec. There are a few minor places where this surfaces:
* Custom repr functions for those types (some of which ignore the
tp_name in favor of using a string literal, such as _io.TextIOWrapper)
* Pickling error messages
The test suite only tests the former. This commit modifies the test
suite to allow Python implementations to omit the module prefix.
https://bugs.python.org/issue36929
* Fix encoding and locale tests in test_embed.InitConfigTests.
* InitConfigTests now only computes EXPECTED_CONFIG once.
* Add tests for PYTHONWARNINGS and PYTHONPATH env vars
* Add _PyPreConfig.parse_argv
* Add _PyCoreConfig._config_init field and _PyCoreConfigInitEnum enum
type
* Initialization functions: reject preconfig=NULL and config=NULL
* Add config parameter to _PyCoreConfig_DecodeLocaleErr(): pass
config->argv to _Py_PreInitializeFromPyArgv(), to parse config
command line arguments in preinitialization.
* Add config parameter to _PyCoreConfig_SetString(). It now
preinitializes Python.
* _PyCoreConfig_SetPyArgv() now also preinitializes Python for wide
argv
* Fix _Py_PreInitializeFromCoreConfig(): don't pass args to
_Py_PreInitializeFromPyArgv() if config.parse_argv=0.
* Use "char * const *" and "wchar_t * const *" types for 'argv'
parameters and _PyArgv.argv.
* Add unit test on preinitialization from argv.
* _PyPreConfig.allocator type becomes int
* Add _PyPreConfig_InitFromPreConfig() and
_PyPreConfig_InitFromCoreConfig() helper functions
* Add math.isqrt function computing the integer square root.
* Code cleanup: remove redundant comments, rename some variables.
* Tighten up code a bit more; use Py_XDECREF to simplify error handling.
* Update Modules/mathmodule.c
Co-Authored-By: Serhiy Storchaka <storchaka@gmail.com>
* Update Modules/mathmodule.c
Use real argument clinic type instead of an alias
Co-Authored-By: Serhiy Storchaka <storchaka@gmail.com>
* Add proof sketch
* Updates from review.
* Correct and expand documentation.
* Fix bad reference handling on error; make some variables block-local; other tidying.
* Style and consistency fixes.
* Add missing error check; don't try to DECREF a NULL a
* Simplify some error returns.
* Another two test cases:
- clarify that floats are rejected even if they happen to be
squares of small integers
- TypeError beats ValueError for a negative float
* Add fast path for small inputs. Needs tests.
* Speed up isqrt for n >= 2**64 as well; add extra tests.
* Reduce number of test-cases to avoid dominating the run-time of test_math.
* Don't perform unnecessary extra iterations when computing c_bit_length.
* Abstract common uint64_t code out into a separate function.
* Cleanup.
* Add a missing Py_DECREF in an error branch. More cleanup.
* Update Modules/mathmodule.c
Add missing `static` declaration to helper function.
Co-Authored-By: Serhiy Storchaka <storchaka@gmail.com>
* Add missing backtick.