Make ssl tests less strict and also accept TLSv1 as system default. The
changes unbreaks test_min_max_version on Fedora 29.
Signed-off-by: Christian Heimes <christian@python.org>
The subprocess module can now use the os.posix_spawn() function
in some cases for better performance. Currently, it is only used on macOS
and Linux (using glibc 2.24 or newer) if all these conditions are met:
* executable path contains a directory
* close_fds=False
* preexec_fn, pass_fds, cwd, stdin, stdout, stderr
and start_new_session parameters are not set
Co-authored-by: Joannah Nanjekye <nanjekyejoannah@gmail.com>
Fix a NULL pointer deref in ssl module. The cert parser did not handle CRL
distribution points with empty DP or URI correctly. A malicious or buggy
certificate can result into segfault.
Signed-off-by: Christian Heimes <christian@python.org>
https://bugs.python.org/issue35746
asyncio.ProactorEventLoop now catchs and logs send errors when the
self-pipe is full: BaseProactorEventLoop._write_to_self() now catchs
and logs OSError exceptions, as done by
BaseSelectorEventLoop._write_to_self().
IocpProactor.close() now uses time to decide when to log: wait 1
second before the first log, then log every second. Log also the
number of seconds since close() was called.
Previously, calling the strftime() method on a datetime object with a
trailing '%' in the format string would result in an exception. However,
this only occured when the datetime C module was being used; the python
implementation did not match this behavior. Datetime is now PEP-399
compliant, and will not throw an exception on a trailing '%'.
Format characters "%s" and "%V" in PyUnicode_FromFormat() and "%s" in PyBytes_FromFormat()
no longer read memory past the limit if precision is specified.
Use the fast call convention for math functions atan2(),
copysign(), hypot() and remainder() and inline unpacking
arguments. This sped up them by 1.3--2.5 times.
This change separates the signal handling trigger in the eval loop from the "pending calls" machinery. There is no semantic change and the difference in performance is insignificant.
The change makes both components less confusing. It also eliminates the risk of changes to the pending calls affecting signal handling. This is particularly relevant for some upcoming pending calls changes I have in the works.
Fix memory leaks in asyncio ProactorEventLoop on overlapped operation
failures.
Changes:
* Implement the tp_traverse slot in the _overlapped.Overlapped type
to help to break reference cycles and identify referrers in the
garbage collector.
* Always clear overlapped on failure: not only set type to
TYPE_NOT_STARTED, but release also resources.
As in title, expose C `raise` function as `raise_function` in `signal` module. Also drop existing `raise_signal` in `_testcapi` module and replace all usages with new function.
https://bugs.python.org/issue35568
bpo-32622, bpo-35682: Fix asyncio.ProactorEventLoop.sendfile(): don't
attempt to set the result of an internal future if it's already done.
Fix asyncio _ProactorBasePipeTransport._force_close(): don't set the
result of _empty_waiter if it's already done.
test_asyncio/test_sendfile.py now resets the event loop policy using
tearDownModule() as done in other tests, to prevent a warning when
running tests on Windows.
Fix an assertion error in format() in debug build for floating point
formatting with "n" format, zero padding and small width. Release build is
not impacted. Patch by Karthikeyan Singaravelan.
* bpo-35660: IDLE: Remove * import from window.py
* sys was being imported through the *, so also added an import sys.
* Update 2019-01-04-19-14-29.bpo-35660.hMxI7N.rst
Anyone who wants details can check the issue, where I added the point about the sys import bug.
* bpo-35588: Implement mod and divmod operations for Fraction type by spelling out the numerator/denominator calculation, instead of instantiating and normalising Fractions along the way. This speeds up '%' and divmod() by 2-3x.
* bpo-35588: Also reimplement Fraction.__floordiv__() using integer operations to make it ~4x faster.
* Improve code formatting.
Co-Authored-By: scoder <stefan_ml@behnel.de>
* bpo-35588: Fix return type of divmod(): the result of the integer division should be an integer.
* bpo-35588: Further specialise __mod__() and inline the original helper function _flat_divmod() since it's no longer reused.
* bpo-35588: Add some tests with large numerators and/or denominators.
* bpo-35588: Use builtin "divmod()" function for implementing __divmod__() in order to simplify the implementation, even though performance results are mixed.
* Rremove accidentally added empty line.
* bpo-35588: Try to provide more informative output on test failures.
* bpo-35588: Improve wording in News entry.
Co-Authored-By: scoder <stefan_ml@behnel.de>
* Remove stray space.
There is already a `Py_ssize_t i` defined at function scope that is used
for similar loops. By removing the local `int i` declaration that `i` is
used, which has the appropriate type.
Add Clang Memory Sanitizer build instrumentation to work around
false positives from the socket and time modules as well as skipping
a couple test_faulthandler tests.
Use crypt_r() when available instead of crypt() in the crypt module.
As a nice side effect: This also avoids a memory sanitizer flake as clang msan doesn't know about crypt's internal libc allocated buffer.
* bpo-32492: 2.5x speed up in namedtuple attribute access using C fast path
* Add News entry
* fixup! bpo-32492: 2.5x speed up in namedtuple attribute access using C fast path
* Check for tuple in the __get__ of the new descriptor and don't cache the descriptor itself
* Don't inherit from property. Implement GC methods to handle __doc__
* Add a test for the docstring substitution in descriptors
* Update NEWS entry to reflect time against 3.7 branch
* Simplify implementation with argument clinic, better error messages, only __new__
* Use positional-only parameters for the __new__
* Use PyTuple_GET_SIZE and PyTuple_GET_ITEM to tighter the implementation of tuplegetterdescr_get
* Implement __set__ to make tuplegetter a data descriptor
* Use Py_INCREF now that we inline PyTuple_GetItem
* Apply the valid_index() function, saving one test
* Move Py_None test out of the critical path.
Command line options for the xlc compiler behave differently from gcc and clang,
so skip this test case for now when xlc is the compiler.
Patch by aixtools (Michael Felt)
* Fix test_mktime on AIX by adding code to get mktime to behave the
same way as it does on other *nix systems
* Fix test_pthread_getcpuclickid in AIX by adjusting the test case
expectations when running on AIX in 32-bit mode
Patch by Michael Felt.
This reverts commit 7cf3d8e251.
Due to regressions found with using Tk 8.6.9.1, build the python.org macOS installers with Tcl/Tk 8.6.8 as used in previous releases.
https://bugs.python.org/issue35402
Depending on system config, a missing candidate compiler name may be
reported as the empty string rather than as None, so adjust the test
helper accordingly.
AIX allows a trailing slash on local file system paths, which isn't what we want
in http.server. Accordingly, check explicitly for this case in the server code,
rather than relying on the OS raising an exception.
Patch by Michael Felt.
This test case needs "signed short" bitfields, but the
IBM XLC compiler (on AIX) does not support this.
Skip the code and test when AIX and XLC are used.
Use __xlc__ as identifier to detect the XLC compiler.
* use platform.system() as runtime test, rather than sys.platform() build-time test
* IPv6 zone id support on AIX is limited to inet_pton6_zone(), so skip related
getaddrinfo() and getnameinfo() tests as not supported
"Include/token.h", "Lib/token.py" (containing now some data moved from
"Lib/tokenize.py") and new files "Parser/token.c" (containing the code
moved from "Parser/tokenizer.c") and "Doc/library/token-list.inc" (included
in "Doc/library/token.rst") are now generated from "Grammar/Tokens" by
"Tools/scripts/generate_token.py". The script overwrites files only if
needed and can be used on the read-only sources tree.
"Lib/symbol.py" is now generated by "Tools/scripts/generate_symbol_py.py"
instead of been executable itself.
Added new make targets "regen-token" and "regen-symbol" which are now
dependencies of "regen-all".
The documentation contains now strings for operators and punctuation tokens.
The Code Context menu label now toggles between Show/Hide Code Context.
The Zoom Height menu now toggles between Zoom/Restore Height.
Zoom Height has moved from the Window menu to the Options menu.
https://bugs.python.org/issue22703
When compiling 3rd party C extensions, the linker flags used by the
compiler for the interpreter and the stdlib modules, will get
leaked into distutils. In order to avoid that, the PY_CORE_LDFLAGS
and PY_LDFLAGS_NODIST are introduced to keep those flags separated.
Fix a race condition in check_interrupted_write() of test_io:
create directly the thread with SIGALRM signal blocked,
rather than blocking the signal later from the thread. Previously, it
was possible that the thread gets the signal before the signal is
blocked.
Rename test.bisect module to test.bisect_cmd to avoid conflict with
bisect module when running directly a test like
"./python Lib/test/test_xmlrpc.py".
Make platform.architecture() parsing of "file" command output more
reliable:
* Add the "-b" option to the "file" command to omit the filename;
* Force the usage of the C locale;
* Search also the "shared object" pattern.
Co-Authored-By: Serhiy Storchaka <storchaka@gmail.com>
TextTestRunner of unittest.runner now uses time.perf_counter() rather
than time.time() to measure the execution time of a test: time.time()
can go backwards, whereas time.perf_counter() is monotonic.
Similar change made in libregrtest, pprint and random.
Replace time.time() with time.monotonic() in tests to measure time
delta.
test_zipfile64: display progress every minute (60 secs) rather than
every 5 minutes (5*60 seconds).
platform.dist() is deprecated and slated for removal in Python 3.8. The
upload command itself should also not be used to upload to PyPI, but
while it continues to exist it should not use deprecated functions.
"make profile-opt" no longer replaces CFLAGS_NODIST with CFLAGS. It
now adds profile-guided optimization (PGO) flags to CFLAGS_NODIST,
existing CFLAGS_NODIST flags are kept.
bpo-34279, bpo-35412: support.run_unittest() no longer raises
TestDidNotRun if a test result contains skipped tests. The
exception is now only raised if no test have been run and no test
have been skipped.
* Fix partial and partial method signatures in mock
* Add more calls
* Add NEWS entry
* Use assertEquals and fix markup in NEWS
* Refactor branching and add markup reference for functools
* Revert partial object related changes and fix pr comments
The length check for AF_ALG salg_name and salg_type had a off-by-one
error. The code assumed that both values are not necessarily NULL
terminated. However the Kernel code for alg_bind() ensures that the last
byte of both strings are NULL terminated.
Signed-off-by: Christian Heimes <christian@python.org>
Fix xml.dom.minidom cloneNode() on a document with an entity: pass
the correct arguments to the user data handler of an entity (fix an
old copy/paste mistake).
Bug spotted and fix proposed by Charalampos Stratakis, initial
reproducer written by Petr Viktorin.
Co-Authored-By: Charalampos Stratakis <cstratak@redhat.com>
Co-Authored-By: Petr Viktorin <encukou@gmail.com>
* tests: Further validate `wraps` functionality in `unittest.mock.Mock`
Add more tests to validate how `wraps` interacts with other features of
mocks.
* Don't call the wrapped object if `side_effect` is set
When a object is wrapped using `Mock(wraps=...)`, if an user sets a
`side_effect` in one of their methods, return the value of `side_effect`
and don't call the original object.
* Refactor what to be called on `mock_call`
When a `Mock` is called, it should return looking up in the following
order: `side_effect`, `return_value`, `wraps`. If any of the first two
return `mock.DEFAULT`, lookup in the next option.
It makes no sense to check for `wraps` returning default, as it is
supposed to be the original implementation and there is nothing to
fallback to.
Replace os.popen() with subprocess.check_output() in the platform module:
* platform.uname() (its _syscmd_ver() helper function) now redirects
stderr to DEVNULL.
* Remove platform.DEV_NULL.
* _syscmd_uname() and _syscmd_file() no longer catch AttributeError.
The "except AttributeError:" was only needed in Python 2, when
os.popen() was not always available. In Python 3,
subprocess.check_output() is always available.
select() calls are retried on EINTR (per PEP 475). However, if a
timeout was provided and the deadline has passed after running the
signal handlers, rlist, wlist and xlist should be cleared since select(2)
left them unmodified.
* posixpath.expanduser() now returns the input path unchanged if
the HOME environment variable is not set and pwd.getpwuid() raises
KeyError (the current user identifier doesn't exist in the password
database).
* Add test_no_home_directory() to test_site.
platform.libc_ver() now uses os.confstr('CS_GNU_LIBC_VERSION') if
available and the *executable* parameter is not set. The default
value of the libc_ver() *executable* parameter becomes None.
Quick benchmark on Fedora 29:
python3 -m perf command ./python -S -c 'import platform; platform.libc_ver()'
94.9 ms +- 4.3 ms -> 33.2 ms +- 1.4 ms: 2.86x faster (-65%)
When using link time optimizations, the -flto flag is passed to
BASECFLAGS, which makes it propagate to distutils. Those flags
should be reserved for the interpreter and the stdlib extension
modules only, thus moving those flags to CFLAGS_NODIST.
Modify asyncio tests to utilize the certificates from the test directory
instead of its own set, as they are the same and with each update they had
to be updated as well.
Fix an undefined behaviour in the pthread implementation of
PyThread_start_new_thread(): add a function wrapper to always return
NULL.
Add pythread_callback struct and pythread_wrapper() to thread_pthread.h.
Remove platform.popen() function, it was deprecated since Python 3.3:
use os.popen() instead.
Rename also the "Removed" section to "API and Feature Removals"
of What's New in Python 3.8.
Fix memory leak in PyUnicode_EncodeLocale() and
PyUnicode_EncodeFSDefault() on error handling.
Changes:
* Fix unicode_encode_locale() error handling
* Fix test_codecs.LocaleCodecTest
Fix mktime() overflow error in test_email: run
test_localtime_daylight_true_dst_true() and
test_localtime_daylight_false_dst_true() with a specific timezone.
The lineno and col_offset attributes of AST nodes for list comprehensions,
generator expressions and tuples are now point to the opening parenthesis or
square brace. For tuples without parenthesis they point to the position
of the first item.
Regression introduced in e3ce695 and 25b804a, where the old parameter
update_tryorder to _synthesize was first ignored, then given the opposite
value in the attempt to fix bpo-31014.
Fix str.format(), float.__format__() and complex.__format__() methods
for non-ASCII decimal point when using the "n" formatter.
Changes:
* Rewrite _PyUnicode_InsertThousandsGrouping(): it now requires
a _PyUnicodeWriter object for the buffer and a Python str object
for digits.
* Rename FILL() macro to unicode_fill(), convert it to static inline function,
add "assert(0 <= start);" and rework its code.
There are some same consts in a module. This commit merges them into
single instance. It reduces number of objects in memory after loading modules.
https://bugs.python.org/issue34100
Importing ProcessPoolExecutor may hang or cause an error when the import
accesses urandom on a low resource platform
https://bugs.python.org/issue29877
The "-I" command line option (run Python in isolated mode) is now
also copied by the multiprocessing and distutils modules when
spawning child processes. Previously, only -E and -s options (enabled
by -I) were copied.
subprocess._args_from_interpreter_flags() now copies the -I flag.
Include/*.h should be the "portable Python API", whereas
Include/cpython/*.h should be the "CPython API": CPython
implementation details.
Changes:
* Create Include/cpython/ subdirectory
* "make install" now creates $prefix/include/cpython and copy
Include/cpython/* to $prefix/include/cpython
* Create Include/cpython/objimpl.h: move objimpl.h code
surrounded by "#ifndef Py_LIMITED_API" to cpython/objimpl.h.
* objimpl.h now includes cpython/objimpl.h
* Windows installer (MSI) now also install Include/ subdirectories:
Include/cpython/ and Include/internal/.
locale.localeconv() now sets temporarily the LC_CTYPE locale to the
LC_MONETARY locale if the two locales are different and monetary
strings are non-ASCII. This temporary change affects other threads.
Changes:
* locale.localeconv() can now set LC_CTYPE to LC_MONETARY to decode
monetary fields.
* Add LocaleInfo.grouping_buffer: copy localeconv() grouping string
since it can be replaced anytime if a different thread calls
localeconv().
* _Py_GetLocaleconvNumeric() now requires a "struct lconv *"
structure, so locale.localeconv() now longer calls localeconv()
twice. Moreover, the function now requires all arguments to be
non-NULL.
* Rename STATIC_LOCALE_INFO_INIT to LocaleInfo_STATIC_INIT.
* Move _Py_GetLocaleconvNumeric() definition from fileutils.h
to pycore_fileutils.h. pycore_fileutils.h now includes locale.h.
* The _locale module is now built with Py_BUILD_CORE defined.
coro->cr_origin wasn't initialized if compute_cr_origin() failed in
PyCoro_New(), which would cause a crash during the coroutine's
deallocation.
https://bugs.python.org/issue35269
Discovered using clang's MemorySanitizer when it ran python3's
test_fstring test_misformed_unicode_character_name.
An msan build will fail by simply executing: ./python -c 'u"\N"'
The warnings module now suggests to enable tracemalloc if the source
is specified, tracemalloc module is available, but tracemalloc is not
tracing memory allocations.
Adds configure flags for msan and ubsan builds to make it easier to enable.
These also encode the detail that address sanitizer and memory sanitizer
should disable pymalloc.
Define MEMORY_SANITIZER when appropriate at build time and adds workarounds
to existing code to mark things as initialized where the sanitizer is otherwise unable to
determine that. This lets our build succeed under the memory sanitizer. not all tests
pass without sanitizer failures yet but we're in pretty good shape after this.
The System Preferences Dock "prefer tabs always" setting disables some
IDLE features. Menus are a bit different than as described for Windows
and Linux.
Fix an off by one error in the peephole optimizer when checking for unreachable code beyond a return.
Do a bounds check within find_op so it can return before going past the end as a safety measure.
7db3c48833 (diff-a33329ae6ae0bb295d742f0caf93c137)
introduced this off by one error while fixing another one nearby.
This bug was shipped in all Python 3.6 and 3.7 releases.
The included unittest won't fail unless you do a clang msan build.
The call to `_untrack_reader` is performed too soon, causing the protocol
to forget about the reader before `connection_lost` can run and feed the
EOF to the reader. See bpo-35065.
Current support for hash-based bytecode files in `zipimport` is rather
sparse, which leads to test failures when the test suite is ran with
the ``SOURCE_DATE_EPOCH`` environment variable set.
This teaches zipimport to handle hash-based pycs properly.
Handle Unicode contents on localised Windows systems when activating a
venv. activate.bat currently breaks on German Windows systems, as chcp.com does
not return a plain number as on English systems, but (arbitrarily) appends a dot at the end
(for example "Aktive Codepage: 850." instead of "Active Codepage: 850"). The
dependency to chcp.com is removed and ctypes is used to get, set and restore the
console output code page. The code page for console input is not changed.
We can't use __VENV_PYTHON__ to find python.exe, since it's UTF-8. cmd.exe decodes
the script using the console output code page.
Without setting mtime, time.time() will be used as the timestamp which will
end up in the compressed data and each invocation of the compress() function
will vary over time.
The new option in the CLI of the profile module allow to profile
executable modules. This change follows the same implementation as the
one already present in `cProfile`.
As the argument is now present on both modules, move the tests to the
common test case to be run with profile as well.
Some methods in the os module can accept path-like objects. This is documented in the general documentation but not in the function docstrings. To keep both in sync, the docstrings need to be updated to reflect that path-like objects are also accepted.
This implements getstate and setstate for the cjkcodecs multibyte incremental encoders/decoders, primarily to fix issues with seek/tell.
The encoder getstate/setstate is slightly tricky as the "state" is pending bytes + MultibyteCodec_State but only an integer can be returned. The approach I've taken is to encode this data into a long, similar to how .tell() encodes a "cookie_type" as a long.
https://bugs.python.org/issue33578
Modules imported last are now cleared first at interpreter shutdown.
A newly imported module is moved to the end of sys.modules, behind
modules on which it depends.
The list() constructor isn't taking full advantage of known input
lengths or length hints. This commit makes the constructor
pre-size and not over-allocate when the input size is known (the
input collection implements __len__). One on the main advantages is
that this provides 12% difference in memory savings due to the difference
between overallocating and allocating exactly the input size.
For efficiency purposes and to avoid a performance regression for small
generators and collections, the size of the input object is calculated using
__len__ and not __length_hint__, as the later is considerably slower.
There is only one trivial change to idle.rst. Nearly all the changes to help.html are the elimination of chapter and section numbers on headers due to changes in the build system. help.py no longer requires header numbering.
Since `SourceFileLoader.set_data()` catches exceptions raised by `_write_atomic()` and logs an informative message consequently, always logging successful outcome in 'SourceLoader.get_code()' seems redundant.
https://bugs.python.org/issue35024
Prior to this revision, after the shutdown of a `BaseServer`,
the server accepted a last single request
if it was sent between the server socket polling
and the polling timeout.
This can be problematic for instance for a server restart
for which you do not want to interrupt the service,
by not closing the listening socket during the restart.
One request failed because of this behavior.
Note that only one request failed,
following requests were not accepted, as expected.
Visual Studio solution: Set InlineFunctionExpansion to
OnlyExplicitInline ("/Ob1" option) on all projects (in
pyproject.props) in Debug mode on Win32 and x64 platforms to expand
functions marked as inline.
This change should make Python compiled in Debug mode a little bit
faster on Windows. On Unix, GCC uses -Og optimization level for
./configure --with-pydebug.
inspect.isfunction() processes both inspect.isfunction(func) and
inspect.isfunction(partial(func, arg)) correctly but some other functions in the
inspect module (iscoroutinefunction, isgeneratorfunction and isasyncgenfunction)
lack this functionality. This commits adds a new check in the mentioned functions
in the inspect module so they can work correctly with arbitrarily nested partial
functions.
The MagicMock class supports many magic methods, but not __fspath__. To ease
testing with modules such as os.path, this function is now supported by default.
tracemalloc now tries to update the traceback when an object is
reused from a "free list" (optimization for faster object creation,
used by the builtin list type for example).
Changes:
* Add _PyTraceMalloc_NewReference() function which tries to update
the Python traceback of a Python object.
* _Py_NewReference() now calls _PyTraceMalloc_NewReference().
* Add an unit test.
.o generated by clang in LTO mode actually are LLVM bitcode files, which
leads to a few errors during configure/build step:
- add lto flags to the BASECFLAGS instead of CFLAGS, as CFLAGS are used
to build autoconf test case, and some are not compatible with clang LTO
(they assume binary in the .o, not bitcode)
- force llvm-ar instead of ar, as ar is not aware of .o files generated
by clang -flto
Raise ValueError OverflowError in case of a negative
_length_ in a ctypes.Array subclass. Also raise TypeError
instead of AttributeError for non-integer _length_.
Co-authored-by: Oren Milman <orenmn@gmail.com>
For builtin types with builtin subclasses, help() on the type now shows up
to 4 of the subclasses. This partially replaces the exception hierarchy
information previously displayed in Python 2.7.
path_error() uses GetLastError() on Windows, but some os functions
are implemented via CRT APIs which report errors via errno.
This may result in raising OSError with invalid error code (such
as zero).
Introduce posix_path_error() function and use it where appropriate.
If buffering=1 is specified for open() in binary mode, it is silently
treated as buffering=-1 (i.e., the default buffer size).
Coupled with the fact that line buffering is always supported in Python 2,
such behavior caused several issues (e.g., bpo-10344, bpo-21332).
Warn that line buffering is not supported if open() is called with
binary mode and buffering=1.
The reprlib code was copied here instead of importing reprlib. I'm not sure if we really need to avoid the import, but since I expect dataclasses to be more common that reprlib, it seems wise. Plus, the code is small.
Adding `max_num_fields` to `cgi.FieldStorage` to make DOS attacks harder by
limiting the number of `MiniFieldStorage` objects created by `FieldStorage`.
Restores the use of pyexpatns.h to isolate our embedded copy of the expat C
library so that its symbols do not conflict at link or dynamic loading time
with an embedding application or other extension modules with their own
version of libexpat.
5dc3f23b5f (diff-3afaf7274c90ce1b7405f75ad825f545) inadvertently removed it when upgrading expat.
python-gdb.py now handles errors on computing the line number
of a Python frame.
Changes:
* PyFrameObjectPtr.current_line_num() now catchs any Exception on
calling addr2line(), instead of failing with a surprising "<class
'TypeError'> 'FakeRepr' object is not subscriptable" error.
* All callers of current_line_num() now handle current_line_num()
returning None.
* PyFrameObjectPtr.current_line() now also catchs IndexError on
getting a line from the Python source file.
Allow annotated global names in the module namespace after the symbol is
declared as global. Previously, only symbols annotated before they are declared
as global (i.e. inside a function) were allowed. This change allows symbols to be
declared as global before the annotation happens in the global scope.
Unconditional forcing of ``CHECKED_HASH`` invalidation was introduced in
3.7.0 in bpo-29708. The change is bad, as it unconditionally overrides
*invalidation_mode*, even if it was passed as an explicit argument to
``py_compile.compile()`` or ``compileall``. An environment variable
should *never* override an explicit argument to a library function.
That change leads to multiple test failures if the ``SOURCE_DATE_EPOCH``
environment variable is set.
This changes ``py_compile.compile()`` to only look at
``SOURCE_DATE_EPOCH`` if no explicit *invalidation_mode* was specified.
I also made various relevant tests run with explicit control over the
value of ``SOURCE_DATE_EPOCH``.
While looking at this, I noticed that ``zipimport`` does not work
with hash-based .pycs _at all_, though I left the fixes for
subsequent commits.
When Python is built with the intel control-flow protection flags,
-mcet -fcf-protection, gdb is not able to read the stack without
actually jumping inside the function. This means an extra
'next' command is required to make the $pc (program counter)
enter the function and make the stack of the function exposed to gdb.
Co-Authored-By: Marcel Plch <gmarcel.plch@gmail.com>
(cherry picked from commit 9b7c74ca32)
Let .chm document display non-ASCII characters properly
Escape the `body` part of .chm source file to 7-bit ASCII, to fix visual effect on some MBCS Windows systems.
This is needed to even the run the test suite on buildbots for affected platforms; e.g.:
```
./python.exe ./Tools/scripts/run_tests.py -j 1 -u all -W --slowest --fail-env-changed --timeout=11700 -j2
/home/embray/src/python/test-worker/3.x.test-worker/build/python -u -W default -bb -E -W error::BytesWarning -m test -r -w -j 1 -u all -W --slowest --fail-env-changed --timeout=11700 -j2
Traceback (most recent call last):
File "./Tools/scripts/run_tests.py", line 56, in <module>
main(sys.argv[1:])
File "./Tools/scripts/run_tests.py", line 52, in main
os.execv(sys.executable, args)
PermissionError: [Errno 13] Permission denied
make: *** [Makefile:1073: buildbottest] Error 1
```
The C implementation of asyncio.Task currently fails to perform the
cancellation cleanup correctly in the following scenario.
async def task1():
async def task2():
await task3 # task3 is never cancelled
asyncio.current_task().cancel()
await asyncio.create_task(task2())
The actuall error is a hardcoded call to `future_cancel()` instead of
calling the `cancel()` method of a future-like object.
Thanks to Vladimir Matveev for noticing the code discrepancy and to
Yury Selivanov for coming up with a pathological scenario.
Fix a reference issue inside multiprocessing.Pool that caused the pool to remain alive if it was deleted without being closed or terminated explicitly.
property_descr_get() uses a "cached" tuple to optimize function
calls. But this tuple can be discovered in debug mode with
sys.getobjects(). Remove the optimization, it's not really worth it
and it causes 3 different crashes last years.
Microbenchmark:
./python -m perf timeit -v \
-s "from collections import namedtuple; P = namedtuple('P', 'x y'); p = P(1, 2)" \
--duplicate 1024 "p.x"
Result:
Mean +- std dev: [ref] 32.8 ns +- 0.8 ns -> [patch] 40.4 ns +- 1.3 ns: 1.23x slower (+23%)
* Compiling a string annotation containing a lambda with keyword-only
argument without default value caused a crash.
* Remove the final "*" (it is incorrect syntax) in the representation of
lambda without *args and keyword-only arguments when compile from AST.
* Improve the representation of lambda without arguments.
The waiting is pretty normal for any asyncio program, logging its time just adds
a noise to logs without any useful information provided.
https://bugs.python.org/issue34849
Report the filename to the exception when raising {gdbm,dbm.ndbm}.error in
dbm.gnu.open() and dbm.ndbm.open() functions, so it gets printed when the
exception is raised, and can also be obtained by the filename attribute of the
exception object.
Use a monotonic clock to compute timeouts in :meth:`Executor.map` and :func:`as_completed`, in order to prevent timeouts from deviating when the system clock is adjusted.
This may not be sufficient on all systems. On POSIX for example, the actual waiting (e.g. in ``sem_timedwait``) is specified to rely on the CLOCK_REALTIME clock.
When dict subclass overrides order (`__iter__()`, `keys()`, and `items()`), `dict(o)`
should use it instead of dict ordering.
https://bugs.python.org/issue34320
The xml.sax and xml.dom.domreg modules now obey
sys.flags.ignore_environment.
Signed-off-by: Christian Heimes <christian@python.org>
https://bugs.python.org/issue34791
* Insert the warn in the asyncio.sleep when the loop argument is used
* Insert the warn in the asyncio.wait and asyncio.wait_for when the loop argument is used
* Better format of the code
* Add news file
* change calls for get_event_loop() to calls for get_running_loop()
* Change message to be more clear in News
* Improve the comments in test_tasks
The SAX parser no longer processes general external entities by default
to increase security. Before, the parser created network connections
to fetch remote files or loaded local files from the file system for DTD
and entities.
Signed-off-by: Christian Heimes <christian@python.org>
https://bugs.python.org/issue17239
Add SSLContext.post_handshake_auth and
SSLSocket.verify_client_post_handshake for TLS 1.3 post-handshake
authentication.
Signed-off-by: Christian Heimes <christian@python.org>q
https://bugs.python.org/issue34670
Iterable unpacking is now allowed without parentheses in yield and return
statements, e.g. ``yield 1, 2, 3, *rest``. Thanks to David Cuthbert for the
change and jChapman for added tests.
We cannot simply call locale.getpreferredencoding() here,
as GDB might have been linked against a different version
of Python with a different encoding and coercion policy
with respect to PEP 538 and PEP 540.
Thanks to Victor Stinner for a hint on how to fix this.
PowerShell Core 6.1 is the cross-platform port of Windows PowerShell. This change updates Activate.ps1 to not make Windows assumptions as well as installing it into the bin/Scripts directory on all operating systems.
Requires PowerShell Core 6.1 for proper readline support once the shell has been activated for the virtual environment.
* Revert "bpo-34589: Add -X coerce_c_locale command line option (GH-9378)"
This reverts commit dbdee0073c.
* Revert "bpo-34589: C locale coercion off by default (GH-9073)"
This reverts commit 7a0791b699.
* Revert "bpo-34589: Make _PyCoreConfig.coerce_c_locale private (GH-9371)"
This reverts commit 188ebfa475.
`list.append([], None)` was profiled but `list.append([], None, **{})` was not profiled.
Enable profiling for later case.
https://bugs.python.org/issue34125
Currently configure.ac uses AC_RUN_IFELSE to determine the byte order of doubles, but this silently fails under cross compilation and Python doesn't do floats properly.
Instead, steal a macro from autoconf-archive which compiles code using magic doubles (which encode to ASCII) and grep for the representation in the binary.
RFC because this doesn't yet handle the weird ancient ARMv4 OABI 'mixed-endian' encoding properly. This encoding is ancient and I don't believe the union of "Python 3.8 users" and "OABI users" has anything in. Should the support for this just be dropped too? Alternatively, someone will need to find an OABI toolchain to verify the encoding of the magic double.
The C accelerated _elementtree module now initializes hash randomization
salt from _Py_HashSecret instead of libexpat's default CPRNG.
Signed-off-by: Christian Heimes <christian@python.org>
https://bugs.python.org/issue34623
Such functions as os.path.exists(), os.path.lexists(), os.path.isdir(),
os.path.isfile(), os.path.islink(), and os.path.ismount() now return False
instead of raising ValueError or its subclasses UnicodeEncodeError
and UnicodeDecodeError for paths that contain characters or bytes
unrepresentative at the OS level.
Py_Initialize() and Py_Main() cannot enable the C locale coercion
(PEP 538) anymore: it is always disabled. It can now only be enabled
by the Python program ("python3).
test_embed: get_filesystem_encoding() doesn't have to set PYTHONUTF8
nor PYTHONCOERCECLOCALE, these variables are already set in the
parent.
The test tries to fill the receiver's socket buffer and expects an
error. But the RDS protocol doesn't require that. Moreover, the Linux
implementation of RDS expects that the producer of the messages
reduces its rate, it's not the role of the receiver to trigger an
error.
The test fails on Fedora 28 by design, so remove it.
* Modify DEFAULT_CONFIG for AIX
* bedevere/news did not like old name
* Modify NEWS entry
* Modified per peer review
* Define and use NULL_STR constant to account for AIX libc behavior
* Modify per peer review
* Modify NEWS
When os.fork() is called (on platforms that support it) all threads but the current one are destroyed in the child process. Consequently we must ensure that all but the associated interpreter are likewise destroyed. The main interpreter is critical for runtime operation, so we must ensure that fork only happens in the main interpreter.
https://bugs.python.org/issue34651
bpo-6721: When os.fork() was called while another thread holds a logging lock, the child process may deadlock when it tries to log. This fixes that by acquiring all logging locks before fork and releasing them afterwards.
A regression test that fails before this change is included.
Within the new unittest itself: There is a small _potential_ due to mixing of fork and a thread in the child process if the parent's thread happened to hold a non-reentrant library call lock (malloc?) when the os.fork() happens. buildbots and time will tell if this actually manifests itself in this test or not. :/ A functionality test that avoids that would be a challenge.
An alternate test that isn't trying to produce the deadlock itself but just checking that the release and acquire calls are made would be the next best alternative if so.
[bpo-34658](https://www.bugs.python.org/issue34658): Fix a rare interpreter unhandled exception state SystemError only
seen when using subprocess with a preexec_fn while an after_parent handler has
been registered with os.register_at_fork and the fork system call fails.
https://bugs.python.org/issue34658
This causes the tearDown code to only unimport the test modules specifically created as part of each test via the self.mkhier method rather than abusing test.support.modules_setup() and the scary test.support.modules_cleanup() code.
https://bugs.python.org/issue34200
Store a weak reference to stream readerfor breaking strong references
It breaks the strong reference loop between reader and protocol and allows to detect and close the socket if the stream is deleted (garbage collected)
Address a C undefined behavior signed integer overflow issue in set object table resizing. Our -fwrapv compiler flag and practical reasons why sets are unlikely to get this large should mean this was never an issue but it was incorrect code that generates code analysis warnings.
<!-- issue-number: [bpo-1621](https://www.bugs.python.org/issue1621) -->
https://bugs.python.org/issue1621
<!-- /issue-number -->
When subprocess.Popen() stdin= stdout= or stderr= handles are specified
and appear in pass_fds=, don't close the original fds after dup'ing them.
This implementation and unittest primarily came from @izbyshev (see the PR)
See also b89b52f284
This also removes the old manual p2cread, c2pwrite, and errwrite closing logic
as inheritable flags and _close_open_fds takes care of that properly today without special treatment.
This code is within child_exec() where it is the only thread so there is no
race condition between the dup and _Py_set_inheritable_async_safe call.
Christian Heimes
Terry Jan Reedy
Serhiy Storchaka
Timo Furrer
Dong-hee Na
Joannah Nanjekye
Victor Stinner
MichaelSaah
Tal Einat
Anthony Sottile
Eric Snow
Michael Felt
Bo Bayles
Sanyam Khurana
Steve Dower
Vladimir Matveev
Gregory Szorc
Raymond Hettinger
Xtreak
Cheryl Sabella
Anthony Shaw
Emmanuel Arias
Mickaël Schoentgen
Stefan Behnel
sth
Gregory P. Smith
Jakub Kulík
Pablo Galindo
jab
Ned Deily
Andrew Svetlov
leodema
Chris Rands
Jean-François B
Xtreak
Arthur Neufeld
stratakis
Manjusaka
Zackery Spytz
Dima Tisnek
Paul Ganssle
Beomsoo Kim
Xiang Zhang
jdemeyer
Mario Corchero
Oran Avraham
William Grzybowski
Sergey Fedoseev
Andrew Dunai
Chris Withers
Ismo Toijala
Siddhesh Poyarekar
INADA Naoki
Zhiming Wang
Lisa Roach
Dustin Spicuzza
nierob
Zhiming Wang
Brendan Gerrity
Donald Stufft
Stefano Rivera
Julien Palard
Eddie Elizondo
Giampaolo Rodola
Srinivas Thatiparthy (శ్రీనివాస్ తాటిపర్తి)
Vincent Michel
Elvis Pranskevichus
samstagern
Diego Rojas
HongWeipeng
guoci
Alexander Buchkovsky
Rémi Lapeyre
Max Bélanger
Stéphane Wirtel
Lysandros Nikolaou
BNMetrics
Christopher Thorne
Petter Strandmark
Sebastian Rittau
Quentin Agren
Andrés Delfino
Denis Ledoux
Tim Graham
madman-bob
serge-sans-paille
Alexey Izbyshev
Matthias Bussonnier
David Herberth
Srinivas Thatiparthy (శ్రీనివాస్ తాటిపర్తి)
matthewbelisle-wf
Juliette Monsel
Bruno Oliveira
Mayank Asthana
twisteroid ambassador
Romuald Brunet
travisoneill
animalize
Bradley Meck
Paul Bailey
E. M. Bray
tzickel
Zsolt Cserna
orlnub123
Bjorn Andersson
Yury Selivanov
Joe Pamer
Ammar Askar
João Júnior
Tim Hoffmann
jChapman
Ethan Furman
David Cuthbert
Brett Cannon
Berker Peksag
Ross Burton
Silas Sewell
Alexandru Ardelean
Monson Shao
Benjamin Peterson
Eric V. Smith
Windson yang
Tony Flury
Oren Milman
wim glenn
Lew Kurtz
Zachary Ware
Rahul Jha