On Windows, when the Python test suite is run with the -jN option,
the ANSI code page is now used as the encoding for the stdout
temporary file, rather than using UTF-8 which can lead to decoding
errors.
Linux abstract sockets are insecure as they lack any form of filesystem
permissions so their use allows anyone on the system to inject code into
the process.
This removes the default preference for abstract sockets in
multiprocessing introduced in Python 3.9+ via
https://github.com/python/cpython/pull/18866 while fixing
https://github.com/python/cpython/issues/84031.
Explicit use of an abstract socket by a user now generates a
RuntimeWarning. If we choose to keep this warning, it should be
backported to the 3.7 and 3.8 branches.
* The compiler analyzes the usage of the first 64 local variables all at once using bit masks.
* Local variables beyond the first 64 are only partially analyzed, achieving linear time.
Added os.setns and os.unshare to easily switch between namespaces
on Linux.
Co-authored-by: Christian Heimes <christian@python.org>
Co-authored-by: CAM Gerlach <CAM.Gerlach@Gerlach.CAM>
Co-authored-by: Victor Stinner <vstinner@python.org>
Make sys.setprofile() and sys.settrace() functions reentrant. They
can no long fail with: RuntimeError("Cannot install a trace function
while another trace function is being installed").
Make _PyEval_SetTrace() and _PyEval_SetProfile() functions reentrant,
rather than detecting and rejecting reentrant calls. Only delete the
reference to function arguments once the new function is fully set,
when a reentrant call is safe. Call also _PySys_Audit() earlier.
The os module and the PyUnicode_FSDecoder() function no longer accept
bytes-like paths, like bytearray and memoryview types: only the exact
bytes type is accepted for bytes strings.
A small example of what a full date and time would look like would help a lot of developers who may not realize that they should investigate `time.h`'s `strftime`, run `man strftime`, or click through a series of docs on the python docs before they get to the actual [definition here](https://docs.python.org/3/library/datetime.html#strftime-and-strptime-format-codes) which still doesn't have an obvious copy-pastable example of "what the heck format does this thing actually expect?".
Automerge-Triggered-By: GH:rhettinger
Alas, warnings.catch_warnings() has global scope, not thread scope, so this is still not perfect, but it reduces the time during which warnings are ignored. Better solution welcome.
Change summary:
+ There is now a `gzip.READ_BUFFER_SIZE` constant that is 128KB. Other programs that read in 128KB chunks: pigz and cat. So this seems best practice among good programs. Also it is faster than 8 kb chunks.
+ a zlib._ZlibDecompressor was added. This is the _bz2.BZ2Decompressor ported to zlib. Since the zlib.Decompress object is better for in-memory decompression, the _ZlibDecompressor is hidden. It only makes sense in file decompression, and that is already implemented now in the gzip library. No need to bother the users with this.
+ The ZlibDecompressor uses the older Cpython arrange_output_buffer functions, as those are faster and more appropriate for the use case.
+ GzipFile.read has been optimized. There is no longer a `unconsumed_tail` member to write back to padded file. This is instead handled by the ZlibDecompressor itself, which has an internal buffer. `_add_read_data` has been inlined, as it was just two calls.
EDIT: While I am adding improvements anyway, I figured I could add another one-liner optimization now to the python -m gzip application. That read chunks in io.DEFAULT_BUFFER_SIZE previously, but has been updated now to use READ_BUFFER_SIZE chunks.
#97530 fixed IDLE tests possibly crashing on a Mac without a GUI.
But it resulted in IDLE not starting in 3.10.8, 3.12.0a1, and
Microsoft Python 3.10.2288.0 when test/* is not installed.
After this patch, test.* is only imported when testing on Mac.
This is the next step for deprecating child watchers.
Until we've removed the API completely we have to use it, so this PR is mostly suppressing a lot of warnings when using the API internally.
Once the child watcher API is totally removed, the two child watcher implementations we actually use and need (Pidfd and Thread) will be turned into internal helpers.
test_tools.test_sundry() now uses an unittest mock to prevent the
logging module to register a real "atfork" function which kept the
logging module dictionary alive. So the logging module can be
properly unloaded. Previously, the logging module was loaded before
test_sundry(), but it's no longer the case since recent test_tools
sub-tests removals.
Remove outdated example scripts of the Tools/scripts/ directory:
* gprof2html.py
* md5sum.py
* nm2def.py
* pathfix.py
* win_add2path.py
Remove test_gprof2html, test_md5sum and test_pathfix of test_tools.
There is no reason for this watcher to be attached to any particular loop.
This should make it safe to use regardless of the lifetime of the event loop running in the main thread
(relative to other loops).
Co-authored-by: Yury Selivanov <yury@edgedb.com>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
The test failed on a buildbot because the pointer was only 7 hex characters. To be safe,
I bumped it down to 3: 4 in case we have 32-bit platforms, and 3 in case the pointer is very small.
This PR reverts gh-93369 and gh-97896 because they've made asyncio tests unstable. After these PRs were merged, random GitHub action jobs of random commits started to fail unrelated tests and test framework methods.
The reverting is necessary because such shrapnel failures are a symptom of some underlying bug that must be found and fixed first.
I had a hope that it's a server overload because we already have extremely rare disc access errors. However, one and a half day passed, and the failures continue to emerge both in PRs and commits.
Affected issue: gh-93357.
First reported in https://github.com/python/cpython/pull/97940#issuecomment-1270004134.
* Revert "gh-93357: Port test cases to IsolatedAsyncioTestCase, part 2 (#97896)"
This reverts commit 09aea94d29.
* Revert "gh-93357: Start porting asyncio server test cases to IsolatedAsyncioTestCase (#93369)"
This reverts commit ce8fc186ac.
In `_warnings.c`, in the C equivalent of `warnings.warn_explicit()`, if the module globals are given (and not None), the warning will attempt to get the source line for the issued warning. To do this, it needs the module's loader.
Previously, it would only look up `__loader__` in the module globals. In https://github.com/python/cpython/issues/86298 we want to defer to the `__spec__.loader` if available.
The first step on this journey is to check that `loader == __spec__.loader` and issue another warning if it is not. This commit does that.
Since this is a PoC, only manual testing for now.
```python
# /tmp/foo.py
import warnings
import bar
warnings.warn_explicit(
'warning!',
RuntimeWarning,
'bar.py', 2,
module='bar knee',
module_globals=bar.__dict__,
)
```
```python
# /tmp/bar.py
import sys
import os
import pathlib
# __loader__ = pathlib.Path()
```
Then running this: `./python.exe -Wdefault /tmp/foo.py`
Produces:
```
bar.py:2: RuntimeWarning: warning!
import os
```
Uncomment the `__loader__ = ` line in `bar.py` and try it again:
```
sys:1: ImportWarning: Module bar; __loader__ != __spec__.loader (<_frozen_importlib_external.SourceFileLoader object at 0x109f7dfa0> != PosixPath('.'))
bar.py:2: RuntimeWarning: warning!
import os
```
Automerge-Triggered-By: GH:warsaw
This is a small performance improvement, especially for one or two hot
places such as _handle_fromlist() that are called a lot and the
.format() method was being used just to join two strings with a dot.
Otherwise it is merely a readability improvement.
We keep `_ERR_MSG` and `_ERR_MSG_PREFIX` as those may be used elsewhere for canonical looking error messages.
The macOS 13 SDK includes support for the `mkfifoat` and `mknodat` system calls.
Using the `dir_fd` option with either `os.mkfifo` or `os.mknod` could result in a
segfault if cpython is built with the macOS 13 SDK but run on an earlier
version of macOS. Prevent this by adding runtime support for detection of
these system calls ("weaklinking") as is done for other newer syscalls on
macOS.
* improve performance of get_proxies_environment when there are many environment variables
* 📜🤖 Added by blurb_it.
* fix case of short env name
* fix formatting
* fix whitespace
* whitespace
* Update Lib/urllib/request.py
Co-authored-by: Carl Meyer <carl@oddbird.net>
* Update Lib/urllib/request.py
Co-authored-by: Carl Meyer <carl@oddbird.net>
* Update Lib/urllib/request.py
Co-authored-by: Carl Meyer <carl@oddbird.net>
* Update Lib/urllib/request.py
Co-authored-by: Carl Meyer <carl@oddbird.net>
* whitespace
* Update Misc/NEWS.d/next/Library/2022-04-15-11-29-38.gh-issue-91539.7WgVuA.rst
Co-authored-by: Carl Meyer <carl@oddbird.net>
* Update Lib/urllib/request.py
Co-authored-by: Carl Meyer <carl@oddbird.net>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Carl Meyer <carl@oddbird.net>
Relevant tests moved from test_exceptions to test_traceback to be able to
compare both implementations.
Co-authored-by: Carl Friedrich Bolz-Tereick <cfbolz@gmx.de>
Remove the sys.getdxp() function and the Tools/scripts/analyze_dxp.py
script. DXP stands for "dynamic execution pairs". They were related
to DYNAMIC_EXECUTION_PROFILE and DXPAIRS macros which have been
removed in Python 3.11. Python can now be built with "./configure
--enable-pystats" to gather statistics on Python opcodes.
dataclass used to get the annotations on a class object using
cls.__dict__.get('__annotations__'). Now that it always imports
inspect, it can use inspect.get_annotations, which is modern
best practice for coping with annotations.
It had to live as a global outside of PyConfig for stable ABI reasons in
the pre-3.12 backports.
This removes the `_Py_global_config_int_max_str_digits` and gets rid of
the equivalent field in the internal `struct _is PyInterpreterState` as
code can just use the existing nested config struct within that.
Adds tests to verify unique settings and configs in subinterpreters.
Ensure that the event loop's `_thread_id` attribute and the asyncgen hooks set by `sys.set_asyncgen_hooks()` are always restored no matter where a KeyboardInterrupt exception is raised.
Fix multiplying a list by an integer (list *= int): detect the
integer overflow when the new allocated length is close to the
maximum size. Issue reported by Jordan Limor.
list_resize() now checks for integer overflow before multiplying the
new allocated length by the list item size (sizeof(PyObject*)).
Previously, checkbuttons in different parent widgets could have the same
short name and share the same state if arguments "name" and "variable" are
not specified. Now they are globally unique.
Fix command line parsing: reject "-X int_max_str_digits" option with
no value (invalid) when the PYTHONINTMAXSTRDIGITS environment
variable is set to a valid limit.
Only call tkinter.tk and its follow-up code in _init_tk_type when requires('gui')
does not raise. This function can be called as an unintended side-effect of
calling other idlelib code as part of tests on macOS without a GUI enabled.
Stanley
Raymond Hettinger
Victor Stinner
Carl Meyer
Gregory P. Smith
Dennis Sweeney
Ben Kallus
Irit Katriel
Steve Dower
Serhiy Storchaka
Noam Cohen
Dong-hee Na
Nikita Sobolev
chgnrdv
Jack Hindmarch
Paul Moore
Alex Zvorygin
fancidev
Kumar Aditya
Ruben Vorderman
Jason R. Coombs
Terry Jan Reedy
Vinay Sajip
Pablo Galindo Salgado
Jakub Kuczys
Jeong YunWon
Joannah Nanjekye
Tony Solomonik
Christoph Hamsen
Michael Droettboom
Thomas Grainger
Jelle Zijlstra
Shantanu
Frazer McLean
Guido van Rossum
Anh71me
andrei kulakov
Oleg Iarygin
Barry Warsaw
Tiger
Brandt Bucher
Brett Cannon
Ned Deily
Ethan Furman
Pieter Eendebak
Mark Shannon
Ev2geny
Łukasz Langa
Koki Saito
Benjamin Peterson
larryhastings
Boris Verkhovskiy
MonadChains
hetmankp
Ofey Chan
Cheryl Sabella
Shin-myoung-serp
Jakob Weigert
Cyker Way
Michael Curran