When sys.stdout.encoding is None compile_file will fall back to
sys.getdefaultencoding to encode/decode error messages.
Co-authored-by: Stefan Hoelzl <stefan.hoelzl@posteo.de>
Co-authored-by: Mickaël Schoentgen <contact@tiger-222.fr>
* rlcompleter was calling these methods to identify whether to add
parenthesis to the completion, based on if the attribute is callable.
* for property objects, completion with parenthesis are never desirable.
* property methods with print statements behaved very strangely, which
was especially unfriendly to language newcomers. <tab> could suddenly
produce output unexpectedly.
`pathlib.PureWindowsPath.is_reserved()` now identifies as reserved
filenames with trailing spaces or colons.
Co-authored-by: Barney Gale <barney.gale@foundry.com>
Co-authored-by: Eryk Sun <eryksun@gmail.com>
* bpo-44461: Fix bug with pdb's handling of import error due to a package which does not have a __main__ module
* 📜🤖 Added by blurb_it.
* remove "else"
Co-authored-by: Jason R. Coombs <jaraco@jaraco.com>
* If running as a module, first check that it can run as a module. Alternate fix for bpo-44461.
Co-authored-by: Irit Katriel <iritkatriel@yahoo.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>
* Convert "specials" array to InterpreterFrame struct, adding f_lasti, f_state and other non-debug FrameObject fields to it.
* Refactor, calls pushing the call to the interpreter upward toward _PyEval_Vector.
* Compute f_back when on thread stack, only filling in value when frame object outlives stack invocation.
* Move ownership of InterpreterFrame in generator from frame object to generator object.
* Do not create frame objects for Python calls.
* Do not create frame objects for generators.
Replace sentence with confusing "pseudo-module" with two sentences
separating future statements and the __future__ module.
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
* Remove code that checks Py_TPFLAGS_HAVE_VERSION_TAG
The field is always present in the type struct, as explained
in the added comment.
* Remove Py_TPFLAGS_HAVE_AM_SEND
The flag is not needed, and since it was added in 3.10 it can be removed now.
* Move to a static argparse.Namespace subclass
* Roughly annotate runtest.py
* Refactor libregrtest to use lossless test result objects
* Only re-run test methods that match names of previously failing test methods
* Adopt tests to cover test method name matching
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* Renamed assertLeadingPadding function to match logic
* Added a separate error message for discontinuous padding
* Updated the tests for discontinuous padding
- Refactor module/script handling to share an interface (check method).
- Import functools and adjust tests for the new line number for find_function.
- Use cached_property for details.
- Add blurb.
Automerge-Triggered-By: GH:jaraco
binascii.a2b_base64 gains a strict_mode= parameter. When enabled it will raise an
error on input that deviates from the base64 spec in any way. The default remains
False for backward compatibility.
Code reviews and minor tweaks by: Gregory P. Smith <greg@krypto.org> [Google]
This fixes TypedDict to work with get_type_hints and postponed evaluation of annotations across modules.
This is done by adding the module name to ForwardRef at the time the object is created and using that to resolve the globals during the evaluation.
Co-authored-by: Ken Jin <28750310+Fidget-Spinner@users.noreply.github.com>
It no longer depends on the order of arguments.
hash(int | str) == hash(str | int)
Co-authored-by: Jack DeVries <58614260+jdevries3133@users.noreply.github.com>
Patch by Erik Welch.
bpo-19072 (#8405) allows `classmethod` to wrap other descriptors, but this does
not work when the wrapped descriptor mimics classmethod. The current PR fixes
this.
In Python 3.8 and before, one could create a callable descriptor such that this
works as expected (see Lib/test/test_decorators.py for examples):
```python
class A:
@myclassmethod
def f1(cls):
return cls
@classmethod
@myclassmethod
def f2(cls):
return cls
```
In Python 3.8 and before, `A.f2()` return `A`. Currently in Python 3.9, it
returns `type(A)`. This PR make `A.f2()` return `A` again.
As of #8405, classmethod calls `obj.__get__(type)` if `obj` has `__get__`.
This allows one to chain `@classmethod` and `@property` together. When
using classmethod-like descriptors, it's the second argument to `__get__`--the
owner or the type--that is important, but this argument is currently missing.
Since it is None, the "owner" argument is assumed to be the type of the first
argument, which, in this case, is wrong (we want `A`, not `type(A)`).
This PR updates classmethod to call `obj.__get__(type, type)` if `obj` has
`__get__`.
Co-authored-by: Erik Welch <erik.n.welch@gmail.com>
* Fix issubclass() for None.
E.g. issubclass(type(None), int | None) returns now True.
* Fix issubclass() for virtual subclasses.
E.g. issubclass(dict, int | collections.abc.Mapping) returns now True.
* Fix crash in isinstance() if the check for one of items raises exception.
Heap types with the Py_TPFLAGS_IMMUTABLETYPE flag can now inherit the
PEP 590 vectorcall protocol. Previously, this was only possible for static types.
Co-authored-by: Victor Stinner <vstinner@python.org>
2021-07-08 12:48:01 +02:00
Srinivas Reddy Thatiparthy (శ్రీనివాస్ రెడ్డి తాటిపర్తి)
* zlib uses an UINT32_MAX sliding window for the output buffer
These funtions have an initial output buffer size parameter:
- zlib.decompress(data, /, wbits=MAX_WBITS, bufsize=DEF_BUF_SIZE)
- zlib.Decompress.flush([length])
If the initial size > UINT32_MAX, use an UINT32_MAX sliding window, instead of clamping to UINT32_MAX.
Speed up when (the initial size == the actual size).
This fixes a memory consumption and copying performance regression in earlier 3.10 beta releases if someone used an output buffer larger than 4GiB with zlib.decompress.
Reviewed-by: Gregory P. Smith
Remove the @asyncio.coroutine decorator
enabling legacy generator-based coroutines to be compatible with async/await
code; remove asyncio.coroutines.CoroWrapper used for wrapping
legacy coroutine objects in the debug mode.
The decorator has been deprecated
since Python 3.8 and the removal was initially scheduled for Python 3.10.
A TypeError is now raised instead of an AttributeError in
ExitStack.enter_context() and AsyncExitStack.enter_async_context()
for objects which do not support the context manager or
asynchronous context manager protocols correspondingly.
A TypeError is now raised instead of an AttributeError in
"with" and "async with" statements for objects which do not
support the context manager or asynchronous context manager
protocols correspondingly.
I tried to be relatively thorough and give lots of links.
One reason is that this wasn't deprecated very long; also it seems people running into this tend to not be familiar with similar APIs.
Co-authored-by: Ken Jin <28750310+Fidget-Spinner@users.noreply.github.com>
Remove 4 C API private trashcan functions which were only kept for
the backward compatibility of the stable ABI with Python 3.8 and
older, since the trashcan API was not usable with the limited C API
on Python 3.8 and older. The trashcan API was excluded from the
limited C API in Python 3.9.
Removed functions:
* _PyTrash_deposit_object()
* _PyTrash_destroy_chain()
* _PyTrash_thread_deposit_object()
* _PyTrash_thread_destroy_chain()
The trashcan C API was never usable with the limited C API, since old
trashcan macros accessed directly PyThreadState members like
"_tstate->trash_delete_nesting", whereas the PyThreadState structure
is opaque in the limited C API.
Exclude also the PyTrash_UNWIND_LEVEL constant from the C API.
The trashcan C API was modified in Python 3.9 by commit
38965ec541 and in Python 3.10 by commit
ed1a5a5bac to hide implementation
details.
Py_RunMain() now resets PyImport_Inittab to its initial value at
exit. It must be possible to call PyImport_AppendInittab() or
PyImport_ExtendInittab() at each Python initialization.
Fix asyncio test_popen() of test_windows_utils by using a longer
timeout. Use military grade battle-tested test.support.SHORT_TIMEOUT
timeout rather than a hardcoded timeout of 10 seconds: it's 30
seconds by default, but it is made longer on slow buildbots.
WaitForMultipleObjects() timeout argument is in milliseconds.
_thread.start_new_thread() no longer calls PyThread_exit_thread()
explicitly at the thread exit, the call was redundant.
On Linux with the glibc, pthread_cancel() loads dynamically the
libgcc_s.so.1 library. dlopen() can fail if there is no more
available file descriptor to open the file. In this case, the process
aborts with the error message:
"libgcc_s.so.1 must be installed for pthread_cancel to work"
pthread_cancel() unwinds back to the thread's wrapping function that
calls the thread entry point.
The unwind function is dynamically loaded from the libgcc_s library
since it is tightly coupled to the C compiler (GCC). The unwinder
depends on DWARF, the compiler generates DWARF, so the unwinder
belongs to the compiler.
Thanks Florian Weimer and Carlos O'Donell for their help on
investigating this issue.
This avoids the following error if DeprecationWarnings are ignored.
======================================================================
ERROR: test_entry_points_by_index (test.test_importlib.test_metadata_api.APITests)
Prior versions of Distribution.entry_points would return a
----------------------------------------------------------------------
Traceback (most recent call last):
File "/builddir/build/BUILD/Python-3.10.0b3/Lib/test/test_importlib/test_metadata_api.py", line 145, in test_entry_points_by_index
expected = next(iter(caught))
StopIteration
----------------------------------------------------------------------
Ran 1402 tests in 2.125s
FAILED (errors=1, skipped=18, expected failures=1)
* Remove struct _node from the stable ABI list
This struct was removed along with the old parser in Python 3.9 (PEP 617)
* Stable ABI list: Use the public name "PyFrameObject" rather than "_frame"
* Ensure limited API doesn't contain private names
Names prefixed by an underscore are private by definition.
* Add a blurb
Currently, if an arg value escapes (into the closure for an inner function) we end up allocating two indices in the fast locals even though only one gets used. Additionally, using the lower index would be better in some cases, such as with no-arg `super()`. To address this, we update the compiler to fix the offsets so each variable only gets one "fast local". As a consequence, now some cell offsets are interspersed with the locals (only when an arg escapes to an inner function).
https://bugs.python.org/issue43693
Change the behaviour of `math.pow(0.0, -math.inf)` and `math.pow(-0.0, -math.inf)` to return positive infinity instead of raising `ValueError`. This makes `math.pow` consistent with the built-in `pow` (and the `**` operator) for this particular special case, and brings the `math.pow` special-case handling into compliance with IEEE 754.
by-value lookups could fail on complex enums, necessitating a check for
__reduce__ and possibly sabotaging the final enum;
by-name lookups should never fail, and sabotaging is no longer necessary
for class-based enum creation.
This enables, for example, two base Enums to both inherit from `str`, and then both be mixed into the same final Enum:
class Str1Enum(str, Enum):
# some behavior here
class Str2Enum(str, Enum):
# some more behavior here
class FinalStrEnum(Str1Enum, Str2Enum):
# this now works
* Specialize LOAD_ATTR with LOAD_ATTR_SLOT and LOAD_ATTR_SPLIT_KEYS
* Move dict-common.h to internal/pycore_dict.h
* Add LOAD_ATTR_WITH_HINT specialized opcode.
* Quicken in function if loopy
* Specialize LOAD_ATTR for module attributes.
* Add specialization stats
Replace it with Windows tab for Shell and Editor options
and Shell/Ed for options exclusive to one of them.
Create room for more options and make dialog shorter,
to better fit small windows.
Emit a deprecation warning if the numeric literal is immediately followed by
one of keywords: and, else, for, if, in, is, or. Raise a syntax error with
more informative message if it is immediately followed by other keyword or
identifier.
Automerge-Triggered-By: GH:pablogsal
This was reverted in GH-26596 (commit 6d518bb) due to some bad memory accesses.
* Add the MAKE_CELL opcode. (gh-26396)
The memory accesses have been fixed.
https://bugs.python.org/issue43693
This moves logic out of the frame initialization code and into the compiler and eval loop. Doing so simplifies the runtime code and allows us to optimize it better.
https://bugs.python.org/issue43693
These were reverted in gh-26530 (commit 17c4edc) due to refleaks.
* 2c1e258 - Compute deref offsets in compiler (gh-25152)
* b2bf2bc - Add new internal code objects fields: co_fastlocalnames and co_fastlocalkinds. (gh-26388)
This change fixes the refleaks.
https://bugs.python.org/issue43693
* Add co_firstinstr field to code object.
* Implement barebones quickening.
* Use non-quickened bytecode when tracing.
* Add NEWS item
* Add new file to Windows build.
* Don't specialize instructions with EXTENDED_ARG.
* bpo-44258: support PEP 515 for Fraction's initialization from string
* regexps's version
* A different regexps version, which doesn't suffer from catastrophic backtracking
* revert denom -> den
* strip "_" from the decimal str, add few tests
* drop redundant tests
* Add versionchanged & whatsnew entry
* Amend Fraction constructor docs
* Change .. versionchanged:...
* `ensurepip` now uses `importlib.resources.files()` traversable APIs
* Update Lib/ensurepip/__init__.py
Co-authored-by: Jason R. Coombs <jaraco@jaraco.com>
* Update Misc/NEWS.d/next/Library/2020-10-11-20-23-48.bpo-37449.f-t3V6.rst
Co-authored-by: Jason R. Coombs <jaraco@jaraco.com>
Co-authored-by: Jason R. Coombs <jaraco@jaraco.com>
1. SyntaxError args have a tuple of other attributes.
2. Attributes are adjusted for errors in f-string field expressions.
3. Compile() can raise SyntaxErrors.
* Revert "bpo-43693: Compute deref offsets in compiler (gh-25152)"
This reverts commit b2bf2bc1ec.
* Revert "bpo-43693: Add new internal code objects fields: co_fastlocalnames and co_fastlocalkinds. (gh-26388)"
This reverts commit 2c1e2583fd.
These two commits are breaking the refleak buildbots.
Merges locals and cells into a single array.
Saves a pointer in the interpreter and means that we don't need the LOAD_CLOSURE opcode any more
https://bugs.python.org/issue43693
When compiling an AST object with a direct / indirect reference
cycles, on the conversion phase because of exceeding amount of
calls, a segfault was raised. This patch adds recursion guards to
places for preventing user inputs to not to crash AST but instead
raise a RecursionError.
Convert the Py_TYPE() and Py_SIZE() macros to static inline
functions. The Py_SET_TYPE() and Py_SET_SIZE() functions must now be
used to set an object type and size.
A number of places in the code base (notably ceval.c and frameobject.c) rely on mapping variable names to indices in the frame "locals plus" array (AKA fast locals), and thus opargs. Currently the compiler indirectly encodes that information on the code object as the tuples co_varnames, co_cellvars, and co_freevars. At runtime the dependent code must calculate the proper mapping from those, which isn't ideal and impacts performance-sensitive sections. This is something we can easily address in the compiler instead.
This change addresses the situation by replacing internal use of co_varnames, etc. with a single combined tuple of names in locals-plus order, along with a minimal array mapping each to its kind (local vs. cell vs. free). These two new PyCodeObject fields, co_fastlocalnames and co_fastllocalkinds, are not exposed to Python code for now, but co_varnames, etc. are still available with the same values as before (though computed lazily).
Aside from the (mild) performance impact, there are a number of other benefits:
* there's now a clear, direct relationship between locals-plus and variables
* code that relies on the locals-plus-to-name mapping is simpler
* marshaled code objects are smaller and serialize/de-serialize faster
Also note that we can take this approach further by expanding the possible values in co_fastlocalkinds to include specific argument types (e.g. positional-only, kwargs). Doing so would allow further speed-ups in _PyEval_MakeFrameVector(), which is where args get unpacked into the locals-plus array. It would also allow us to shrink marshaled code objects even further.
https://bugs.python.org/issue43693
The PyType_Ready() function now raises an error if a type is defined
with the Py_TPFLAGS_HAVE_GC flag set but has no traverse function
(PyTypeObject.tp_traverse).
On macOS, tk defers to system setting for button background when in normal state.
Give turtledemo button text a color that works on either light or dark background.
Fix a regression in type() when a metaclass raises an exception. The
C function type_new() must properly report the exception when a
metaclass constructor raises an exception and the winner class is not
the metaclass.
Py_FrozenMain was added to the Limited C API in [bpo-42591]() (3.10.0a4);
but to fix that issue it would be enough to add it to the regular C API.
The function is undocumented, tests were added very recently ([bpo-44131]()),
and most importantly, it is not present in all builds of Python, as
the linker sometimes omits it as unused.
It should be added back when these issues are fixed.
Note that this does not affect Python's regular C API.
The fix only applies to ``isinstance``. ``issubclass`` isn't affected (because it was always working to begin with). So I also fixed the news to reflect that.
A previous commit broke a check in sysconfig when building cpython itself.
This caused builds of the standard library modules to search a wrong
location (the installed location rather than the source directory) for
header files with the net effect that a ``make install``
incorrectly caused all extension modules to be rebuilt again and
with incorrect include file paths.
When building Python, we need two distinct "include" directories:
- source .h files
- install target for .h files
Note that this doesn't matter except when building Python from source.
Historically:
- source .h files were in the distutils scheme under 'include'
- the install directory was in the distutils.command.install scheme
under 'headers'
GH-24549 merged these; sysconfig is now the single source of truth and
distutils is derived from it.
This commit introduces a "secret" scheme path, 'headers', which contains
the install target. It is only present when building Python.
The distutils code uses it if present, and falls back to 'include'.
Co-authored-by: Ned Deily <nad@python.org>
When the parser does a second pass to check for errors, these rules can
have some small side-effects as they may advance the parser more than
the point reached in the first pass. This can cause the tokenizer to ask
for extra tokens in interactive mode causing the tokenizer to show the
prompt instead of failing instantly.
To avoid this, add a new mode to the tokenizer that is activated in the
second pass and deactivates asking for new tokens when the interactive
line is finished. As the parsing should have reached the last line in
the first pass, the second pass should not need to ask for more tokens.
Fix a crash at Python exit when a deallocator function removes the
last strong reference to a heap type.
Don't read type memory after calling basedealloc() since
basedealloc() can deallocate the type and free its memory.
_PyMem_IsPtrFreed() argument is now constant.
* Remove 'zombie' frames. We won't need them once we are allocating fixed-size frames.
* Add co_nlocalplus field to code object to avoid recomputing size of locals + frees + cells.
* Move locals, cells and freevars out of frame object into separate memory buffer.
* Use per-threadstate allocated memory chunks for local variables.
* Move globals and builtins from frame object to per-thread stack.
* Move (slow) locals frame object to per-thread stack.
* Move internal frame functions to internal header.
`shutil.unpack_archive()` tries to read the whole file into memory, making no use of any kind of smaller buffer. Process crashes for really large files: I.e. archive: ~1.7G, unpacked: ~10G. Before the crash it can easily take away all available RAM on smaller systems. Had to pull the code form `zipfile.Zipfile.extractall()` to fix this
Automerge-Triggered-By: GH:gpshead
andrei kulakov
Stefan Hoelzl
Yurii Karabas
Jason R. Coombs
Irit Katriel
Jack DeVries
Inada Naoki
Hai Shi
Erlend Egeberg Aasland
Ray Donnelly
Barney Gale
Geoffrey Thomas
Pablo Galindo Salgado
Mariusz Felisiak
Nick Coghlan
Jelle Zijlstra
T. Wouters
Hasan
Logan Jones
Mark Shannon
Steven Hsu
Charles Burkland
Filipe Laíns
Dennis Sweeney
Dong-hee Na
Petr Viktorin
Anthony Sottile
Serhiy Storchaka
Łukasz Langa
Thomas Grainger
Mohamad Mansour
Leonardo Freua
Idan Moral
Zackery Spytz
Julien Palard
Hasan
Germán Méndez Bravo
Batuhan Taskaya
Weipeng Hong
Ammar Askar
Tzu-ping Chung
Shane Harvey
Konstantin-Glukhov
jsnklln
Barry Warsaw
Steve Dower
Srinivas Reddy Thatiparthy (శ్రీనివాస్ రెడ్డి తాటిపర్తి)
Ian Henriksen
Rupert Tombs
Gabriele N. Tornetta
Sergey Fedoseev
Ma Lin
Ken Jin
finefoot
Illia Volochii
Miguel Brito
will-ca
Stéphane Bidoul
Brandt Bucher
Victor Stinner
E-Paine
Erik Welch
Russell Keith-Magee
Terry Jan Reedy
Georg Sauthoff
Sebastian Rittau
Miro Hrončok
Eric Snow
Ethan Furman
Christian Heimes
Tim Peters
Mark Dickinson
Kaustubh J
huzhaojie
Lumír 'Frenzy' Balhar
Daniel Hahler
Mark Roseman
Ajith Ramachandran
Furkan Onder
Ryan Hileman
Sergey B Kirpichev
wim glenn
Wm. Keith van der Meulen
Tal Einat
Jakub Kulík
Ned Deily
Irit Katriel
Raymond Hettinger
Catherine Devlin
Jürgen Gmach
Igor Bolshakov