* Revert "bpo-43693: Compute deref offsets in compiler (gh-25152)"
This reverts commit b2bf2bc1ec.
* Revert "bpo-43693: Add new internal code objects fields: co_fastlocalnames and co_fastlocalkinds. (gh-26388)"
This reverts commit 2c1e2583fd.
These two commits are breaking the refleak buildbots.
Merges locals and cells into a single array.
Saves a pointer in the interpreter and means that we don't need the LOAD_CLOSURE opcode any more
https://bugs.python.org/issue43693
A number of places in the code base (notably ceval.c and frameobject.c) rely on mapping variable names to indices in the frame "locals plus" array (AKA fast locals), and thus opargs. Currently the compiler indirectly encodes that information on the code object as the tuples co_varnames, co_cellvars, and co_freevars. At runtime the dependent code must calculate the proper mapping from those, which isn't ideal and impacts performance-sensitive sections. This is something we can easily address in the compiler instead.
This change addresses the situation by replacing internal use of co_varnames, etc. with a single combined tuple of names in locals-plus order, along with a minimal array mapping each to its kind (local vs. cell vs. free). These two new PyCodeObject fields, co_fastlocalnames and co_fastllocalkinds, are not exposed to Python code for now, but co_varnames, etc. are still available with the same values as before (though computed lazily).
Aside from the (mild) performance impact, there are a number of other benefits:
* there's now a clear, direct relationship between locals-plus and variables
* code that relies on the locals-plus-to-name mapping is simpler
* marshaled code objects are smaller and serialize/de-serialize faster
Also note that we can take this approach further by expanding the possible values in co_fastlocalkinds to include specific argument types (e.g. positional-only, kwargs). Doing so would allow further speed-ups in _PyEval_MakeFrameVector(), which is where args get unpacked into the locals-plus array. It would also allow us to shrink marshaled code objects even further.
https://bugs.python.org/issue43693
The timeit doc references Tim Peters introduction to the Chapter 18,
Algorithms, of the second edition. The first editiion was before timeit.
The third edition instead has Chapter 1, Data Structures and Algorithms,
without Tim's introduction.
The previous example did not fully showcase the interest of using gather.
Here the example showcases "the result is an aggregate list of returned values".
* Document os-system, subprocess Patch
* Update Doc/library/os.rst
Co-authored-by: Ken Jin <28750310+Fidget-Spinner@users.noreply.github.com>
Co-authored-by: Ken Jin <28750310+Fidget-Spinner@users.noreply.github.com>
"Zero cost" exception handling.
* Uses a lookup table to determine how to handle exceptions.
* Removes SETUP_FINALLY and POP_TOP block instructions, eliminating (most of) the runtime overhead of try statements.
* Reduces the size of the frame object by about 60%.
* bpo-43926: Cleaner metadata with PEP 566 JSON support.
* Add blurb
* Add versionchanged and versionadded declarations for changes to metadata.
* Use descriptor for PEP 566
Reverts commit e653d4d8e8 and makes
parsing even more strict. Like socket.inet_pton() any leading zero
is now treated as invalid input.
Signed-off-by: Christian Heimes <christian@python.org>
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
Add inspect.get_annotations, which safely computes the annotations defined on an object. It works around the quirks of accessing the annotations from various types of objects, and makes very few assumptions about the object passed in. inspect.get_annotations can also correctly un-stringize stringized annotations.
inspect.signature, inspect.from_callable, and inspect.from_function now call inspect.get_annotations to retrieve annotations. This means inspect.signature and inspect.from_callable can now un-stringize stringized annotations, too.
Accessing the following attributes will now fire PEP 578 style audit hooks as ("object.__getattr__", obj, name):
* PyTracebackObject: tb_frame
* PyFrameObject: f_code
* PyGenObject: gi_code, gi_frame
* PyCoroObject: cr_code, cr_frame
* PyAsyncGenObject: ag_code, ag_frame
Add an AUDIT_READ attribute flag aliased to READ_RESTRICTED.
Update obsolete flag documentation.
Links for 'raise Exception from x' target to 'The raise statement' (7.8) section instead of 'The import statement' (7.11) section.
There are more modified links than in the bug report because I searched some other ones which can get the same improvement.
In 3.12 ``True`` or ``False`` will be returned for all containment checks,
with ``True`` being returned if the value is either a member of that enum
or one of its members' value.
The check about the f argument type was removed in this commit:
2c94aa567e
Thanks for Pedro Arthur Duarte (pedroarthur.jedi at gmail.com) by the help with
this bug.
Previous ID (5233) refers to "Sieve Email Filtering: Subaddress
Extension". It seems that the actual reference should be "Internet
Message Format" RFC 5322 (https://tools.ietf.org/html/rfc5322).
(The typo probably comes from commit 29d1bc0842 in which the ID of
this RFC got updated from the obsolete 2822.)
Co-authored-by: Ambrose Chua <ambrose@hey.com>
asyncio.get_event_loop() emits now a deprecation warning when it creates a new event loop.
In future releases it will became an alias of asyncio.get_running_loop().
The argument order of `link_to()` is reversed compared to what one may expect, so:
a.link_to(b)
Might be expected to create *a* as a link to *b*, in fact it creates *b* as a link to *a*, making it function more like a "link from". This doesn't match `symlink_to()` nor the documentation and doesn't seem to be the original author's intent.
This PR deprecates `link_to()` and introduces `hardlink_to()`, which has the same argument order as `symlink_to()`.
To improve the user experience understanding what part of the error messages associated with SyntaxErrors is wrong, we can highlight the whole error range and not only place the caret at the first character. In this way:
>>> foo(x, z for z in range(10), t, w)
File "<stdin>", line 1
foo(x, z for z in range(10), t, w)
^
SyntaxError: Generator expression must be parenthesized
becomes
>>> foo(x, z for z in range(10), t, w)
File "<stdin>", line 1
foo(x, z for z in range(10), t, w)
^^^^^^^^^^^^^^^^^^^^
SyntaxError: Generator expression must be parenthesized
This change:
* merges `distutils.sysconfig` into `sysconfig` while keeping the original functionality and
* marks `distutils.sysconfig` as deprecated
https://bugs.python.org/issue41282
The sys module uses the kernel32.dll version number, which can vary from the "actual" Windows version.
Since the best option for getting the version is WMI (which is expensive), we switch back to launching cmd.exe (which is also expensive, but a lot less code on our part).
sys.getwindowsversion() is not updated to avoid launching executables from that module.
add:
* `_simple_enum` decorator to transform a normal class into an enum
* `_test_simple_enum` function to compare
* `_old_convert_` to enable checking `_convert_` generated enums
`_simple_enum` takes a normal class and converts it into an enum:
@simple_enum(Enum)
class Color:
RED = 1
GREEN = 2
BLUE = 3
`_old_convert_` works much like` _convert_` does, using the original logic:
# in a test file
import socket, enum
CheckedAddressFamily = enum._old_convert_(
enum.IntEnum, 'AddressFamily', 'socket',
lambda C: C.isupper() and C.startswith('AF_'),
source=_socket,
)
`_test_simple_enum` takes a traditional enum and a simple enum and
compares the two:
# in the REPL or the same module as Color
class CheckedColor(Enum):
RED = 1
GREEN = 2
BLUE = 3
_test_simple_enum(CheckedColor, Color)
_test_simple_enum(CheckedAddressFamily, socket.AddressFamily)
Any important differences will raise a TypeError
add:
_simple_enum decorator to transform a normal class into an enum
_test_simple_enum function to compare
_old_convert_ to enable checking _convert_ generated enums
_simple_enum takes a normal class and converts it into an enum:
@simple_enum(Enum)
class Color:
RED = 1
GREEN = 2
BLUE = 3
_old_convert_ works much like _convert_ does, using the original logic:
# in a test file
import socket, enum
CheckedAddressFamily = enum._old_convert_(
enum.IntEnum, 'AddressFamily', 'socket',
lambda C: C.isupper() and C.startswith('AF_'),
source=_socket,
)
test_simple_enum takes a traditional enum and a simple enum and
compares the two:
# in the REPL or the same module as Color
class CheckedColor(Enum):
RED = 1
GREEN = 2
BLUE = 3
_test_simple_enum(CheckedColor, Color)
_test_simple_enum(CheckedAddressFamily, socket.AddressFamily)
Any important differences will raise a TypeError
The ssl module now uses ``SSL_read_ex`` and ``SSL_write_ex``
internally. The functions support reading and writing of data larger
than 2 GB. Writing zero-length data no longer fails with a protocol
violation error.
Signed-off-by: Christian Heimes <christian@python.org>
Fix problem with ssl.SSLContext.hostname_checks_common_name. OpenSSL does not
copy hostflags from *struct SSL_CTX* to *struct SSL*.
Signed-off-by: Christian Heimes <christian@python.org>
Use a versionadded directive to generate the text "New in version
3.8." (to match with the documentation of other modules).
Automerge-Triggered-By: GH:jaraco
When printing NameError raised by the interpreter, PyErr_Display
will offer suggestions of simmilar variable names in the function that the exception
was raised from:
>>> schwarzschild_black_hole = None
>>> schwarschild_black_hole
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'schwarschild_black_hole' is not defined. Did you mean: schwarzschild_black_hole?
When printing AttributeError, PyErr_Display will offer suggestions of similar
attribute names in the object that the exception was raised from:
>>> collections.namedtoplo
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: module 'collections' has no attribute 'namedtoplo'. Did you mean: namedtuple?
The snake_case names have existed since Python 2.6, so there is
no reason to keep the old camelCase names around. One similar
method, threading.Thread.isAlive, was already removed in
Python 3.9 (bpo-37804).
ripemd160 is not available in OpenSSL 3.0.0's default crypto provider.
It's only present in legacy provider.
Signed-off-by: Christian Heimes <christian@python.org>
Static methods (@staticmethod) and class methods (@classmethod) now
inherit the method attributes (__module__, __name__, __qualname__,
__doc__, __annotations__) and have a new __wrapped__ attribute.
Changes:
* Add a repr() method to staticmethod and classmethod types.
* Add tests on the @classmethod decorator.
Add Doc/using/configure.rst documentation to document configure,
preprocessor, compiler and linker options.
Add a new section about the "Python debug build".
This makes `ntpath.expanduser()` match `pathlib.Path.expanduser()` in this regard, and is more in line with `posixpath.expanduser()`'s cautious approach.
Also remove the near-duplicate implementation of `expanduser()` in pathlib, and by doing so fix a bug where KeyError could be raised when expanding another user's home directory.
* Handle check for sending None to starting generator and coroutine into bytecode.
* Document new bytecode and make it fail gracefully if mis-compiled.
* Enum: streamline repr() and str(); improve docs
- repr() is now ``enum_class.member_name``
- stdlib global enums are ``module_name.member_name``
- str() is now ``member_name``
- add HOW-TO section for ``Enum``
- change main documentation to be an API reference
See [PEP 597](https://www.python.org/dev/peps/pep-0597/).
* Add `-X warn_default_encoding` and `PYTHONWARNDEFAULTENCODING`.
* Add EncodingWarning
* Add io.text_encoding()
* open(), TextIOWrapper() emits EncodingWarning when encoding is omitted and warn_default_encoding is enabled.
* _pyio.TextIOWrapper() uses UTF-8 as fallback default encoding used when failed to import locale module. (used during building Python)
* bz2, configparser, gzip, lzma, pathlib, tempfile modules use io.text_encoding().
* What's new entry
It doesn't actually affect whether match_hostname() is called (it
never is in this context any longer), but whether hostname
verification occurs in the first place.
pprint() gains a new boolean underscore_numbers kwarg to emit
integers with thousands separated by an underscore character
for improved readability (for example 1_000_000 instead of 1000000).
* bpo-43428: Sync with importlib_metadata 3.7.3 (16ac3a95)
* Add 'versionadded' for importlib.metadata.packages_distributions
* Add section in what's new for Python 3.10 highlighting most salient changes and relevant backport.
Added an invalidate_caches() method to the zipimport.zipimporter class based on the implementation of importlib.FileFinder.invalidate_caches(). This was done by adding a get_files() method and an _archive_mtime attribute to zipimport.zipimporter to check for updates or cache invalidation whenever the cache of files and toc entry information in the zipimporter is accessed.
The case of tempfile.tempdir variable being bytes is now handled consistently.
The getters return the right type and no more error of mixing str and bytes unless explicitly caused by the user.
Adds a regression test.
Expands the documentation to clarify the behavior.
Co-authored-by: Eric L <ewl+git@lavar.de>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
* bpo-42128: Add documentation for the new match-based AST nodes
* Update Doc/library/ast.rst
Co-authored-by: Carol Willing <carolcode@willingconsulting.com>
* Fix trailing whitespace
Co-authored-by: Carol Willing <carolcode@willingconsulting.com>
The note about the GIL was buried pretty deep in the threading documentation,
and this made it hard for first time users to discover why their attempts
at using threading to parallelizing their application did not work.
In this commit, the note is moved to the top of the module documention for
visibility.
Printing to IDLE's Shell is often slower than printing to a system
terminal, but it can be made faster by pre-formatting a single
string before printing.
Unittest discovery support namespace package as start
directory. But it doesn't find namespace package in
the start directory automatically.
Otherwise, unittest discovery search into unexpected
directories like `vendor/` or `node_modules/`.
Expose the new PyFunctionObject.func_builtins member in Python as a
new __builtins__ attribute on functions.
Document also the behavior change in What's New in Python 3.10.
bpo-42967: [security] Address a web cache-poisoning issue reported in urllib.parse.parse_qsl().
urllib.parse will only us "&" as query string separator by default instead of both ";" and "&" as allowed in earlier versions. An optional argument seperator with default value "&" is added to specify the separator.
Co-authored-by: Éric Araujo <merwok@netwok.org>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Ken Jin <28750310+Fidget-Spinner@users.noreply.github.com>
Co-authored-by: Éric Araujo <merwok@netwok.org>
bpo-43172: readline now passes its tests when built against libedit.
Existing irreconcilable API differences remain in readline.get_begidx
and readline.get_endidx behavior based on libreadline vs libedit use.
A note about that has been documented.
The documentation for some parts of the logging.config formatters has
fallen behind the code. For example, the dictionary-schema section
does not list the "class" attribute, however it is discussed in the
file/ini discussion; and neither references the style argument which
has been added.
This modifies the dictionary-schema formatters documentation to list
the keys available and overall makes it clearer these are passed to
create a logging.Formatter object.
The logging.Formatter documentation describes the default values of
format/datefmt and the various formatting options. Since we have now
more clearly described how the configuration is created via this type
of object, we remove the discussion in this document to avoid
duplication and rely on users reading the referenced logging.Formatter
documenation directly for such details.
Instead of duplicating the discussion for the two config types, the
file/ini section is modified to link back to the dictionary-schema
discussion, making it clear the same arguments are accepted.
Automerge-Triggered-By: GH:vsajip
Flag members are now divided by one-bit verses multi-bit, with multi-bit being treated as aliases. Iterating over a flag only returns the contained single-bit flags.
Iterating, repr(), and str() show members in definition order.
When constructing combined-member flags, any extra integer values are either discarded (CONFORM), turned into ints (EJECT) or treated as errors (STRICT). Flag classes can specify which of those three behaviors is desired:
>>> class Test(Flag, boundary=CONFORM):
... ONE = 1
... TWO = 2
...
>>> Test(5)
<Test.ONE: 1>
Besides the three above behaviors, there is also KEEP, which should not be used unless necessary -- for example, _convert_ specifies KEEP as there are flag sets in the stdlib that are incomplete and/or inconsistent (e.g. ssl.Options). KEEP will, as the name suggests, keep all bits; however, iterating over a flag with extra bits will only return the canonical flags contained, not the extra bits.
Iteration is now in member definition order. If member definition order
matches increasing value order, then a more efficient method of flag
decomposition is used; otherwise, sort() is called on the results of
that method to get definition order.
``re`` module:
repr() has been modified to support as closely as possible its previous
output; the big difference is that inverted flags cannot be output as
before because the inversion operation now always returns the comparable
positive result; i.e.
re.A|re.I|re.M|re.S is ~(re.L|re.U|re.S|re.T|re.DEBUG)
in both of the above terms, the ``value`` is 282.
re's tests have been updated to reflect the modifications to repr().
Co-authored-by: Éric Araujo <merwok@netwok.org>
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
Co-authored-by: Tal Einat <532281+taleinat@users.noreply.github.com>
Add --with-wheel-pkg-dir=PATH option to the ./configure script. If
specified, the ensurepip module looks for setuptools and pip wheel
packages in this directory: if both are present, these wheel packages
are used instead of ensurepip bundled wheel packages.
Some Linux distribution packaging policies recommend against bundling
dependencies. For example, Fedora installs wheel packages in the
/usr/share/python-wheels/ directory and don't install the
ensurepip._bundled package.
ensurepip: Remove unused runpy import.
Pablo Galindo
Mark Shannon
Erlend Egeberg Aasland
Eric Snow
andrei kulakov
Zac Bentley
MapleCCC
Jules Lasne
Dong-hee Na
Zackery Spytz
Jürgen Gmach
Ken Jin
Peter Law
Mariusz Felisiak
Raymond Hettinger
Zack Kneupper
Jason R. Coombs
Junnosuke Kuroda
Irit Katriel
E-Paine
Terry Jan Reedy
Numerlor
Bruno
naglis
Catherine Devlin
Tal Einat
Rafael Fontenelle
Miguel Brito
Rory Yorke
kudavid
Géry Ogam
josephernest
Shantanu
uniocto
Julien Palard
dhoekstra2000
Eric V. Smith
Senthil Kumaran
Scott Noyes
Jean-Abou-Samra
Jelle Zijlstra
Brandt Bucher
Shreyan Avigyan
Christian Heimes
Inada Naoki
larryhastings
Shreyash Sharma
Yurii Karabas
Mohamed Moselhy
Ryan Hileman
sblondon
Nick Coghlan
Andre Delfino
Paul Moore
Barney Gale
Ethan Furman
Tzu-ping Chung
Llandy Riveron Del Risco
Adorilson Bezerra
Gabriel R F
Etienne Gautier
Taneli Hukkinen
Denis Laxalde
Sergey Fedoseev
Saiyang Gou
Tymoteusz Wołodźko
Serhiy Storchaka
Steve Dower
Lumír 'Frenzy' Balhar
Ned Batchelder
Joonas Paalasmaa
l0x
Illia Volochii
Lewis Gaul
Victor Stinner
Cédric Krier
DevilXD
Gregory P. Smith
Brett Cannon
Ammar Askar
Ville Skyttä
Jared Sutton
Joshua Bronson
Rui Cunha
Chris Burr
CAM Gerlach
Kamil Turek
Antoine Pitrou
Adrian Freund
Guilherme Martins Crocetti
Desmond Cheong
Alex Willmer
Eddie Peters
Eric L
Erik Soma
Mariatta Wijaya
Paul Bryan
Brandt Bucher
takahashi
Guanzhong Chen
David CARLIER
Adam Goldschmidt
L
Ian Wienand
Ken
Борис Верховский
Conchylicultor
Hong Xu