check_set_special_type_attr() and type_set_annotations()
now check for immutable flag (Py_TPFLAGS_IMMUTABLETYPE).
Co-authored-by: Victor Stinner <vstinner@python.org>
The PyStdPrinter_Type type now uses the
Py_TPFLAGS_DISALLOW_INSTANTIATION flag to disallow instantiation,
rather than seting a tp_init method which always fail.
Write also unit tests for PyStdPrinter_Type.
Add a new Py_TPFLAGS_DISALLOW_INSTANTIATION type flag to disallow
creating type instances: set tp_new to NULL and don't create the
"__new__" key in the type dictionary.
The flag is set automatically on static types if tp_base is NULL or
&PyBaseObject_Type and tp_new is NULL.
Use the flag on the following types:
* _curses.ncurses_version type
* _curses_panel.panel
* _tkinter.Tcl_Obj
* _tkinter.tkapp
* _tkinter.tktimertoken
* _xxsubinterpretersmodule.ChannelID
* sys.flags type
* sys.getwindowsversion() type
* sys.version_info type
Update MyStr example in the C API documentation to use
Py_TPFLAGS_DISALLOW_INSTANTIATION.
Add _PyStructSequence_InitType() function to create a structseq type
with the Py_TPFLAGS_DISALLOW_INSTANTIATION flag set.
type_new() calls _PyType_CheckConsistency() at exit.
* Add Py_TPFLAGS_SEQUENCE and Py_TPFLAGS_MAPPING, add to all relevant standard builtin classes.
* Set relevant flags on collections.abc.Sequence and Mapping.
* Use flags in MATCH_SEQUENCE and MATCH_MAPPING opcodes.
* Inherit Py_TPFLAGS_SEQUENCE and Py_TPFLAGS_MAPPING.
* Add NEWS
* Remove interpreter-state map_abc and seq_abc fields.
Add inspect.get_annotations, which safely computes the annotations defined on an object. It works around the quirks of accessing the annotations from various types of objects, and makes very few assumptions about the object passed in. inspect.get_annotations can also correctly un-stringize stringized annotations.
inspect.signature, inspect.from_callable, and inspect.from_function now call inspect.get_annotations to retrieve annotations. This means inspect.signature and inspect.from_callable can now un-stringize stringized annotations, too.
Remove call to macosx.setupApp, which calls macosc.overrideRootMenu, which modifies
the menus, which results in two failures in the second round of the leak test.
While working on another issue, I noticed two minor nits in the C implementation of the module object. Both are related to getting a module's name.
First, the C function module_dir() (module.__dir__) starts by ensuring the module dict is valid. If the module dict is invalid, it wants to format an exception using the name of the module, which it gets from PyModule_GetName(). However, PyModule_GetName() gets the name of the module from the dict. So getting the name in this circumstance will never succeed.
When module_dir() wants to format the error but can't get the name, it knows that PyModule_GetName() must have already raised an exception. So it leaves that exception alone and returns an error. The end result is that the exception raised here is kind of useless and misleading: dir(module) on a module with no __dict__ raises SystemError("nameless module"). I changed the code to actually raise the exception it wanted to raise, just without a real module name: TypeError("<module>.__dict__ is not a dictionary"). This seems more useful, and would do a better job putting the programmer who encountered this on the right track of figuring out what was going on.
Second, the C API function PyModule_GetNameObject() checks to see if the module has a dict. If m->md_dict is not NULL, it calls _PyDict_GetItemIdWithError(). However, it's possible for m->md_dict to be None. And if you call _PyDict_GetItemIdWithError(Py_None, ...) it will *crash*.
Unfortunately, this crash was due to my own bug in the other branch. Fixing my code made the crash go away. I assert that this is still possible at the API level.
The fix is easy: add a PyDict_Check() to PyModule_GetNameObject().
Unfortunately, I don't know how to add a unit test for this. Having changed module_dir() above, I can't find any other interfaces callable from Python that eventually call PyModule_GetNameObject(). So I don't know how to trick the runtime into reproducing this error.
Since both these changes are minor--each entails only a small edit to only one line--I didn't bother with a news item.
Change class and module objects to lazy-create empty annotations dicts on demand. The annotations dicts are stored in the object's `__dict__` for backwards compatibility.
* Add length parameter to PyLineTable_InitAddressRange and doen't use sentinel values at end of table. Makes the line number table more robust.
* Update PyCodeAddressRange to match PEP 626.
Removes the `list` call in the Popen `repr`.
Current implementation:
For cmd = `python --version`, with `shell=True`.
```bash
<Popen: returncode: None args: ['p', 'y', 't', 'h', 'o', 'n', ' ', '-', '-',...>
```
For `shell=False` and args=`['python', '--version']`, the output is correct:
```bash
<Popen: returncode: None args: ['python', '--version']>
```
With the new changes the `repr` yields:
For cmd = `python --version`, with `shell=True`:
```bash
<Popen: returncode: None args: 'python --version'>
```
For `shell=False` and args=`['python', '--version']`, the output:
```bash
<Popen: returncode: None args: ['python', '--version']>
```
Automerge-Triggered-By: GH:gpshead
Faster bz2/lzma/zlib via new output buffering.
Also adds .readall() function to _compression.DecompressReader class
to take best advantage of this in the consume-all-output at once scenario.
Often a 5-20% speedup in common scenarios due to less data copying.
Contributed by Ma Lin.
In 3.12 ``True`` or ``False`` will be returned for all containment checks,
with ``True`` being returned if the value is either a member of that enum
or one of its members' value.
In 3.12 the enum member, not the member's value, will be used for
format() calls. Format specifiers can be used to retain the current
display of enum members:
Example enumeration:
class Color(IntEnum):
RED = 1
GREEN = 2
BLUE = 3
Current behavior:
f'{Color.RED}' --> '1'
Future behavior:
f'{Color.RED}' --> 'RED'
Using d specifier:
f'{Color.RED:d}' --> '1'
Using specifiers can be done now and is future-compatible.
The internal `_ssl._SSLSocket` object now provides methods to retrieve
the peer cert chain and verified cert chain as a list of Certificate
objects. Certificate objects have methods to convert the cert to a dict,
PEM, or DER (ASN.1).
These are private APIs for now. There is a slim chance to stabilize the
approach and provide a public API for 3.10. Otherwise I'll provide a
stable API in 3.11.
Signed-off-by: Christian Heimes <christian@python.org>
* Add space after period to warning in _tzpath.py
Currently:
InvalidTZPathWarning: Invalid paths specified in PYTHONTZPATH environment variable.Paths should be absolute but found the following relative paths: ...
* Update _tzpath.py
asyncio.get_event_loop() emits now a deprecation warning when it creates a new event loop.
In future releases it will became an alias of asyncio.get_running_loop().
Revert 73ea546, increase logging, and improve stability of test.
Handle all OSErrors in a single block. OSError also takes care of
SSLError and socket's connection errors.
Partly reverts commit fb7e750. The
threaded connection handler must not raise an unhandled exception.
Depending on usage, it's possible for Flag members to have the _inverted_ attribute when they are testing, while the Flag being testing against will not have that attribute on its members -- so skip that comparison.
The argument order of `link_to()` is reversed compared to what one may expect, so:
a.link_to(b)
Might be expected to create *a* as a link to *b*, in fact it creates *b* as a link to *a*, making it function more like a "link from". This doesn't match `symlink_to()` nor the documentation and doesn't seem to be the original author's intent.
This PR deprecates `link_to()` and introduces `hardlink_to()`, which has the same argument order as `symlink_to()`.
test_wrong_cert_tls13 sometimes fails on some Windows buildbots. Turn
failing test case into skipped test case until we have more time to
investigate.
Signed-off-by: Christian Heimes <christian@python.org>
To improve the user experience understanding what part of the error messages associated with SyntaxErrors is wrong, we can highlight the whole error range and not only place the caret at the first character. In this way:
>>> foo(x, z for z in range(10), t, w)
File "<stdin>", line 1
foo(x, z for z in range(10), t, w)
^
SyntaxError: Generator expression must be parenthesized
becomes
>>> foo(x, z for z in range(10), t, w)
File "<stdin>", line 1
foo(x, z for z in range(10), t, w)
^^^^^^^^^^^^^^^^^^^^
SyntaxError: Generator expression must be parenthesized
This change:
* merges `distutils.sysconfig` into `sysconfig` while keeping the original functionality and
* marks `distutils.sysconfig` as deprecated
https://bugs.python.org/issue41282
The sys module uses the kernel32.dll version number, which can vary from the "actual" Windows version.
Since the best option for getting the version is WMI (which is expensive), we switch back to launching cmd.exe (which is also expensive, but a lot less code on our part).
sys.getwindowsversion() is not updated to avoid launching executables from that module.
Previously TestIntEnumConvert and TestStrEnumConvert would end up
converting the module level variables from their regular int form
to a `test.test_enum.X` instance after _convert would run. This
meant that after a single test ran, the next set of _convert
functions would be operating on the enum instances rather than
ints. This would cause some tests such as the one involving format
to fail when running under a mode that repeatedly runs test such
as the refleak finder.
add:
* `_simple_enum` decorator to transform a normal class into an enum
* `_test_simple_enum` function to compare
* `_old_convert_` to enable checking `_convert_` generated enums
`_simple_enum` takes a normal class and converts it into an enum:
@simple_enum(Enum)
class Color:
RED = 1
GREEN = 2
BLUE = 3
`_old_convert_` works much like` _convert_` does, using the original logic:
# in a test file
import socket, enum
CheckedAddressFamily = enum._old_convert_(
enum.IntEnum, 'AddressFamily', 'socket',
lambda C: C.isupper() and C.startswith('AF_'),
source=_socket,
)
`_test_simple_enum` takes a traditional enum and a simple enum and
compares the two:
# in the REPL or the same module as Color
class CheckedColor(Enum):
RED = 1
GREEN = 2
BLUE = 3
_test_simple_enum(CheckedColor, Color)
_test_simple_enum(CheckedAddressFamily, socket.AddressFamily)
Any important differences will raise a TypeError
add:
_simple_enum decorator to transform a normal class into an enum
_test_simple_enum function to compare
_old_convert_ to enable checking _convert_ generated enums
_simple_enum takes a normal class and converts it into an enum:
@simple_enum(Enum)
class Color:
RED = 1
GREEN = 2
BLUE = 3
_old_convert_ works much like _convert_ does, using the original logic:
# in a test file
import socket, enum
CheckedAddressFamily = enum._old_convert_(
enum.IntEnum, 'AddressFamily', 'socket',
lambda C: C.isupper() and C.startswith('AF_'),
source=_socket,
)
test_simple_enum takes a traditional enum and a simple enum and
compares the two:
# in the REPL or the same module as Color
class CheckedColor(Enum):
RED = 1
GREEN = 2
BLUE = 3
_test_simple_enum(CheckedColor, Color)
_test_simple_enum(CheckedAddressFamily, socket.AddressFamily)
Any important differences will raise a TypeError
The ssl module now uses ``SSL_read_ex`` and ``SSL_write_ex``
internally. The functions support reading and writing of data larger
than 2 GB. Writing zero-length data no longer fails with a protocol
violation error.
Signed-off-by: Christian Heimes <christian@python.org>
Many servers in the cloud environment require SNI to be used during the
SSL/TLS handshake, therefore it is not possible to fetch their certificates
using the ssl.get_server_certificate interface.
This change adds an additional optional hostname argument that can be used to
set the SNI. Note that it is intentionally a separate argument instead of
using the host part of the addr tuple, because one might want to explicitly
fetch the default certificate or fetch a certificate from a specific IP
address with the specified SNI hostname. A separate argument also works better
for backwards compatibility.
Automerge-Triggered-By: GH:tiran
Since we can use now the sha3 and shake algorithms from OpenSSL, we shouldn't
skip the tests in case the builtin algorithm was not compiled or removed.
- Remove HAVE_X509_VERIFY_PARAM_SET1_HOST check
- Update hashopenssl to require OpenSSL 1.1.1
- multissltests only OpenSSL > 1.1.0
- ALPN is always supported
- SNI is always supported
- Remove deprecated NPN code. Python wrappers are no-op.
- ECDH is always supported
- Remove OPENSSL_VERSION_1_1 macro
- Remove locking callbacks
- Drop PY_OPENSSL_1_1_API macro
- Drop HAVE_SSL_CTX_CLEAR_OPTIONS macro
- SSL_CTRL_GET_MAX_PROTO_VERSION is always defined now
- security level is always available now
- get_num_tickets is available with TLS 1.3
- X509_V_ERR MISMATCH is always available now
- Always set SSL_MODE_RELEASE_BUFFERS
- X509_V_FLAG_TRUSTED_FIRST is always available
- get_ciphers is always supported
- SSL_CTX_set_keylog_callback is always available
- Update Modules/Setup with static link example
- Mention PEP in whatsnew
- Drop 1.0.2 and 1.1.0 from GHA tests
Fix problem with ssl.SSLContext.hostname_checks_common_name. OpenSSL does not
copy hostflags from *struct SSL_CTX* to *struct SSL*.
Signed-off-by: Christian Heimes <christian@python.org>
The multiprocessing Server class now explicitly catchs SystemExit and
closes the client connection in this case. It happens when the
Server.serve_client() method reachs the end of file (EOF).
test.libregrtest now marks a test as ENV_CHANGED (altered the
execution environment) if a thread raises an exception but does not
catch it. It sets a hook on threading.excepthook. Use
--fail-env-changed option to mark the test as failed.
libregrtest regrtest_unraisable_hook() explicitly flushs
sys.stdout, sys.stderr and sys.__stderr__.
Fix a race condition in the SMTP test of test_logging. Don't close a
file descriptor (socket) from a different thread while
asyncore.loop() is polling the file descriptor.
* Modify compiler to reduce stack consumption for large expressions.
* Add more tests for stack usage.
* Add NEWS item.
* Raise SystemError for truly excessive stack use.
With this patch, `distutils.command.install.INSTALL_SCHEMES` are loaded from
`sysconfig._INSTALL_SCHEMES`.
The distutils module is deprecated and will be removed in 3.12 (PEP 632).
This change makes the `sysconfig._INSTALL_SCHEMES` the single point of truth
for install schemes while keeping `distutils.command.install.INSTALL_SCHEMES`
exactly the same. If we, during the transition to the sysconfig, change
something, this makes sure that it also propagates to distutils until the
module gets removed.
Moreover, as discussed [on Discourse], Linux distros need to patch
distutils/sysconfig to make sure the packages will land in proper locations.
This patch makes it easier because it leaves only one location where install
schemes are defined which is much easier to patch/adjust.
[on Discourse]: https://discuss.python.org/t/pep-632-deprecate-distutils-module/5134
The implementation is slightly different than the plan but I think it's the
easiest way how to do it and it also makes the downstream patch simple,
flexible and easy to maintain.
It's also necessary to implement this before setuptools starts bundling
the distutils module so the default install schemes stay in the standard library.
The removed code from sysconfig does not seem to have any negative effect
because, honestly, it seems that nothing actually uses the install schemes
from sysconfig at all. There were many big changes in these modules where
they were trying to include packaging in stdlib and then reverted that.
Also, the test of distutils install command does not count with the different
locations which is good evidence that the reason to have this piece of code
is no longer valid.
https://bugs.python.org/issue41282
When printing NameError raised by the interpreter, PyErr_Display
will offer suggestions of simmilar variable names in the function that the exception
was raised from:
>>> schwarzschild_black_hole = None
>>> schwarschild_black_hole
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'schwarschild_black_hole' is not defined. Did you mean: schwarzschild_black_hole?
When printing AttributeError, PyErr_Display will offer suggestions of similar
attribute names in the object that the exception was raised from:
>>> collections.namedtoplo
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: module 'collections' has no attribute 'namedtoplo'. Did you mean: namedtuple?
Deprecate io.OpenWrapper and _pyio.OpenWrapper: use io.open and
_pyio.open instead. Until Python 3.9, _pyio.open was not a static
method and builtins.open was set to OpenWrapper to not become a bound
method when set to a class variable. _io.open is a built-in function
whereas _pyio.open is a Python function. In Python 3.10, _pyio.open()
is now a static method, and builtins.open() is now io.open().
Zackery Spytz
Christian Heimes
Inada Naoki
Ethan Furman
Roberto Hueso
Pablo Galindo
Eric V. Smith
Stéphane Bidoul
Yurii Karabas
Erlend Egeberg Aasland
Tarjei Bærland
Dong-hee Na
Victor Stinner
Mark Shannon
larryhastings
Terry Jan Reedy
Brandt Bucher
Senthil Kumaran
Nick Coghlan
Hristo Venev
kfollstad
Tal Einat
Barney Gale
Ken Jin
Steve Dower
Irit Katriel
M. Kocher
Ma Lin
Anthony Sottile
Shreyan Avigyan
Tzu-ping Chung
Dennis Sweeney
uy-rrodriguez
Allen
Serhiy Storchaka
Tymoteusz Wołodźko
Batuhan Taskaya
Stéphane Bidoul
Jason R. Coombs
Brett Cannon
Carl Friedrich Bolz-Tereick
Lumír 'Frenzy' Balhar
Saiyang Gou
E-Paine
Raymond Hettinger
Ammar Askar
juhovh
stratakis
Karthikeyan Singaravelan
Lewis Gaul