Commit Graph

46832 Commits

Author SHA1 Message Date
Christian Heimes c8666cfa7c
bpo-37322: Fix ResourceWarning and exception handling in test (GH-25553)
Revert 73ea546, increase logging, and improve stability of test.

Handle all OSErrors in a single block. OSError also takes care of
SSLError and socket's connection errors.

Partly reverts commit fb7e750. The
threaded connection handler must not raise an unhandled exception.
2021-04-24 09:17:54 +02:00
Christian Heimes f05c2aed7e
bpo-31870: Fix test_get_server_certificate_timeout on Windows (GH-25570)
Some OS do not support millisecond granularity in select(). Use 100ms
timeout and a server callback with sleep to emulate a slow server.
2021-04-24 07:54:08 +02:00
Zackery Spytz b2fac1afaa
bpo-31870: Add a timeout parameter to ssl.get_server_certificate() (GH-22270) 2021-04-23 21:46:01 -07:00
Ethan Furman 6c681e1a4a
bpo-38659: [Enum] do not check '_inverted_' during simple test (GH-25566)
Depending on usage, it's possible for Flag members to have the _inverted_ attribute when they are testing, while the Flag being testing against will not have that attribute on its members -- so skip that comparison.
2021-04-23 19:08:22 -07:00
Brett Cannon 22556d84bc
Silence find_module() DeprecationWarning on Windows tests (GH-25563) 2021-04-23 14:40:18 -07:00
Carl Friedrich Bolz-Tereick 1e9f093309
bpo-43907: add missing memoize call in pure python pickling of bytearray (GH-25501) 2021-04-23 23:27:14 +02:00
Barney Gale f24e2e5464
bpo-39950: add `pathlib.Path.hardlink_to()` method that supersedes `link_to()` (GH-18909)
The argument order of `link_to()` is reversed compared to what one may expect, so:

    a.link_to(b)

Might be expected to create *a* as a link to *b*, in fact it creates *b* as a link to *a*, making it function more like a "link from". This doesn't match `symlink_to()` nor the documentation and doesn't seem to be the original author's intent.

This PR deprecates `link_to()` and introduces `hardlink_to()`, which has the same argument order as `symlink_to()`.
2021-04-23 13:48:52 -07:00
Christian Heimes e047239eaf
bpo-43921: ignore failing test_wrong_cert_tls13 on Windows (GH-25561)
test_wrong_cert_tls13 sometimes fails on some Windows buildbots. Turn
failing test case into skipped test case until we have more time to
investigate.

Signed-off-by: Christian Heimes <christian@python.org>
2021-04-23 20:03:25 +02:00
Steve Dower 019e9e8168
bpo-43538: Add extra arguments to os.startfile (GH-25538) 2021-04-23 18:03:17 +01:00
Steve Dower 3513d55a61
bpo-43607: Fix urllib handling of Windows paths with \\?\ prefix (GH-25539) 2021-04-23 18:02:47 +01:00
Pablo Galindo a77aac4fca
bpo-43914: Highlight invalid ranges in SyntaxErrors (#25525)
To improve the user experience understanding what part of the error messages associated with SyntaxErrors is wrong, we can highlight the whole error range and not only place the caret at the first character. In this way:

>>> foo(x, z for z in range(10), t, w)
  File "<stdin>", line 1
    foo(x, z for z in range(10), t, w)
           ^
SyntaxError: Generator expression must be parenthesized

becomes

>>> foo(x, z for z in range(10), t, w)
  File "<stdin>", line 1
    foo(x, z for z in range(10), t, w)
           ^^^^^^^^^^^^^^^^^^^^
SyntaxError: Generator expression must be parenthesized
2021-04-23 14:27:05 +01:00
Lumír 'Frenzy' Balhar 90d02e5e63
bpo-41282: (PEP 632) Deprecate distutils.sysconfig (partial implementation of the PEP) (GH-23142)
This change:
* merges `distutils.sysconfig` into `sysconfig` while keeping the original functionality and
* marks `distutils.sysconfig` as deprecated

https://bugs.python.org/issue41282
2021-04-23 14:02:41 +02:00
Christian Heimes b9ad88be03
bpo-43920: Make load_verify_locations(cadata) error message consistent (GH-25554)
Signed-off-by: Christian Heimes <christian@python.org>
2021-04-23 13:51:40 +02:00
Saiyang Gou 927b841c21
bpo-37363: Add audit events to the `http.client` module (GH-21321)
Add audit events to the `http.client` module

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2021-04-23 12:19:08 +02:00
E-Paine 702a0885ba
Reformat idlelib colorizer (GH-25479)
Also replace if-then and and-or with conditional expressions.

Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
2021-04-22 21:21:53 -04:00
Steve Dower 4696f1285d
bpo-35306: Avoid raising OSError from pathlib.Path.exists when passed an invalid filename (GH-25529) 2021-04-22 21:04:44 +01:00
Shreyan Avigyan 2a3f4899c6
bpo-43284: Update platform.win32_ver to use _syscmd_ver instead of sys.getwindowsversion() (GH-25500)
The sys module uses the kernel32.dll version number, which can vary from the "actual" Windows version.
Since the best option for getting the version is WMI (which is expensive), we switch back to launching cmd.exe (which is also expensive, but a lot less code on our part).
sys.getwindowsversion() is not updated to avoid launching executables from that module.
2021-04-22 17:43:37 +01:00
Raymond Hettinger a07da09ad5
bpo-43475: Fix worst case collision behavior for NaN instances (GH-25493) 2021-04-22 08:34:57 -07:00
Ammar Askar 37b173c523
bpo-38659: Properly re-intialize module variables in test_enum (GH-25516)
Previously TestIntEnumConvert and TestStrEnumConvert would end up
converting the module level variables from their regular int form
to a `test.test_enum.X` instance after _convert would run. This
meant that after a single test ran, the next set of _convert
functions would be operating on the enum instances rather than
ints. This would cause some tests such as the one involving format
to fail when running under a mode that repeatedly runs test such
as the refleak finder.
2021-04-21 20:22:58 -07:00
Saiyang Gou a32f8fe713
bpo-43756: Add new audit event for new arguments added to glob.glob (GH-25239) 2021-04-21 23:42:55 +01:00
Ethan Furman a02cb474f9
bpo-38659: [Enum] add _simple_enum decorator (GH-25497)
add:

* `_simple_enum` decorator to transform a normal class into an enum
* `_test_simple_enum` function to compare
* `_old_convert_` to enable checking `_convert_` generated enums

`_simple_enum` takes a normal class and converts it into an enum:

    @simple_enum(Enum)
    class Color:
        RED = 1
        GREEN = 2
        BLUE = 3

`_old_convert_` works much like` _convert_` does, using the original logic:

    # in a test file
    import socket, enum
    CheckedAddressFamily = enum._old_convert_(
            enum.IntEnum, 'AddressFamily', 'socket',
            lambda C: C.isupper() and C.startswith('AF_'),
            source=_socket,
            )

`_test_simple_enum` takes a traditional enum and a simple enum and
compares the two:

    # in the REPL or the same module as Color
    class CheckedColor(Enum):
        RED = 1
        GREEN = 2
        BLUE = 3

    _test_simple_enum(CheckedColor, Color)

    _test_simple_enum(CheckedAddressFamily, socket.AddressFamily)

Any important differences will raise a TypeError
2021-04-21 10:20:44 -07:00
Pablo Galindo 56c95dfe27
bpo-43859: Improve the error message for IndentationError exceptions (GH-25431) 2021-04-21 15:28:21 +01:00
Pablo Galindo b0544ba77c
bpo-38605: Revert making 'from __future__ import annotations' the default (GH-25490)
This reverts commits 044a1048ca and 1be456ae9d, adapting the code to changes that happened after it.
2021-04-21 12:41:19 +01:00
Christian Heimes 3309113d61
bpo-43799: Also define SSLv3_method() (GH-25481)
Signed-off-by: Christian Heimes <christian@python.org>
2021-04-20 18:10:10 +02:00
Raymond Hettinger 2a36b09ce7
Improve the error message for choices(population, 10) (GH-25267) 2021-04-19 20:29:48 -07:00
Ethan Furman 503cdc7c12
Revert "bpo-38659: [Enum] add _simple_enum decorator (GH-25285)" (GH-25476)
This reverts commit dbac8f40e8.
2021-04-19 19:12:24 -07:00
Ethan Furman dbac8f40e8
bpo-38659: [Enum] add _simple_enum decorator (GH-25285)
add:

_simple_enum decorator to transform a normal class into an enum
_test_simple_enum function to compare
_old_convert_ to enable checking _convert_ generated enums
_simple_enum takes a normal class and converts it into an enum:

@simple_enum(Enum)
class Color:
    RED = 1
    GREEN = 2
    BLUE = 3

_old_convert_ works much like _convert_ does, using the original logic:

# in a test file
import socket, enum
CheckedAddressFamily = enum._old_convert_(
        enum.IntEnum, 'AddressFamily', 'socket',
        lambda C: C.isupper() and C.startswith('AF_'),
        source=_socket,
        )

test_simple_enum takes a traditional enum and a simple enum and
compares the two:

# in the REPL or the same module as Color
class CheckedColor(Enum):
    RED = 1
    GREEN = 2
    BLUE = 3

_test_simple_enum(CheckedColor, Color)

_test_simple_enum(CheckedAddressFamily, socket.AddressFamily)

Any important differences will raise a TypeError
2021-04-19 18:04:53 -07:00
Pablo Galindo 7a04116246
bpo-25460: Surround suggestions by quotes (GH-25473) 2021-04-19 23:35:53 +01:00
Christian Heimes d37b74f341
bpo-43669: More test_ssl cleanups (GH-25470)
Signed-off-by: Christian Heimes <christian@python.org>
2021-04-19 08:31:29 +02:00
Christian Heimes 2875c603b2
bpo-43880: Show DeprecationWarnings for deprecated ssl module features (GH-25455)
* ssl.OP_NO_SSLv2
* ssl.OP_NO_SSLv3
* ssl.OP_NO_TLSv1
* ssl.OP_NO_TLSv1_1
* ssl.OP_NO_TLSv1_2
* ssl.OP_NO_TLSv1_3
* ssl.PROTOCOL_SSLv2
* ssl.PROTOCOL_SSLv3
* ssl.PROTOCOL_SSLv23 (alias for PROTOCOL_TLS)
* ssl.PROTOCOL_TLS
* ssl.PROTOCOL_TLSv1
* ssl.PROTOCOL_TLSv1_1
* ssl.PROTOCOL_TLSv1_2
* ssl.TLSVersion.SSLv3
* ssl.TLSVersion.TLSv1
* ssl.TLSVersion.TLSv1_1
* ssl.wrap_socket()
* ssl.RAND_pseudo_bytes()
* ssl.RAND_egd() (already removed since it's not supported by OpenSSL 1.1.1)
* ssl.SSLContext() without a protocol argument
* ssl.match_hostname()
* hashlib.pbkdf2_hmac() (pure Python implementation, fast OpenSSL
  function will stay)

Signed-off-by: Christian Heimes <christian@python.org>
2021-04-19 07:27:10 +02:00
Christian Heimes 89d1550d14
bpo-42854: Use SSL_read/write_ex() (GH-25468)
The ssl module now uses ``SSL_read_ex`` and ``SSL_write_ex``
internally. The functions support reading and writing of data larger
than 2 GB. Writing zero-length data no longer fails with a protocol
violation error.

Signed-off-by: Christian Heimes <christian@python.org>
2021-04-19 06:55:30 +02:00
juhovh 49fdf118ae
bpo-36076: Add SNI support to ssl.get_server_certificate. (GH-16820)
Many servers in the cloud environment require SNI to be used during the
SSL/TLS handshake, therefore it is not possible to fetch their certificates
using the ssl.get_server_certificate interface.

This change adds an additional optional hostname argument that can be used to
set the SNI. Note that it is intentionally a separate argument instead of
using the host part of the addr tuple, because one might want to explicitly
fetch the default certificate or fetch a certificate from a specific IP
address with the specified SNI hostname. A separate argument also works better
for backwards compatibility.

Automerge-Triggered-By: GH:tiran
2021-04-18 04:11:48 -07:00
Pablo Galindo 0b1c169c4a
bpo-38530: Cover more error paths in error suggestion functions (GH-25462) 2021-04-17 23:28:45 +01:00
stratakis 685719871a
bpo-37630: Do not skip the sha3 tests in case of missing builtin sha3 module (GH-20986)
Since we can use now the sha3 and shake algorithms from OpenSSL, we shouldn't
skip the tests in case the builtin algorithm was not compiled or removed.
2021-04-17 14:27:07 -07:00
Pablo Galindo 3ab4bea5a3
bpo-38530: Include builtins in NameError suggestions (GH-25460) 2021-04-17 22:26:54 +01:00
Eric V. Smith 76beadb8ff
Fix typo in a dataclasses comment. (GH-25454) 2021-04-17 09:53:24 -04:00
Christian Heimes 39258d3595
bpo-43669: PEP 644: Require OpenSSL 1.1.1 or newer (GH-23014)
- Remove HAVE_X509_VERIFY_PARAM_SET1_HOST check
- Update hashopenssl to require OpenSSL 1.1.1
- multissltests only OpenSSL > 1.1.0
- ALPN is always supported
- SNI is always supported
- Remove deprecated NPN code. Python wrappers are no-op.
- ECDH is always supported
- Remove OPENSSL_VERSION_1_1 macro
- Remove locking callbacks
- Drop PY_OPENSSL_1_1_API macro
- Drop HAVE_SSL_CTX_CLEAR_OPTIONS macro
- SSL_CTRL_GET_MAX_PROTO_VERSION is always defined now
- security level is always available now
- get_num_tickets is available with TLS 1.3
- X509_V_ERR MISMATCH is always available now
- Always set SSL_MODE_RELEASE_BUFFERS
- X509_V_FLAG_TRUSTED_FIRST is always available
- get_ciphers is always supported
- SSL_CTX_set_keylog_callback is always available
- Update Modules/Setup with static link example
- Mention PEP in whatsnew
- Drop 1.0.2 and 1.1.0 from GHA tests
2021-04-17 11:36:35 +02:00
Christian Heimes b467d9a240
bpo-43522: Fix SSLContext.hostname_checks_common_name (GH-24899)
Fix problem with ssl.SSLContext.hostname_checks_common_name. OpenSSL does not
copy hostflags from *struct SSL_CTX* to *struct SSL*.

Signed-off-by: Christian Heimes <christian@python.org>
2021-04-17 10:07:19 +02:00
Victor Stinner 7c29ae1f05
bpo-43867: multiprocessing Server catchs SystemExit (GH-25441)
The multiprocessing Server class now explicitly catchs SystemExit and
closes the client connection in this case. It happens when the
Server.serve_client() method reachs the end of file (EOF).
2021-04-16 19:42:34 +02:00
Pablo Galindo 0ad81d4db2
bpo-38530: Match exactly AttributeError and NameError when offering suggestions (GH-25443) 2021-04-16 17:12:03 +01:00
Victor Stinner b136b1aac4
bpo-43843: libregrtest uses threading.excepthook (GH-25400)
test.libregrtest now marks a test as ENV_CHANGED (altered the
execution environment) if a thread raises an exception but does not
catch it. It sets a hook on threading.excepthook. Use
--fail-env-changed option to mark the test as failed.

libregrtest regrtest_unraisable_hook() explicitly flushs
sys.stdout, sys.stderr and sys.__stderr__.
2021-04-16 14:33:10 +02:00
Victor Stinner 75ec103b3a
bpo-43842: Fix race condition in test_logging SMTP test (GH-25436)
Fix a race condition in the SMTP test of test_logging. Don't close a
file descriptor (socket) from a different thread while
asyncore.loop() is polling the file descriptor.
2021-04-16 14:32:01 +02:00
Victor Stinner 471870fc50
bpo-43844: Fix PendingDeprecationWarning in test_lib2to3 (GH-25407) 2021-04-16 11:27:00 +02:00
Victor Stinner 69ca32e0d3
bpo-41282: Fix distutils.utils.byte_compile() DeprecationWarning (GH-25406)
* byte_compile() of distutils.utils no longer logs a
  DeprecationWarning
* test_distutils no longer logs a DeprecationWarning
2021-04-16 11:26:40 +02:00
Victor Stinner a6a5c91b1e
bpo-40443: Remove unused imports (GH-25429)
* pyclbr no longer uses copy
* typing no longer uses ast

Issue discovered by pyflakes.
2021-04-16 11:26:06 +02:00
Pablo Galindo b280248be8
bpo-43822: Improve syntax errors for missing commas (GH-25377) 2021-04-15 21:38:45 +01:00
Ethan Furman ec09973f5b
bpo-43744: [Enum] fix ``_is_private`` (GH-25349)
``_is_private`` now returns ``False`` instead of raising an exception when enum name matches beginning of class name
as used in a private variable
2021-04-15 06:58:33 -07:00
Ethan Furman 0dca5eb54b
[Enum] fix doc string (GH-25376) 2021-04-15 06:49:54 -07:00
Mark Shannon 11e0b295de
bpo-43846: Use less stack for large literals and calls (GH-25403)
* Modify compiler to reduce stack consumption for large expressions.

* Add more tests for stack usage.

* Add NEWS item.

* Raise SystemError for truly excessive stack use.
2021-04-15 14:28:56 +01:00
Pablo Galindo da74350174
bpo-43823: Improve syntax errors for invalid dictionary literals (GH-25378) 2021-04-15 14:06:39 +01:00