Commit Graph

24552 Commits

Author SHA1 Message Date
Terry Jan Reedy 1b4a9c7956
bpo-42225: IDLE - document two unix-related problems. (#25078)
1. Bad IP masquerade rules can prevent startup.
2. X cannot handle some complex colored chars.
2021-03-31 01:19:38 -04:00
Ethan Furman b775106d94
bpo-40066: Enum: modify `repr()` and `str()` (GH-22392)
* Enum: streamline repr() and str(); improve docs

- repr() is now ``enum_class.member_name``
- stdlib global enums are ``module_name.member_name``
- str() is now ``member_name``
- add HOW-TO section for ``Enum``
- change main documentation to be an API reference
2021-03-30 21:17:26 -07:00
Alex Prengère 51a85ddce8
bpo-43399: Fix ElementTree.extend not working on iterators (GH-24751) 2021-03-31 00:11:29 +03:00
Antoine Pitrou 73b20ae2fb
bpo-41369: Finish updating the vendored libmpdec to version 2.5.1 (GH-24962)
Complete the update to libmpdec-2.5.1.

Co-authored-by: Stefan Krah <skrah@bytereef.org>
2021-03-30 18:11:06 +02:00
Brett Cannon a7ff6df60c
bpo-42134: Raise ImportWarning when calling find_module() in the import system (GH-25044) 2021-03-30 08:43:03 -07:00
Grégory Starck cf35e05f89
bpo-43125: Fix: return expected type (str), not original value (bytes) in email/base64mime.py::body_encode (GH-24476) 2021-03-30 17:37:37 +09:00
Neil Schemenauer 85b6b70589
bpo-37448: Use radix tree for pymalloc address_in_range(). (GH-14474)
The radix tree approach is a relatively simple and memory sanitary
alternative to the old (slightly) unsanitary address_in_range().
To disable the radix tree map, set a preprocessor flag as follows:
-DWITH_PYMALLOC_RADIX_TREE=0.

Co-authored-by: Tim Peters <tim.peters@gmail.com>
2021-03-29 19:51:15 -07:00
Christian Heimes a54fc683f2
bpo-43631: Update to OpenSSL 1.1.1k (GH-25024)
- [x] Build OpenSSL 1.1.1k for macOS
- [x] Build OpenSSL 1.1.1k for Windows

I have also updated multissl tester and various CI configurations to use latest OpenSSL. The versions were all over the place.

Signed-off-by: Christian Heimes <christian@python.org>

Automerge-Triggered-By: GH:tiran
2021-03-29 17:00:34 -07:00
Pablo Galindo 09b90a037d
bpo-43660: Fix crash when displaying exceptions with custom values for sys.stderr (GH-25075) 2021-03-29 23:38:51 +01:00
Jesús Cea 32430aadad
bpo-35930: Raising an exception raised in a "future" instance will create reference cycles (#24995)
Before: https://lists.es.python.org/pipermail/general/attachments/20201229/0c14bc58/attachment-0002.png

After: https://lists.es.python.org/pipermail/general/attachments/20201229/0c14bc58/attachment-0003.png
2021-03-29 19:22:13 +02:00
Serhiy Storchaka c1b073a630
bpo-43433: Preserve query and fragment in the URL of the server in ServerProxy. (GH-25057) 2021-03-29 22:39:31 +09:00
Pablo Galindo 70cdf1812c
bpo-40645: Fix reference leak in the _hashopenssl extension (GH-25063) 2021-03-29 06:17:40 -07:00
Victor Stinner 9b999479c0
bpo-42988: Remove the pydoc getfile feature (GH-25015)
CVE-2021-3426: Remove the "getfile" feature of the pydoc module which
could be abused to read arbitrary files on the disk (directory
traversal vulnerability). Moreover, even source code of Python
modules can contain sensitive data like passwords. Vulnerability
reported by David Schwörer.
2021-03-29 14:40:40 +02:00
Inada Naoki 4827483f47
bpo-43510: Implement PEP 597 opt-in EncodingWarning. (GH-19481)
See [PEP 597](https://www.python.org/dev/peps/pep-0597/).

* Add `-X warn_default_encoding` and `PYTHONWARNDEFAULTENCODING`.
* Add EncodingWarning
* Add io.text_encoding()
* open(), TextIOWrapper() emits EncodingWarning when encoding is omitted and warn_default_encoding is enabled.
* _pyio.TextIOWrapper() uses UTF-8 as fallback default encoding used when failed to import locale module. (used during building Python)
* bz2, configparser, gzip, lzma, pathlib, tempfile modules use io.text_encoding().
* What's new entry
2021-03-29 12:28:14 +09:00
Christian Heimes 933dfd7504
bpo-40645: use C implementation of HMAC (GH-24920)
- [x] fix tests
- [ ] add test scenarios for old/new code.

Signed-off-by: Christian Heimes <christian@python.org>
2021-03-27 06:55:03 -07:00
Christian Heimes 5d6e8c1c1a
bpo-43617: Check autoconf-archive package in configure.ac (GH-25016)
Signed-off-by: Christian Heimes <christian@python.org>
2021-03-27 14:44:04 +01:00
Jürgen Gmach 027b669927
bpo-43354: xmlrpc: Fix type documentation for Fault.faultCode (GH-24707)
The type of `faultCode` has to be an `int` instead of a `str`.

cf http://xmlrpc.com/spec.md

Pinging @pganssle
2021-03-26 14:09:09 -07:00
Brett Cannon 1899087b21
bpo-42136: Deprecate module_repr() as found in importlib (GH-25022) 2021-03-26 11:55:07 -07:00
Pablo Galindo 8efad61963
bpo-41064: Improve syntax error for invalid usage of '**' in f-strings (GH-25006) 2021-03-24 19:34:17 +00:00
Mark Shannon 4958f5d69d
Only check evalbreaker after calls and on backwards egdes. Makes sure that __exit__ or __aexit__ is called in with statments in case of interrupt. (GH-18334) 2021-03-24 17:56:12 +00:00
Brett Cannon 9cb31d6716
bpo-42137: have ModuleType.__repr__ prefer __spec__ over module_repr() (GH-24953)
This is to work towards the removal of the use of  module_repr() in Python 3.12 (documented as deprecated since 3.4).
2021-03-24 08:26:56 -07:00
sblondon 3ba3d513b1
bpo-42914: add a pprint underscore_numbers option (GH-24864)
pprint() gains a new boolean underscore_numbers kwarg to emit
integers with thousands separated by an underscore character
for improved readability (for example 1_000_000 instead of 1000000).
2021-03-24 01:23:20 -07:00
Victor Stinner 8370e07e1e
bpo-43244: Remove the pyarena.h header (GH-25007)
Remove the pyarena.h header file with functions:

* PyArena_New()
* PyArena_Free()
* PyArena_Malloc()
* PyArena_AddPyObject()

These functions were undocumented, excluded from the limited C API,
and were only used internally by the compiler.

Add pycore_pyarena.h header. Rename functions:

* PyArena_New() => _PyArena_New()
* PyArena_Free() => _PyArena_Free()
* PyArena_Malloc() => _PyArena_Malloc()
* PyArena_AddPyObject() => _PyArena_AddPyObject()
2021-03-24 02:23:01 +01:00
Victor Stinner 57364ce34e
bpo-43244: Remove parser_interface.h header file (GH-25001)
Remove parser functions using the "struct _mod" type, because the
AST C API was removed:

* PyParser_ASTFromFile()
* PyParser_ASTFromFileObject()
* PyParser_ASTFromFilename()
* PyParser_ASTFromString()
* PyParser_ASTFromStringObject()

These functions were undocumented and excluded from the limited C
API.

Add pycore_parser.h internal header file. Rename functions:

* PyParser_ASTFromFileObject() => _PyParser_ASTFromFile()
* PyParser_ASTFromStringObject() => _PyParser_ASTFromString()

These functions are no longer exported (replace PyAPI_FUNC() with
extern).

Remove also _PyPegen_run_parser_from_file() function. Update
test_peg_generator to use _PyPegen_run_parser_from_file_pointer()
instead.
2021-03-24 01:29:09 +01:00
Victor Stinner a81fca6ec8
bpo-43244: Add pycore_compile.h header file (GH-25000)
Remove the compiler functions using "struct _mod" type, because the
public AST C API was removed:

* PyAST_Compile()
* PyAST_CompileEx()
* PyAST_CompileObject()
* PyFuture_FromAST()
* PyFuture_FromASTObject()

These functions were undocumented and excluded from the limited C API.

Rename functions:

* PyAST_CompileObject() => _PyAST_Compile()
* PyFuture_FromASTObject() => _PyFuture_FromAST()

Moreover, _PyFuture_FromAST() is no longer exported (replace
PyAPI_FUNC() with extern). _PyAST_Compile() remains exported for
test_peg_generator.

Remove also compatibility functions:

* PyAST_Compile()
* PyAST_CompileEx()
* PyFuture_FromAST()
2021-03-24 00:51:50 +01:00
Joshua Bronson f0a6fde882
bpo-31861: Add aiter and anext to builtins (#23847)
Co-authored-by: jab <jab@users.noreply.github.com>
Co-authored-by: Daniel Pope <mauve@mauveweb.co.uk>
Co-authored-by: Justin Wang <justin39@gmail.com>
2021-03-23 15:47:21 -07:00
Victor Stinner 94faa0724f
bpo-43244: Remove ast.h, asdl.h, Python-ast.h headers (GH-24933)
These functions were undocumented and excluded from the limited C
API.

Most names defined by these header files were not prefixed by "Py"
and so could create names conflicts. For example, Python-ast.h
defined a "Yield" macro which was conflict with the "Yield" name used
by the Windows <winbase.h> header.

Use the Python ast module instead.

* Move Include/asdl.h to Include/internal/pycore_asdl.h.
* Move Include/Python-ast.h to Include/internal/pycore_ast.h.
* Remove ast.h header file.
* pycore_symtable.h no longer includes Python-ast.h.
2021-03-23 20:47:40 +01:00
Pablo Galindo 96eeff5162
bpo-43555: Report the column offset for invalid line continuation character (GH-24939) 2021-03-22 17:28:11 +00:00
Dong-hee Na 86883d40e9
bpo-43575: Use PEP 590 vectorcall to speed up map() (GH-24955) 2021-03-22 19:01:14 +09:00
Sergey B Kirpichev 690aca7811
bpo-43420: Simple optimizations for Fraction's arithmetics (GH-24779)
bpo-43420: Implement standard transformations in + - * / that can often reduce the size of intermediate integers needed. For rationals with large components, this can yield dramatic speed improvements, but for small rationals can run 10-20% slower, due to increased fixed overheads in the longer-winded code. If those slowdowns turn out to be a problem, see the PR discussion for low-level implementation tricks that could cut other fixed overheads.

Co-authored-by: Tim Peters <tim.peters@gmail.com>
Co-authored-by: Mark Dickinson <dickinsm@gmail.com>
2021-03-21 21:30:55 -05:00
Antoine Pitrou cdddc2b742
bpo-43422: Revert _decimal C API addition (GH-24960)
Stefan Krah requested the reversal of these (unreleased) changes, quoting him:
> The capsule API does not meet my testing standards, since I've focused
on the upstream mpdecimal in the last couple of months.
> Additionally, I'd like to refine the API, perhaps together with the
Arrow community.

Automerge-Triggered-By: GH:pitrou
2021-03-21 09:27:54 -07:00
Christian Heimes 77cde5042a
bpo-43577: Fix deadlock with SSLContext._msg_callback and sni_callback (GH-24957)
OpenSSL copies the internal message callback from SSL_CTX->msg_callback to
SSL->msg_callback. SSL_set_SSL_CTX() does not update SSL->msg_callback
to use the callback value of the new context.

PySSL_set_context() now resets the callback and _PySSL_msg_callback()
resets thread state in error path.

Signed-off-by: Christian Heimes <christian@python.org>
2021-03-21 16:13:09 +01:00
Ilya Stepin 20a5b7e986
bpo-43542: Add heif/heic formats in mimetypes (GH-24917)
* bpo-43542: Add heif/heic formats in mimetypes

Add HEIF and HEIC format to list of media types. It has IANA registration.

IANA: https://www.iana.org/assignments/media-types/image/heic
HEIF Github: https://github.com/nokiatech/heif

Co-authored-by: Gregory P. Smith <greg@krypto.org>
2021-03-20 16:06:27 -07:00
Rui Cunha b05b48dd7a
bpo-43571: Add IPPROTO_MPTCP macro (GH-24946)
Add IPPROTO_MPTCP constant when defined in the system headers.
2021-03-20 15:04:56 -07:00
Dino Viehland ee48c7d541
bpo-43452: Micro-optimizations to PyType_Lookup (GH-24804)
The common case going through _PyType_Lookup is to have a cache hit. There are some small tweaks that can make this a little cheaper:

* The name field identity is used for a cache hit and is kept alive by the cache. So there's no need to read the hash code o the name - instead, the address can be used as the hash.

*  There's no need to check if the name is cachable on the lookup either, it probably is, and if it is, it'll be in the cache.

*  If we clear the version tag when invalidating a type then we don't actually need to check for a valid version tag bit.
2021-03-20 19:12:05 +00:00
Antoine Pitrou 2fd16ef406
bpo-43517: Fix false positive in detection of circular imports (#24895) 2021-03-20 20:07:44 +01:00
Victor Stinner 28ad12f8fe
bpo-43244: Remove symtable.h header file (GH-24910)
Rename Include/symtable.h to to Include/internal/pycore_symtable.h,
don't export symbols anymore (replace PyAPI_FUNC and PyAPI_DATA with
extern) and rename functions:

* PyST_GetScope() to _PyST_GetScope()
* PySymtable_BuildObject() to _PySymtable_Build()
* PySymtable_Free() to _PySymtable_Free()

Remove PySymtable_Build(), Py_SymtableString() and
Py_SymtableStringObject() functions.

The Py_SymtableString() function was part the stable ABI by mistake
but it could not be used, since the symtable.h header file was
excluded from the limited C API.

The Python symtable module remains available and is unchanged.
2021-03-19 12:41:49 +01:00
Christian Heimes 32eba61ea4
bpo-43466: Add --with-openssl-rpath configure option (GH-24820) 2021-03-19 10:29:25 +01:00
Christian Heimes f6c6b5821b
bpo-41561: Add workaround for Ubuntu's custom security level (GH-24915)
Ubuntu 20.04 comes with a patched OpenSSL 1.1.1. Default security level
2 blocks TLS 1.0 and 1.1 connections. Regular OpenSSL 1.1.1 builds allow
TLS 1.0 and 1.1 on security level 2.

See: 
See: https://bugs.launchpad.net/ubuntu/+source/openssl/+bug/1899878
See: https://bugs.launchpad.net/ubuntu/+source/openssl/+bug/1917625
Signed-off-by: Christian Heimes <christian@python.org>
2021-03-18 15:06:50 -07:00
Kodi Arfer 08ff4369af
bpo-43521: Allow ast.unparse with empty sets and NaN (GH-24897)
Automerge-Triggered-By: GH:pablogsal
2021-03-18 10:36:06 -07:00
Victor Stinner eec8e61992
bpo-43244: Remove the PyAST_Validate() function (GH-24911)
Remove the PyAST_Validate() function. It is no longer possible to
build a AST object (mod_ty type) with the public C API. The function
was already excluded from the limited C API (PEP 384).

Rename PyAST_Validate() function to _PyAST_Validate(), move it to the
internal C API, and don't export it anymore (replace PyAPI_FUNC with
extern).

The function was added in bpo-12575 by
the commit 832bfe2ebd.
2021-03-18 14:57:49 +01:00
Victor Stinner fc980e0be1
bpo-43541: Fix PyEval_EvalCodeEx() regression (GH-24918)
* Remove an assertion which required CO_NEWLOCALS and CO_OPTIMIZED
  code flags. It is ok to call this function on a code with these
  flags set.
* Fix reference counting on builtins: remove Py_DECREF().
  Fix regression introduced in the
  commit 46496f9d12.

Add also a comment to document that _PyEval_BuiltinsFromGlobals()
returns a borrowed reference.
2021-03-18 14:51:24 +01:00
Chris Burr e0b4aa0f5c
bpo-39342: Expose X509_V_FLAG_ALLOW_PROXY_CERTS in ssl module (GH-18011)
Exposes the `X509_V_FLAG_ALLOW_PROXY_CERTS` constant as `ssl.VERIFY_ALLOW_PROXY_CERTS` to allow for proxy certificate validation as described in: https://www.openssl.org/docs/man1.1.1/man7/proxy-certificates.html
2021-03-18 01:24:01 -07:00
Victor Stinner 9976834f80
bpo-35883: Py_DecodeLocale() escapes invalid Unicode characters (GH-24843)
Python no longer fails at startup with a fatal error if a command
line argument contains an invalid Unicode character.

The Py_DecodeLocale() function now escapes byte sequences which would
be decoded as Unicode characters outside the [U+0000; U+10ffff]
range.

Use MAX_UNICODE constant in unicodeobject.c.
2021-03-17 21:46:53 +01:00
tsukasa-au a8ef4572a6
bpo-43497: Emit SyntaxWarnings for assertions with tuple constants. (GH-24867)
* bpo-43497: Emit SyntaxWarnings for assertions with tuple constants.

Add a test that shows that a tuple constant (a tuple, where all of its
members are also compile-time constants) produces a SyntaxWarning. Then
fix this failure.

* Make SyntaxWarnings also work when "optimized".

* Split tests for SyntaxWarning to SyntaxError conversion

SyntaxWarnings emitted by the compiler when configured to be errors are
actually raised as SyntaxError exceptions.

Move these tests into their own method and add a test to ensure they are
raised. Previously we only tested that they were not raised for a
"valid" assertion statement.
2021-03-16 11:14:41 +00:00
Dennis Sweeney 448801da96
bpo-41361: Optimized argument parsing for deque_rotate (GH-24796) 2021-03-15 21:02:25 -07:00
Chavdar Yotov 0269ce87c9
bpo-41933: Clarify wording for s * n in Common Sequence Operations (GH-22570) 2021-03-15 23:50:49 +01:00
Gregory P. Smith 0ab152c6b5
bpo-43285 Make ftplib not trust the PASV response. (GH-24838)
bpo-43285: Make ftplib not trust the PASV response.

The IPv4 address value returned from the server in response to the PASV command
should not be trusted.  This prevents a malicious FTP server from using the
response to probe IPv4 address and port combinations on the client network.

Instead of using the returned address, we use the IP address we're
already connected to.  This is the strategy other ftp clients adopted,
and matches the only strategy available for the modern IPv6 EPSV command
where the server response must return a port number and nothing else.

For the rare user who _wants_ this ugly behavior, set a `trust_server_pasv_ipv4_address`
attribute on your `ftplib.FTP` instance to True.
2021-03-15 11:39:31 -07:00
Terry Jan Reedy 5e29021a5e
bpo-43199: Briefly explain why no goto (GH-24852)
Answer "Why is there no goto?" in the Design and History FAQ.
2021-03-14 18:12:04 -04:00
CAM Gerlach bd2fa3c416
bpo-29982: Add "ignore_cleanup_errors" param to tempfile.TemporaryDirectory() (GH-24793) 2021-03-14 11:06:56 -07:00
Mark Shannon d48848c83e
bpo-39316: Make sure that attribute accesses and stores, including method calls, conform to PEP 626. (GH-24859) 2021-03-14 18:01:30 +00:00
Pablo Galindo cd8dcbc851
bpo-43410: Fix crash in the parser when producing syntax errors when reading from stdin (GH-24763) 2021-03-14 04:38:40 +01:00
Kamil Turek 9923df9641
bpo-43245: Add keyword argument support to ChainMap.new_child() (GH-24788) 2021-03-13 19:15:44 -08:00
Jason R. Coombs f917efccf8
bpo-43428: Sync with importlib_metadata 3.7. (GH-24782)
* bpo-43428: Sync with importlib_metadata 3.7.2 (67234b6)

* Add blurb

* Reformat blurb to create separate paragraphs for each change included.
2021-03-13 11:31:45 -05:00
Pandu E POLUAN 7591d9455e
bpo-27820: Fix AUTH LOGIN logic in smtplib.SMTP (GH-24118)
* Fix auth_login logic (bpo-27820)

* Also fix a longstanding bug in the SimSMTPChannel.found_terminator() method that causes inability to test
SMTP AUTH with initial_response_ok=False.
2021-03-12 15:25:49 -08:00
Antoine Pitrou ba251c2ae6
bpo-43356: Allow passing a signal number to interrupt_main() (GH-24755)
Also introduce a new C API ``PyErr_SetInterruptEx(int signum)``.
2021-03-11 23:35:45 +01:00
Chris Griffith b4fc44bb2d
bpo-43423 Fix IndexError in subprocess _communicate function (GH-24777)
Check to make sure stdout and stderr are not empty before selecting an item from them in Windows subprocess._communicate.

Co-authored-by: Gregory P. Smith <greg@krypto.org>
2021-03-11 11:43:29 -08:00
Dong-hee Na 9a9c11ad41
bpo-43287: Use PEP 590 vectorcall to speed up filter() (GH-24611) 2021-03-11 01:39:52 +09:00
Victor Stinner 307745aa42
bpo-43445: Add frozen modules to sys.stdlib_module_names (GH-24798)
Add frozen modules to sys.stdlib_module_names. For example, add
"_frozen_importlib" and "_frozen_importlib_external" names.

Add "list_frozen" command to Programs/_testembed.
2021-03-10 11:14:07 +01:00
Pablo Galindo b4f9089d4a
bpo-43439: Add audit hooks for gc functions (GH-24794) 2021-03-10 00:53:57 +00:00
Erlend Egeberg Aasland 31818e98d3
bpo-43440 : Enable SQLite R*Tree support for windows builds (GH-24797) 2021-03-09 20:59:44 +00:00
Desmond Cheong 3abf6f0102
bpo-14678: Update zipimport to support importlib.invalidate_caches() (GH-24159)
Added an invalidate_caches() method to the zipimport.zipimporter class based on the implementation of importlib.FileFinder.invalidate_caches(). This was done by adding a get_files() method and an _archive_mtime attribute to zipimport.zipimporter to check for updates or cache invalidation whenever the cache of files and toc entry information in the zipimporter is accessed.
2021-03-08 12:06:02 -08:00
Gregory P. Smith c25910a135
bpo-43332: Buffer proxy connection setup packets before sending. (GH-24780)
We now buffer the CONNECT request + tunnel HTTP headers into a single
send call.  This prevents the OS from generating multiple network
packets for connection setup when not necessary, improving efficiency.
2021-03-07 23:35:13 -08:00
Neil Schemenauer 87ec26b812
bpo-43372: Use _freeze_importlib for regen-frozen. (GH-24759)
This approach ensures the code matches the interpreter version.
Previously, PYTHON_FOR_REGEN was used to generate the code, which might
be wrong. The marshal format for code objects has changed with
bpo-42246, commit 877df851. Update the code and the expected code sizes
in ctypes test_frozentable.
2021-03-06 13:34:03 -08:00
Alex Willmer ff5f05934d
bpo-43407: Clarify comparisons of time.monotonic() et al results (GH-24757)
Previous wording implied that only the result of call N and N+1 could be
meaningfully compared, whereas comparing call N and N+M is fine.
2021-03-05 20:22:13 -05:00
Antoine Pitrou 68245b7a10
bpo-43406: Fix possible race condition where ``PyErr_CheckSignals`` tries to execute a non-Python signal handler (GH-24756)
We can receive signals (at the C level, in `trip_signal()` in signalmodule.c) while `signal.signal` is being called to modify the corresponding handler.  Later when `PyErr_CheckSignals()` is called to handle the given signal, the handler may be a non-callable object and would raise a cryptic asynchronous exception.
2021-03-05 10:32:50 +01:00
Gregory P. Smith 02ac6f41e5
bpo-43390: Set SA_ONSTACK in PyOS_setsig (GH-24730)
This is friendlier to other in-process code that an extension module or
embedding use could pull in such as CGo where tiny stacks are the norm
and sigaltstack() has been used to provide for signal handlers.

Without this, signals received by a process using tiny stacks may lead
to stack overflow crashes.
2021-03-04 21:49:30 -08:00
Jason R. Coombs 6714825414
bpo-42129: Add support for resources in namespaces (GH-24670)
* Unify behavior in ResourceReaderDefaultsTests and align with the behavior found in importlib_resources.
* Equip NamespaceLoader with a NamespaceReader.
* Apply changes from importlib_resources 5.0.4
2021-03-04 13:43:00 -05:00
Jason R. Coombs fbf75b9997
Revert "bpo-42405: fix C extensions build on Windows ARM64 (GH-23399)" (#24753)
This reverts commit cb7bc76409.
2021-03-04 13:41:49 -05:00
Adrian Vladu cb7bc76409
bpo-42405: fix C extensions build on Windows ARM64 (GH-23399)
The following changes are required:

      * add a new platform win-arm64
      * replace the emulated compiler executable paths
      * bump the linker base addressed as ARM64 requires more memory
        this change might not be needed (investigation required)

    On Windows 10 ARM64, VS compiler paths look like this:
    C:\Program Files (x86)\Microsoft Visual
    Studio\2019\Community\VC\Tools\MSVC\14.27.29110\bin\HostX86\ARM64\cl.exe

    Note that the cl.exe for ARM64 is an x32 binary, which can run emulated
    on Windows 10 ARM64 (it has builtin emulation for x32).

    The rc.exe and mc.exe paths have to also be changed, as the initial
    discovery has to be fixed.

    Work in progress to remove the hardcoded bits and to change the path
    query fixes to the proper location.

Automerge-Triggered-By: GH:jaraco
2021-03-04 08:59:12 -08:00
Erlend Egeberg Aasland e161ec5dd7
bpo-43369: sqlite3_column_{text,blob} failures now raise MemoryError (GH-24723) 2021-03-04 11:50:25 +02:00
Pablo Galindo 0632b1012d
bpo-42128: Add __match_args__ to structseq-based classes (GH-24732) 2021-03-04 00:03:44 +00:00
Eric L 9c7927400c
bpo-40701: tempfile mixes str and bytes in an inconsistent manner (GH-20442)
The case of tempfile.tempdir variable being bytes is now handled consistently.
The getters return the right type and no more error of mixing str and bytes unless explicitly caused by the user.

Adds a regression test.

Expands the documentation to clarify the behavior.

Co-authored-by: Eric L <ewl+git@lavar.de>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
2021-03-03 12:36:22 -08:00
Noor Michael 04f6fbb696
bpo-43295: Fix error handling of datetime.strptime format string '%z' (GH-24627)
Previously, `datetime.strptime` would match `'z'` with the format string `'%z'` (for UTC offsets), throwing an `IndexError` by erroneously trying to parse `'z'` as a timestamp. As a special case, `'%z'` matches the string `'Z'` which is equivalent to the offset `'+00:00'`, however this behavior is not defined for lowercase `'z'`.

This change ensures a `ValueError` is thrown when encountering the original example, as follows:

```
>>> from datetime import datetime
>>> datetime.strptime('z', '%z')
ValueError: time data 'z' does not match format '%z'
```

Automerge-Triggered-By: GH:pganssle
2021-03-03 08:58:57 -08:00
Mariusz Felisiak 3b4b2cf418
bpo-43368: Fix fetching empty bytes in sqlite3 (GH-24706)
Regression introduced in 47feb1feb2.
2021-03-03 16:16:24 +02:00
Nathan Beals 3a87e562ea
bpo-42994: Add MIME types for opus, AAC, 3gpp and 3gpp2 (#24287)
Co-authored-by: Nathan Beals <ndbeals@users.noreply.github.com>
2021-03-02 16:20:18 -08:00
Winson Luk 132131b404
bpo-42782: Fail fast for permission errors in shutil.move() (GH-24001)
* Fail fast in shutil.move() to avoid creating destination directories on failure.

Co-authored-by: Zackery Spytz <zspytz@gmail.com>
2021-03-02 12:53:15 -08:00
Inada Naoki 5bfa945605
bpo-41282: distutils: Fix stacklevel for DeprecationWarning (GH-24657)
DeprecationWarning is emit for `import distutils`, not for `distutils` itself.
2021-03-02 11:49:10 +09:00
Pablo Galindo 06c245fb67
Python 3.10.0a6 2021-03-01 16:45:40 +00:00
Ned Deily 0242494a15
bpo-41837: Update macOS installer build to use OpenSSL 1.1.1j. (GH-24677) 2021-03-01 02:39:08 -05:00
Ned Deily a65b050516
bpo-42603: Add whatsnew and ACKS entries. (GH-24675) 2021-03-01 00:27:20 -05:00
Manolis Stamatogiannakis d20279494a
bpo-42603: Use pkg-config to get TCL/TK paths for tkinter. (GH-23721) 2021-02-28 22:29:57 -05:00
Filipe Laíns 0d7ad9fb38
bpo-29753: fix merging packed bitfields in ctypes struct/union (GH-19850)
From the commit message:

> When the structure is packed we should always expand when needed,
> otherwise we will add some padding between the fields. This patch makes
> sure we always merge bitfields together. It also changes the field merging
> algorithm so that it handles bitfields correctly.

Automerge-Triggered-By: GH:jaraco
2021-02-28 14:43:19 -08:00
Dennis Sweeney 73a85c4e1d
bpo-41972: Use the two-way algorithm for string searching (GH-22904)
Implement an enhanced variant of Crochemore and Perrin's Two-Way string searching algorithm, which reduces worst-case time from quadratic (the product of the string and pattern lengths) to linear. This applies to forward searches (like``find``, ``index``, ``replace``); the algorithm for reverse searches (like ``rfind``) is not changed.

Co-authored-by: Tim Peters <tim.peters@gmail.com>
2021-02-28 12:20:50 -06:00
Erlend Egeberg Aasland 2183d06bc8
bpo-43251: sqlite3_column_name() failures now raise MemoryError (GH-24609) 2021-02-28 19:01:06 +02:00
Inada Naoki c71d24f558
bpo-43321: Fix SystemError in getargs.c (GH-24656) 2021-02-27 20:31:03 +09:00
Brandt Bucher 145bf269df
bpo-42128: Structural Pattern Matching (PEP 634) (GH-22917)
Co-authored-by: Guido van Rossum <guido@python.org>
Co-authored-by: Talin <viridia@gmail.com>
Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
2021-02-26 14:51:55 -08:00
Alex cc02b4f2e8
bpo-38302: __pow__/__rpow__ now called when __ipow__ returns NotImplemented (#16459) 2021-02-26 11:58:39 -08:00
Ruben Vorderman 7956ef8849
bpo-43317: Use io.DEFAULT_BUFFER_SIZE instead of 1024 in gzip CLI (#24645)
This improves the performance slightly.
2021-02-26 21:17:51 +09:00
Joseph Shen 28a30bc2e2
closes bpo-43278: remove unnecessary leading '\n' from COMPILER when build with GCC/Clang (GH-24606)
Automerge-Triggered-By: GH:benjaminp
2021-02-25 20:24:21 -08:00
Ruben Vorderman cc3df6368d
bpo-43316: gzip: CLI uses non-zero return code on error. (GH-24647)
Exit code is now 1 instead of 0. A message is printed to stderr instead of stdout. This is
the proper behaviour for a tool that can be used in scripts.
2021-02-25 20:30:24 +09:00
Felix C. Stegerman 1f433406bd
bpo-42151: don't set specified_attributes=1 in pure Python ElementTree (GH-22987) 2021-02-24 11:25:31 +09:00
Terry Jan Reedy 2827e8a177
bpo-43283: Add IDLE doc paragraph about print speed (GH-24615)
Printing to IDLE's Shell is often slower than printing to a system
terminal, but it can be made faster by pre-formatting a single
string before printing.
2021-02-23 19:39:51 -05:00
Irit Katriel b798ab0693
bpo-43146: fix None-handling in single-arg traceback.print_exception(None) (GH-24629)
(The previous commit fixed print_exception(None, None, None).)
2021-02-23 09:43:04 -08:00
Irit Katriel 26f18b8540
bpo-43146: fix regression in traceback.print_exception(None) (GH-24463) 2021-02-23 06:58:47 -08:00
Petr Viktorin 7bb1cafa4e
bpo-43239: Export PyCFunction_New with PyAPI_FUNC (GH-24551) 2021-02-23 13:23:56 +01:00
Inada Naoki 2d6f2eed14
bpo-36346: Document removal schedule of deprecate APIs (GH-20879)
We will remove wstr cache in Python 3.12. See PEP 623.
2021-02-23 08:06:51 +09:00
Inada Naoki 91a639a094
bpo-36346: Emit DeprecationWarning for PyArg_Parse() with 'u' or 'Z'. (GH-20927)
Emit DeprecationWarning when PyArg_Parse*() is called with 'u', 'Z' format.

See PEP 623.
2021-02-22 22:11:48 +09:00
Dennis Sweeney b19855bb6f
bpo-42808: Add PyType_Type.tp_vectorcall for type(obj) performance (GH-24058) 2021-02-22 11:59:16 +09:00
Inada Naoki 01806d5beb
bpo-43260: io: Prevent large data remains in textio buffer. (GH-24592)
When very large data remains in TextIOWrapper, flush() may fail forever.

So prevent that data larger than chunk_size is remained in TextIOWrapper internal
buffer.

Co-Authored-By: Eryk Sun
2021-02-22 08:29:30 +09:00
Neil Schemenauer 50288aa8c9
bpo-43288: Fix bug in test_importlib test. (GH-24612) 2021-02-21 13:48:18 -08:00
Terry Jan Reedy 2f9ef514fb
bpo-27646: Say that 'yield from' expression can be any iterable (GH-24595)
Previously, the doc at least strongly implied that it had to be an iterator.
2021-02-20 21:33:25 -05:00
Pablo Galindo d439fb304c
bpo-43277: Add PySet_CheckExact to the C-API (GH-24598)
For some mysterious reason we have PySet_Check, PyFrozenSet_Check, PyAnySet_Check, PyAnySet_CheckExact and PyFrozenSet_CheckExact but no PySet_CheckExact.
2021-02-20 18:03:08 +00:00
Victor Stinner 46496f9d12
bpo-42990: Functions inherit current builtins (GH-24564)
The types.FunctionType constructor now inherits the current builtins
if the globals dictionary has no "__builtins__" key, rather than
using {"None": None} as builtins: same behavior as eval() and exec()
functions.

Defining a function with "def function(...): ..." in Python is not
affected, globals cannot be overriden with this syntax: it also
inherits the current builtins.

PyFrame_New(), PyEval_EvalCode(), PyEval_EvalCodeEx(),
PyFunction_New() and PyFunction_NewWithQualName() now inherits the
current builtins namespace if the globals dictionary has no
"__builtins__" key.

* Add _PyEval_GetBuiltins() function.
* _PyEval_BuiltinsFromGlobals() now uses _PyEval_GetBuiltins() if
  builtins cannot be found in globals.
* Add tstate parameter to _PyEval_BuiltinsFromGlobals().
2021-02-20 15:17:18 +01:00
Nicholas Sim 4a6bf276ed
bpo-35134: Move non-limited C API files to Include/cpython/ (GH-24561)
Include/{odictobject.h,parser_interface.h,picklebufobject.h,pydebug.h,pyfpe.h}
into Include/cpython/.

Parser: peg_api: include Python.h instead of parser_interface.h.
2021-02-19 15:55:46 +01:00
Victor Stinner a486054b24
bpo-43270: Remove private _PyErr_OCCURRED() macro (GH-24579)
Remove the private _PyErr_OCCURRED() macro: use the public
PyErr_Occurred() function instead.

CPython internals must use the internal _PyErr_Occurred(tstate)
function instead: it is the most efficient way to check if an
exception was raised.
2021-02-19 15:08:54 +01:00
Erlend Egeberg Aasland 979b23cbe4
bpo-43258: Don't allocate sqlite3 aggregate context for empty queries (GH-24569) 2021-02-19 13:20:32 +02:00
Neil Schemenauer ffa55d21b4
bpo-39448: Add regen-frozen makefile target. (GH-18174)
Add the "regen-frozen" makefile target that regenerates the code for the
frozen __hello__ module.
2021-02-18 16:49:12 -08:00
David CARLIER 7be00ee64a
bpo-42960: Add resource.RLIMIT_KQUEUES constant from FreeBSD (GH-24251) 2021-02-18 16:26:20 +01:00
Victor Stinner a3c3ffa68e
bpo-42990: Add __builtins__ attribute to functions (GH-24559)
Expose the new PyFunctionObject.func_builtins member in Python as a
new __builtins__ attribute on functions.

Document also the behavior change in What's New in Python 3.10.
2021-02-18 12:35:37 +01:00
Nicholas Sim 366dc3a135
bpo-35134: Move Include/{pyarena.h,pyctype.h} to Include/cpython/ (GH-24550)
Move non-limited C API headers pyarena.h and pyctype.h
into Include/cpython/ directory.
2021-02-17 19:30:31 +01:00
Erlend Egeberg Aasland 630264a152
bpo-40170: Move 3 NEWS entries to the C API section (GH-24555) 2021-02-17 19:15:39 +01:00
Erlend Egeberg Aasland cd80f430da
bpo-40170: Always define PyExceptionClass_Name() as a function (GH-24553)
Remove macro variant of PyExceptionClass_Name().
2021-02-17 11:51:08 +01:00
Victor Stinner 801bb0b503
bpo-43103: Add configure --without-static-libpython (GH-24418)
Add a new configure --without-static-libpython option to not build
the libpythonMAJOR.MINOR.a static library and not install the
python.o object file.

Fix smelly.py and stable_abi.py tools when libpython3.10.a is
missing.
2021-02-17 11:14:42 +01:00
Erlend Egeberg Aasland cc54001c2e
bpo-40170: Always define PyIter_Check() as a function (GH-24548) 2021-02-16 16:05:58 +01:00
Erlend Egeberg Aasland 871eb4237b
bpo-40170: Convert PyDescr_IsData() to static inline function (GH-24535) 2021-02-16 08:50:00 +01:00
Zackery Spytz 8a8b5df93f
bpo-43155: Add PyCMethod_New to PC/python3dll.c (GH-24500) 2021-02-16 01:14:13 +01:00
Dustin Rodrigues 755f3c1521
bpo-42819, readline: Disable bracketed paste (GH-24108) 2021-02-16 00:28:24 +01:00
Erlend Egeberg Aasland 4bb2a1ebc5
bpo-43181: Convert PyObject_TypeCheck to static inline function (GH-24533) 2021-02-15 17:19:24 +01:00
Adam Goldschmidt fcbe0cb04d
bpo-42967: only use '&' as a query string separator (#24297)
bpo-42967: [security] Address a web cache-poisoning issue reported in urllib.parse.parse_qsl().

urllib.parse will only us "&" as query string separator by default instead of both ";" and "&" as allowed in earlier versions. An optional argument seperator with default value "&" is added to specify the separator.


Co-authored-by: Éric Araujo <merwok@netwok.org>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Ken Jin <28750310+Fidget-Spinner@users.noreply.github.com>
Co-authored-by: Éric Araujo <merwok@netwok.org>
2021-02-14 14:41:57 -08:00
Gregory P. Smith fd053fdd39
bpo-43172: readline now passes its tests when built against libedit (GH-24499)
bpo-43172: readline now passes its tests when built against libedit.

Existing irreconcilable API differences remain in readline.get_begidx
and readline.get_endidx behavior based on libreadline vs libedit use.
A note about that has been documented.
2021-02-12 12:04:46 -08:00
Inada Naoki fedd86df24
bpo-43174: Windows: Use /utf-8 compiler option. (GH-24498) 2021-02-12 09:06:47 +09:00
Erlend Egeberg Aasland ea46579067
bpo-40956: Fix segfault when Connection.backup is called without target (GH-24503) 2021-02-11 01:04:02 +02:00
Inada Naoki bdb941be42
bpo-42217: compiler: merge same co_code and co_linetable objects (GH-23056) 2021-02-10 09:20:42 +09:00
Pablo Galindo dbb228189b
bpo-43163: Handle unclosed parentheses in codeop (GH-24483) 2021-02-09 20:07:38 +00:00
Ethan Furman d65b9033d6
bpo-43162: [Enum] deprecate enum member.member access (GH-24486)
In 3.5 (?) a speed optimization made it possible to access members as
attributes of other members, i.e. ``Color.RED.BLUE``.  This was always
discouraged in the docs, and other recent optimizations has made that
one no longer necessary.  Because some may be relying on it anyway, it
is being deprecated in 3.10, and will be removed in 3.11.
2021-02-08 17:32:38 -08:00
Roland Hieber e1f7769513
bpo-13501: allow choosing between readline and libedit (GH-24189)
In contrast to macOS, libedit is available as its own include file and
library on Linux systems to prevent file name clashes. So if both
libraries are available on the system, readline is currently chosen by
default; and if only libedit is available, it is not found at all. This
patch adds a way to link against libedit by adding the following
arguments to configure:

  --with-readline           link against libreadline (the default)
  --with-readline=editline  link against libeditline
  --with-readline=no        disable building the readline module
  --without-readline        (same)

The runtime detection of libedit vs. readline was already done in commit
7105319ada (2019-12-04, serge-sans-paille: "bpo-38634: Allow
non-apple build to cope with libedit (GH-16986)").

Fixes: GH-12076 ("bpo-13501 Build or disable readline with Editline")
Fixes: bpo-13501 ("Make libedit support more generic; port readline / libedit to FreeBSD")
Co-authored-by: Enji Cooper (ngie-eign)
Co-authored-by: Martin Panter (vadmium)
Co-authored-by: Robert Marshall (kellinm)
2021-02-08 17:05:25 -08:00
Asheesh Laroia bf2e7e55d7
bpo-40692: Run more test_concurrent_futures tests (GH-20239)
In the case of multiprocessing.synchronize() being missing, the
test_concurrent_futures test suite now skips only the tests that
require multiprocessing.synchronize().

Validate that multiprocessing.synchronize exists as part of
_check_system_limits(), allowing ProcessPoolExecutor to raise
NotImplementedError during __init__, rather than crashing with
ImportError during __init__ when creating a lock imported from
multiprocessing.synchronize.

Use _check_system_limits() to disable tests of
ProcessPoolExecutor on systems without multiprocessing.synchronize.

Running the test suite without multiprocessing.synchronize reveals
that Lib/compileall.py crashes when it uses a ProcessPoolExecutor.
Therefore, change Lib/compileall.py to call _check_system_limits()
before creating the ProcessPoolExecutor.

Note that both Lib/compileall.py and Lib/test/test_compileall.py
were attempting to sanity-check ProcessPoolExecutor by expecting
ImportError. In multiprocessing.resource_tracker, sem_unlink() is also absent
on platforms where POSIX semaphores aren't available. Avoid using
sem_unlink() if it, too, does not exist.

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2021-02-08 03:15:51 +00:00
Pablo Galindo 206cbdab16
bpo-43149: Improve error message for exception group without parentheses (GH-24467) 2021-02-07 18:42:21 +00:00
Raymond Hettinger b6d68aa08b
bpo-43102: Set namedtuple __new__'s internal builtins to a dict. (GH-24439) 2021-02-04 15:52:16 -08:00
Dong-hee Na f917c243c5
bpo-43106: Add os.O_EVTONLY/O_FSYNC/O_SYMLINK/O_NOFOLLOW_ANY (GH-24428) 2021-02-04 08:32:55 +09:00
Pablo Galindo d4e6ed7e5f
bpo-43121: Fix incorrect SyntaxError message for missing comma (GH-24436) 2021-02-03 23:29:26 +00:00
Pablo Galindo 22dbd9e8c0
Python 3.10.0a5 2021-02-02 20:41:05 +00:00
Pablo Galindo bb739ec922
bpo-43108: Fix a reference leak in the curses module (GH-24420) 2021-02-02 20:38:26 +00:00
BarneyStratford 01c4fddc4b
bpo-41149: Fix a bug in threading that causes fals-y threads callables to fail to start. (GH-21201) 2021-02-02 20:24:24 +00:00
Pablo Galindo 58fb156edd
bpo-42997: Improve error message for missing : before suites (GH-24292)
* Add to the peg generator a new directive ('&&') that allows to expect
  a token and hard fail the parsing if the token is not found. This
  allows to quickly emmit syntax errors for missing tokens.

* Use the new grammar element to hard-fail if the ':' is missing before
  suites.
2021-02-02 19:54:22 +00:00
Karl Dubost 9eb11a139f
bpo-41748: Handles unquoted attributes with commas (#24072)
* bpo-41748: Adds tests for unquoted attributes with comma

* bpo-41748: Handles unquoted attributes with comma

* bpo-41748: Addresses review comments

* bpo-41748: Addresses review comments

* Adds more test cases
* Simplifies the regex for handling spaces

* bpo-41748: Moves attributes tests under the right class

* bpo-41748: Addresses review about duplicate attributes

* bpo-41748: Adds NEWS.d entry for this patch
2021-02-01 21:32:50 +01:00
Aviral Srivastava 000cde5984
bpo-38307: Add end_lineno attribute to pyclbr Objects (GH-24348)
For back-compatibility, make the new constructor parameter for public classes Function and Class
keyword-only with a default of None.

Co-authored-by: Aviral Srivastava <aviralsrivastava@Avirals-MacBook-Air.local
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
2021-02-01 12:38:44 -05:00
Ken Jin b5931f1d9f
bpo-42834: Fix _json internal caches for subinterpreters (GH-24121)
Make internal caches of the _json extension module
compatible with subinterpreters.
2021-02-01 17:26:56 +01:00
Mark Shannon 0332e569c1
bpo-42990: Further refactoring of PyEval_ functions. (GH-24368)
* Further refactoring of PyEval_EvalCode and friends. Break into make-frame, and eval-frame parts.

* Simplify function vector call using new _PyEval_Vector.

* Remove unused internal functions: _PyEval_EvalCodeWithName and _PyEval_EvalCode.

* Don't use legacy function PyEval_EvalCodeEx.
2021-02-01 10:42:03 +00:00
Ronald Oussoren 49926cf2bc
bpo-42504: Ensure that get_config_var('MACOSX_DEPLOYMENT_TARGET') is a string (GH-24341)
* bpo-42504: Ensure that get_config_var('MACOSX_DEPLOYMENT_TARGET') is a string
2021-01-31 22:29:44 -05:00
Pablo Galindo 835f14ff8e
bpo-43017: Improve error message for unparenthesised tuples in comprehensions (GH24314) 2021-01-31 22:52:56 +00:00
Pablo Galindo 4090151816
bpo-42986: Fix parser crash when reporting syntax errors in f-string with newlines (GH-24279) 2021-01-31 22:48:23 +00:00
Anonymous Maarten 3243e8a4b4
bpo-41604: Don't decrement the reference count of the previous user_ptr when set_panel_usertpr fails (GH-21933) 2021-01-31 19:55:15 +02:00
Serhiy Storchaka 42b1806af9
bpo-43030: Fixed a compiler warning in Py_UNICODE_ISSPACE with signed wchar_t (GH-24350) 2021-01-31 15:55:13 +02:00
Paul Moore 4d11ecbb5e
bpo-43077: Update bundled pip to 21.0.1 and setuptools to 52.0.0 (GH-24386)
Update bundled pip to 21.0.1 and setuptools to 52.0.0
2021-01-30 16:42:36 +00:00
Guido van Rossum 5c5a938573
bpo-42927: Inline cache for attributes defined with '__slots__' (#24216) 2021-01-29 18:02:29 -08:00
Victor Stinner ba7a99ddb5
bpo-38631: Replace compiler fatal errors with exceptions (GH-24369)
* Replace Py_FatalError() calls with regular SystemError exceptions.
* compiler_exit_scope() calls _PyErr_WriteUnraisableMsg() to log the
  PySequence_DelItem() failure.
* compiler_unit_check() uses _PyMem_IsPtrFreed().
* compiler_make_closure(): remove "(reftype == FREE)" comment since
  reftype can also be LOCAL or GLOBAL_EXPLICIT.
2021-01-30 01:46:44 +01:00
Steve Dower 62949f697f
bpo-41282: Add deprecation warning and docs for distutils (PEP 632) (GH-24355) 2021-01-29 21:48:55 +00:00
Terry Jan Reedy 11d75ec807
bpo-43008: Add 'Patch by Ken Hilton' (GH-24370) 2021-01-29 13:02:05 -05:00
Victor Stinner a6192635f1
bpo-42979: Use _Py_CheckSlotResult() to check slots result (GH-24356)
When Python is built in debug mode (with C assertions), calling a
type slot like sq_length (__len__() in Python) now fails with a fatal
error if the slot succeeded with an exception set, or failed with no
exception set. The error message contains the slot, the type name,
and the current exception (if an exception is set).

* Check the result of all slots using _Py_CheckSlotResult().
* No longer pass op_name to ternary_op() in release mode.
* Replace operator with dunder Python method name in error messages.
  For example, replace "*" with "__mul__".
* Fix compiler_exit_scope() when an exception is set.
* Fix bytearray.extend() when an exception is set: don't call
  bytearray_setslice() with an exception set.
2021-01-29 16:53:03 +01:00
Zackery Spytz 23a567c11c
bpo-23544: Disable IDLE Stack Viewer when running user code (GH-17163)
Starting stack viewer when user code is running, including when Debugger is active, hangs or crashes IDLE.

Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
2021-01-28 18:13:22 -05:00
Victor Stinner 6790005a9a
bpo-43031: Set a timeout when running tests in PGO build (GH-24339)
Pass --timeout=$(TESTTIMEOUT) option to the default profile task
"./python -m test --pgo" command.
2021-01-27 11:16:15 +01:00
Ken 7a34380ad7
bpo-43008: Make IDLE respect sys.excepthook (GH-24302)
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
2021-01-26 18:55:52 -05:00
Ethan Furman 7aaeb2a3d6
bpo-38250: [Enum] single-bit flags are canonical (GH-24215)
Flag members are now divided by one-bit verses multi-bit, with multi-bit being treated as aliases. Iterating over a flag only returns the contained single-bit flags.

Iterating, repr(), and str() show members in definition order.

When constructing combined-member flags, any extra integer values are either discarded (CONFORM), turned into ints (EJECT) or treated as errors (STRICT). Flag classes can specify which of those three behaviors is desired:

>>> class Test(Flag, boundary=CONFORM):
...     ONE = 1
...     TWO = 2
...
>>> Test(5)
<Test.ONE: 1>

Besides the three above behaviors, there is also KEEP, which should not be used unless necessary -- for example, _convert_ specifies KEEP as there are flag sets in the stdlib that are incomplete and/or inconsistent (e.g. ssl.Options). KEEP will, as the name suggests, keep all bits; however, iterating over a flag with extra bits will only return the canonical flags contained, not the extra bits.

Iteration is now in member definition order.  If member definition order
matches increasing value order, then a more efficient method of flag
decomposition is used; otherwise, sort() is called on the results of
that method to get definition order.


``re`` module:

repr() has been modified to support as closely as possible its previous
output; the big difference is that inverted flags cannot be output as
before because the inversion operation now always returns the comparable
positive result; i.e.

   re.A|re.I|re.M|re.S is ~(re.L|re.U|re.S|re.T|re.DEBUG)

in both of the above terms, the ``value`` is 282.

re's tests have been updated to reflect the modifications to repr().
2021-01-25 14:26:19 -08:00
Victor Stinner 9852cb3811
bpo-42955: Rename module_names to sys.stdlib_module_names (GH-24332)
* Rename _Py_module_names to _Py_stdlib_module_names.
* Rename Python/module_names.h to Python/stdlib_module_names.h.
2021-01-25 23:12:50 +01:00
Andrey Bienkowski 501d4a51e3
bpo-42383: pdb: do not fail to restart the target if the current directory changed (#23412)
This commit only adds tests and a news entry. The actual bug was fixed in the earlier commit.
2021-01-25 13:08:01 -08:00
Serhiy Storchaka f066bd94b9
bpo-37319: Improve documentation, code and tests of randrange. (GH-19112) 2021-01-25 23:02:04 +02:00
Victor Stinner db584bdad3
bpo-42955: Add sys.modules_names (GH-24238)
Add sys.module_names, containing the list of the standard library
module names.
2021-01-25 13:24:42 +01:00
Serhiy Storchaka cf19cc3b92
bpo-27772: Make preceding width with 0 valid in string format. (GH-11270)
Previously it was an error with confusing error message.
2021-01-25 11:56:33 +02:00
Anthony Sottile 15bd9efd01
bpo-43014: Improve performance of tokenize.tokenize by 20-30% 2021-01-24 12:23:17 +03:00
Борис Верховский 644d52818a
bpo-40304: Correct type(name, bases, dict) doc (GH-19553)
Co-authored-by: Éric Araujo <merwok@netwok.org>
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
Co-authored-by: Tal Einat <532281+taleinat@users.noreply.github.com>
2021-01-22 00:47:23 -05:00
Andrey Bienkowski 8603dfb421
bpo-42384: pdb: correctly populate sys.path[0] (GH-23338)
Automerge-Triggered-By: GH:gvanrossum
2021-01-21 17:19:51 -08:00
pxinwr 5e45f1c8e7
bpo-31904: setup.py: fix cross-compilation on VxWorks (GH-24191)
Add library search path by wr-cc in add_cross_compiling_paths().
2021-01-22 01:55:52 +01:00
Cheryl Sabella 6713e869c4
bpo-33289: Return RGB triplet of ints instead of floats from tkinter.colorchooser (GH-6578) 2021-01-21 21:14:04 +02:00
Batuhan Taskaya a698d52c39
bpo-40176: Improve error messages for unclosed string literals (GH-19346)
Automerge-Triggered-By: GH:isidentical
2021-01-20 13:38:47 -08:00
Victor Stinner 75e59a97f5
bpo-42856: Add --with-wheel-pkg-dir=PATH configure option (GH-24210)
Add --with-wheel-pkg-dir=PATH option to the ./configure script. If
specified, the ensurepip module looks for setuptools and pip wheel
packages in this directory: if both are present, these wheel packages
are used instead of ensurepip bundled wheel packages.

Some Linux distribution packaging policies recommend against bundling
dependencies. For example, Fedora installs wheel packages in the
/usr/share/python-wheels/ directory and don't install the
ensurepip._bundled package.

ensurepip: Remove unused runpy import.
2021-01-20 17:07:21 +01:00
Victor Stinner c1c3493fb7
bpo-42323: Fix math.nextafter() for NaN on AIX (GH-24265) 2021-01-20 15:20:13 +01:00
cptpcrd 7dc71c425c
bpo-42780: Fix set_inheritable() for O_PATH file descriptors on Linux (GH-24172) 2021-01-20 15:05:51 +01:00
Zhiming Wang 3554fa4abe
bpo-42005: profile and cProfile catch BrokenPipeError (GH-22643) 2021-01-20 09:56:21 +01:00
Pablo Galindo d6d6371447
bpo-42864: Improve error messages regarding unclosed parentheses (GH-24161) 2021-01-19 23:59:33 +00:00
Benjamin Peterson 916610ef90
closes bpo-42938: Replace snprintf with Python unicode formatting in ctypes param reprs. (24239) 2021-01-18 14:47:05 -06:00
Victor Stinner 6a809fa01f
bpo-36143: make regen-all now also runs regen-keyword (GH-24245) 2021-01-18 21:23:35 +01:00
Victor Stinner 250035d134
bpo-42923: Dump extension modules on fatal error (GH-24207)
The Py_FatalError() function and the faulthandler module now dump the
list of extension modules on a fatal error.

Add _Py_DumpExtensionModules() and _PyModule_IsExtension() internal
functions.
2021-01-18 20:47:13 +01:00
jonanifranco f7b5bacd7a
bpo-42944 Fix Random.sample when counts is not None (GH-24235) 2021-01-18 10:04:29 -08:00
Setrak Balian 998ae1fa3f
bpo-42931: randbytes missing from random.__all__ (GH-24219) 2021-01-15 09:50:42 -08:00
Irit Katriel ba876c44a8
bpo-42934: use TracebackException(compact=True) in unittest.TestResult (GH-24221) 2021-01-15 07:59:44 -08:00
Irit Katriel 4c94d74152
bpo-42877: add the 'compact' param to TracebackException's __init__ (#24179)
Use it to reduce the time and memory taken up by several of traceback's module-level functions.
2021-01-14 18:45:02 -08:00
Lysandros Nikolaou e5fe509054
bpo-42827: Fix crash on SyntaxError in multiline expressions (GH-24140)
When trying to extract the error line for the error message there
are two distinct cases:

1. The input comes from a file, which means that we can extract the
   error line by using `PyErr_ProgramTextObject` and which we already
   do.
2. The input does not come from a file, at which point we need to get
   the source code from the tokenizer:
   * If the tokenizer's current line number is the same with the line
     of the error, we get the line from `tok->buf` and we're ready.
   * Else, we can extract the error line from the source code in the
     following two ways:
     * If the input comes from a string we have all the input
       in `tok->str` and we can extract the error line from it.
     * If the input comes from stdin, i.e. the interactive prompt, we
       do not have access to the previous line. That's why a new
       field `tok->stdin_content` is added which holds the whole input for the
       current (multiline) statement or expression. We can then extract the
       error line from `tok->stdin_content` like we do in the string case above.

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2021-01-14 21:36:30 +00:00
Zackery Spytz 14cfa325c2
bpo-39273: Expose BUTTON5_* constants in the curses module if available (GH-17996) 2021-01-14 11:40:09 +02:00
Tobias Holl 61d8c54f43
bpo-42924: Fix incorrect copy in bytearray_repeat (GH-24208)
Before, using the * operator to repeat a bytearray would copy data from the start of
the internal buffer (ob_bytes) and not from the start of the actual data (ob_start).
2021-01-13 18:16:40 +02:00
Ethan Furman c314e60388
bpo-42901: [Enum] move member creation to `__set_name__` (GH-24196)
`type.__new__` calls `__set_name__` and `__init_subclass__`, which means
that any work metaclasses do after calling `super().__new__()` will not
be available to those two methods.  In particular, `Enum` classes that
want to make use of `__init_subclass__` will not see any members.

Almost all customization is therefore moved to before the
`type.__new__()` call, including changing all members to a proto member
descriptor with a `__set_name__` that will do the final conversion of a
member to be an instance of the `Enum` class.
2021-01-12 23:47:57 -08:00
Hugo van Kemenade c47c78b878
bpo-37324: Remove ABC aliases from collections (GH-23754)
Remove deprecated aliases to Abstract Base Classes from the
collections module.
2021-01-13 00:16:37 +01:00
Irit Katriel 6dfd1734f5
bpo-42848: remove recursion from TracebackException (GH-24158) 2021-01-12 14:14:27 -08:00
Paul Ganssle 0f66498fd8
bpo-42874: Remove grep -qE options for Solaris 10 compatibility (GH-24200)
According to [bpo-42874](), some versions of grep do not support the `-q` and `-E` options. Although both options are used elsewhere in the configure script, this particular bit of validation can be achieved without them,
so there's no real harm in using a grep call with no flags.

Would be good to get some people taking advantage of the `--with-tzpath` arguments in the wild to try this out.. Local testing seems to indicate that this does the same thing, but I don't know that we have any buildbots using this option. Maybe @pablogsal?

[bpo-42874]():
2021-01-12 10:17:52 -08:00
Serhiy Storchaka 4db8988420
bpo-41994: Fix refcount issues in Python/import.c (GH-22632)
https://bugs.python.org/issue41994
2021-01-12 15:43:32 +01:00
Victor Stinner 44bf57aca6
bpo-42882: _PyRuntimeState_Init() leaves unicode next_index unchanged (GH-24193)
Fix the _PyUnicode_FromId() function (_Py_IDENTIFIER(var) API) when
Py_Initialize() / Py_Finalize() is called multiple times:
preserve _PyRuntime.unicode_ids.next_index value.

Use _PyRuntimeState_INIT macro instead memset(0) to reset
_PyRuntimeState members to zero.
2021-01-12 10:29:45 +01:00
Tomáš Hrnčiar fb35fa49d1
bpo-42870: Document change in argparse help output. (GH-24190) 2021-01-11 16:41:35 -08:00
Terry Jan Reedy 81f87bbf9f
bpo-33065: Fix problem debugging user classes with __repr__ method (GH-24183)
If __repr__ uses instance attributes, as normal, and one steps
through the __init__ method, debugger may try to get repr before
the instance attributes exist.  reprlib.repr handles the error.
2021-01-10 01:59:47 -05:00
Victor Stinner 0e2a0f72cc
bpo-42802: Remove distutils bdist_wininst command (GH-24043)
The distutils bdist_wininst command deprecated in Python 3.8 has been
removed. The distutils bidst_wheel command is now recommended to
distribute binary packages on Windows.

* Remove Lib/distutils/command/bdist_wininst.py
* Remove PC/bdist_wininst/ project
* Remove Lib/distutils/command/wininst-*.exe programs
* Remove all references to bdist_wininst
2021-01-09 00:35:01 +01:00
Victor Stinner e542d417b9
bpo-42866: Fix refleak in CJK getcodec() (GH-24165)
Fix a reference leak in the getcodec() function of CJK codecs.
2021-01-08 15:01:38 +01:00
Victor Stinner 07f2cee93f
bpo-42846: Convert CJK codec extensions to multiphase init (GH-24157)
Convert the 6 CJK codec extension modules (_codecs_cn, _codecs_hk,
_codecs_iso2022, _codecs_jp, _codecs_kr and _codecs_tw) to the
multiphase initialization API (PEP 489).

Remove getmultibytecodec() local cache: always import
_multibytecodec. It should be uncommon to get a codec. For example,
this function is only called once per CJK codec module.

Fix a reference leak in register_maps() error path.
2021-01-08 00:15:22 +01:00
Ethan Furman a581a868d9
bpo-42851: [Enum] remove brittle __init_subclass__ support (GH-24154)
Solution to support calls to `__init_subclass__` with members defined is too brittle and breaks with certain mixins.
2021-01-07 13:17:55 -08:00
Erlend Egeberg Aasland 849e339a92
bpo-40823: Use loadTestsFromTestCase() iso. makeSuite() in sqlite3 tests (GH-20538) 2021-01-07 02:05:07 +02:00
Yair Frid ff8458b918
bpo-42811: Update importlib.utils.resolve_name() docs to use __spec__.parent (GH-24100)
Automerge-Triggered-By: GH:brettcannon
2021-01-06 09:42:10 -08:00
Erlend Egeberg Aasland ddb5e11683
bpo-24464: Deprecate sqlite3.enable_shared_cache (GH-24008) 2021-01-06 02:36:04 +02:00
Erlend Egeberg Aasland cf0b23908c
bpo-40810: Require SQLite 3.7.15 (GH-24106) 2021-01-06 02:02:43 +02:00
Erlend Egeberg Aasland dd74c01d3b
bpo-42584: Update Windows installer to use SQLite 3.34.0 (GH-23675) 2021-01-05 22:01:56 +00:00
Steve Dower afb7144378
bpo-41837: Updated Windows installer to include OpenSSL 1.1.1i (GH-24125) 2021-01-05 21:35:01 +00:00
Mark Shannon ee9f98d9f4
bpo-42823: Fix frame lineno when frame.f_trace is set (GH-24099)
* Add test for frame.f_lineno with/without tracing.

* Make sure that frame.f_lineno is correct regardless of whether frame.f_trace is set.

* Update importlib

* Add NEWS
2021-01-05 12:04:10 +00:00