Commit Graph

109398 Commits

Author SHA1 Message Date
Brett Cannon 9cb31d6716
bpo-42137: have ModuleType.__repr__ prefer __spec__ over module_repr() (GH-24953)
This is to work towards the removal of the use of  module_repr() in Python 3.12 (documented as deprecated since 3.4).
2021-03-24 08:26:56 -07:00
sblondon 3ba3d513b1
bpo-42914: add a pprint underscore_numbers option (GH-24864)
pprint() gains a new boolean underscore_numbers kwarg to emit
integers with thousands separated by an underscore character
for improved readability (for example 1_000_000 instead of 1000000).
2021-03-24 01:23:20 -07:00
Pablo Galindo a02683ac38
bpo-31861: Fix reference leak in builtin_anext_impl() (GH-25008) 2021-03-24 01:42:13 +00:00
Victor Stinner 8370e07e1e
bpo-43244: Remove the pyarena.h header (GH-25007)
Remove the pyarena.h header file with functions:

* PyArena_New()
* PyArena_Free()
* PyArena_Malloc()
* PyArena_AddPyObject()

These functions were undocumented, excluded from the limited C API,
and were only used internally by the compiler.

Add pycore_pyarena.h header. Rename functions:

* PyArena_New() => _PyArena_New()
* PyArena_Free() => _PyArena_Free()
* PyArena_Malloc() => _PyArena_Malloc()
* PyArena_AddPyObject() => _PyArena_AddPyObject()
2021-03-24 02:23:01 +01:00
Pablo Galindo 919d42d477
bpo-31861: Fix possible crash in PyAnextAwaitable_New (GH-25005) 2021-03-24 00:30:02 +00:00
Victor Stinner 57364ce34e
bpo-43244: Remove parser_interface.h header file (GH-25001)
Remove parser functions using the "struct _mod" type, because the
AST C API was removed:

* PyParser_ASTFromFile()
* PyParser_ASTFromFileObject()
* PyParser_ASTFromFilename()
* PyParser_ASTFromString()
* PyParser_ASTFromStringObject()

These functions were undocumented and excluded from the limited C
API.

Add pycore_parser.h internal header file. Rename functions:

* PyParser_ASTFromFileObject() => _PyParser_ASTFromFile()
* PyParser_ASTFromStringObject() => _PyParser_ASTFromString()

These functions are no longer exported (replace PyAPI_FUNC() with
extern).

Remove also _PyPegen_run_parser_from_file() function. Update
test_peg_generator to use _PyPegen_run_parser_from_file_pointer()
instead.
2021-03-24 01:29:09 +01:00
Pablo Galindo a054f6b2b1
bpo-43452: Document the PyType_Lookup optimizations in the What's New for 3.10 (GH-24949) 2021-03-24 00:04:52 +00:00
Pablo Galindo d9692027f4
bpo-31861: Complete the C-API docs for PyObject_GetAiter and PyAiter_Check (GH-25004) 2021-03-23 23:57:03 +00:00
Victor Stinner a81fca6ec8
bpo-43244: Add pycore_compile.h header file (GH-25000)
Remove the compiler functions using "struct _mod" type, because the
public AST C API was removed:

* PyAST_Compile()
* PyAST_CompileEx()
* PyAST_CompileObject()
* PyFuture_FromAST()
* PyFuture_FromASTObject()

These functions were undocumented and excluded from the limited C API.

Rename functions:

* PyAST_CompileObject() => _PyAST_Compile()
* PyFuture_FromASTObject() => _PyFuture_FromAST()

Moreover, _PyFuture_FromAST() is no longer exported (replace
PyAPI_FUNC() with extern). _PyAST_Compile() remains exported for
test_peg_generator.

Remove also compatibility functions:

* PyAST_Compile()
* PyAST_CompileEx()
* PyFuture_FromAST()
2021-03-24 00:51:50 +01:00
Joshua Bronson f0a6fde882
bpo-31861: Add aiter and anext to builtins (#23847)
Co-authored-by: jab <jab@users.noreply.github.com>
Co-authored-by: Daniel Pope <mauve@mauveweb.co.uk>
Co-authored-by: Justin Wang <justin39@gmail.com>
2021-03-23 15:47:21 -07:00
Victor Stinner 94faa0724f
bpo-43244: Remove ast.h, asdl.h, Python-ast.h headers (GH-24933)
These functions were undocumented and excluded from the limited C
API.

Most names defined by these header files were not prefixed by "Py"
and so could create names conflicts. For example, Python-ast.h
defined a "Yield" macro which was conflict with the "Yield" name used
by the Windows <winbase.h> header.

Use the Python ast module instead.

* Move Include/asdl.h to Include/internal/pycore_asdl.h.
* Move Include/Python-ast.h to Include/internal/pycore_ast.h.
* Remove ast.h header file.
* pycore_symtable.h no longer includes Python-ast.h.
2021-03-23 20:47:40 +01:00
Victor Stinner cd27af70d5
bpo-41718: Update runpy startup time What's New (GH-24998) 2021-03-23 20:22:40 +01:00
Victor Stinner bd9154a495
bpo-41718: runpy now imports pkgutil in functions (GH-24996)
Reduce the number of modules imported by "python3 -m module".

The runpy module no longer imports at startup (in the module body),
but only in functions using it: _get_code_from_file() and run_path().
2021-03-23 19:22:57 +01:00
Victor Stinner d72e8d4875
bpo-41718: subprocess imports grp and pwd on demand (GH-24987)
The shutil and subprocess modules now only import the grp and pwd
modules when they are needed, which is not the case by default in
subprocess.
2021-03-23 17:42:51 +01:00
Brett Cannon 76b5d714e4
Clarify attribute docs on types.ModuleType (GH-24974) 2021-03-23 08:25:39 -07:00
Victor Stinner 9feae41c4f
bpo-41718: libregrtest avoids importing datetime (GH-24985)
* libregrtest reimplements datetime.timedelta.__str__()
* support.testresult only imports datetime if USE_XML is true.
2021-03-23 01:40:31 +01:00
Victor Stinner 30793e81bd
bpo-41718: Disable support.testresult XML output by default (GH-24982)
RegressionTestResult.USE_XML must now be set to True to get the JUnit
XML output.

Reduce the number of imports when --junit-xml=FILE option is not
used: 153 => 144 (-9).
2021-03-23 01:11:31 +01:00
Victor Stinner 0473fb2229
bpo-41718: libregrtest runtest avoids import_helper (GH-24983)
Inline import_helper.unload() in libregrtest.runtest to avoid one
import.
2021-03-23 01:08:49 +01:00
Victor Stinner 10417dd15f
bpo-41718: Reduce libregrtest runtest imports (GH-24980)
Move clear_caches() from libregrtest.refleak to libregrtest.utils to
avoid importing libregrtest.refleak when it's not needed.

clear_caches() now only calls re.purge() if 're' is in sys.modules.
2021-03-23 00:17:05 +01:00
Victor Stinner 532e063fc2
bpo-41718: regrtest saved_test_environment avoids imports (GH-24934)
Reduce the number of modules imported by libregrtest.

saved_test_environment no longer imports modules at startup, but try
to get them from sys.modules. If an module is missing, skip the test.
It also sets directly support.environment_altered.

runtest() now now two saved_test_environment instances: one before
importing the test module, one after importing it.

Remove imports from test.libregrtest.save_env:

* asyncio
* logging
* multiprocessing
* shutil
* sysconfig
* urllib.request
* warnings

When a test method imports a module (ex: warnings) and the test
has a side effect (ex: add a warnings filter), the side effect is not
detected, because the module was not imported when Python
enters the saved_test_environment context manager.
2021-03-22 23:52:13 +01:00
Pablo Galindo 96eeff5162
bpo-43555: Report the column offset for invalid line continuation character (GH-24939) 2021-03-22 17:28:11 +00:00
Pablo Galindo 123ff266cd
bpo-43591: Fix error location in interactive mode for errors at the end of the line (GH-24973)
Co-authored-by: Erlend Egeberg Aasland
2021-03-22 16:24:39 +00:00
Victor Stinner 39f643614d
Revert "bpo-40521: Make dtoa bigint free list per-interpreter (GH-24821)" (GH-24964)
This reverts commit 5bd1059184.
2021-03-22 11:58:59 +01:00
Dong-hee Na 86883d40e9
bpo-43575: Use PEP 590 vectorcall to speed up map() (GH-24955) 2021-03-22 19:01:14 +09:00
junyixie 88d9983b56
bpo-43551: Fix PyImport_Import() for subinterpreters (GH-24929)
Avoid static variables.
2021-03-22 10:47:10 +01:00
Hai Shi 56f031ec52
bpo-35134: Add include/cpython/compile.h (GH-24922)
Move C API excluded from the limited C API from Include/compile.h
to a new Include/cpython/compile.h header file.
2021-03-22 09:32:11 +01:00
Sergey B Kirpichev 690aca7811
bpo-43420: Simple optimizations for Fraction's arithmetics (GH-24779)
bpo-43420: Implement standard transformations in + - * / that can often reduce the size of intermediate integers needed. For rationals with large components, this can yield dramatic speed improvements, but for small rationals can run 10-20% slower, due to increased fixed overheads in the longer-winded code. If those slowdowns turn out to be a problem, see the PR discussion for low-level implementation tricks that could cut other fixed overheads.

Co-authored-by: Tim Peters <tim.peters@gmail.com>
Co-authored-by: Mark Dickinson <dickinsm@gmail.com>
2021-03-21 21:30:55 -05:00
Jason R. Coombs 9a50ef43e4
bpo-43569: Add test/test_importlib/namespacedata01 to TESTSUBDIRS (GH-24952) 2021-03-21 18:26:45 +01:00
Antoine Pitrou cdddc2b742
bpo-43422: Revert _decimal C API addition (GH-24960)
Stefan Krah requested the reversal of these (unreleased) changes, quoting him:
> The capsule API does not meet my testing standards, since I've focused
on the upstream mpdecimal in the last couple of months.
> Additionally, I'd like to refine the API, perhaps together with the
Arrow community.

Automerge-Triggered-By: GH:pitrou
2021-03-21 09:27:54 -07:00
Christian Heimes 77cde5042a
bpo-43577: Fix deadlock with SSLContext._msg_callback and sni_callback (GH-24957)
OpenSSL copies the internal message callback from SSL_CTX->msg_callback to
SSL->msg_callback. SSL_set_SSL_CTX() does not update SSL->msg_callback
to use the callback value of the new context.

PySSL_set_context() now resets the callback and _PySSL_msg_callback()
resets thread state in error path.

Signed-off-by: Christian Heimes <christian@python.org>
2021-03-21 16:13:09 +01:00
Ilya Stepin 20a5b7e986
bpo-43542: Add heif/heic formats in mimetypes (GH-24917)
* bpo-43542: Add heif/heic formats in mimetypes

Add HEIF and HEIC format to list of media types. It has IANA registration.

IANA: https://www.iana.org/assignments/media-types/image/heic
HEIF Github: https://github.com/nokiatech/heif

Co-authored-by: Gregory P. Smith <greg@krypto.org>
2021-03-20 16:06:27 -07:00
Rui Cunha b05b48dd7a
bpo-43571: Add IPPROTO_MPTCP macro (GH-24946)
Add IPPROTO_MPTCP constant when defined in the system headers.
2021-03-20 15:04:56 -07:00
Antoine Pitrou 66c8adfa27
bpo-43517 (followup): Install additional test directory (GH-24950)
Should fix some CI failures on buildbots that test an installed version of Python.

Automerge-Triggered-By: GH:pitrou
2021-03-20 13:45:58 -07:00
Dino Viehland ee48c7d541
bpo-43452: Micro-optimizations to PyType_Lookup (GH-24804)
The common case going through _PyType_Lookup is to have a cache hit. There are some small tweaks that can make this a little cheaper:

* The name field identity is used for a cache hit and is kept alive by the cache. So there's no need to read the hash code o the name - instead, the address can be used as the hash.

*  There's no need to check if the name is cachable on the lookup either, it probably is, and if it is, it'll be in the cache.

*  If we clear the version tag when invalidating a type then we don't actually need to check for a valid version tag bit.
2021-03-20 19:12:05 +00:00
Antoine Pitrou 2fd16ef406
bpo-43517: Fix false positive in detection of circular imports (#24895) 2021-03-20 20:07:44 +01:00
Skip Montanaro 7cb033c423
bpo-43494: Make some minor changes to lnotab notes (GH-24861)
This cleanup makes no substantive changes.
2021-03-19 19:10:54 -04:00
Mark Shannon 148bc05844
Mention that code.co_lnotab is deprecated in what's new for 3.10. (#24902) 2021-03-19 17:30:24 +00:00
Victor Stinner 28ad12f8fe
bpo-43244: Remove symtable.h header file (GH-24910)
Rename Include/symtable.h to to Include/internal/pycore_symtable.h,
don't export symbols anymore (replace PyAPI_FUNC and PyAPI_DATA with
extern) and rename functions:

* PyST_GetScope() to _PyST_GetScope()
* PySymtable_BuildObject() to _PySymtable_Build()
* PySymtable_Free() to _PySymtable_Free()

Remove PySymtable_Build(), Py_SymtableString() and
Py_SymtableStringObject() functions.

The Py_SymtableString() function was part the stable ABI by mistake
but it could not be used, since the symtable.h header file was
excluded from the limited C API.

The Python symtable module remains available and is unchanged.
2021-03-19 12:41:49 +01:00
Christian Heimes 32eba61ea4
bpo-43466: Add --with-openssl-rpath configure option (GH-24820) 2021-03-19 10:29:25 +01:00
Ikko Ashimine ff8c77fe96
Fix a typo in c-analyzer (GH-24468) 2021-03-19 16:38:12 +09:00
Christian Heimes f6c6b5821b
bpo-41561: Add workaround for Ubuntu's custom security level (GH-24915)
Ubuntu 20.04 comes with a patched OpenSSL 1.1.1. Default security level
2 blocks TLS 1.0 and 1.1 connections. Regular OpenSSL 1.1.1 builds allow
TLS 1.0 and 1.1 on security level 2.

See: 
See: https://bugs.launchpad.net/ubuntu/+source/openssl/+bug/1899878
See: https://bugs.launchpad.net/ubuntu/+source/openssl/+bug/1917625
Signed-off-by: Christian Heimes <christian@python.org>
2021-03-18 15:06:50 -07:00
Kodi Arfer 08ff4369af
bpo-43521: Allow ast.unparse with empty sets and NaN (GH-24897)
Automerge-Triggered-By: GH:pablogsal
2021-03-18 10:36:06 -07:00
Victor Stinner eec8e61992
bpo-43244: Remove the PyAST_Validate() function (GH-24911)
Remove the PyAST_Validate() function. It is no longer possible to
build a AST object (mod_ty type) with the public C API. The function
was already excluded from the limited C API (PEP 384).

Rename PyAST_Validate() function to _PyAST_Validate(), move it to the
internal C API, and don't export it anymore (replace PyAPI_FUNC with
extern).

The function was added in bpo-12575 by
the commit 832bfe2ebd.
2021-03-18 14:57:49 +01:00
Victor Stinner fc980e0be1
bpo-43541: Fix PyEval_EvalCodeEx() regression (GH-24918)
* Remove an assertion which required CO_NEWLOCALS and CO_OPTIMIZED
  code flags. It is ok to call this function on a code with these
  flags set.
* Fix reference counting on builtins: remove Py_DECREF().
  Fix regression introduced in the
  commit 46496f9d12.

Add also a comment to document that _PyEval_BuiltinsFromGlobals()
returns a borrowed reference.
2021-03-18 14:51:24 +01:00
Victor Stinner 6af528b4ab
bpo-43244: Fix test_peg_generators on Windows (GH-24913)
Don't redefine Py_DebugFlag, it's already defined in pydebug.h which
is included by Python.h
2021-03-18 09:54:13 +01:00
Chris Burr e0b4aa0f5c
bpo-39342: Expose X509_V_FLAG_ALLOW_PROXY_CERTS in ssl module (GH-18011)
Exposes the `X509_V_FLAG_ALLOW_PROXY_CERTS` constant as `ssl.VERIFY_ALLOW_PROXY_CERTS` to allow for proxy certificate validation as described in: https://www.openssl.org/docs/man1.1.1/man7/proxy-certificates.html
2021-03-18 01:24:01 -07:00
Victor Stinner e0bf70d08c
bpo-43244: Fix test_peg_generator for PyAST_Validate() (GH-24912)
test_peg_generator now defines _Py_TEST_PEGEN macro when building C
code to not call PyAST_Validate() in Parser/pegen.c. Moreover, it
defines Py_BUILD_CORE_MODULE macro to get access to the internal
C API.

Remove "global_ast_state" from Python-ast.c when it's built by
test_peg_generator: always get the AST state from the current interpreter.
2021-03-18 02:46:06 +01:00
Pablo Galindo 08fb8ac99a
bpo-42128: Add 'missing :' syntax error message to match statements (GH-24733) 2021-03-18 01:03:11 +00:00
Victor Stinner 526fdeb227
bpo-43244: Add pycore_ast.h header file (GH-24908)
Move _PyAST_GetDocString() and _PyAST_ExprAsUnicode() functions the
internal C API: from Include/ast.h to a new
Include/internal/pycore_ast.h header file. Don't export these
functions anymore: replace PyAPI_FUNC() with extern.

Remove also unused includes.
2021-03-17 23:50:50 +01:00
Victor Stinner b4536e1c6a
bpo-43244: Rename pycore_ast.h to pycore_ast_state.h (GH-24907) 2021-03-17 23:11:03 +01:00