Fix encoding name when running a ".pyc" file on Windows:
PyRun_SimpleFileExFlags() now uses the correct encoding to decode the
filename.
* Add pyrun_file() subfunction.
* Add pyrun_simple_file() subfunction.
* PyRun_SimpleFileExFlags() now calls _Py_fopen_obj() rather than
_Py_fopen().
(cherry picked from commit b6d98c10ff)
Several built-in and standard library types now ensure that their internal result tuples are always tracked by the garbage collector:
- collections.OrderedDict.items
- dict.items
- enumerate
- functools.reduce
- itertools.combinations
- itertools.combinations_with_replacement
- itertools.permutations
- itertools.product
- itertools.zip_longest
- zip
Previously, they could have become untracked by a prior garbage collection.
(cherry picked from commit 226a012d1c)
* [3.9] bpo-41100: Support macOS 11 and Apple Silicon (GH-22855)
Co-authored-by: Lawrence D’Anna <lawrence_danna@apple.com>
* Add support for macOS 11 and Apple Silicon (aka arm64)
As a side effect of this work use the system copy of libffi on macOS, and remove the vendored copy
* Support building on recent versions of macOS while deploying to older versions
This allows building installers on macOS 11 while still supporting macOS 10.9..
(cherry picked from commit 41761933c1)
Co-authored-by: Ronald Oussoren <ronaldoussoren@mac.com>
* Back port of changes to _decimal to support arm64
* temp_dir is in test.support in 3.9
Signed-off-by: Christian Heimes <christian@python.org>
Automerge-Triggered-By: GH:tiran
(cherry picked from commit 07f2adedf0)
Co-authored-by: Christian Heimes <christian@python.org>
bpo-41686, bpo-41713: On Windows, the SIGINT event,
_PyOS_SigintEvent(), is now created even if Python is configured to
not install signal handlers (PyConfig.install_signal_handlers=0 or
Py_InitializeEx(0)).
On Windows, fix a regression in signal handling which prevented to
interrupt a program using CTRL+C. The signal handler can be run in a
thread different than the Python thread, in which case the test
deciding if the thread can handle signals is wrong.
On Windows, _PyEval_SignalReceived() now always sets eval_breaker to
1 since it cannot test _Py_ThreadCanHandleSignals(), and
eval_frame_handle_pending() always calls
_Py_ThreadCanHandleSignals() to recompute eval_breaker.
(cherry picked from commit d96a7a8313)
Co-authored-by: Victor Stinner <vstinner@python.org>
Use wide-char _W_* fields of lconv structure on Windows
Remove "ps_AF" from test__locale.known_numerics on Windows
(cherry picked from commit f2312037e3)
Co-authored-by: TIGirardi <tiagoigirardi@gmail.com>
When running in a non-UTF-8 locale, if an error occurs while importing a
native Python module (say because a dependent share library is missing),
the error message string returned may contain non-ASCII code points
causing a UnicodeDecodeError.
PyUnicode_DecodeFSDefault is used for buffers which may contain
filesystem paths. For consistency with os.strerror(),
PyUnicode_DecodeLocale is used for buffers which contain system error
messages. While the shortname parameter is always encoded in ASCII
according to PEP 489, it is left decoded using PyUnicode_FromString to
minimize the changes and since it should not affect the decoding (albeit
_potentially_ slower).
In dynload_hpux, since the error buffer contains a message generated
from a static ASCII string and the module filesystem path,
PyUnicode_DecodeFSDefault is used instead of PyUnicode_DecodeLocale as
is used elsewhere.
* bpo-41894: Fix bugs in dynload error msg handling
For both dynload_aix and dynload_hpux, properly handle the possibility
that decoding strings may return NULL and when such an error happens,
properly decrement any previously decoded strings and return early.
In addition, in dynload_aix, ensure that we pass the decoded string
*object* pathname_ob to PyErr_SetImportError instead of the original
pathname buffer.
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
(cherry picked from commit 2d2af320d9)
Co-authored-by: Kevin Adler <kadler@us.ibm.com>
* PyMapping_HasKey() is not safe because it silences all exceptions and can return incorrect result.
* Informative exceptions from PyMapping_DelItem() are overridden with RuntimeError and
the original exception raised before calling remove_module() is lost.
* There is a race condition between PyMapping_HasKey() and PyMapping_DelItem().
(cherry picked from commit 8287aadb75)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Fix the compiler warning:
format specifies type `wint_t` (aka `int`) but the argument has type `unsigned int`
(cherry picked from commit c322948892)
Co-authored-by: Samuel Marks <807580+SamuelMarks@users.noreply.github.com>
Partially revert commit ac46eb4ad6662cf6d771b20d8963658b2186c48c:
"bpo-38113: Update the Python-ast.c generator to PEP384 (gh-15957)".
Using a module state per module instance is causing subtle practical
problems.
For example, the Mercurial project replaces the __import__() function
to implement lazy import, whereas Python expected that "import _ast"
always return a fully initialized _ast module.
Add _PyAST_Fini() to clear the state at exit.
The _ast module has no state (set _astmodule.m_size to 0). Remove
astmodule_traverse(), astmodule_clear() and astmodule_free()
functions..
(cherry picked from commit e5fbe0cbd4)
Co-authored-by: Victor Stinner <vstinner@python.org>
Fix GCC 9.3 (using -O3) warnings on x86:
initconfig.c: In function ‘init_dump_ascii_wstr’:
initconfig.c:2679:34: warning: format ‘%lc’ expects argument of type
‘wint_t’, but argument 2 has type ‘wchar_t’ {aka ‘long int’}
2679 | PySys_WriteStderr("%lc", ch);
initconfig.c:2682:38: warning: format ‘%x’ expects argument of type
‘unsigned int’, but argument 2 has type ‘wchar_t’ {aka ‘long int’}
2682 | PySys_WriteStderr("\\x%02x", ch);
initconfig.c:2686:38: warning: format ‘%x’ expects argument of type
‘unsigned int’, but argument 2 has type ‘wchar_t’ {aka ‘long int’}
2686 | PySys_WriteStderr("\\U%08x", ch);
initconfig.c:2690:38: warning: format ‘%x’ expects argument of type
‘unsigned int’, but argument 2 has type ‘wchar_t’ {aka ‘long int’}
2690 | PySys_WriteStderr("\\u%04x", ch);
(cherry picked from commit 640e8e1d5f)
Co-authored-by: Victor Stinner <vstinner@python.org>
Free the stack allocated in va_build_stack if do_mkstack fails
and the stack is not a small_stack
(cherry picked from commit 75c80b0bda)
Co-authored-by: Tony Solomonik <tony.solomonik@gmail.com>
* Fixed `f-string/str.format` error description when using two `,` in format specifier.
Co-authored-by: millefalcon <hanish0019@hmail.com>
(cherry picked from commit 0d6aa7f0ee)
Co-authored-by: han-solo <hanish0019@gmail.com>
Co-authored-by: han-solo <hanish0019@gmail.com>
* bpo-41194: Convert _ast extension to PEP 489 (GH-21293)
Convert the _ast extension module to PEP 489 "Multiphase
initialization". Replace the global _ast state with a module state.
(cherry picked from commit b1cc6ba73a)
* bpo-41204: Fix compiler warning in ast_type_init() (GH-21307)
(cherry picked from commit 1f76453173)
3.8.3 had a regression where compiling with
ast.PyCF_ALLOW_TOP_LEVEL_AWAIT woudl agressively mark things are
coroutine even if there were not.
(cherry picked from commit bd46174)
Co-authored-by: Matthias Bussonnier <bussonniermatthias@gmail.com>
Co-authored-by: Matthias Bussonnier <bussonniermatthias@gmail.com>
Also enables using debug build of `python3_d.dll`
Reference: CVE-2020-15523
(cherry picked from commit dcbaa1b49c)
Co-authored-by: Steve Dower <steve.dower@python.org>
* bpo-41194: Pass module state in Python-ast.c (GH-21284)
Rework asdl_c.py to pass the module state to functions in
Python-ast.c, instead of using astmodulestate_global.
Handle also PyState_AddModule() failure in init_types().
(cherry picked from commit 74419f0c64)
* bpo-41194: The _ast module cannot be loaded more than once (GH-21290)
Fix a crash in the _ast module: it can no longer be loaded more than
once. It now uses a global state rather than a module state.
* Move _ast module state: use a global state instead.
* Set _astmodule.m_size to -1, so the extension cannot be loaded more
than once.
(cherry picked from commit 91e1bc18bd)
This commit changes the parsing of f-string expressions with the new parser. The parser gets pre-fed with the location of the expression itself (not the f-string, which was what we were doing before). This allows us to completely skip the shifting of the AST nodes after the parsing is completed..
(cherry picked from commit 1f0f4abb11)
Reported by Coverity. (CID 1457554 RETURN_LOCAL)
path0 is assigned as a pointer to this right before it goes out of scope.
(cherry picked from commit 81328f3070)
Co-authored-by: Gregory P. Smith <greg@krypto.org>
When a file ends with a line that contains a line continuation character
the text of the emitted SyntaxError is empty, contrary to the old
parser, where the error text contained the text of the last line.
(cherry picked from commit 113e2b0a07)
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
On Windows, GH-include "pyerrors.h" no longer defines "snprintf" and
"vsnprintf" macros.
PyOS_snprintf() and PyOS_vsnprintf() should be used to get portable
behavior.
Replace snprintf() calls with PyOS_snprintf() and replace vsnprintf()
calls with PyOS_vsnprintf().
(cherry picked from commit e822e37946)
Co-authored-by: Victor Stinner <vstinner@python.org>
Export explicitly the Py_GetArgcArgv() function to the C API and
document the function. Previously, it was exported implicitly which
no longer works since Python is built with -fvisibility=hidden.
* Add PyConfig._orig_argv member.
* Py_InitializeFromConfig() no longer calls _PyConfig_Write() twice.
* PyConfig_Read() no longer initializes Py_GetArgcArgv(): it is now
_PyConfig_Write() responsibility.
* _PyConfig_Write() result type becomes PyStatus instead of void.
* Write an unit test on Py_GetArgcArgv().
(cherry picked from commit e81f6e687d)
* Provide native .files support on SourceFileLoader.
* Add native importlib.resources.files() support to zipimporter. Remove fallback support.
* make regen-all
* 📜🤖 Added by blurb_it.
* Move 'files' into the ResourceReader so it can carry the relevant module name context.
* Create 'importlib.readers' module and add FileReader to it.
* Add zip reader and rely on it for a TraversableResources object on zipimporter.
* Remove TraversableAdapter, no longer needed.
* Update blurb.
* Replace backslashes with forward slashes.
* Incorporate changes from importlib_metadata 2.0, finalizing the interface for extension via get_resource_reader.
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
(cherry picked from commit 843c277656)
Co-authored-by: Jason R. Coombs <jaraco@jaraco.com>
* bpo-40826: Add _Py_EnsureTstateNotNULL() macro (GH-20571)
Add _Py_EnsureTstateNotNULL(tstate) macro: call Py_FatalError() if
tstate is NULL, the error message contains the current function name.
(cherry picked from commit 3026cad59b)
* bpo-40826: PyOS_InterruptOccurred() requires GIL (GH-20578)
PyOS_InterruptOccurred() now fails with a fatal error if it is called
with the GIL released.
(cherry picked from commit cbe1296922)
Miss Islington (bot)
Victor Stinner
Brandt Bucher
Ronald Oussoren
Christian Heimes
Irit Katriel
Inada Naoki
Pablo Galindo
Steve Dower
Guido van Rossum
Lysandros Nikolaou
Łukasz Langa
Ammar Askar