This change partically reverts
commit ad3252bad9
and the commit fe2978b3b9.
Many third party C extension modules rely on the ability of using
Py_TYPE() to set an object type: "Py_TYPE(obj) = type;" or to set an
object type using: "Py_SIZE(obj) = size;".
Remove the undocumented PyOS_InitInterrupts() C function.
* Rename PyOS_InitInterrupts() to _PySignal_Init(). It now installs
other signal handlers, not only SIGINT.
* Rename PyOS_FiniInterrupts() to _PySignal_Fini()
Fix _PyConfig_Read() if compute_path_config=0: use values set by
Py_SetPath(), Py_SetPythonHome() and Py_SetProgramName(). Add
compute_path_config parameter to _PyConfig_InitPathConfig().
The following functions now return NULL if called before
Py_Initialize():
* Py_GetExecPrefix()
* Py_GetPath()
* Py_GetPrefix()
* Py_GetProgramFullPath()
* Py_GetProgramName()
* Py_GetPythonHome()
These functions no longer automatically computes the Python Path
Configuration. Moreover, Py_SetPath() no longer computes
program_full_path.
The format_exception(), format_exception_only(), and
print_exception() functions can now take an exception object as a positional-only argument.
Co-Authored-By: Matthias Bussonnier <bussonniermatthias@gmail.com>
Enhance the documentation of the Python startup, filesystem encoding
and error handling, locale encoding. Add a new "Python UTF-8 Mode"
section.
* Add "locale encoding" and "filesystem encoding and error handler"
to the glossary
* Remove documentation from Include/cpython/initconfig.h: move it to
Doc/c-api/init_config.rst.
* Doc/c-api/init_config.rst:
* Document command line options and environment variables
* Document default values.
* Add a new "Python UTF-8 Mode" section in Doc/library/os.rst.
* Add warnings to Py_DecodeLocale() and Py_EncodeLocale() docs.
* Document how Python selects the filesystem encoding and error
handler at a single place: PyConfig.filesystem_encoding and
PyConfig.filesystem_errors.
* PyConfig: move orig_argv member at the right place.
No backport is required since union is only in 3.10.
This addresses "3. Consistency nitpicks for Union's docs" in the bpo.
Please skip news. Thank you.
Removed the unicodedata.ucnhash_CAPI attribute which was an internal
PyCapsule object. The related private _PyUnicode_Name_CAPI structure
was moved to the internal C API.
Rename unicodedata.ucnhash_CAPI as unicodedata._ucnhash_CAPI.
* UCD_Check() uses PyModule_Check()
* Simplify the internal _PyUnicode_Name_CAPI structure:
* Remove size and state members
* Remove state and self parameters of getcode() and getname()
functions
* Remove global_module_state
The private _PyUnicode_Name_CAPI structure of the PyCapsule API
unicodedata.ucnhash_CAPI moves to the internal C API. Moreover, the
structure gets a new state member which must be passed to the
getcode() and getname() functions.
* Move Include/ucnhash.h to Include/internal/pycore_ucnhash.h
* unicodedata module is now built with Py_BUILD_CORE_MODULE.
* unicodedata: move hashAPI variable into unicodedata_module_state.
Remove complex special methods __int__, __float__, __floordiv__,
__mod__, __divmod__, __rfloordiv__, __rmod__ and __rdivmod__
which always raised a TypeError.
This special marker annotation is intended to help in distinguishing
proper PEP 484-compliant type aliases from regular top-level variable
assignments.
The hard part was making all the tests pass; there are some subtle issues here, because apparently the future import wasn't tested very thoroughly in previous Python versions.
For example, `inspect.signature()` returned type objects normally (except for forward references), but strings with the future import. We changed it to try and return type objects by calling `typing.get_type_hints()`, but fall back on returning strings if that function fails (which it may do if there are future references in the annotations that require passing in a specific namespace to resolve).
This commit reverts commit ac0333e1e1 as the original links are working again and they provide extended features such as comments and alternative versions.
Remove the global _Py_CheckRecursionLimit variable: it has been
replaced by ceval.recursion_limit of the PyInterpreterState
structure.
There is no need to keep the variable for the stable ABI, since
Py_EnterRecursiveCall() and Py_LeaveRecursiveCall() were not usable
in Python 3.8 and older: these macros accessed PyThreadState members,
whereas the PyThreadState structure is opaque in the limited C API.
Sphinx 3 requires to refer to terms with the exact case.
For example, fix the Sphinx 3 warning:
Doc/library/pkgutil.rst:71: WARNING: term Loader not found in case
sensitive match.made a reference to loader instead.
For example, fix the following Sphinx 3 errors:
Doc/c-api/buffer.rst:102: WARNING: Error in declarator or parameters
Invalid C declaration: Expected identifier in nested name. [error at 5]
void \*obj
-----^
Doc/c-api/arg.rst:130: WARNING: Unparseable C cross-reference: 'PyObject*'
Invalid C declaration: Expected end of definition. [error at 8]
PyObject*
--------^
The modified documentation is compatible with Sphinx 2 and Sphinx 3.
#msg373510
[bpo-32528]()/#13528 changed `asyncio.CancelledError` such that it no longer inherits from `concurrent.futures.CancelledError`. As this affects existing code, specifically when catching the latter instead of the former in exception handling, it should be documented in the "What's new in 3.8?" document.
Automerge-Triggered-By: @1st1
Add sys.orig_argv attribute: the list of the original command line
arguments passed to the Python executable.
Rename also PyConfig._orig_argv to PyConfig.orig_argv and
document it.
The PY_SSIZE_T_CLEAN macro must now be defined to use
PyArg_ParseTuple() and Py_BuildValue() "#" formats: "es#", "et#",
"s#", "u#", "y#", "z#", "U#" and "Z#". See the PEP 353.
Update _testcapi.test_buildvalue_issue38913().
The C99 functions snprintf() and vsnprintf() are now required
to build Python.
PyOS_snprintf() and PyOS_vsnprintf() no longer call Py_FatalError().
Previously, they called Py_FatalError() on a buffer overflow on platforms
which don't provide vsnprintf().
The topological sort functionality that was introduced initially in the
functools module has been moved to a new graphlib module to
better accommodate the new tools and keep the original scope of the
functools module.
* bpo-29882: Add an efficient popcount method for integers
* Update 'sign bit' and versionadded in docs
* Add entry to whatsnew document
* Doc: use positive example, mention population count
* Minor cleanups of the core code
* Move popcount_digit closer to where it's used
* Use z instead of self after conversion
* Add 'absolute value' and 'population count' to docstring
* Fix clinic error about missing summary line
* Ensure popcount_digit is portable with 64-bit ints
Co-authored-by: Mark Dickinson <dickinsm@gmail.com>
Try to make the meaning of platlibdir clear. The previous wording could
be misinterpreted to suggest that it will be used to find all shared
libraries on the system, and not just Python extensions. Furthermore,
it was unclear whether it affects third-party (site-packages) extensions
or not. The new wording tries to make its dual purpose clear,
and provide the additional example of extensions in site-packages.
Previously, the result could have been an instance of a subclass of int.
Also revert bpo-26202 and make attributes start, stop and step of the range
object having exact type int.
Add private function _PyNumber_Index() which preserves the old behavior
of PyNumber_Index() for performance to use it in the conversion functions
like PyLong_AsLong().
Convert Py_REFCNT() and Py_SIZE() macros to static inline functions.
They cannot be used as l-value anymore: use Py_SET_REFCNT() and
Py_SET_SIZE() to set an object reference count and size.
Replace &Py_SIZE(self) with &((PyVarObject*)self)->ob_size
in arraymodule.c.
This change is backward incompatible on purpose, to prepare the C API
for an opaque PyObject structure.
The reset_peak function sets the peak memory size to the current size,
representing a resetting of that metric. This allows for recording the
peak of specific sections of code, ignoring other code that may have
had a higher peak (since the most recent `tracemalloc.start()` or
tracemalloc.clear_traces()` call).
- Fix upload test on systems that blocks MD5
- Add SHA2-256 and Blake2b-256 digests based on new Warehous and twine
specs.
Signed-off-by: Christian Heimes <christian@python.org>
Implements `asyncio.to_thread`, a coroutine for asynchronously running IO-bound functions in a separate thread without blocking the event loop. See the discussion starting from [here](https://github.com/python/cpython/pull/18410#issuecomment-628930973) in GH-18410 for context.
Automerge-Triggered-By: @aeros
This reverts commit 0da5466650.
The commit is causing make failures on a FreeBSD buildbot.
Due to the imminent 3.9.0b1 cutoff, revert this commit for
now pending further investigation.
Add support to the configure script for OBJC and OBJCXX command line options so that the macOS builds can use the clang compiler for the macOS-specific Objective C source files. This allows third-party compilers, like GNU gcc, to be used to build the rest of the project since some of the Objective C system header files are not compilable by GNU gcc.
Co-authored-by: Jeffrey Kintscher <websurfer@surf2c.net>
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
{date, datetime}.isocalendar() now return a private custom named tuple object
IsoCalendarDate rather than a simple tuple.
In order to leave IsocalendarDate as a private class and to improve what
backwards compatibility is offered for pickling the result of a
datetime.isocalendar() call, add a __reduce__ method to the named tuples that
reduces them to plain tuples. (This is the part of this PR most likely to cause
problems — if it causes major issues, switching to a strucseq or equivalent
would be prudent).
The pure python implementation of IsoCalendarDate uses positional-only
arguments, since it is private and only constructed by position anyway; the
equivalent change in the argument clinic on the C side would require us to move
the forward declaration of the type above the clinic import for whatever
reason, so it seems preferable to hold off on that for now.
bpo-24416: https://bugs.python.org/issue24416
Original PR by Dong-hee Na with only minor alterations by Paul Ganssle.
Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>
On AIX, time.thread_time() is now implemented with thread_cputime()
which has nanosecond resolution, rather than
clock_gettime(CLOCK_THREAD_CPUTIME_ID) which has a resolution of 10 ms.
compileall is now able to use hardlinks to prevent duplicates in a
case when .pyc files for different optimization levels have the same content.
Co-authored-by: Miro Hrončok <miro@hroncok.cz>
Co-authored-by: Victor Stinner <vstinner@python.org>
Move Build Changes and C API Changes to the end of the document.
Most Python users don't build Python themselves and don't use the C
API. Other changes:
* Add Build Changes section
* Add sub-sections to the C API Changes
* Sort modules in Improved Modules section: move nntplib after
multiprocessing
New PyFrame_GetBack() function: get the frame next outer frame.
Replace frame->f_back with PyFrame_GetBack(frame) in most code but
frameobject.c, ceval.c and genobject.c.
Remove the following function from the C API:
* PyAsyncGen_ClearFreeLists()
* PyContext_ClearFreeList()
* PyDict_ClearFreeList()
* PyFloat_ClearFreeList()
* PyFrame_ClearFreeList()
* PyList_ClearFreeList()
* PySet_ClearFreeList()
* PyTuple_ClearFreeList()
Make these functions private, move them to the internal C API and
change their return type to void.
Call explicitly PyGC_Collect() to free all free lists.
Note: PySet_ClearFreeList() did nothing.
PyFrame_GetCode(frame): return a borrowed reference to the frame
code.
Replace frame->f_code with PyFrame_GetCode(frame) in most code,
except in frameobject.c, genobject.c and ceval.c.
Also add PyFrame_GetLineNumber() to the limited C API.
Add a new separated pyframe.h header file of the PyFrame public C
API: it is included by Python.h.
Add PyFrame_GetLineNumber() to the limited C API.
Replace "struct _frame" with "PyFrameObject" in header files.
PyFrameObject is now defined as struct _frame by pyframe.h which is
included early enough in Python.h.
Optimize the subprocess module on FreeBSD using closefrom().
A single close(fd) syscall is cheap, but when sysconf(_SC_OPEN_MAX)
is high, the loop calling close(fd) on each file descriptor can take
several milliseconds.
The workaround on FreeBSD to improve performance was to load and
mount the fdescfs kernel module, but this is not enabled by default.
Initial patch by Ed Maste (emaste), Conrad Meyer (cem), Kyle Evans
(kevans) and Kubilay Kocak (koobs):
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=242274
Added str.removeprefix and str.removesuffix methods and corresponding
bytes, bytearray, and collections.UserString methods to remove affixes
from a string if present. See PEP 616 for a full description.
The constant values of future flags in the __future__ module
is updated in order to prevent collision with compiler flags.
Previously PyCF_ALLOW_TOP_LEVEL_AWAIT was clashing
with CO_FUTURE_DIVISION.
Add random.randbytes() function and random.Random.randbytes()
method to generate random bytes.
Modify secrets.token_bytes() to use SystemRandom.randbytes()
rather than calling directly os.urandom().
Rename also genrand_int32() to genrand_uint32(), since it returns an
unsigned 32-bit integer, not a signed integer.
The _random module is now built with Py_BUILD_CORE_MODULE defined.
Add the encoding in ftplib.FTP and ftplib.FTP_TLS to the
constructor as keyword-only and change the default from "latin-1" to "utf-8"
to follow RFC 2640.
* bpo-39011: Preserve line endings within attributes
Line endings within attributes were previously normalized to "\n" in Py3.7/3.8.
This patch removes that normalization, as line endings which were
replaced by entity numbers should be preserved in original form.
Add the functions PyObject_GC_IsTracked and PyObject_GC_IsFinalized to the public API to allow to query if Python objects are being currently tracked or have been already finalized by the garbage collector respectively.
Add os.waitstatus_to_exitcode() function to convert a wait status to an
exitcode.
Suggest waitstatus_to_exitcode() usage in the documentation when
appropriate.
Use waitstatus_to_exitcode() in:
* multiprocessing, os, subprocess and _bootsubprocess modules;
* test.support.wait_process();
* setup.py: run_command();
* and many tests.
Remove daemon threads from :mod:`concurrent.futures` by adding
an internal `threading._register_atexit()`, which calls registered functions
prior to joining all non-daemon threads. This allows for compatibility
with subinterpreters, which don't support daemon threads.
* Re-add removed classes Suite, slice, Param, AugLoad and AugStore.
* Add docstrings for dummy classes.
* Add docstrings for attribute aliases.
* Set __module__ to "ast" instead of "_ast".
Remove _PyRuntime.getframe hook and remove _PyThreadState_GetFrame
macro which was an alias to _PyRuntime.getframe. They were only
exposed by the internal C API. Remove also PyThreadFrameGetter type.
If a thread different than the main thread gets a signal, the
bytecode evaluation loop is no longer interrupted at each bytecode
instruction to check for pending signals which cannot be handled.
Only the main thread of the main interpreter can handle signals.
Previously, the bytecode evaluation loop was interrupted at each
instruction until the main thread handles signals.
Changes:
* COMPUTE_EVAL_BREAKER() and SIGNAL_PENDING_SIGNALS() no longer set
eval_breaker to 1 if the current thread cannot handle signals.
* take_gil() now always recomputes eval_breaker.
If Py_AddPendingCall() is called in a subinterpreter, the function is
now scheduled to be called from the subinterpreter, rather than being
called from the main interpreter.
Each subinterpreter now has its own list of scheduled calls.
* Move pending and eval_breaker fields from _PyRuntimeState.ceval
to PyInterpreterState.ceval.
* new_interpreter() now calls _PyEval_InitThreads() to create
pending calls lock.
* Fix Py_AddPendingCall() for subinterpreters. It now calls
_PyThreadState_GET() which works in a subinterpreter if the
caller holds the GIL, and only falls back on
PyGILState_GetThisThreadState() if _PyThreadState_GET()
returns NULL.
Extension modules: m_traverse, m_clear and m_free functions of
PyModuleDef are no longer called if the module state was requested
but is not allocated yet. This is the case immediately after the
module is created and before the module is executed (Py_mod_exec
function). More precisely, these functions are not called if m_size is
greater than 0 and the module state (as returned by
PyModule_GetState()) is NULL.
Extension modules without module state (m_size <= 0) are not affected.
Co-Authored-By: Petr Viktorin <encukou@gmail.com>
* Rename _PyInterpreterState_Get() to PyInterpreterState_Get() and
move it the limited C API.
* Add _PyInterpreterState_Get() alias to PyInterpreterState_Get() for
backward compatibility with Python 3.8.
PyInterpreterState.eval_frame function now requires a tstate (Python
thread state) parameter.
Add private functions to the C API to get and set the frame
evaluation function:
* Add tstate parameter to _PyFrameEvalFunction function type.
* Add _PyInterpreterState_GetEvalFrameFunc() and
_PyInterpreterState_SetEvalFrameFunc() functions.
* Add tstate parameter to _PyEval_EvalFrameDefault().
Received email on the docs mailing list to fix a typo from `sys.platlitdir` which doesn't exist to the correct new attribute `sys.platlibdir`
Automerge-Triggered-By: @vstinner
* Remove the slice type.
* Make Slice a kind of the expr type instead of the slice type.
* Replace ExtSlice(slices) with Tuple(slices, Load()).
* Replace Index(value) with a value itself.
All non-terminal nodes in AST for expressions are now of the expr type.
Add --with-platlibdir option to the configure script: name of the
platform-specific library directory, stored in the new sys.platlitdir
attribute. It is used to build the path of platform-specific dynamic
libraries and the path of the standard library.
It is equal to "lib" on most platforms. On Fedora and SuSE, it is
equal to "lib64" on 64-bit systems.
Co-Authored-By: Jan Matějek <jmatejek@suse.com>
Co-Authored-By: Matěj Cepl <mcepl@cepl.eu>
Co-Authored-By: Charalampos Stratakis <cstratak@redhat.com>
The AST "Suite" node is no longer used and it can be removed from the ASDL definition and related structures (compiler, visitors, ...).
Co-Authored-By: Victor Stinner <vstinner@python.org>
Co-authored-by: Brett Cannon <54418+brettcannon@users.noreply.github.com>
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
Add a section in What's New In Python 3.9 to strongly advice to check
for DeprecationWarning in your Python projects.
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
Remove reference to sys.path[0] being absolute path in whatsnew
Co-Authored-By: Kyle Stanley <aeros167@gmail.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Kyle Stanley <aeros167@gmail.com>
* bpo-39648: Expand math.gcd() and math.lcm() to handle multiple arguments.
* Simplify fast path.
* Difine lcm() without arguments returning 1.
* Apply suggestions from code review
Co-Authored-By: Mark Dickinson <dickinsm@gmail.com>
Co-authored-by: Mark Dickinson <dickinsm@gmail.com>
* Hard reset + cherry piciking the changes.
* 📜🤖 Added by blurb_it.
* Added @vstinner News
* Update Misc/NEWS.d/next/Library/2020-02-11-13-01-38.bpo-38691.oND8Sk.rst
Co-Authored-By: Victor Stinner <vstinner@python.org>
* Hard reset to master
* Hard reset to master + latest changes
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
* bpo-39491: Merge PEP 593 (typing.Annotated) support
PEP 593 has been accepted some time ago. I got a green light for merging
this from Till, so I went ahead and combined the code contributed to
typing_extensions[1] and the documentation from the PEP 593 text[2].
My changes were limited to:
* removing code designed for typing_extensions to run on older Python
versions
* removing some irrelevant parts of the PEP text when copying it over as
documentation and otherwise changing few small bits to better serve
the purpose
* changing the get_type_hints signature to match reality (parameter
names)
I wasn't entirely sure how to go about crediting the authors but I used
my best judgment, let me know if something needs changing in this
regard.
[1] 8280de241f/typing_extensions/src_py3/typing_extensions.py
[2] 17710b8798/pep-0593.rst
The os.putenv() and os.unsetenv() functions are now always available.
On non-Windows platforms, Python now requires setenv() and unsetenv()
functions to build.
Remove putenv_dict from posixmodule.c: it's not longer needed.
Deprecate binhex4 and hexbin4 standards. Deprecate the binhex module
and the following binascii functions:
* b2a_hqx(), a2b_hqx()
* rlecode_hqx(), rledecode_hqx()
* crc_hqx()
Remove the buffering parameter of bz2.BZ2File. Since Python 3.0, it
was ignored and using it was emitting a DeprecationWarning. Pass an
open file object to control how the file is opened.
The compresslevel parameter becomes keyword-only.
Remove base64.encodestring() and base64.decodestring(), aliases
deprecated since Python 3.1: use base64.encodebytes() and
base64.decodebytes() instead.
nntplib.NNTP and nntplib.NNTP_SSL now raise a ValueError
if the given timeout for their constructor is zero to
prevent the creation of a non-blocking socket.
poplib.POP3 and poplib.POP3_SSL now raise a ValueError
if the given timeout for their constructor is zero to
prevent the creation of a non-blocking socket.
imaplib.IMAP4 and imaplib.IMAP4_SSL now have an
optional *timeout* parameter for their constructors.
Also, the imaplib.IMAP4.open() method now has an optional *timeout* parameter
with this change. The overridden methods of imaplib.IMAP4_SSL and
imaplib.IMAP4_stream were applied to this change.
Add ast.unparse() as a function in the ast module that can be used to unparse an
ast.AST object and produce a string with code that would produce an equivalent ast.AST
object when parsed.
Remove PyMethod_ClearFreeList() and PyCFunction_ClearFreeList()
functions: the free lists of bound method objects have been removed.
Remove also _PyMethod_Fini() and _PyCFunction_Fini() functions.
This exposes a Linux-specific syscall for sending a signal to a process
identified by a file descriptor rather than a pid.
For simplicity, we don't support the siginfo_t parameter to the syscall. This
parameter allows implementing a pidfd version of rt_sigqueueinfo(2), which
Python also doesn't support.
Lysandros Nikolaou
Miro Hrončok
Victor Stinner
Tom Gringauz
Erlend Egeberg Aasland
Pablo Galindo
Brett Cannon
Christian Heimes
Mario Corchero
Hai Shi
Joongi Kim
Zackery Spytz
Dong-hee Na
kj
Matti Picus
Terry Jan Reedy
Alex Gaynor
Shantanu
Raymond Hettinger
Vladimir Matveev
Serhiy Storchaka
Mikhail Golubev
Batuhan Taskaya
Andre Delfino
Bas van Beek
Mark Dickinson
Ram Rachum
Petr Viktorin
Filipe Laíns
pxinwr
Hans Petter Jansson
Gregory Schevchenko
Berker Peksag
JustAnotherArchivist
Dong-hee Na
Rémi Lapeyre
Inada Naoki
Dennis Sweeney
Huon Wilson
Niklas Fiekas
Michał Górny
Elazar Gershuni
Florian Dahlitz
karl ding
Ned Deily
Kyle Stanley
Shantanu
Paul Ganssle
Batuhan Taskaya
Lumír 'Frenzy' Balhar
Javier Buzzi
Guido van Rossum
Sebastian Pedersen
mefistotelis
Ammar Askar
Batuhan Taşkaya
Ross
Rahul Kumaresan
Raúl Cumplido
Benjamin Peterson
idomic
Brandt Bucher
MojoVampire
Ananthakrishnan
opavlyuk
Hugo van Kemenade
Jakub Stasiak
Baljak
Chris Withers
Andrew Svetlov
fireattack
Yoni Lavi