Add sys.orig_argv attribute: the list of the original command line
arguments passed to the Python executable.
Rename also PyConfig._orig_argv to PyConfig.orig_argv and
document it.
- Mention that some compiler optimizations might not roundtrip
exactly (such as constant tuples and frozensets).
- Add a warning about it might raise RecursionError on very
complex expressions due to recursive unparsing aspect of ast.unparse
* Add a glossary entry for the term "callback"
* Link to it in loop.call_soon() and in the "Concurrency and Multithreading" section
Co-authored-by: Kyle Stanley <aeros167@gmail.com>
…choices' argument of 'argparse.ArgumentParser.add_argument'.
Here's a short first proposal of doc. enhancement addressing [bpo-41024]().
Automerge-Triggered-By: @csabella
The PY_SSIZE_T_CLEAN macro must now be defined to use
PyArg_ParseTuple() and Py_BuildValue() "#" formats: "es#", "et#",
"s#", "u#", "y#", "z#", "U#" and "Z#". See the PEP 353.
Update _testcapi.test_buildvalue_issue38913().
On Linux, skip tests using multiprocessing if the current user cannot
create a file in /dev/shm/ directory. Add the
skip_if_broken_multiprocessing_synchronize() function to the
test.support module.
The C99 functions snprintf() and vsnprintf() are now required
to build Python.
PyOS_snprintf() and PyOS_vsnprintf() no longer call Py_FatalError().
Previously, they called Py_FatalError() on a buffer overflow on platforms
which don't provide vsnprintf().
Remove duplication in `threading.Thread.native_id` documentation, so resulting documentation is more consistent with the `threading.Thread.ident`.
Issue initially raised [here](https://github.com/python/python-docs-fr/pull/1122#discussion_r369236634) (in French).
No issue associated to this PR.
Automerge-Triggered-By: @csabella
This commit removes the old parser, the deprecated parser module, the old parser compatibility flags and environment variables and all associated support code and documentation.
Move TransientResource, time_out, socket_peer_reset and
ioerror_peer_reset from test.support to test_urllib2net.
Remove "import errno" from test.support.
The doccumentation at https://docs.python.org/3.10/library/weakref.html cautions that the `WeakKeyDictionary` and `WeakValueDictionary` are susceptible to the problem of dictionary mutation during iteration.
These notes present the user with a problem that has no easy solution.
I dug into the implementation and found that fortunately, Antoine Pitrou already addressed this challenge (10 years ago!) by introducing an `_IterationGuard` context manager to the implementation, which delays mutation while an iteration is in progress.
I asked for confirmation and @pitrou agreed that these notes could be removed:
c1baa601e2 (commitcomment-39514438)
Export explicitly the Py_GetArgcArgv() function to the C API and
document the function. Previously, it was exported implicitly which
no longer works since Python is built with -fvisibility=hidden.
* Add PyConfig._orig_argv member.
* Py_InitializeFromConfig() no longer calls _PyConfig_Write() twice.
* PyConfig_Read() no longer initializes Py_GetArgcArgv(): it is now
_PyConfig_Write() responsibility.
* _PyConfig_Write() result type becomes PyStatus instead of void.
* Write an unit test on Py_GetArgcArgv().
Before
```
content.txt is 42 bytes in size and isa regular file.
folder is 420 bytes in size and isa directory.
magic is 4200 bytes in size and issomething else.
```
After:
```
content.txt is 42 bytes in size and is a regular file.
folder is 420 bytes in size and is a directory.
magic is 4200 bytes in size and is something else.
```
Automerge-Triggered-By: @orsenthil
The topological sort functionality that was introduced initially in the
functools module has been moved to a new graphlib module to
better accommodate the new tools and keep the original scope of the
functools module.
When I wrote the documentation for `asyncio.to_thread()`, I mistakenly assumed that `return await loop.run_in_executor(...)` within an async def function would return a Future. In reality, it returns a coroutine.
This likely won't affect typical usage of `asyncio.to_thread()`, but it's important for the documentation to be correct here. In general, we also tend to avoid returning futures from high-level APIs in asyncio.
* bpo-29882: Add an efficient popcount method for integers
* Update 'sign bit' and versionadded in docs
* Add entry to whatsnew document
* Doc: use positive example, mention population count
* Minor cleanups of the core code
* Move popcount_digit closer to where it's used
* Use z instead of self after conversion
* Add 'absolute value' and 'population count' to docstring
* Fix clinic error about missing summary line
* Ensure popcount_digit is portable with 64-bit ints
Co-authored-by: Mark Dickinson <dickinsm@gmail.com>
Try to make the meaning of platlibdir clear. The previous wording could
be misinterpreted to suggest that it will be used to find all shared
libraries on the system, and not just Python extensions. Furthermore,
it was unclear whether it affects third-party (site-packages) extensions
or not. The new wording tries to make its dual purpose clear,
and provide the additional example of extensions in site-packages.
Previously, the result could have been an instance of a subclass of int.
Also revert bpo-26202 and make attributes start, stop and step of the range
object having exact type int.
Add private function _PyNumber_Index() which preserves the old behavior
of PyNumber_Index() for performance to use it in the conversion functions
like PyLong_AsLong().
* Use a more universal explanation of string interpolation rather than specifically referencing sprintf(), which depends on the reader having a C background.
Co-authored-by: Kyle Stanley <aeros167@gmail.com>
hashlib.compare_digest uses OpenSSL's CRYPTO_memcmp() function
when OpenSSL is available.
Note: The _operator module is a builtin module. I don't want to add
libcrypto dependency to libpython. Therefore I duplicated the wrapper
function and added a copy to _hashopenssl.c.
Convert Py_REFCNT() and Py_SIZE() macros to static inline functions.
They cannot be used as l-value anymore: use Py_SET_REFCNT() and
Py_SET_SIZE() to set an object reference count and size.
Replace &Py_SIZE(self) with &((PyVarObject*)self)->ob_size
in arraymodule.c.
This change is backward incompatible on purpose, to prepare the C API
for an opaque PyObject structure.
* Fix outdated __int__ and nb_int references in comments
* Also update C-API documentation
* Add back missing 'method' word
* Remove .. deprecated notices
* Fix as_completed docs to correctly state the function return value.
* Also, improves the general wording of the as_completed documentation.
Co-Authored-By: Rémi Lapeyre <remi.lapeyre@henki.fr>
Co-Authored-By: Kyle Stanley <aeros167@gmail.com>
Co-Authored-By: Yury Selivanov <yury@edgedb.com>
The reset_peak function sets the peak memory size to the current size,
representing a resetting of that metric. This allows for recording the
peak of specific sections of code, ignoring other code that may have
had a higher peak (since the most recent `tracemalloc.start()` or
tracemalloc.clear_traces()` call).
- Fix upload test on systems that blocks MD5
- Add SHA2-256 and Blake2b-256 digests based on new Warehous and twine
specs.
Signed-off-by: Christian Heimes <christian@python.org>
Implements `asyncio.to_thread`, a coroutine for asynchronously running IO-bound functions in a separate thread without blocking the event loop. See the discussion starting from [here](https://github.com/python/cpython/pull/18410#issuecomment-628930973) in GH-18410 for context.
Automerge-Triggered-By: @aeros
```
Warning, treated as error:
/tmp/code/Doc/library/zoneinfo.rst:303:Error in "code-block" directive:
1 argument(s) required, 0 supplied.
.. code-block::
>>> a = ZoneInfo("Europe/Berlin")
>>> b = pickle.loads(europe_berlin_pkl)
>>> a is b
True
```
This reverts commit 0da5466650.
The commit is causing make failures on a FreeBSD buildbot.
Due to the imminent 3.9.0b1 cutoff, revert this commit for
now pending further investigation.
Add support to the configure script for OBJC and OBJCXX command line options so that the macOS builds can use the clang compiler for the macOS-specific Objective C source files. This allows third-party compilers, like GNU gcc, to be used to build the rest of the project since some of the Objective C system header files are not compilable by GNU gcc.
Co-authored-by: Jeffrey Kintscher <websurfer@surf2c.net>
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
This was not specified in the PEP, but it will likely be a frequently requested feature if it's not included.
This includes only the "canonical" zones, not a simple listing of every valid value of `key` that can be passed to `Zoneinfo`, because it seems likely that that's what people will want.
{date, datetime}.isocalendar() now return a private custom named tuple object
IsoCalendarDate rather than a simple tuple.
In order to leave IsocalendarDate as a private class and to improve what
backwards compatibility is offered for pickling the result of a
datetime.isocalendar() call, add a __reduce__ method to the named tuples that
reduces them to plain tuples. (This is the part of this PR most likely to cause
problems — if it causes major issues, switching to a strucseq or equivalent
would be prudent).
The pure python implementation of IsoCalendarDate uses positional-only
arguments, since it is private and only constructed by position anyway; the
equivalent change in the argument clinic on the C side would require us to move
the forward declaration of the type above the clinic import for whatever
reason, so it seems preferable to hold off on that for now.
bpo-24416: https://bugs.python.org/issue24416
Original PR by Dong-hee Na with only minor alterations by Paul Ganssle.
Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>
On AIX, time.thread_time() is now implemented with thread_cputime()
which has nanosecond resolution, rather than
clock_gettime(CLOCK_THREAD_CPUTIME_ID) which has a resolution of 10 ms.
Currently, if asyncio.wait_for() timeout expires, it cancels
inner future and then always raises TimeoutError. In case
those future is task, it can handle cancelation mannually,
and those process can lead to some other exception. Current
implementation silently loses thoses exception.
To resolve this, wait_for will check was the cancelation
successfull or not. In case there was exception, wait_for
will reraise it.
Co-authored-by: Roman Skurikhin <roman.skurikhin@cruxlab.com>
compileall is now able to use hardlinks to prevent duplicates in a
case when .pyc files for different optimization levels have the same content.
Co-authored-by: Miro Hrončok <miro@hroncok.cz>
Co-authored-by: Victor Stinner <vstinner@python.org>
* bpo-39791: Update importlib.resources to support files() API (importlib_resources 1.5).
* 📜🤖 Added by blurb_it.
* Add some documentation about the new objects added.
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Module C state is now accessible from C-defined heap type methods (PEP 573).
Patch by Marcel Plch and Petr Viktorin.
Co-authored-by: Marcel Plch <mplch@redhat.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
An isolated subinterpreter cannot spawn threads, spawn a child
process or call os.fork().
* Add private _Py_NewInterpreter(isolated_subinterpreter) function.
* Add isolated=True keyword-only parameter to
_xxsubinterpreters.create().
* Allow again os.fork() in "non-isolated" subinterpreters.
Move Build Changes and C API Changes to the end of the document.
Most Python users don't build Python themselves and don't use the C
API. Other changes:
* Add Build Changes section
* Add sub-sections to the C API Changes
* Sort modules in Improved Modules section: move nntplib after
multiprocessing
New PyFrame_GetBack() function: get the frame next outer frame.
Replace frame->f_back with PyFrame_GetBack(frame) in most code but
frameobject.c, ceval.c and genobject.c.
Remove the following function from the C API:
* PyAsyncGen_ClearFreeLists()
* PyContext_ClearFreeList()
* PyDict_ClearFreeList()
* PyFloat_ClearFreeList()
* PyFrame_ClearFreeList()
* PyList_ClearFreeList()
* PySet_ClearFreeList()
* PyTuple_ClearFreeList()
Make these functions private, move them to the internal C API and
change their return type to void.
Call explicitly PyGC_Collect() to free all free lists.
Note: PySet_ClearFreeList() did nothing.
PyFrame_GetCode(frame): return a borrowed reference to the frame
code.
Replace frame->f_code with PyFrame_GetCode(frame) in most code,
except in frameobject.c, genobject.c and ceval.c.
Also add PyFrame_GetLineNumber() to the limited C API.
Add a new separated pyframe.h header file of the PyFrame public C
API: it is included by Python.h.
Add PyFrame_GetLineNumber() to the limited C API.
Replace "struct _frame" with "PyFrameObject" in header files.
PyFrameObject is now defined as struct _frame by pyframe.h which is
included early enough in Python.h.
Adds a short description of `PyDoc_STRVAR` and `PyDoc_STR` to "Useful macros" section of C-API docs.
Currently, there is [one lone mention](https://docs.python.org/3/c-api/module.html?highlight=pydoc_strvar#c.PyModuleDef) in the C-API reference, despite the fact that `PyDoc_STRVAR` is ubiquitous to `Modules/`.
Additionally, this properly uses `c:macro` within `Doc/c-api/module.rst` to link.
* Move socket related functions from test.support to socket_helper.
* Import socket, nntplib and urllib.error lazily in transient_internet().
* Remove importing multiprocess.
Optimize the subprocess module on FreeBSD using closefrom().
A single close(fd) syscall is cheap, but when sysconf(_SC_OPEN_MAX)
is high, the loop calling close(fd) on each file descriptor can take
several milliseconds.
The workaround on FreeBSD to improve performance was to load and
mount the fdescfs kernel module, but this is not enabled by default.
Initial patch by Ed Maste (emaste), Conrad Meyer (cem), Kyle Evans
(kevans) and Kubilay Kocak (koobs):
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=242274
Log "Warning -- ..." test warnings into sys.__stderr__ rather than
sys.stderr, to ensure to display them even if sys.stderr is captured.
test.libregrtest.utils.print_warning() now calls
test.support.print_warning().
* Rename PyConfig.use_peg to _use_peg_parser
* Document PyConfig._use_peg_parser and mark it a deprecated
* Mark -X oldparser option and PYTHONOLDPARSER env var as deprecated
in the documentation.
* Add use_old_parser() and skip_if_new_parser() to test.support
* Remove sys.flags.use_peg: use_old_parser() uses
_testinternalcapi.get_configs() instead.
* Enhance test_embed tests
* subprocess._args_from_interpreter_flags() copies -X oldparser
Added str.removeprefix and str.removesuffix methods and corresponding
bytes, bytearray, and collections.UserString methods to remove affixes
from a string if present. See PEP 616 for a full description.
The constant values of future flags in the __future__ module
is updated in order to prevent collision with compiler flags.
Previously PyCF_ALLOW_TOP_LEVEL_AWAIT was clashing
with CO_FUTURE_DIVISION.
* Add underscores to long numbers to improve readability
* Use bigger dataset in the bootstrapping example
* Convert single-server queue example to more useful multi-server queue
Fixes Issue39285
The example incorrectly returned True for match.
Furthermore the example is ambiguous in its usage of PureWindowsPath.
Windows is case-insensitve, however the underlying match functionality
utilizes fnmatch.fnmatchcase.
Automerge-Triggered-By: @pitrou
* Replace PY_INT64_T with int64_t
* Replace PY_UINT32_T with uint32_t
* Replace PY_UINT64_T with uint64_t
sha3module.c no longer checks if PY_UINT64_T is defined since it's
always defined and uint64_t is always available on platforms
supported by Python.
Add random.randbytes() function and random.Random.randbytes()
method to generate random bytes.
Modify secrets.token_bytes() to use SystemRandom.randbytes()
rather than calling directly os.urandom().
Rename also genrand_int32() to genrand_uint32(), since it returns an
unsigned 32-bit integer, not a signed integer.
The _random module is now built with Py_BUILD_CORE_MODULE defined.
The names "member" and "container" for the arguments are also used in the module and shown with the help() function, and are immediately understandable in this context, contrary to "first" and "second".
Add the encoding in ftplib.FTP and ftplib.FTP_TLS to the
constructor as keyword-only and change the default from "latin-1" to "utf-8"
to follow RFC 2640.
* bpo-39011: Preserve line endings within attributes
Line endings within attributes were previously normalized to "\n" in Py3.7/3.8.
This patch removes that normalization, as line endings which were
replaced by entity numbers should be preserved in original form.
Add the functions PyObject_GC_IsTracked and PyObject_GC_IsFinalized to the public API to allow to query if Python objects are being currently tracked or have been already finalized by the garbage collector respectively.
Add os.waitstatus_to_exitcode() function to convert a wait status to an
exitcode.
Suggest waitstatus_to_exitcode() usage in the documentation when
appropriate.
Use waitstatus_to_exitcode() in:
* multiprocessing, os, subprocess and _bootsubprocess modules;
* test.support.wait_process();
* setup.py: run_command();
* and many tests.
Moreover, the following tests now check the child process exit code:
* test_os.PtyTests
* test_mailbox.test_lock_conflict()
* test_tempfile.test_process_awareness()
* test_uuid.testIssue8621()
* multiprocessing resource tracker tests
Zackery Spytz
Pablo Galindo
Rémi Lapeyre
Kit Choi
tomerv
E-Paine
Inada Naoki
Victor Stinner
Stefan Krah
Batuhan Taskaya
Ram Rachum
Ned Deily
Arisaka97
Gareth Rees
Roger Iyengar
Brian Rutledge
Vincent Férotin
Serhiy Storchaka
Bar Harel
kevin seelbach
Antoine
Dennis Sweeney
Géry Ogam
native-api
Hai Shi
Daniel Fortunov
Éric Araujo
Brett Cannon
Sandro Mani
Shantanu
Lysandros Nikolaou
Jason R. Coombs
Harsha Laxman
Steve Dower
Huon Wilson
Raymond Hettinger
Kyle Stanley
Florian Dahlitz
Samuel Gaist
Niklas Fiekas
Windson yang
Ruaridh Williamson
Ramil Nugmanov
Michał Górny
Adorilson Bezerra
Elazar Gershuni
YoSTEALTH
Christian Heimes
Mark Dickinson
Ethan Steinberg
ziheng
sth
Rotuna
Dong-hee Na
Miro Hrončok
Jakub Stasiak
Albert
Matteo Bertucci
Sardorbek Imomaliev
Ville Skyttä
Andre Delfino
Mathieu Dupuy
karl ding
Joe DeCapo
Julien Palard
Anthony Sottile
Lumír 'Frenzy' Balhar
qudongfang
Paul Ganssle
Rahul Kumaresan
Batuhan Taskaya
Chris Jerdonek
romasku
Joannah Nanjekye
jack1142
Allen Guo
Ned Batchelder
Petr Viktorin
Naglis
Javier Buzzi
Joel Rosdahl
Mathieu Dupuy
Sander
Noah Doersing
lrjball
Alex Povel
Ethan Onstott
Brad Solomon
Heshy Roskes
Cajetan Rodrigues
Guido van Rossum
Carl Meyer
Tim Hoffmann
Tim Lo
Furkan Önder
Antoine Pitrou
Mariusz Felisiak
Christoph Zwerschke
Sebastian Pedersen
laike9m
mefistotelis
Rémi Lapeyre
amaajemyfren
Derek Keeler