Avoid linear runtime of ShareableList.__getitem__ and
ShareableList.__setitem__ by storing running allocated bytes in
ShareableList._allocated_bytes instead of the number of bytes for
a particular stored item.
Co-authored-by: Antoine Pitrou <antoine@python.org>
* Use ast module to find class definition
* Add NEWS entry
* Fix class with multiple children and move decorator code to the method
* Fix PR comments
1. Use node.decorator_list to select decorators
2. Remove unwanted variables in ClassVisitor
3. Simplify stack management as per review
* Add test for nested functions and async calls
* Fix pydoc test since comments are returned now correctly
* Set event loop policy as None to fix environment related change
* Refactor visit_AsyncFunctionDef and tests
* Refactor to use local variables and fix tests
* Add patch attribution
* Use self.addCleanup for asyncio
* Rename ClassVisitor to ClassFinder and fix asyncio cleanup
* Return first class inside conditional in case of multiple definitions. Remove decorator for class source.
* Add docstring to make the test correct
* Modify NEWS entry regarding decorators
* Return decorators too for bpo-15856
* Move ast and the class source code to top. Use proper Exception.
Add random.randbytes() function and random.Random.randbytes()
method to generate random bytes.
Modify secrets.token_bytes() to use SystemRandom.randbytes()
rather than calling directly os.urandom().
Rename also genrand_int32() to genrand_uint32(), since it returns an
unsigned 32-bit integer, not a signed integer.
The _random module is now built with Py_BUILD_CORE_MODULE defined.
* Replace flag-flip indirection with direct inspection
* Use any for simpler code
* Avoid flag flip and set results directly.
* Resolve processor in a single function.
* Extract processor handling into a namespace (class)
* Remove _syscmd_uname, unused
* Restore platform.processor behavior to match prior expectation (reliant on uname -p in a subprocess).
* Extract '_unknown_as_blank' function.
* Override uname_result to resolve the processor late.
* Add a test intended to capture the expected values from 'uname -p'
* Instead of trying to keep track of all of the possible outputs on different systems (probably a fool's errand), simply assert that except for the known platform variance, uname().processor matches the output of 'uname -p'
* Use a skipIf directive
* Use contextlib.suppress to suppress the error. Inline strip call.
* 📜🤖 Added by blurb_it.
* Remove use of contextlib.suppress (it would fail with NameError if it had any effect). Rely on _unknown_as_blank to replace unknown with blank.
Co-authored-by: blurb-it[bot] <blurb-it[bot]@users.noreply.github.com>
When there is a SyntaxError after reading the last input character from
the tokenizer and if no newline follows it, the error message used to be
`unexpected EOF while parsing`, which is wrong.
Add the encoding in ftplib.FTP and ftplib.FTP_TLS to the
constructor as keyword-only and change the default from "latin-1" to "utf-8"
to follow RFC 2640.
Fix a hang at fork in the logging module: the new private
_at_fork_reinit() method is now used to reinitialize locks at fork in
the child process.
The createLock() method is no longer used at fork.
Move the PyGC_Head structure and the following private macros to the
internal C API:
* _PyGCHead_FINALIZED()
* _PyGCHead_NEXT()
* _PyGCHead_PREV()
* _PyGCHead_SET_FINALIZED()
* _PyGCHead_SET_NEXT()
* _PyGCHead_SET_PREV()
* _PyGC_FINALIZED()
* _PyGC_PREV_MASK
* _PyGC_PREV_MASK_COLLECTING
* _PyGC_PREV_MASK_FINALIZED
* _PyGC_PREV_SHIFT
* _PyGC_SET_FINALIZED()
* _PyObject_GC_IS_TRACKED()
* _PyObject_GC_MAY_BE_TRACKED()
* _Py_AS_GC(o)
Keep the private _PyGC_FINALIZED() macro in the public C API for
backward compatibility with Python 3.8: make it an alias to the new
PyObject_GC_IsFinalized() function.
Move the SIZEOF_PYGC_HEAD constant from _testcapi module to
_testinternalcapi module.
* bpo-39011: Preserve line endings within attributes
Line endings within attributes were previously normalized to "\n" in Py3.7/3.8.
This patch removes that normalization, as line endings which were
replaced by entity numbers should be preserved in original form.
Add the functions PyObject_GC_IsTracked and PyObject_GC_IsFinalized to the public API to allow to query if Python objects are being currently tracked or have been already finalized by the garbage collector respectively.
Fix the signal handler: it now always uses the main interpreter,
rather than trying to get the current Python thread state.
The following function now accepts an interpreter, instead of a
Python thread state:
* _PyEval_SignalReceived()
* _Py_ThreadCanHandleSignals()
* _PyEval_AddPendingCall()
* COMPUTE_EVAL_BREAKER()
* SET_GIL_DROP_REQUEST(), RESET_GIL_DROP_REQUEST()
* SIGNAL_PENDING_CALLS(), UNSIGNAL_PENDING_CALLS()
* SIGNAL_PENDING_SIGNALS(), UNSIGNAL_PENDING_SIGNALS()
* SIGNAL_ASYNC_EXC(), UNSIGNAL_ASYNC_EXC()
Py_AddPendingCall() now uses the main interpreter if it fails to the
current Python thread state.
Convert _PyThreadState_GET() and PyInterpreterState_GET_UNSAFE()
macros to static inline functions.
Always declare PyIndex_Check() as an opaque function to hide
implementation details: remove PyIndex_Check() macro. The macro
accessed directly the PyTypeObject.tp_as_number member.
PyType_HasFeature() now always calls PyType_GetFlags() to hide
implementation details. Previously, it accessed directly the
PyTypeObject.tp_flags member when the limited C API was not used.
Add fast inlined version _PyType_HasFeature() and _PyType_IS_GC()
for object.c and typeobject.c.
The PyObject_NEW() macro becomes an alias to the PyObject_New()
macro, and the PyObject_NEW_VAR() macro becomes an alias to the
PyObject_NewVar() macro, to hide implementation details. They no
longer access directly the PyTypeObject.tp_basicsize member.
Exclude _PyObject_SIZE() and _PyObject_VAR_SIZE() macros from
the limited C API.
Replace PyObject_NEW() with PyObject_New() and replace
PyObject_NEW_VAR() with PyObject_NewVar().
Add a private _at_fork_reinit() method to _thread.Lock,
_thread.RLock, threading.RLock and threading.Condition classes:
reinitialize the lock after fork in the child process; reset the lock
to the unlocked state.
Rename also the private _reset_internal_locks() method of
threading.Event to _at_fork_reinit().
* Add _PyThread_at_fork_reinit() private function. It is excluded
from the limited C API.
* threading.Thread._reset_internal_locks() now calls
_at_fork_reinit() on self._tstate_lock rather than creating a new
Python lock object.
This implements things like `list[int]`,
which returns an object of type `types.GenericAlias`.
This object mostly acts as a proxy for `list`,
but has attributes `__origin__` and `__args__`
that allow recovering the parts (with values `list` and `(int,)`.
There is also an approximate notion of type variables;
e.g. `list[T]` has a `__parameters__` attribute equal to `(T,)`.
Type variables are objects of type `typing.TypeVar`.
str.encode() and str.decode() no longer check the encoding and errors
in development mode or in debug mode during Python finalization. The
codecs machinery can no longer work on very late calls to
str.encode() and str.decode().
This change should help to call _PyObject_Dump() to debug during late
Python finalization.
Convert the PyObject_GET_WEAKREFS_LISTPTR() macro to a function to
hide implementation details: the macro accessed directly to the
PyTypeObject.tp_weaklistoffset member.
Add _PyObject_GET_WEAKREFS_LISTPTR() static inline function to the
internal C API.
There are two copies of the grammar -- the one used by Python itself as
Grammar/Grammar, and the one used by lib2to3 which has necessarily diverged at
Lib/lib2to3/Grammar.txt because it needs to support older syntax an we want it
to be reasonable stable to avoid requiring fixer rewrites.
This brings suport for syntax like `if x:= foo():` to match what the live
Python grammar does.
This should've been added at the time of the walrus operator itself, but lib2to3 being
independent is often overlooked. So we do consider this a bugfix rather than enhancement.
CGIHTTPRequestHandler of http.server now logs the CGI script exit
code, rather than the CGI script exit status of os.waitpid().
For example, if the script is killed by signal 11, it now logs:
"CGI script exit code -11."
The AbstractBasicAuthHandler class of the urllib.request module uses
an inefficient regular expression which can be exploited by an
attacker to cause a denial of service. Fix the regex to prevent the
catastrophic backtracking. Vulnerability reported by Ben Caller
and Matt Schwager.
AbstractBasicAuthHandler of urllib.request now parses all
WWW-Authenticate HTTP headers and accepts multiple challenges per
header: use the realm of the first Basic challenge.
Co-Authored-By: Serhiy Storchaka <storchaka@gmail.com>
Add os.waitstatus_to_exitcode() function to convert a wait status to an
exitcode.
Suggest waitstatus_to_exitcode() usage in the documentation when
appropriate.
Use waitstatus_to_exitcode() in:
* multiprocessing, os, subprocess and _bootsubprocess modules;
* test.support.wait_process();
* setup.py: run_command();
* and many tests.
On Solaris, the regular "grep" command may be an old version that fails to search a binary file. We need to use the correct command (ggrep, in our case), which is found by the configure script earlier.
Automerge-Triggered-By: @pablogsal
Moreover, the following tests now check the child process exit code:
* test_os.PtyTests
* test_mailbox.test_lock_conflict()
* test_tempfile.test_process_awareness()
* test_uuid.testIssue8621()
* multiprocessing resource tracker tests
Fix a leak and subsequent crash in parsetok.c caused by realloc misuse on a rare codepath.
Realloc returns a null pointer on failure, and then growable_comment_array_deallocate crashes later when it dereferences it.
Speed up calls to list() by using the PEP 590 vectorcall
calling convention. Patch by Mark Shannon.
Co-authored-by: Mark Shannon <mark@hotpy.org>
Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>
Remove daemon threads from :mod:`concurrent.futures` by adding
an internal `threading._register_atexit()`, which calls registered functions
prior to joining all non-daemon threads. This allows for compatibility
with subinterpreters, which don't support daemon threads.
If fork was not called by a thread spawned by threading.Thread,
threading._after_fork() now creates a _MainThread instance for
_main_thread, instead of a _DummyThread instance.
Replace statically allocated types with heap allocated types:
use PyType_FromSpec().
Add a module state to store the Scanner and Encoder types.
Add traverse, clear and free functions to the module.
This pull request fixes the newline conversion bug originally reported in bpo-1812. When that issue was originally submitted, the open builtin did not default to universal newline mode; now it does, which makes the issue fix simpler, since the only code path that needs to be changed is the one in doctest._load_testfile where the file is loaded from a package whose loader has a get_data method.
Fix os.getgrouplist(): if getgrouplist() function fails because the
group list is too small, retry with a larger group list.
On failure, the glibc implementation of getgrouplist() sets ngroups
to the total number of groups. For other implementations, double the
group list size.
Remove two unused imports: _thread and _weakref. Avoid creating a new
winreg builtin module if it's already available in sys.modules.
The winreg module is now stored as "winreg" rather than "_winreg".
PyThreadState.frame is a borrowed reference, not a strong reference:
PyThreadState_Clear() must not call Py_CLEAR(tstate->frame).
Remove test_threading.test_warnings_at_exit(): we cannot warranty
that the Python thread state of daemon threads is cleared in a
reliable way during Python shutdown.
sweeneyde
HongWeipeng
Antoine Pitrou
Tim Hoffmann
Thomas Krennwallner
Kyle Stanley
Raymond Hettinger
Furkan Önder
Karthikeyan Singaravelan
Serhiy Storchaka
Victor Stinner
Inada Naoki
Batuhan Taşkaya
Jason R. Coombs
Ammar Askar
Russell Davis
Lysandros Nikolaou
Barry
Hai Shi
Sebastian Pedersen
Oren Milman
Alex Itkes
mefistotelis
Benjamin Peterson
Pablo Galindo
pxinwr
Zackery Spytz
Guido van Rossum
Steve Dower
Tal Einat
Chris Martinez
Tim Hatch
Dong-hee Na
Arnon Yaari
Barney Gale
Alexander Riccio
Petr Viktorin
Ashwin Ramaswami
Peter Donis
Lahfa Samy
Furkan Önder
Ram Rachum
Paulo Henrique Silva
Curtis Bucher
Russell Owen
Łukasz Langa