Commit Graph

118382 Commits

Author SHA1 Message Date
Victor Stinner c55e73112c
gh-106320: Remove private PyLong C API functions (#108429)
Remove private PyLong C API functions:

* _PyLong_AsByteArray()
* _PyLong_DivmodNear()
* _PyLong_Format()
* _PyLong_Frexp()
* _PyLong_FromByteArray()
* _PyLong_FromBytes()
* _PyLong_GCD()
* _PyLong_Lshift()
* _PyLong_Rshift()

Move these functions to the internal C API. No longer export
_PyLong_FromBytes() function.
2023-08-24 18:53:50 +02:00
A 7f31676340
Fix a code snippet typo in asyncio docs (#108427) 2023-08-24 16:27:54 +00:00
Victor Stinner b7808820b1
gh-107211: No longer export internal functions (5) (#108423)
No longer export _PyCompile_AstOptimize() internal C API function.

Change comment style to "// comment" and add comment explaining why
other functions have to be exported.
2023-08-24 16:06:53 +00:00
Victor Stinner f1ae706ca5
gh-107211: No longer export internal functions (7) (#108425)
No longer export _PyUnicode_FromId() internal C API function.

Change comment style to "// comment" and add comment explaining why
other functions have to be exported.

Update Tools/build/generate_token.py to update Include/internal/pycore_token.h
comments.
2023-08-24 17:40:56 +02:00
Victor Stinner 52c6a6e48a
gh-108308: Remove _PyDict_GetItemStringWithError() function (#108426)
Remove the internal _PyDict_GetItemStringWithError() function. It can
now be replaced with the new public PyDict_ContainsString() and
PyDict_GetItemStringRef() functions.

getargs.c now now uses a strong reference for current_arg.
find_keyword() returns a strong reference.
2023-08-24 17:34:22 +02:00
Victor Stinner ea871c9b0f
gh-107211: No longer export internal functions (6) (#108424)
No longer export these 5 internal C API functions:

* _PyArena_AddPyObject()
* _PyArena_Free()
* _PyArena_Malloc()
* _PyArena_New()
* _Py_FatalRefcountErrorFunc()

Change comment style to "// comment" and add comment explaining why
other functions have to be exported.
2023-08-24 17:28:35 +02:00
Victor Stinner 3f7e93be51
gh-107211: No longer export PyTime internal functions (#108422)
No longer export these 2 internal C API functions:

* _PyTime_AsNanoseconds()
* _PyTime_GetSystemClockWithInfo()

Change comment style to "// comment" and add comment explaining why
other functions have to be exported.
2023-08-24 17:17:40 +02:00
Victor Stinner a35d48d4bd
gh-108240: _PyCapsule_SetTraverse() rejects NULL callbacks (#108417) 2023-08-24 16:37:59 +02:00
Victor Stinner 6726626646
gh-108314: Add PyDict_ContainsString() function (#108323) 2023-08-24 15:59:12 +02:00
Jeffery To c163d7f0b6
gh-95855: Refactor platform triplet detection code, add detection for MIPS soft float and musl libc (#107221)
- Move platform triplet detection code into Misc/platform_triplet.c
- Refactor MIPS detection, use defined(__mips64) to detect MIPS64
- Compute libc values in separate section
- Add detection for MIPS soft float
- Add detection for musl

musl supports SPE with its soft-float ABI:
https://git.musl-libc.org/cgit/musl/commit/?id=7be59733d71ada3a32a98622507399253f1d5e48

Original patch by Christian Heimes.

Co-authored-by: Christian Heimes <christian@python.org>
Co-authored-by: Erlend E. Aasland <erlend@python.org>
2023-08-24 14:22:50 +02:00
Daniele Procida 809ea7c4b6
gh-107432 Update Porting Python 2 Code to Python 3 how-to (GH-107434)
https://docs.python.org/3/howto/pyporting.html#porting-python-2-code-to-python-3 was written for another time. In this patch:

- material that frames Python 3 as "new" is removed
- descriptions and directions have been trimmed
2023-08-24 11:28:23 +02:00
Chris Markiewicz 2eb60c1934
gh-108111: Flush gzip write buffer before seeking, fixing bad writes (#108341)
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2023-08-24 10:23:01 +02:00
Victor Stinner aa9a359ca2
gh-108388: Split test_multiprocessing_spawn (#108396)
Split test_multiprocessing_fork, test_multiprocessing_forkserver and
test_multiprocessing_spawn into test packages. Each package is made
of 4 sub-tests: processes, threads, manager and misc. It allows
running more tests in parallel and so reduce the total test duration.
2023-08-24 05:35:39 +02:00
Victor Stinner 174e9da083
gh-108388: regrtest splits test_asyncio package (#108393)
Currently, test_asyncio package is only splitted into sub-tests when
using command "./python -m test". With this change, it's also
splitted when passing it on the command line:
"./python -m test test_asyncio".

Remove the concept of "STDTESTS". Python is now mature enough to not
have to bother with that anymore. Removing STDTESTS simplify the
code.
2023-08-24 02:44:58 +00:00
Victor Stinner 7a6cc3eb66
test_peg_generator and test_freeze require cpu (#108386)
The test_peg_generator and test_tools.test_freeze tests now require
the 'cpu' resource. Skip these tests unless the 'cpu' resource is
enabled (it is disabled by default).

These tests are no longer skipped if Python is built with ASAN or
MSAN sanitizer.
2023-08-24 00:52:48 +02:00
Guido van Rossum 3107b453bc
gh-108253: Fix reads of uninitialized memory in funcobject.c (#108383) 2023-08-23 22:36:19 +00:00
Victor Stinner 513c89d012
gh-108240: Add _PyCapsule_SetTraverse() internal function (#108339)
The _socket extension uses _PyCapsule_SetTraverse() to visit and clear
the socket type in the garbage collector. So the _socket.socket type
can be cleared in some corner cases when it wasn't possible before.
2023-08-24 00:19:11 +02:00
Gregory P. Smith b6be18812c
gh-70766: Mention the object getstate caveat in 3.11 What's new. (#108379) 2023-08-23 15:00:09 -07:00
Victor Stinner 4890f65ecf
gh-108308: Use PyDict_GetItemRef() in moduleobject.c (#108381)
Replace PyDict_GetItemWithError() with PyDict_GetItemRef() which
returns a strong reference.

Cleanup also the function: move variable definition to their first
assignation, rename variable names to use name longer than 1
character.
2023-08-23 21:57:40 +00:00
Victor Stinner 592bacb6fc
gh-108342: Make ssl TestPreHandshakeClose more reliable (#108370)
* In preauth tests of test_ssl, explicitly break reference cycles
  invoving SingleConnectionTestServerThread to make sure that the
  thread is deleted. Otherwise, the test marks the environment as
  altered because the threading module sees a "dangling thread"
  (SingleConnectionTestServerThread). This test leak was introduced
  by the test added for the fix of issue gh-108310.
* Use support.SHORT_TIMEOUT instead of hardcoded 1.0 or 2.0 seconds
  timeout.
* SingleConnectionTestServerThread.run() catchs TimeoutError
* Fix a race condition (missing synchronization) in
  test_preauth_data_to_tls_client(): the server now waits until the
  client connect() completed in call_after_accept().
* test_https_client_non_tls_response_ignored() calls server.join()
  explicitly.
* Replace "localhost" with server.listener.getsockname()[0].
2023-08-23 21:57:11 +00:00
Victor Stinner ec3527d196
gh-108308: config_dict_get() uses PyDict_GetItemRef() (#108371)
Replace _PyDict_GetItemStringWithError() with PyDict_GetItemRef() in
config_dict_get() to get a strong reference to the item.
2023-08-23 23:12:08 +02:00
Victor Stinner 4dc9f48930
gh-108308: Replace _PyDict_GetItemStringWithError() (#108372)
Replace _PyDict_GetItemStringWithError() calls with
PyDict_GetItemStringRef() which returns a strong reference to the
item.

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2023-08-23 22:59:00 +02:00
albanD 1700d34d31
gh-77377: Ensure multiprocessing SemLock is valid for spawn-based Process before serializing it (#107275)
Ensure multiprocessing SemLock is valid for spawn Process before serializing it.

Creating a multiprocessing SemLock with a fork context, and then trying to pass it to a spawn-created Process, would segfault if not detected early.

---------

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Antoine Pitrou <pitrou@free.fr>
2023-08-23 20:27:35 +00:00
Petr Viktorin 5d18715765
gh-107811: tarfile: treat overflow in UID/GID as failure to set it (#108369) 2023-08-23 20:00:07 +02:00
Irit Katriel 72119d16a5
gh-105481: remove regen-opcode. Generated _PyOpcode_Caches in regen-cases. (#108367) 2023-08-23 18:39:00 +01:00
Serhiy Storchaka 422f81b5d2
gh-107298: Fix a few more refs in the C API docs (GH-108361)
gh-107298: Fix few more refs in the C API docs
2023-08-23 20:12:42 +03:00
Dong-hee Na 2135bcd3ca
gh-107265: Ensure de_instrument does not handle ENTER_EXECUTOR (#108366) 2023-08-23 08:45:20 -07:00
Nikita Sobolev 3f61cf646d
gh-108303: Move `ann_module*.py` files to `typinganndata/` folder (#108354) 2023-08-23 17:42:08 +02:00
Victor Stinner f5559f38d9
gh-108308: Replace PyDict_GetItem() with PyDict_GetItemRef() (#108309)
Replace PyDict_GetItem() calls with PyDict_GetItemRef()
or PyDict_GetItemWithError() to handle errors.

* Replace PyLong_AS_LONG() with _PyLong_AsInt()
  and check for errors.
* Check for PyDict_Contains() error.
* pycore_init_builtins() checks for _PyType_Lookup() failure.
2023-08-23 17:40:26 +02:00
Serhiy Storchaka 154477be72
gh-50002: xml.dom.minidom now preserves whitespaces in attributes (GH-107947)
Also double quotes (") are now only quoted in attributes.
2023-08-23 15:23:41 +03:00
Mariusz Felisiak 29bc6165ab
gh-105539: Fix ResourceWarning from unclosed SQLite connections in test_sqlite3 (#108360)
Follow up to 1a1bfc2891.

Explicitly manage connections in:

- test_audit.test_sqlite3
- test_sqlite3.test_audit
- test_sqlite3.test_backup

Co-authored-by: Erlend E. Aasland <erlend@python.org>
2023-08-23 10:10:08 +00:00
Petr Viktorin 31b61d19ab
gh-108294: Add time.sleep audit event (GH-108298) 2023-08-23 11:00:22 +02:00
Irit Katriel 2dfbd4f36d
gh-108113: Make it possible to optimize an AST (#108282) 2023-08-23 09:01:17 +01:00
FrozenBob 79fdacc005
gh-108267: Dataclasses docs: Fix object.__setattr__ typo (#108355)
Fixed a sentence in dataclasses.rst

Changed "__setattr__" to "object.__setattr__" in a section that was specifically supposed to refer to the __setattr__ method of the object class. Also suppressed the link to the data model docs for __setattr__, since we're talking about a specific __setattr__ implementation, not __setattr__ methods in general.
2023-08-23 08:11:15 +01:00
Victor Stinner 64f9935035
gh-108342: Break ref cycle in SSLSocket._create() exc (#108344)
Explicitly break a reference cycle when SSLSocket._create() raises an
exception. Clear the variable storing the exception, since the
exception traceback contains the variables and so creates a reference
cycle.

This test leak was introduced by the test added for the fix of #108310.
2023-08-23 05:26:01 +00:00
Victor Stinner 9173b2bbe1
gh-105776: Fix test_cppext when CC contains -std=c11 option (#108343)
Fix test_cppext when the C compiler command has the "-std=c11" option.
Remove "-std=" options from the compiler command.
2023-08-23 02:52:56 +00:00
Victor Stinner 3a1ac87f8f
gh-90791: test.pythoninfo logs ASAN_OPTIONS env var (#108289)
* Cleanup libregrtest code logging ASAN_OPTIONS.
* Fix a typo on "ASAN_OPTIONS" vs "MSAN_OPTIONS".
2023-08-23 01:39:50 +02:00
Alex Waygood a0bb4a39d1
Fix spurious diff if the cases generator is run on Windows (#108319) 2023-08-22 20:49:39 +01:00
Hugo van Kemenade fc23f34cc9
gh-107136: Remove Plausible for docs metrics (#107856) 2023-08-22 20:45:22 +02:00
Adam Turner e97b7bef4f
GH-92584: Remove distutils from the newtypes tutorial includes (#108024) 2023-08-22 20:42:51 +02:00
Hadházy Tamás 13966da71b
gh-105857: Document that asyncio subprocess std{in,out,err} can be file handles (#107986)
stdin/out can be filehandles -> add to docs.
2023-08-22 20:41:56 +02:00
Hugo van Kemenade 39de79b345
Document 3.13, 3.14 and future removals (#108055) 2023-08-22 20:40:46 +02:00
Hugo van Kemenade 35cb1605d0
Docs: Add link to skip to datetime's format codes (#108027) 2023-08-22 20:38:38 +02:00
Victor Stinner 21dda09600
gh-108303: Add Lib/test/test_cppext/ sub-directory (#108325)
* Move test_cppext to its own directory
* Rename setup_testcppext.py to setup.py
* Rename _testcppext.cpp to extension.cpp
* The source (extension.cpp) is now also copied by the test.
2023-08-22 18:30:18 +00:00
Victor Stinner c1c9cd61da
gh-89745: Remove commented code in getpath.c (#108307) 2023-08-22 20:25:06 +02:00
Victor Stinner 615f6e946d
gh-106320: Remove _PyDict_GetItemStringWithError() function (#108313)
Remove private _PyDict_GetItemStringWithError() function of the
public C API: the new PyDict_GetItemStringRef() can be used instead.

* Move private _PyDict_GetItemStringWithError() to the internal C API.
* _testcapi get_code_extra_index() uses PyDict_GetItemStringRef().
  Avoid using private functions in _testcapi which tests the public C
  API.
2023-08-22 18:17:25 +00:00
Łukasz Langa 0cb0c238d5
gh-108310: Fix CVE-2023-40217: Check for & avoid the ssl pre-close flaw (#108315)
Instances of `ssl.SSLSocket` were vulnerable to a bypass of the TLS handshake
and included protections (like certificate verification) and treating sent
unencrypted data as if it were post-handshake TLS encrypted data.

The vulnerability is caused when a socket is connected, data is sent by the
malicious peer and stored in a buffer, and then the malicious peer closes the
socket within a small timing window before the other peers’ TLS handshake can
begin. After this sequence of events the closed socket will not immediately
attempt a TLS handshake due to not being connected but will also allow the
buffered data to be read as if a successful TLS handshake had occurred.

Co-authored-by: Gregory P. Smith [Google LLC] <greg@krypto.org>
2023-08-22 19:53:15 +02:00
Victor Stinner d2879f2095
gh-108303: Remove unused Lib/test/sgml_input.html (#108305)
In Python 2.7, the file was used by Lib/test/test_sgmllib.py to test
Lib/sgmllib.py. The sgmllib module and its tests have been removed in
Python 3.0.
2023-08-22 19:12:51 +02:00
Guido van Rossum b8f96b5eda
gh-108253: Fix bug in func version cache (#108296)
When a function object changed its version, a stale pointer might remain in the cache.
Zap these whenever `func_version` changes (even when set to 0).
2023-08-22 08:29:49 -07:00
Victor Stinner adfc118fda
gh-106016: Add Lib/test/test_module/ directory (#108293)
* Move Python scripts related to test_module to this new directory:
  good_getattr.py and bad_getattrX.py scripts.
* Move Lib/test/test_module.py to Lib/test/test_module/__init__.py.
2023-08-22 16:53:49 +02:00