Commit Graph

7938 Commits

Author SHA1 Message Date
Victor Stinner 86e76ab8af
gh-110031: Skip test_threading fork tests if ASAN (#110100)
Skip test_threading tests using thread+fork if Python is built with
Address Sanitizer (ASAN).
2023-09-29 12:41:33 +00:00
Victor Stinner db0a258e79
gh-110088, gh-109878: Fix test_asyncio timeouts (#110092)
Fix test_asyncio timeouts: don't measure the maximum duration, a test
should not measure a CI performance. Only measure the minimum
duration when a task has a timeout or delay. Add CLOCK_RES to
test_asyncio.utils.
2023-09-29 11:49:30 +00:00
Jelle Zijlstra 7dc2c5093e
gh-110045: Update symtable module for PEP 695 (#110066) 2023-09-29 02:08:04 +00:00
Victor Stinner bd4518c60c
gh-110036: multiprocessing Popen.terminate() catches PermissionError (#110037)
On Windows, multiprocessing Popen.terminate() now catchs
PermissionError and get the process exit code. If the process is
still running, raise again the PermissionError. Otherwise, the
process terminated as expected: store its exit code.
2023-09-29 02:41:12 +02:00
Victor Stinner 4e356ad183
gh-109974: Fix threading lock_tests race conditions (#110057)
Fix race conditions in test_threading lock tests. Wait until a
condition is met rather than using time.sleep() with a hardcoded
number of seconds.

* Replace sleeping loops with support.sleeping_retry() which raises
  an exception on timeout.
* Add wait_threads_blocked(nthread) which computes a sleep depending
  on the number of threads. Remove _wait() function.
* test_set_and_clear(): use a way longer Event.wait() timeout.
* BarrierTests.test_repr(): wait until the 2 threads are waiting for
  the barrier. Use a way longer timeout for Barrier.wait() timeout.
* test_thread_leak() no longer needs to count
  len(threading.enumerate()): Bunch uses
  threading_helper.wait_threads_exit() internally which does it in
  wait_for_finished().
* Add BaseLockTests.wait_phase() which implements a timeout.
  test_reacquire() and test_recursion_count() use wait_phase().
2023-09-29 02:34:27 +02:00
Zachary Ware b488c0d761
gh-109991: Remove obsolete NEWS entries for OpenSSL 3.0.10 (GH-110055) 2023-09-28 17:58:49 -05:00
Zachary Ware cf4c297256
gh-109991: Update Windows build to use OpenSSL 3.0.11 (GH-110054) 2023-09-28 22:58:13 +00:00
Irit Katriel f580edcc6a
gh-109889: fix compiler's redundant NOP detection to look past NOPs with no lineno when looking for the next instruction's lineno (#109987) 2023-09-28 20:33:28 +01:00
Davide Rizzo b14f0ab51c
gh-110038: KqueueSelector must count all read/write events (#110039) 2023-09-28 17:25:10 +00:00
Victor Stinner 7e0fbf5175
gh-110033: Fix signal test_interprocess_signal() (#110035)
Fix test_interprocess_signal() of test_signal. Make sure that the
subprocess.Popen object is deleted before the test raising an
exception in a signal handler. Otherwise, Popen.__del__() can get the
exception which is logged as "Exception ignored in: ...." and the
test fails.
2023-09-28 19:12:11 +02:00
Amin Alaee 7df8b16d28
gh-109782: Ensure `os.path.isdir` has the same signature on all platforms (GH-109790) 2023-09-28 16:17:30 +01:00
Victor Stinner 9be283e5e1
gh-109594: Fix concurrent.futures test_timeout() (#110018)
Fix test_timeout() of test_concurrent_futures.test_wait. Remove the
future which may or may not complete depending if it takes longer
than the timeout ot not. Keep the second future which does not
complete before wait(). Make also the test faster: 0.5 second instead
of 6 seconds, so remove @support.requires_resource('walltime')
decorator.
2023-09-28 15:21:15 +02:00
Victor Stinner 8f324b7ecd
gh-109972: Split test_gdb.py into test_gdb package (#109977)
Split test_gdb.py file into a test_gdb package made of multiple
tests, so tests can now be run in parallel.

* Create Lib/test/test_gdb/ directory.
* Split test_gdb.py into multiple files in Lib/test/test_gdb/
  directory.
* Move Lib/test/gdb_sample.py to Lib/test/test_gdb/ directory.
  Update get_sample_script(): use __file__ to locate gdb_sample.py.
* Move gdb_has_frame_select() and HAS_PYUP_PYDOWN to test_misc.py.
* Explicitly skip test_gdb on Windows. Previously, test_gdb was
  skipped even if gdb was available because of
  gdb_has_frame_select().
2023-09-28 11:24:15 +00:00
Ned Deily 98c0c1de18
gh-109991: Update macOS installer to use OpenSSL 3.0.10. (GH-110003) 2023-09-28 02:08:39 -04:00
Ned Deily c88037d137
gh-109991: Update GitHub CI workflows to use OpenSSL 3.0.11 and multissltests to use 1.1.1w, 3.0.11, and 3.1.3. (gh-110002) 2023-09-28 00:45:13 -04:00
Nikita Sobolev f65f9e80fe
gh-109818: `reprlib.recursive_repr` copies `__type_params__` (#109819) 2023-09-27 19:26:42 -07:00
Eric Snow 32466c97c0
gh-109793: Allow Switching Interpreters During Finalization (gh-109794)
Essentially, we should check the thread ID rather than the thread state pointer.
2023-09-27 13:41:06 -06:00
Dale Collison 74723e1110
gh-109461: Update logging module lock to use context manager (#109462)
Co-authored-by: Victor Stinner <vstinner@python.org>
2023-09-27 18:26:41 +02:00
Sam Gross 773614e03a
gh-109740: Use 't' in `--disable-gil` SOABI (#109922)
Shared libraries for CPython 3.13 are now marked with a 't' for
threading. For example, `binascii.cpython-313t-darwin.so`.
2023-09-27 15:24:12 +00:00
Irit Katriel ea285ad8b6
gh-109923: set line number on the POP_TOP that follows a RETURN_GENERATOR (#109924) 2023-09-27 13:24:33 +01:00
Victor Stinner fbfec5642e
gh-109566: regrtest reexecutes the process (#109909)
When --fast-ci or --slow-ci option is used, regrtest now replaces the
current process with a new process to add "-u -W default -bb -E"
options to Python.

Changes:

* PCbuild/rt.bat and Tools/scripts/run_tests.py no longer need to add
  "-u -W default -bb -E" options to Python: it's now done by
  regrtest.
* Fix Tools/scripts/run_tests.py: flush stdout before replacing the
  process. Previously, buffered messages were lost.
2023-09-26 20:46:52 +02:00
Barney Gale ecd813f054
GH-109187: Improve symlink loop handling in `pathlib.Path.resolve()` (GH-109192)
Treat symlink loops like other errors: in strict mode, raise `OSError`, and
in non-strict mode, do not raise any exception.
2023-09-26 17:57:17 +01:00
Victor Stinner 859618c8cd
gh-109566, regrtest: Add --fast-ci and --slow-ci options (#109570)
* Add --fast-ci and --slow-ci options to libregrtest:

  * --fast-ci uses a default timeout of 10 minutes and "-u all,-cpu"
    (skip slowest tests).
  * --slow-ci uses a default timeout of 20 minues and "-u all" (run
    all tests).

* regrtest header now lists test resources.
* Makefile changes:

  * "make test", "make hostrunnertest" and "make coverage-report" now
    use --fast-ci option and TESTTIMEOUT variable.
  * "make buildbottest" now uses "--slow-ci". Remove options which
    became redundant with "--slow-ci".
  * "make testall" and "make testuniversal" now use --slow-ci option
    and TESTTIMEOUT variable.
  * "make testall" now uses "find -exec rm ..." instead of
    "find ... -print|xargs rm ...", same as "make clean".

* GitHub Actions workflow:

  * Ubuntu and Address Sanitizer jobs now use "make test". Remove
    options which became redundant with "--fast-ci".
  * Windows jobs now use --fast-ci option.
  * Use -j0 to detect the number of CPUs.

* Set Makefile TESTTIMEOUT default to an empty string, since
  --slow-ci and --fast-ci use different default timeout. It's now
  accepted to pass "--timeout=" to regrtest: treated as not timeout.
* Tools/scripts/run_tests.py now uses --fast-ci option.
* Tools/buildbot/test.bat now uses --slow-ci option. Remove
  --timeout=1200 option, redundant with --slow-ci.
2023-09-26 17:22:50 +02:00
Antoine Pitrou 0eb98837b6
gh-109593: Fix reentrancy issue in multiprocessing resource_tracker (#109629)
---------

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2023-09-26 13:57:25 +02:00
Serhiy Storchaka 8ac2085b80
gh-109631: Allow interruption of short repeated regex matches (GH-109867)
Counting for signal checking now continues in new match from the point where
it ended in the previous match instead of starting from 0.
2023-09-26 10:56:33 +03:00
Victor Stinner 4091deba88
gh-109739: regrtest disables load tracker if refleak (#109871)
regrtest: Fix reference leak check on Windows. Disable the load
tracker on Windows in the reference leak check mode (-R option).
2023-09-26 01:05:07 +00:00
Jason R. Coombs e9791ba351
gh-88233: zipfile: refactor _strip_extra (#102084)
* Refactor zipfile._strip_extra to use higher level abstractions for extras instead of a heavy-state loop.

* Add blurb

* Remove _strip_extra and use _Extra.strip directly.

* Use memoryview to avoid unnecessary copies while splitting Extras.
2023-09-25 19:46:58 -04:00
Victor Stinner 25bb266fc8
gh-109748: Fix venv test_zippath_from_non_installed_posix() (#109872)
Fix test_zippath_from_non_installed_posix() of test_venv: don't copy
__pycache__/ sub-directories, because they can be modified by other
Python tests running in parallel.
2023-09-25 23:16:30 +00:00
Irit Katriel d73c12b88c
gh-109823: Adjust labels in compiler when removing an empty basic block which is a jump target (#109839) 2023-09-25 18:25:05 +00:00
Antoine Pitrou 88a6137cdb
gh-109599: Add types.CapsuleType (#109600)
---------

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2023-09-25 19:50:39 +02:00
Victor Stinner 72fb39c965
gh-109276: regrtest re-runs "env changed" tests (#109831)
When a test fails with "env changed" and --rerun option is used, the
test is now re-run in verbose mode in a fresh process.
2023-09-25 16:21:01 +02:00
Alex Waygood 40d1de7581
gh-109653: Avoid a top-level import of `types` in `functools` (#109804) 2023-09-24 17:18:27 +01:00
Alex Waygood 51863b7d6e
gh-109653: Improve `enum` import time by avoiding import of `functools` (GH-109789) 2023-09-23 11:31:17 -07:00
Alex Waygood e8be0c9c5a
gh-109653: `typing.py`: improve import time by creating soft-deprecated members on demand (#109651)
Co-authored-by: Thomas Grainger <tagrain@gmail.com>
2023-09-23 08:46:35 +01:00
Serhiy Storchaka 62c7015e89
gh-109521: Fix obscure cases handling in PyImport_GetImporter() (GH-109522)
PyImport_GetImporter() now sets RuntimeError if it fails to get sys.path_hooks
or sys.path_importer_cache or they are not list and dict correspondingly.

Previously it could return NULL without setting error in obscure cases,
crash or raise SystemError if these attributes have wrong type.
2023-09-23 09:39:24 +03:00
Tian Gao d5611f2804
GH-107265: Add missing deoptimizations for ENTER_EXECUTOR's original opcode (GH-109420) 2023-09-22 14:13:31 -07:00
Hugo van Kemenade 8a82bff12c
Docs: Update Donghee Na's name (#109743) 2023-09-22 12:52:57 -06:00
Pablo Galindo Salgado b28ffaa193
gh-109596: Ensure repeated rules in the grammar are not allowed and fix incorrect soft keywords (#109606) 2023-09-22 19:03:23 +01:00
Irit Katriel 7c55399172
gh-109719: Fix missing jump target labels when compiler reorders cold/warm blocks (#109734) 2023-09-22 16:59:35 +00:00
Tian Gao 73ccfa28c5
gh-109164: Replace `getopt` with `argparse` in pdb (#109165)
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
2023-09-22 16:55:48 +00:00
elfstrom 405b06375a
gh-105829: Fix concurrent.futures.ProcessPoolExecutor deadlock (#108513)
This fixes issue #105829, https://github.com/python/cpython/issues/105829

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Antoine Pitrou <antoine@python.org>
Co-authored-by: Chris Withers <chris@withers.org>
Co-authored-by: Thomas Moreau <thomas.moreau.2010@gmail.com>
2023-09-22 13:55:56 +01:00
EliseevEgor 8fc071345b
gh-106584: Fix exit code for unittest in Python 3.12 (#106588)
Co-authored-by: sunmy2019 <59365878+sunmy2019@users.noreply.github.com>
Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
2023-09-22 14:26:27 +02:00
AN Long 4230d7ce93
gh-108996: fix and enable test_msvcrt (#109226)
* Add _testconsole.flush_console_input_buffer() function.
* test_kbhit(), test_getwch() and test_getwche() now call
  flush_console_input_buffer().
* Don't override sys.stdin anymore (not needed).
2023-09-22 02:19:48 +02:00
AN Long 869f177b5c
gh-74481: Add missing debug function docs and constants to msvcrt (GH-109650) 2023-09-21 16:44:24 +01:00
Victor Stinner d4cea794a7
gh-109613: _pystat_fromstructstat() checks for exceptions (#109618)
Fix os.stat() and os.DirEntry.stat(): check for exceptions.
Previously, on Python built in debug mode, these functions could
trigger a fatal Python error (and abort the process) when a function
succeeded with an exception set.

_pystat_fromstructstat() now exits immediately if an exception is
raised, rather only checking for exceptions at the end. It fix
following fatal error in fill_time():

    Fatal Python error: _Py_CheckSlotResult:
    Slot * of type int succeeded with an exception set
2023-09-21 09:55:06 +02:00
Adam Turner 712cb173f8
GH-109209: Bump the minimum Sphinx version to 4.2 (#109210) 2023-09-21 09:06:36 +03:00
Irit Katriel 9ccf0545ef
gh-109627: duplicated smalll exit blocks need to be assigned jump target labels (#109630) 2023-09-20 23:08:06 +00:00
Heinz-Alexander Fuetterer ef6d475db3
Fix typos in docs and comments (#109619) 2023-09-20 16:58:23 +00:00
James Gerity def828995a
fixes gh-109559: Update `unicodedata` for Unicode 15.1.0 (GH-109560)
---------

Co-authored-by: Benjamin Peterson <benjamin@python.org>
2023-09-19 22:07:47 -07:00
Jelle Zijlstra 1293fcc3c6
gh-109543: Remove unnecessary hasattr check (#109544)
Also added a new test case covering the scenario I thought this
might be about.
2023-09-19 20:15:52 -07:00
Victor Stinner 81cd1bd713
gh-103053: Skip test_freeze_simple_script() on PGO build (#109591)
Skip test_freeze_simple_script() of test_tools.test_freeze if Python
is built with "./configure --enable-optimizations", which means with
Profile Guided Optimization (PGO): it just makes the test too slow.
The freeze tool is tested by many other CIs with other (faster)
compiler flags.

test.pythoninfo now gets also get_build_info() of
test.libregrtests.utils.
2023-09-20 01:58:34 +00:00
Mateusz Nowak 5a740cd06e
gh-109109: Expose retrieving certificate chains in SSL module (#109113)
Adds APIs to get the TLS certificate chains, verified or full unverified, from SSLSocket and SSLObject.

Co-authored-by: Gregory P. Smith [Google LLC] <greg@krypto.org>
2023-09-20 01:20:54 +00:00
Ronan Pigott ddf2e953c2
gh-109033: Return filename with os.utime errors (#109034)
The filename was previously intentionally omitted from exception because
"it might confuse the user". Uncaught exceptions are not generally a
replacement for user-facing error messages, so obscuring this
information only has the effect of making the programmer's life more
difficult.
2023-09-20 01:18:23 +02:00
Victor Stinner 754519a9f8
gh-109580: Skip test_perf_profiler on ASAN build (#109584)
Skip test_perf_profiler if Python is built with ASAN, MSAN or UBSAN
sanitizer. Python does crash randomly in this test on such build.
2023-09-19 19:42:51 +02:00
Sam Gross 0c89056fe5
gh-108724: Add PyMutex and _PyParkingLot APIs (gh-109344)
PyMutex is a one byte lock with fast, inlineable lock and unlock functions for the common uncontended case.  The design is based on WebKit's WTF::Lock.

PyMutex is built using the _PyParkingLot APIs, which provides a cross-platform futex-like API (based on WebKit's WTF::ParkingLot).  This internal API will be used for building other synchronization primitives used to implement PEP 703, such as one-time initialization and events.

This also includes tests and a mini benchmark in Tools/lockbench/lockbench.py to compare with the existing PyThread_type_lock.

Uncontended acquisition + release:
* Linux (x86-64): PyMutex: 11 ns, PyThread_type_lock: 44 ns
* macOS (arm64): PyMutex: 13 ns, PyThread_type_lock: 18 ns
* Windows (x86-64): PyMutex: 13 ns, PyThread_type_lock: 38 ns

PR Overview:

The primary purpose of this PR is to implement PyMutex, but there are a number of support pieces (described below).

* PyMutex:  A 1-byte lock that doesn't require memory allocation to initialize and is generally faster than the existing PyThread_type_lock.  The API is internal only for now.
* _PyParking_Lot:  A futex-like API based on the API of the same name in WebKit.  Used to implement PyMutex.
* _PyRawMutex:  A word sized lock used to implement _PyParking_Lot.
* PyEvent:  A one time event.  This was used a bunch in the "nogil" fork and is useful for testing the PyMutex implementation, so I've included it as part of the PR.
* pycore_llist.h:  Defines common operations on doubly-linked list.  Not strictly necessary (could do the list operations manually), but they come up frequently in the "nogil" fork. ( Similar to https://man.freebsd.org/cgi/man.cgi?queue)

---------

Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
2023-09-19 09:54:29 -06:00
Xuehai Pan 74f315edd0
gh-102757: fix function signature mismatch for `functools.reduce` between code and documentation (#102759) 2023-09-18 10:42:58 -06:00
Victor Stinner 0bb0d88e2d
gh-109496: Detect Py_DECREF() after dealloc in debug mode (#109539)
On a Python built in debug mode, Py_DECREF() now calls
_Py_NegativeRefcount() if the object is a dangling pointer to
deallocated memory: memory filled with 0xDD "dead byte" by the debug
hook on memory allocators. The fix is to check the reference count
*before* checking for _Py_IsImmortal().

Add test_decref_freed_object() to test_capi.test_misc.
2023-09-18 14:59:09 +00:00
Tian Gao 412f5e85d6
gh-109371: Fix monitoring with instruction events set (gh-109385) 2023-09-18 23:30:08 +09:00
Shantanu 23f9f6f464
gh-108843: fix ast.unparse for f-string with many quotes (#108981) 2023-09-18 14:56:19 +01:00
Serhiy Storchaka add16f1a5e
gh-108511: Add C API functions which do not silently ignore errors (GH-109025)
Add the following functions:

* PyObject_HasAttrWithError()
* PyObject_HasAttrStringWithError()
* PyMapping_HasKeyWithError()
* PyMapping_HasKeyStringWithError()
2023-09-17 14:23:31 +03:00
Brett Cannon e218e5022e
GH-83417: Allow `venv` to add a `.gitignore` file to environments via a new `scm_ignore_file` parameter (GH-108125)
This feature is off by default via code but on by default via the CLI. The `.gitignore` file contains `*` which causes the entire directory to be ignored.

Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
2023-09-15 22:38:08 +00:00
Hood Chatham 6b179adb8c
gh-106213: Make Emscripten trampolines work with JSPI (GH-106219)
There is a WIP proposal to enable webassembly stack switching which have been
implemented in v8:

https://github.com/WebAssembly/js-promise-integration

It is not possible to switch stacks that contain JS frames so the Emscripten JS
trampolines that allow calling functions with the wrong number of arguments
don't work in this case. However, the js-promise-integration proposal requires
the [type reflection for Wasm/JS API](https://github.com/WebAssembly/js-types)
proposal, which allows us to actually count the number of arguments a function
expects.

For better compatibility with stack switching, this PR checks if type reflection
is available, and if so we use a switch block to decide the appropriate
signature. If type reflection is unavailable, we should use the current EMJS
trampoline.

We cache the function argument counts since when I didn't cache them performance
was negatively affected.

Co-authored-by: T. Wouters <thomas@python.org>
Co-authored-by: Brett Cannon <brett@python.org>
2023-09-15 15:04:21 -07:00
Gregory P. Smith 59073c9ab8
gh-109096: Deprecate `http.server.CGIHTTPRequestHandler` (#109387)
Deprecate `http.server.CGIHTTPRequestHandler`.

Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2023-09-15 14:26:45 -07:00
Victor Stinner 74c72a2fc7
gh-109425: regrtest decodes worker stdout with backslashreplace (#109428)
libregrtest now decodes stdout of test worker processes with the
"backslashreplace" error handler to log corrupted stdout, instead of
failing with an error and not logging the stdout.
2023-09-14 23:37:37 +00:00
buermarc 68a6f21f47
gh-109375: Fix bug where pdb registers an alias without an associated command (#109376) 2023-09-14 22:31:30 +01:00
Victor Stinner e091b9f20f
gh-109396: Fix test_socket.test_hmac_sha1() in FIPS mode (#109423)
Use a longer key: FIPS mode requires at least of at least 112 bits.
The previous key was only 32 bits.
2023-09-14 21:24:11 +00:00
Carl Meyer 909adb5092
gh-109219: propagate free vars through type param scopes (#109377)
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2023-09-14 10:20:32 -06:00
Irit Katriel 4a54074a0f
gh-105658: fix excess trace events for except block ending with a conditional block (#109384) 2023-09-14 17:06:08 +01:00
Brandt Bucher 22e65eecaa
GH-105848: Replace KW_NAMES + CALL with LOAD_CONST + CALL_KW (GH-109300) 2023-09-13 10:25:45 -07:00
Jelle Zijlstra 987b4bc087
gh-109341: Fix crash on compiling invalid AST including TypeAlias (#109349) 2023-09-13 09:00:39 -07:00
Jelle Zijlstra 79101edb03
gh-109351: Fix crash when compiling AST with invalid NamedExpr (#109352) 2023-09-13 09:00:15 -07:00
Tian Gao d69805b38a
gh-109156: Add tests for de-instrumenting instructions with instrumented lines (GH-109157) 2023-09-13 09:47:35 +01:00
Victor Stinner 44d9a71ea2
gh-104736: Fix test_gdb tests on ppc64le with clang (#109360)
Fix test_gdb on Python built with LLVM clang 16 on Linux ppc64le (ex:
Fedora 38). Search patterns in gdb "bt" command output to detect
when gdb fails to retrieve the traceback. For example, skip a test if
"Backtrace stopped: frame did not save the PC" is found.
2023-09-13 04:24:32 +00:00
Irit Katriel b303d3ad3e
gh-109319: deprecate dis.HAVE_ARGUMENT (#109320) 2023-09-12 16:45:35 +01:00
Nikita Sobolev 66d1d7eb06
gh-84867: Do not load tests from TestCase and FunctionTestCase (GH-100497) 2023-09-12 16:33:30 +03:00
Irit Katriel 8b55adfa8f
gh-109256: allocate opcode IDs for internal opcodes in their own range (#109269) 2023-09-12 10:36:17 +00:00
Nikita Sobolev 247ee1bf84
gh-109216: Fix possible memory leak in `BUILD_MAP` (#109257) 2023-09-12 15:07:22 +05:30
Victor Stinner a84cb74d42
gh-109276: libregrtest calls random.seed() before each test (#109279)
libregrtest now calls random.seed() before running each test file
when -r/--randomize command line option is used. Moreover, it's also
called in worker processes. It should help to make tests more
deterministic. Previously, it was only called once in the main
process before running all test files and it was not called in worker
processes.

* Convert some f-strings to regular strings in test_regrtest when
  f-string is not needed.
* Remove unused all_methods variable from test_regrtest.
* Add RunTests members are now mandatory.
2023-09-12 05:35:08 +02:00
Tian Gao 391f3e3ca9
GH-106734: Disable tab completion in pdb's multiline mode (GH-106735) 2023-09-11 18:28:43 -07:00
Jelle Zijlstra b88d9e75f6
gh-109118: Disallow nested scopes within PEP 695 scopes within classes (#109196)
Fixes #109118. Fixes #109194.

Co-authored-by: Carl Meyer <carl@oddbird.net>
2023-09-11 17:11:06 -07:00
Carl Meyer ceeb4173ae
gh-109195: fix source location for super load before LOAD_SUPER_ATTR (#109289) 2023-09-11 17:35:49 -06:00
Irit Katriel ecd21a629a
gh-109179: Fix traceback display for SyntaxErrors with notes (#109197) 2023-09-11 19:18:34 +01:00
Victor Stinner de5f8f7d13
gh-109276: libregrtest: use separated file for JSON (#109277)
libregrtest now uses a separated file descriptor to write test result
as JSON.  Previously, if a test wrote debug messages late around the
JSON, the main test process failed to parse JSON.

Rename TestResult.write_json() to TestResult.write_json_into().

worker_process() no longer writes an empty line at the end. There is
no need to separate test process output from the JSON output anymore,
since JSON is now written into a separated file descriptor.

create_worker_process() now always spawn the process with
close_fds=True.
2023-09-11 19:33:42 +02:00
Victor Stinner 517cd82ea7
gh-108987: Fix _thread.start_new_thread() race condition (#109135)
Fix _thread.start_new_thread() race condition. If a thread is created
during Python finalization, the newly spawned thread now exits
immediately instead of trying to access freed memory and lead to a
crash.

thread_run() calls PyEval_AcquireThread() which checks if the thread
must exit. The problem was that tstate was dereferenced earlier in
_PyThreadState_Bind() which leads to a crash most of the time.

Move _PyThreadState_CheckConsistency() from thread_run() to
_PyThreadState_Bind().
2023-09-11 17:27:03 +02:00
Mark Shannon 4a69301ea4
GH-108976. Keep monitoring data structures valid during de-optimization during callback. (GH-109131) 2023-09-11 14:37:09 +01:00
Victor Stinner a9b1f84790
gh-107219: Fix concurrent.futures terminate_broken() (#109244)
Fix a race condition in concurrent.futures. When a process in the
process pool was terminated abruptly (while the future was running or
pending), close the connection write end. If the call queue is
blocked on sending bytes to a worker process, closing the connection
write end interrupts the send, so the queue can be closed.

Changes:

* _ExecutorManagerThread.terminate_broken() now closes
  call_queue._writer.
* multiprocessing PipeConnection.close() now interrupts
  WaitForMultipleObjects() in _send_bytes() by cancelling the
  overlapped operation.
2023-09-11 08:11:31 +00:00
Victor Stinner e55aab9578
gh-109230: test_pyexpat no longer depends on the current directory (#109233)
Fix test_pyexpat.test_exception(): it can now be run from a directory
different than Python source code directory. Before, the test failed
in this case.

Skip the test if Modules/pyexpat.c source is not available. Skip also
the test on Python implementations other than CPython.
2023-09-10 22:16:59 +00:00
Victor Stinner cbb3a6f8ad
gh-109237: Fix test_site for non-ASCII working directory (#109238)
Fix test_site.test_underpth_basic() when the working directory
contains at least one non-ASCII character: encode the "._pth" file to
UTF-8 and enable the UTF-8 Mode to use UTF-8 for the child process
stdout.
2023-09-10 22:04:35 +00:00
Serhiy Storchaka d6892c2b92
gh-50644: Forbid pickling of codecs streams (GH-109180)
Attempts to pickle or create a shallow or deep copy of codecs streams
now raise a TypeError.

Previously, copying failed with a RecursionError, while pickling
produced wrong results that eventually caused unpickling to fail with
a RecursionError.
2023-09-10 20:06:09 +03:00
云line 4297499696
gh-109207: Fix SystemError when printing symtable entry object. (GH-109225) 2023-09-10 15:04:24 +03:00
Pieter Eendebak 85a5d3dbe1
gh-93627: Align Python implementation of pickle with C implementation of pickle (GH-103035)
If a method like __reduce_ex_ or __reduce__ is set to None, a TypeError is raised.
2023-09-10 10:06:08 +03:00
Serhiy Storchaka 92578919a6
gh-109174: Add support of SimpleNamespace in copy.replace() (GH-109175) 2023-09-10 08:09:25 +03:00
Jelle Zijlstra 17f994174d
gh-109118: Fix runtime crash when NameError happens in PEP 695 function (#109123) 2023-09-09 02:49:20 +00:00
Tian Gao 057bc72490
gh-109052: Use the base opcode when comparing code objects (gh-109107) 2023-09-09 10:24:49 +09:00
AN Long bcb2ab5ef8
gh-108996: add tests for msvcrt (#109004)
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
Co-authored-by: Steve Dower <steve.dower@microsoft.com>
2023-09-08 18:38:38 -04:00
Victor Stinner 1f7e42131d
gh-109054: configure checks if libatomic is needed (#109101)
Fix building the _testcapi extension on Linux AArch64 which requires
linking to libatomic when <cpython/pyatomic.h> is used: the
_Py_atomic_or_uint64() function requires libatomic
__atomic_fetch_or_8() on this platform.

The configure script now checks if linking to libatomic is needed and
generates a new LIBATOMIC variable used to build the _testcapi
extension.

Building the _testcapi extension now uses the LIBATOMIC variable in
its LDFLAGS, since Modules/_testcapi/pyatomic.c uses
<cpython/pyatomic.h>.

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-09-08 23:14:33 +02:00
Pablo Galindo Salgado 5bda2f637e
gh-109114: Relax the check for invalid lambdas inside f-strings to avoid false positives (#109121) 2023-09-08 17:00:23 +00:00
Pablo Galindo Salgado 6275c67ea6
gh-106922: Fix error location for constructs with spaces and parentheses (#108959) 2023-09-08 17:18:35 +01:00
Mark Shannon 15d4c9fabc
GH-108716: Turn off deep-freezing of code objects. (GH-108722) 2023-09-08 10:34:40 +01:00
Ethan Furman c74e440168
gh-109022: [Enum] require `names=()` to create empty enum type (GH-109048)
add guard so that ``Enum('bar')`` raises a TypeError instead of
creating a new enum class called `bar`.  To create the new but
empty class, use:

    huh = Enum('bar', names=())
2023-09-07 18:19:03 -07:00
Carl Meyer f2584eade3
gh-108732: include comprehension locals in frame.f_locals (#109026)
Co-authored-by: Radislav Chugunov <52372310+chgnrdv@users.noreply.github.com>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2023-09-07 08:56:43 -06:00
Mark Dickinson b72251de93
gh-102823: Document return type of floor division on floats (#102824)
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
2023-09-07 14:20:33 +00:00
Serhiy Storchaka b2729e93e9
gh-88943: Improve syntax error for non-ASCII character that follows a numerical literal (GH-109081)
It now points on the invalid non-ASCII character, not on the valid numerical literal.
2023-09-07 17:00:13 +03:00
Mark Shannon 0858328ca2
GH-108614: Add `RESUME_CHECK` instruction (GH-108630) 2023-09-07 14:39:03 +01:00
Ijtaba Hussain 3e53ac9903
gh-103186: Suppress and assert expected RuntimeWarnings in test_sys_settrace (GH-103244)
Caused as a result of frame manipulation where locals are never assigned / initialised.
2023-09-07 10:41:38 +03:00
Victor Stinner a52a350977
gh-109015: Add test.support.socket_helper.tcp_blackhole() (#109016)
Skip test_asyncio, test_imaplib and test_socket tests if FreeBSD TCP
blackhole is enabled (net.inet.tcp.blackhole=2).
2023-09-07 01:58:03 +02:00
Brandt Bucher 6971e40c2e
GH-104584: Restore frame->stacktop on optimizer error (GH-108953) 2023-09-06 13:59:50 -07:00
Serhiy Storchaka 6f3c138dfa
gh-108751: Add copy.replace() function (GH-108752)
It creates a modified copy of an object by calling the object's
__replace__() method.

It is a generalization of dataclasses.replace(), named tuple's _replace()
method and replace() methods in various classes, and supports all these
stdlib classes.
2023-09-06 23:55:42 +03:00
Victor Stinner db1ee6a19a
gh-108740: Fix "make regen-all" race condition (#108741)
Fix a race condition in "make regen-all". The deepfreeze.c source and
files generated by Argument Clinic are now generated or updated
before generating "global objects". Previously, some identifiers may
miss depending on the order in which these files were generated.

* "make regen-global-objects": Make sure that deepfreeze.c is
  generated and up to date, and always run "make clinic".
* "make clinic" no longer runs generate_global_objects.py script.
* "make regen-deepfreeze" now only updates deepfreeze.c (C file).
  It doesn't build deepfreeze.o (object) anymore.
* Remove misleading messages in "make regen-global-objects" and
  "make clinic". They are now outdated, these commands are now
  safe to use.
* Document generates files in Doc/using/configure.rst.

Co-authored-by: Erlend E. Aasland <erlend@python.org>
2023-09-06 20:09:21 +02:00
Victor Stinner 8ff1142578
gh-108851: Fix tomllib recursion tests (#108853)
* Add get_recursion_available() and get_recursion_depth() functions
  to the test.support module.
* Change infinite_recursion() default max_depth from 75 to 100.
* Fix test_tomllib recursion tests for WASI buildbots: reduce the
  recursion limit and compute the maximum nested array/dict depending
  on the current available recursion limit.
* test.pythoninfo logs sys.getrecursionlimit().
* Enhance test_sys tests on sys.getrecursionlimit()
  and sys.setrecursionlimit().
2023-09-06 17:34:31 +02:00
Ed Maste 2cd170db40
gh-91960: Add FreeBSD build and test using Cirrus-CI (#91961)
Cirrus-CI is a hosted CI service that supports FreeBSD, Linux, macOS,
and Windows.  Add a .cirrus.yml configuration file to provide CI coverage
on pull requests for FreeBSD 13.2.

Co-authored-by: Victor Stinner <vstinner@python.org>
2023-09-06 16:57:40 +02:00
Victor Stinner fbce43a251
gh-91960: Skip test_gdb if gdb cannot retrive Python frames (#108999)
Skip test_gdb if gdb is unable to retrieve Python frame objects: if a
frame is "<optimized out>". When Python is built with "clang -Og",
gdb can fail to retrive the 'frame' parameter of
_PyEval_EvalFrameDefault(). In this case, tests like py_bt() are
likely to fail. Without getting access to Python frames,
python-gdb.py is mostly clueless on retrieving the Python traceback.
Moreover, test_gdb is no longer skipped on macOS if Python is built
with Clang.
2023-09-06 14:34:35 +00:00
Victor Stinner cd2ef21b07
gh-108962: Skip test_tempfile.test_flags() if not supported (#108964)
Skip test_tempfile.test_flags() if chflags() fails with "OSError:
[Errno 45] Operation not supported" (ex: on FreeBSD 13).
2023-09-05 21:59:40 +00:00
Anthony Shaw 2c4c26c4ce
gh-108469: Update ast.unparse for unescaped quote support from PEP701 [3.12] (#108553)
Co-authored-by: sunmy2019 <59365878+sunmy2019@users.noreply.github.com>
2023-09-05 21:01:23 +01:00
Zachary Ware deea7c8268
gh-107565: Update Windows build to use OpenSSL 3.0.10 (GH-108928)
Also clean up some intermediate NEWS entries about previous versions.
2023-09-05 16:03:06 +00:00
Mark Shannon 5a2a046151
GH-108390: Prevent non-local events being set with `sys.monitoring.set_local_events()` (GH-108420) 2023-09-05 08:03:53 +01:00
Serhiy Storchaka 04a0830b00
gh-89392: Remove support of test_main() in libregrtest (GH-108876) 2023-09-05 08:36:43 +03:00
Tian Gao 6304d983a0
gh-108463: Make expressions/statements work as expected in pdb (#108464) 2023-09-04 21:44:40 +00:00
Victor Stinner 5a79d2ae57
Revert "gh-46376: Return existing pointer when possible in ctypes (#1… (#108688)
This reverts commit 08447b5deb.

Revert also _ctypes.c changes of the PyDict_ContainsString() change,
commit 6726626646.
2023-09-04 11:21:47 +02:00
Victor Stinner 31c2945f14
gh-108834: regrtest reruns failed tests in subprocesses (#108839)
When using --rerun option, regrtest now re-runs failed tests
in verbose mode in fresh worker processes to have more
deterministic behavior. So it can write its final report even
if a test killed a worker progress.

Add --fail-rerun option to regrtest: exit with non-zero exit code
if a test failed pass passed when re-run in verbose mode (in a
fresh process). That's now more useful since tests can pass
when re-run in a fresh worker progress, whereas they failed
when run after other tests when tests are run sequentially.

Rename --verbose2 option (-w) to --rerun. Keep --verbose2 as a
deprecated alias.

Changes:

* Fix and enhance statistics in regrtest summary. Add "(filtered)"
  when --match and/or --ignore options are used.
* Add RunTests class.
* Add TestResult.get_rerun_match_tests() method
* Rewrite code to serialize/deserialize worker arguments as JSON
  using a new WorkerJob class.
* Fix stats when a test is run with --forever --rerun.
* If failed test names cannot be parsed, log a warning and don't
  filter tests.
* test_regrtest.test_rerun_success() now uses a marker file, since
  the test is re-run in a separated process.
* Add tests on normalize_test_name() function.
* Add test_success() and test_skip() tests to test_regrtest.
2023-09-03 21:37:15 +00:00
Victor Stinner 03c4080c71
gh-108765: Python.h no longer includes <ctype.h> (#108831)
Remove <ctype.h> in C files which don't use it; only sre.c and
_decimal.c still use it.

Remove _PY_PORT_CTYPE_UTF8_ISSUE code from pyport.h:

* Code added by commit b5047fd019
  in 2004 for MacOSX and FreeBSD.
* Test removed by commit 52ddaefb6b
  in 2007, since Python str type now uses locale independent
  functions like Py_ISALPHA() and Py_TOLOWER() and the Unicode
  database.

Modules/_sre/sre.c replaces _PY_PORT_CTYPE_UTF8_ISSUE with new
functions: sre_isalnum(), sre_tolower(), sre_toupper().

Remove unused includes:

* _localemodule.c: remove <stdio.h>.
* getargs.c: remove <float.h>.
* dynload_win.c: remove <direct.h>, it no longer calls _getcwd()
  since commit fb1f68ed7c (in 2001).
2023-09-03 18:54:27 +02:00
Victor Stinner 0e6d582b3b
gh-63760: Don't declare gethostname() on Solaris (#108817)
Since 2005, Solaris defines gethostname(). socketmodule.c no longer
has to define gethostname() for Solaris.

Oracle Solaris and OpenSolaris have patches to remove the
gethostname() definition in Python:

* https://github.com/oracle/solaris-userland/blob/master/components/python/python37/patches/15-gethostname.patch
* https://github.com/OpenIndiana/oi-userland/blob/oi/hipster/components/python/python37/patches/15-gethostname.patch
* https://github.com/omniosorg/omnios-build/blob/master/build/python27/patches/24-gethostname.patch

Co-authored-by: Jakub Kulík <Kulikjak@gmail.com>
2023-09-02 17:54:59 +00:00
Victor Stinner f5ddbeeab7
gh-108822: Add Changelog entry for regrtest statistics (#108821) 2023-09-02 19:26:20 +02:00
Victor Stinner e7de0c5901
gh-108765: Python.h no longer includes <sys/time.h> (#108775)
Python.h no longer includes <time.h>, <sys/select.h> and <sys/time.h>
standard header files.

* Add <time.h> include to xxsubtype.c.
* Add <sys/time.h> include to posixmodule.c and semaphore.c.
* readline.c includes <sys/select.h> instead of <sys/time.h>.
* resource.c no longer includes <time.h> and <sys/time.h>.
2023-09-02 17:51:19 +02:00
Barney Gale bdc3c884cd
GH-78722: Raise exceptions from `pathlib.Path.iterdir()` without delay. (#107320)
`pathlib.Path.iterdir()` now immediately raises any `OSError`
exception from `os.listdir()`, rather than waiting until its
result is iterated over.
2023-09-02 16:08:03 +01:00
Victor Stinner 594b00057e
gh-108765: Python.h no longer includes <unistd.h> (#108783) 2023-09-02 16:50:18 +02:00
Victor Stinner 4f9b706c6f
gh-108794: doctest counts skipped tests (#108795)
* Add 'skipped' attribute to TestResults.
* Add 'skips' attribute to DocTestRunner.
* Rename private DocTestRunner._name2ft attribute
  to DocTestRunner._stats.
* Use f-string for string formatting.
* Add some tests.
* Document DocTestRunner attributes and its API for statistics.
* Document TestResults class.

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-09-02 16:42:07 +02:00
Victor Stinner 4ba18099b7
gh-108765: Python.h no longer includes <ieeefp.h> (#108781)
Remove also the HAVE_IEEEFP_H macro: remove ieeefp.h from the
AC_CHECK_HEADERS() check of configure.ac.
2023-09-02 15:48:32 +02:00
Alex Waygood a1cbace91b
gh-105509: Simplify implementation of `typing.Annotated` (#105510) 2023-09-01 13:57:25 -07:00
Irit Katriel 844f4c2e12
gh-108727: Fix segfault due to missing tp_dealloc definition for CounterOptimizer_Type (GH-108734) 2023-09-01 10:16:09 +01:00
Nikita Sobolev 044b8b3b6a
gh-107805: Fix signatures of module-level generated functions in `turtle` (#107807)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-09-01 01:18:15 -07:00
Victor Stinner 2bd960b579
gh-108337: Add pyatomic.h header (#108701)
This adds a new header that provides atomic operations on common data
types. The intention is that this will be exposed through Python.h,
although that is not the case yet. The only immediate use is in
the test file.

Co-authored-by: Sam Gross <colesbury@gmail.com>
2023-08-31 21:41:18 +00:00
Ethan Furman d48760b2f1
gh-108682: [Enum] raise TypeError if super().__new__ called in custom __new__ (GH-108704)
When overriding the `__new__` method of an enum, the underlying data type should be created directly; i.e. .

    member = object.__new__(cls)
    member = int.__new__(cls, value)
    member = str.__new__(cls, value)

Calling `super().__new__()` finds the lookup version of `Enum.__new__`, and will now raise an exception when detected.
2023-08-31 12:45:12 -07:00
Victor Stinner 13a00078b8
gh-108634: Py_TRACE_REFS uses a hash table (#108663)
Python built with "configure --with-trace-refs" (tracing references)
is now ABI compatible with Python release build and debug build.
Moreover, it now also supports the Limited API.

Change Py_TRACE_REFS build:

* Remove _PyObject_EXTRA_INIT macro.
* The PyObject structure no longer has two extra members (_ob_prev
  and _ob_next).
* Use a hash table (_Py_hashtable_t) to trace references (all
  objects): PyInterpreterState.object_state.refchain.
* Py_TRACE_REFS build is now ABI compatible with release build and
  debug build.
* Limited C API extensions can now be built with Py_TRACE_REFS:
  xxlimited, xxlimited_35, _testclinic_limited.
* No longer rename PyModule_Create2() and PyModule_FromDefAndSpec2()
  functions to PyModule_Create2TraceRefs() and
  PyModule_FromDefAndSpec2TraceRefs().
* _Py_PrintReferenceAddresses() is now called before
  finalize_interp_delete() which deletes the refchain hash table.
* test_tracemalloc find_trace() now also filters by size to ignore
  the memory allocated by _PyRefchain_Trace().

Test changes for Py_TRACE_REFS:

* Add test.support.Py_TRACE_REFS constant.
* Add test_sys.test_getobjects() to test sys.getobjects() function.
* test_exceptions skips test_recursion_normalizing_with_no_memory()
  and test_memory_error_in_PyErr_PrintEx() if Python is built with
  Py_TRACE_REFS.
* test_repl skips test_no_memory().
* test_capi skisp test_set_nomemory().
2023-08-31 18:33:34 +02:00
Carl Meyer d52c4482a8
gh-108654: restore comprehension locals before handling exception (#108659)
Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>
2023-08-30 17:50:50 -06:00
albanD add8d45cbe
gh-108520: Fix bad fork detection in nested multiprocessing use case (#108568)
gh-107275 introduced a regression where a SemLock would fail being passed along nested child processes, as the `is_fork_ctx` attribute would be left missing after the first deserialization.

---------

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Antoine Pitrou <pitrou@free.fr>
2023-08-30 17:07:41 +00:00
Erlend E. Aasland 2a3926fa51
gh-108590: Revert gh-108657 (commit 400a1cebc) (#108686)
Reverted per Serhiy's request.
2023-08-30 14:53:10 +00:00
Victor Stinner 2928e5dc65
gh-108494: Argument Clinic: Document how to generate code that uses the limited C API (#108584)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-08-30 16:02:48 +02:00
Corvin 400a1cebc7
gh-108590: Fix sqlite3.iterdump for invalid Unicode in TEXT columns (#108657)
Co-authored-by: Erlend E. Aasland <erlend@python.org>
2023-08-30 09:06:21 +00:00
Erlend E. Aasland 0b0c1d046c
gh-108278: Deprecate passing the first param of sqlite3.Connection callback APIs by keyword (#108632)
Deprecate passing the callback callable by keyword for the following
sqlite3.Connection APIs:

- set_authorizer(authorizer_callback)
- set_progress_handler(progress_handler, ...)
- set_trace_callback(trace_callback)

The affected parameters will become positional-only in Python 3.15.
2023-08-29 22:02:12 +02:00
Guido van Rossum 59e46932c8
gh-108488: Initialize JUMP_BACKWARD cache to 0, not 17 (#108591)
This mis-initialization caused the executor optimization to kick in sooner than intended. It also set the lower 4 bits of the counter to `1` -- those bits are supposed to be reserved (the actual counter is in the upper 12 bits).
2023-08-29 18:14:56 +00:00
Brett Cannon 5f85b443f7
GH-106176, GH-104702: Fix reference leak when importing across multiple threads (#108497) 2023-08-29 09:17:25 +02:00
Matthias Bussonnier f75cefd402
gh-106670: Allow Pdb to move between chained exceptions (#106676) 2023-08-28 18:31:03 +00:00
Guido van Rossum 47d7eba889
gh-108487: Move assert(self != NULL) down beyond DEOPT_IF() (#108510) 2023-08-28 10:17:00 -07:00
Erlend E. Aasland 4116592b6f
gh-108278: Deprecate passing the three first params as keyword args for sqlite3 UDF creation APIs (#108281)
Deprecate passing name, number of arguments, and the callable as keyword
arguments, for the following sqlite3.Connection APIs:

- create_function(name, nargs, callable, ...)
- create_aggregate(name, nargs, callable)

The affected parameters will become positional-only in Python 3.15.
2023-08-28 13:32:07 +00:00
Erlend E. Aasland 8db451ceb1
gh-64662: Fix NEWS entry and remove What's New entry (#108565) 2023-08-28 12:02:26 +00:00
Erlend E. Aasland d0160c7c22
gh-64662: Add virtual table support to sqlite3.Connection.iterdump (#108340)
Co-authored-by: Aviv Palivoda <palaviv@gmail.com>
2023-08-28 00:18:32 +02:00
Ori Avtalion a429eafef2
gh-108542: Fix incorrect module name in NEWS entry for gh-105475 (#108543) 2023-08-27 21:50:59 +02:00
Raymond Hettinger 042aa88bcc
gh-108322: Optimize statistics.NormalDist.samples() (gh-108324) 2023-08-27 08:59:40 -05:00
R 7096a2be33
gh-105052:update timeit function's description (#105060)
---------
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
2023-08-27 03:22:27 -04:00
Jelle Zijlstra 482fad7f01
gh-108295: Fix crashes with TypeVar weakrefs (#108517) 2023-08-26 17:13:33 -07:00
Serhiy Storchaka 2b15536fa9
gh-107913: Fix possible losses of OSError error codes (GH-107930)
Functions like PyErr_SetFromErrno() and SetFromWindowsErr() should be
called immediately after using the C API which sets errno or the Windows
error code.
2023-08-27 00:35:06 +03:00
denballakh e407cea193
gh-107406: Add better `struct.Struct` repr (#107407) 2023-08-26 15:24:16 +05:30
Victor Stinner 1dd9510977
gh-108494: Argument Clinic partial supports of Limited C API (#108495)
Argument Clinic now has a partial support of the
Limited API:

* Add --limited option to clinic.c.
* Add '_testclinic_limited' extension which is built with
  the limited C API version 3.13.
* For now, hardcode in clinic.py that "_testclinic_limited.c" targets
  the limited C API.
2023-08-25 23:22:08 +02:00
Brandt Bucher 4eae1e5342
GH-106581: Fix instrumentation in tier 2 (GH-108493) 2023-08-25 19:12:59 +00:00
Corvin 24aa249a66
gh-107932: Fix dis module for bytecode that does not have an associated source line (GH-107988) 2023-08-25 09:31:26 +01:00
Peeyush Aggarwal 8d4052075e
gh-103384: Generalize the regex pattern `BaseConfigurator.INDEX_PATTERN` to allow spaces and non-alphanumeric characters in keys. (GH-103391)
Co-authored-by: Vinay Sajip <vinay_sajip@yahoo.co.uk>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Erlend E. Aasland <erlend@python.org>
2023-08-25 09:15:26 +01:00
Victor Stinner be436e08b8
gh-108444: Add PyLong_AsInt() public function (#108445)
* Rename _PyLong_AsInt() to PyLong_AsInt().
* Add documentation.
* Add test.
* For now, keep _PyLong_AsInt() as an alias to PyLong_AsInt().
2023-08-24 23:55:30 +02:00
Samuel Thibault edb569e8e1
gh-108447: Detect platform triplets for x86_64 GNU/Hurd (#108045) 2023-08-24 22:37:20 +02:00
Victor Stinner aa6f787faa
gh-108388: Convert test_concurrent_futures to package (#108401)
Convert test_concurrent_futures to a package of sub-tests.
2023-08-24 19:21:44 +02:00
Victor Stinner 6726626646
gh-108314: Add PyDict_ContainsString() function (#108323) 2023-08-24 15:59:12 +02:00
Jeffery To c163d7f0b6
gh-95855: Refactor platform triplet detection code, add detection for MIPS soft float and musl libc (#107221)
- Move platform triplet detection code into Misc/platform_triplet.c
- Refactor MIPS detection, use defined(__mips64) to detect MIPS64
- Compute libc values in separate section
- Add detection for MIPS soft float
- Add detection for musl

musl supports SPE with its soft-float ABI:
https://git.musl-libc.org/cgit/musl/commit/?id=7be59733d71ada3a32a98622507399253f1d5e48

Original patch by Christian Heimes.

Co-authored-by: Christian Heimes <christian@python.org>
Co-authored-by: Erlend E. Aasland <erlend@python.org>
2023-08-24 14:22:50 +02:00
Chris Markiewicz 2eb60c1934
gh-108111: Flush gzip write buffer before seeking, fixing bad writes (#108341)
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2023-08-24 10:23:01 +02:00
Victor Stinner aa9a359ca2
gh-108388: Split test_multiprocessing_spawn (#108396)
Split test_multiprocessing_fork, test_multiprocessing_forkserver and
test_multiprocessing_spawn into test packages. Each package is made
of 4 sub-tests: processes, threads, manager and misc. It allows
running more tests in parallel and so reduce the total test duration.
2023-08-24 05:35:39 +02:00
albanD 1700d34d31
gh-77377: Ensure multiprocessing SemLock is valid for spawn-based Process before serializing it (#107275)
Ensure multiprocessing SemLock is valid for spawn Process before serializing it.

Creating a multiprocessing SemLock with a fork context, and then trying to pass it to a spawn-created Process, would segfault if not detected early.

---------

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Antoine Pitrou <pitrou@free.fr>
2023-08-23 20:27:35 +00:00
Petr Viktorin 5d18715765
gh-107811: tarfile: treat overflow in UID/GID as failure to set it (#108369) 2023-08-23 20:00:07 +02:00
Irit Katriel 72119d16a5
gh-105481: remove regen-opcode. Generated _PyOpcode_Caches in regen-cases. (#108367) 2023-08-23 18:39:00 +01:00
Serhiy Storchaka 154477be72
gh-50002: xml.dom.minidom now preserves whitespaces in attributes (GH-107947)
Also double quotes (") are now only quoted in attributes.
2023-08-23 15:23:41 +03:00
Petr Viktorin 31b61d19ab
gh-108294: Add time.sleep audit event (GH-108298) 2023-08-23 11:00:22 +02:00
Victor Stinner 9173b2bbe1
gh-105776: Fix test_cppext when CC contains -std=c11 option (#108343)
Fix test_cppext when the C compiler command has the "-std=c11" option.
Remove "-std=" options from the compiler command.
2023-08-23 02:52:56 +00:00
Łukasz Langa 0cb0c238d5
gh-108310: Fix CVE-2023-40217: Check for & avoid the ssl pre-close flaw (#108315)
Instances of `ssl.SSLSocket` were vulnerable to a bypass of the TLS handshake
and included protections (like certificate verification) and treating sent
unencrypted data as if it were post-handshake TLS encrypted data.

The vulnerability is caused when a socket is connected, data is sent by the
malicious peer and stored in a buffer, and then the malicious peer closes the
socket within a small timing window before the other peers’ TLS handshake can
begin. After this sequence of events the closed socket will not immediately
attempt a TLS handshake due to not being connected but will also allow the
buffered data to be read as if a successful TLS handshake had occurred.

Co-authored-by: Gregory P. Smith [Google LLC] <greg@krypto.org>
2023-08-22 19:53:15 +02:00
Erlend E. Aasland 1a1bfc2891
gh-105539: Emit ResourceWarning if sqlite3 database is not closed explicitly (#108015) 2023-08-22 13:10:29 +02:00
Steve Dower de33b5c662
gh-106242: Make ntpath.realpath errors consistent with abspath when there are embedded nulls (GH-108248)
* gh-106242: Make ntpath.realpath errors consistent with abspath when there are embedded nulls

* Update 2023-08-22-00-36-57.gh-issue-106242.q24ITw.rst

mention Windows and the former incorrect ValueError.

---------

Co-authored-by: Gregory P. Smith <greg@krypto.org>
2023-08-22 00:16:02 +00:00
Irit Katriel a1cc74c4ee
gh-107901: Fix missing line number on BACKWARD_JUMP at the end of a for loop (#108242) 2023-08-21 23:44:31 +01:00
Irit Katriel 10a91d7e98
gh-108113: Make it possible to create an optimized AST (#108154) 2023-08-21 16:31:30 +00:00
Hugo van Kemenade 71962e5237
Run sphinx-lint on Misc/NEWS.d/next/ (#108212)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-08-21 13:13:55 +00:00
balmeida-nokia 37135d25e2
gh-107396: tarfiles: set self.exception before _init_read_gz() (GH-107485)
In the stack call of: _init_read_gz()
```
_read, tarfile.py:548
read, tarfile.py:526
_init_read_gz, tarfile.py:491
```
a try;except exists that uses `self.exception`, so it needs to be set before
calling _init_read_gz().
2023-08-21 11:39:06 +00:00
Serhiy Storchaka 80bdebdd85
gh-107916: Save the error code before decoding the filename in PyErr_SetFromErrnoWithFilename() etc (GH-107929) 2023-08-21 14:16:31 +03:00
Petr Viktorin acbd3f9c5c
gh-107845: Fix symlink handling for tarfile.data_filter (GH-107846)
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Lumír 'Frenzy' Balhar <frenzy.madness@gmail.com>
2023-08-21 12:56:46 +02:00
Raymond Hettinger 20cc90c0df
gh-105736: Sync pure python version of OrderedDict with the C version (#108098) 2023-08-21 11:49:08 +02:00
Nikita Sobolev db6dc6ce41
gh-107526: Revert "gh-100357: Convert several functions in bltinsmodule to AC" (#107542) 2023-08-20 17:54:10 -07:00
Serhiy Storchaka 633ea217a8
gh-107915: Handle errors in C API functions PyErr_Set*() and PyErr_Format() (GH-107918)
Such C API functions as PyErr_SetString(), PyErr_Format(),
PyErr_SetFromErrnoWithFilename() and many others no longer crash or
ignore errors if it failed to format the error message or decode the
filename. Instead, they keep a corresponding error.
2023-08-19 14:51:03 +03:00
Serhiy Storchaka 79db9d9a0e
gh-72684: Tkinter: provide interface for "tk busy" subcommands (GH-107684)
Add tkinter.Misc methods: tk_busy_hold(), tk_busy_configure(), tk_busy_cget(),
tk_busy_forget(), tk_busy_current(), and tk_busy_status().
2023-08-19 14:48:02 +03:00
Serhiy Storchaka 2f311437cd
gh-107704: Argument Clinic: add support for deprecating keyword use of parameters (GH-107984)
It is now possible to deprecate passing keyword arguments for
keyword-or-positional parameters with Argument Clinic, using the new
'/ [from X.Y]' syntax.
(To be read as "positional-only from Python version X.Y")

Co-authored-by: Erlend E. Aasland <erlend@python.org>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-08-19 10:13:35 +03:00
Ned Deily dc7b630b23
gh-107565: Update macOS installer to use OpenSSL 3.0.10. (GH-107897) 2023-08-18 17:38:24 -04:00
Ned Deily ed25f09716
gh-107565: Update multissltests and GitHub CI workflows to use OpenSSL 1.1.1v, 3.0.10, and 3.1.2. (GH-107896) 2023-08-18 15:48:20 -04:00
Tyler Smart 9bb576cb07
gh-107995: Fix doctest collection of functools.cached_property objects (#107996)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-08-18 15:44:38 +00:00
Erlend E. Aasland fd19509220
gh-108083: Don't ignore exceptions in sqlite3.Connection.__init__() and .close() (#108084)
- Add explanatory comments
- Add return value to connection_close() for propagating errors
- Always check the return value of connection_exec_stmt()
- Assert pre/post state in remove_callbacks()
- Don't log unraisable exceptions in case of interpreter shutdown
- Make sure we're not initialized if reinit fails
- Try to close the database even if ROLLBACK fails

Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2023-08-18 11:39:12 +00:00
Victor Stinner 3ff5ef2ad3
gh-108014: Add Py_IsFinalizing() function (#108032)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2023-08-18 12:34:41 +02:00
Pablo Galindo Salgado 75b3db8445
gh-107944: Improve error message for function calls with bad keyword arguments (#107969) 2023-08-17 19:39:42 +01:00
Nikita Sobolev 80f30cf51b
gh-102029: Deprecate passing arguments to `_PyRLock` in `threading` (#102071) 2023-08-17 09:19:07 -07:00
Mark Shannon 006e44f950
GH-108035: Remove the `_PyCFrame` struct as it is no longer needed for performance. (GH-108036) 2023-08-17 11:16:03 +01:00
Irit Katriel 665a4391e1
gh-105481: generate op IDs from bytecode.c instead of hard coding them in opcode.py (#107971) 2023-08-16 22:25:18 +00:00
Serhiy Storchaka 882cb79afa
gh-56166: Deprecate passing confusing positional arguments in re functions (#107778)
Deprecate passing optional arguments maxsplit, count and flags in
module-level functions re.split(), re.sub() and re.subn() as positional.
They should only be passed by keyword.
2023-08-16 13:35:35 -07:00
Nikita Sobolev bdd8ddfda1
gh-105724: Add location information to `assert` errors (GH-105935) 2023-08-16 11:35:38 +01:00
SKO abd9cc52d9
gh-100061: Proper fix of the bug in the matching of possessive quantifiers (GH-102612)
Restore the global Input Stream pointer after trying to match a sub-pattern.

Co-authored-by: Ma Lin <animalize@users.noreply.github.com>
2023-08-16 10:43:45 +03:00
6t8k a86df298df
gh-99203: shutil.make_archive(): restore select CPython <= 3.10.5 behavior (GH-99802)
Restore following CPython <= 3.10.5 behavior of shutil.make_archive()
that went away as part of gh-93160:

Do not create an empty archive if root_dir is not a directory, and, in
that case, raise FileNotFoundError or NotADirectoryError regardless
of format choice. Beyond the brought-back behavior, the function may
now also raise these exceptions in dry_run mode.
2023-08-16 10:00:03 +03:00
Ken Jin e28b0dc86d
gh-107557: Setup abstract interpretation (#107847)
Co-authored-by: Guido van Rossum <gvanrossum@users.noreply.github.com>
Co-authored-by: Jules <57632293+juliapoo@users.noreply.github.com>
2023-08-15 18:04:17 +00:00
Finn Womack 0932272431
gh-106242: Fix path truncation in os.path.normpath (GH-106816) 2023-08-15 16:33:00 +01:00
Dong-hee Na 6515ec3d3d
gh-107963: Fix set_forkserver_preload to check the type of given list (#107965)
gh-107963: Fix set_forkserver_preload to check the type of given list
2023-08-15 15:58:12 +02:00