Commit Graph

30250 Commits

Author SHA1 Message Date
Serhiy Storchaka 6f3c138dfa
gh-108751: Add copy.replace() function (GH-108752)
It creates a modified copy of an object by calling the object's
__replace__() method.

It is a generalization of dataclasses.replace(), named tuple's _replace()
method and replace() methods in various classes, and supports all these
stdlib classes.
2023-09-06 23:55:42 +03:00
Serhiy Storchaka 3a08db8d13
gh-106307: Fix PyMapping_GetOptionalItemString() (GH-108797)
The resulting pointer was not set to NULL if the creation of a temporary
string object was failed.

The tests were also missed due to oversight.
2023-09-06 19:47:38 +00:00
Victor Stinner a0773b89df
gh-108753: Enhance pystats (#108754)
Statistics gathering is now off by default. Use the "-X pystats"
command line option or set the new PYTHONSTATS environment variable
to 1 to turn statistics gathering on at Python startup.

Statistics are no longer dumped at exit if statistics gathering was
off or statistics have been cleared.

Changes:

* Add PYTHONSTATS environment variable.
* sys._stats_dump() now returns False if statistics are not dumped
  because they are all equal to zero.
* Add PyConfig._pystats member.
* Add tests on sys functions and on setting PyConfig._pystats to 1.
* Add Include/cpython/pystats.h and Include/internal/pycore_pystats.h
  header files.
* Rename '_py_stats' variable to '_Py_stats'.
* Exclude Include/cpython/pystats.h from the Py_LIMITED_API.
* Move pystats.h include from object.h to Python.h.
* Add _Py_StatsOn() and _Py_StatsOff() functions. Remove
  '_py_stats_struct' variable from the API: make it static in
  specialize.c.
* Document API in Include/pystats.h and Include/cpython/pystats.h.
* Complete pystats documentation in Doc/using/configure.rst.
* Don't write "all zeros" stats: if _stats_off() and _stats_clear()
  or _stats_dump() were called.
* _PyEval_Fini() now always call _Py_PrintSpecializationStats() which
  does nothing if stats are all zeros.

Co-authored-by: Michael Droettboom <mdboom@gmail.com>
2023-09-06 15:54:59 +00:00
Victor Stinner 8ff1142578
gh-108851: Fix tomllib recursion tests (#108853)
* Add get_recursion_available() and get_recursion_depth() functions
  to the test.support module.
* Change infinite_recursion() default max_depth from 75 to 100.
* Fix test_tomllib recursion tests for WASI buildbots: reduce the
  recursion limit and compute the maximum nested array/dict depending
  on the current available recursion limit.
* test.pythoninfo logs sys.getrecursionlimit().
* Enhance test_sys tests on sys.getrecursionlimit()
  and sys.setrecursionlimit().
2023-09-06 17:34:31 +02:00
Victor Stinner fbce43a251
gh-91960: Skip test_gdb if gdb cannot retrive Python frames (#108999)
Skip test_gdb if gdb is unable to retrieve Python frame objects: if a
frame is "<optimized out>". When Python is built with "clang -Og",
gdb can fail to retrive the 'frame' parameter of
_PyEval_EvalFrameDefault(). In this case, tests like py_bt() are
likely to fail. Without getting access to Python frames,
python-gdb.py is mostly clueless on retrieving the Python traceback.
Moreover, test_gdb is no longer skipped on macOS if Python is built
with Clang.
2023-09-06 14:34:35 +00:00
Victor Stinner a8cae4071c
gh-107219: Fix concurrent.futures terminate_broken() (#108974)
Fix a race condition in _ExecutorManagerThread.terminate_broken():
ignore the InvalidStateError on future.set_exception(). It can happen
if the future is cancelled before the caller.

Moreover, test_crash_big_data() now waits explicitly until the
executor completes.
2023-09-06 15:57:01 +02:00
Victor Stinner 14d6e197cc
gh-108303: Create Lib/test/test_dataclasses/ directory (#108978)
Move test_dataclasses.py and its "dataclass_*.py" modules into the
new Lib/test/test_dataclasses/ subdirectory.
2023-09-06 15:54:16 +02:00
Nikita Sobolev 1fb20d42c5
gh-108983: Add more PEP 526 tests to `test_grammar` (#108984) 2023-09-06 06:41:38 -07:00
Matthias Bussonnier 5f3433f210
gh-106670: Fix Pdb handling of chained Exceptions with no stacks. (#108865) 2023-09-06 09:41:56 +00:00
Victor Stinner cd2ef21b07
gh-108962: Skip test_tempfile.test_flags() if not supported (#108964)
Skip test_tempfile.test_flags() if chflags() fails with "OSError:
[Errno 45] Operation not supported" (ex: on FreeBSD 13).
2023-09-05 21:59:40 +00:00
Guido van Rossum b87263be9b
gh-106581: Support multiple uops pushing new values (#108895)
Also avoid the need for the awkward .clone() call in the argument
to mgr.adjust_inverse() and mgr.adjust().
2023-09-05 13:58:39 -07:00
Anthony Shaw 2c4c26c4ce
gh-108469: Update ast.unparse for unescaped quote support from PEP701 [3.12] (#108553)
Co-authored-by: sunmy2019 <59365878+sunmy2019@users.noreply.github.com>
2023-09-05 21:01:23 +01:00
Nikita Sobolev 3f89b25763
gh-108927: Fix test_import + test_importlib + test_unittest problem (#108929) 2023-09-05 18:57:48 +00:00
Serhiy Storchaka 1e0d62793a
gh-108416: Mark slow but not CPU bound test methods with requires_resource('walltime') (GH-108480) 2023-09-05 17:56:30 +03:00
Serhiy Storchaka f980cc19b9
gh-89392: Use unittest test runner for doctests in test_getopt (GH-108916) 2023-09-05 17:35:28 +03:00
Serhiy Storchaka eaabaac7c0
gh-89392: Use normal unittest runner in test_type_cache (GH-108911) 2023-09-05 13:46:17 +00:00
Serhiy Storchaka 420c636219
Add missed "f" in an f-string (GH-108906) 2023-09-05 12:00:28 +00:00
Nikita Sobolev b4c8cce9a7
gh-108840: Remove unused `TestEnumTypeSubclassing` from `test_enum` (#108841) 2023-09-05 09:45:54 +01:00
Petr Viktorin 230649f538
gh-108294: Add error handling for time.sleep audit event (GH-108363)
I've also cleaned the tests up a bit to make this easier to test.
2023-09-05 10:25:08 +02:00
Mark Shannon 5a2a046151
GH-108390: Prevent non-local events being set with `sys.monitoring.set_local_events()` (GH-108420) 2023-09-05 08:03:53 +01:00
Serhiy Storchaka 04a0830b00
gh-89392: Remove support of test_main() in libregrtest (GH-108876) 2023-09-05 08:36:43 +03:00
Victor Stinner 1170d5a292
gh-108834: regrtest --fail-rerun exits with code 5 (#108896)
When the --fail-rerun option is used and a test fails and then pass,
regrtest now uses exit code 5 ("rerun) instead of 2 ("bad test").
2023-09-05 01:09:42 +00:00
Victor Stinner 676593859e
gh-106320: Remove private _PyErr_WriteUnraisableMsg() (#108863)
Move the private _PyErr_WriteUnraisableMsg() functions to the
internal C API (pycore_pyerrors.h).

Move write_unraisable_exc() from _testcapi to _testinternalcapi.
2023-09-05 01:54:55 +02:00
Tian Gao 6304d983a0
gh-108463: Make expressions/statements work as expected in pdb (#108464) 2023-09-04 21:44:40 +00:00
Nikita Sobolev 6ead5bd6ae
Disable `differing_test_runners` health check (#108886) 2023-09-04 18:31:58 +00:00
Petr Viktorin 40e52c94a2
gh-107902: Don't test setting suid/sgid on systems that don't support them (GH-108368) 2023-09-04 15:46:00 +02:00
Serhiy Storchaka f3b6608ba2
gh-89392: Fix running test_pep646_syntax as script (GH-108875) 2023-09-04 15:24:03 +03:00
Serhiy Storchaka 074ac1f72e
bpo-45229: Make ElementTree tests discoverable (GH-108859) 2023-09-04 13:04:32 +03:00
Serhiy Storchaka d0b22f6bd8
gh-89392: Make test_pep646_syntax discoverable (GH-108861) 2023-09-04 12:41:58 +03:00
Serhiy Storchaka 76f3c043b6
gh-89392: Remove test_main() in test_netrc (GH-108860) 2023-09-04 12:41:13 +03:00
Victor Stinner 5a79d2ae57
Revert "gh-46376: Return existing pointer when possible in ctypes (#1… (#108688)
This reverts commit 08447b5deb.

Revert also _ctypes.c changes of the PyDict_ContainsString() change,
commit 6726626646.
2023-09-04 11:21:47 +02:00
Victor Stinner 31c2945f14
gh-108834: regrtest reruns failed tests in subprocesses (#108839)
When using --rerun option, regrtest now re-runs failed tests
in verbose mode in fresh worker processes to have more
deterministic behavior. So it can write its final report even
if a test killed a worker progress.

Add --fail-rerun option to regrtest: exit with non-zero exit code
if a test failed pass passed when re-run in verbose mode (in a
fresh process). That's now more useful since tests can pass
when re-run in a fresh worker progress, whereas they failed
when run after other tests when tests are run sequentially.

Rename --verbose2 option (-w) to --rerun. Keep --verbose2 as a
deprecated alias.

Changes:

* Fix and enhance statistics in regrtest summary. Add "(filtered)"
  when --match and/or --ignore options are used.
* Add RunTests class.
* Add TestResult.get_rerun_match_tests() method
* Rewrite code to serialize/deserialize worker arguments as JSON
  using a new WorkerJob class.
* Fix stats when a test is run with --forever --rerun.
* If failed test names cannot be parsed, log a warning and don't
  filter tests.
* test_regrtest.test_rerun_success() now uses a marker file, since
  the test is re-run in a separated process.
* Add tests on normalize_test_name() function.
* Add test_success() and test_skip() tests to test_regrtest.
2023-09-03 21:37:15 +00:00
Sangyun_LEE 0c369d6cb8
Update Lib/test/test_unittest/testmock/testmock.py: fix typo RuntimError to RuntimeError (#108847) 2023-09-03 22:19:49 +01:00
Serhiy Storchaka 509bb61977
Reorder some test's decorators (GH-108804)
For example, do not demand the 'cpu' resource if the test cannot be run
due to non-working threads.
2023-09-03 15:21:43 +03:00
Sergey B Kirpichev 9c995abd78
gh-102837: improve test coverage for math module (#102523)
- input checks for math_1(L989), math_1a(L1023), math_2(L1064,L1071), hypot(L2682), log(L2307), ldexp(L2168), ceil(L1165), floor(L1236,L1239) and dist(L2587,L2588,L2628).
- drop inaccessible "if" branch (L3518) in perm_comb_small()
- improve fsum coverage for exceptional cases (L1433,L1438,L1451,L1497), ditto fmod(L2378)
- rewrite modf to fix inaccessible case(L2229), ditto for pow(L2988)
    
(all line numbers are wrt the main branch at 5e6661bce9)
2023-09-03 01:48:47 -07:00
Victor Stinner d4e534cbb3
regrtest computes statistics (#108793)
test_netrc, test_pep646_syntax and test_xml_etree now return results
in the test_main() function.

Changes:

* Rewrite TestResult as a dataclass with a new State class.
* Add test.support.TestStats class and Regrtest.stats_dict attribute.
* libregrtest.runtest functions now modify a TestResult instance
  in-place.
* libregrtest summary lists the number of run tests and skipped
  tests, and denied resources.
* Add TestResult.has_meaningful_duration() method.
* Compute TestResult duration in the upper function.
* Use time.perf_counter() instead of time.monotonic().
* Regrtest: rename 'resource_denieds' attribute to 'resource_denied'.
* Rename CHILD_ERROR to MULTIPROCESSING_ERROR.
* Use match/case syntadx to have different code depending on the
  test state.

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-09-02 18:09:36 +02:00
Barney Gale bdc3c884cd
GH-78722: Raise exceptions from `pathlib.Path.iterdir()` without delay. (#107320)
`pathlib.Path.iterdir()` now immediately raises any `OSError`
exception from `os.listdir()`, rather than waiting until its
result is iterated over.
2023-09-02 16:08:03 +01:00
Victor Stinner 4f9b706c6f
gh-108794: doctest counts skipped tests (#108795)
* Add 'skipped' attribute to TestResults.
* Add 'skips' attribute to DocTestRunner.
* Rename private DocTestRunner._name2ft attribute
  to DocTestRunner._stats.
* Use f-string for string formatting.
* Add some tests.
* Document DocTestRunner attributes and its API for statistics.
* Document TestResults class.

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-09-02 16:42:07 +02:00
Serhiy Storchaka f3ba0a74cd
gh-108416: Mark slow test methods with @requires_resource('cpu') (GH-108421)
Only mark tests which spend significant system or user time,
by itself or in subprocesses.
2023-09-02 07:45:34 +03:00
Guido van Rossum 76ce537fb1
Fix test_regrtest when run with uops always on (#108778)
The fix has two parts:
- When `-X uops` is detected, pass it on to the subprocess created to run the manufactured test.
  I need this so I can run `./python -Xuops -m test test_regrtest` and see it fail without the next fix.
- Use `-R 6:3:` in `ArgsTestCase.test_huntrleaks` instead of `-R 3:3:` -- it takes longer to settle with `-X uops`.
2023-09-01 16:27:09 -07:00
Nikita Sobolev 3b73f9f00e
gh-107862: Add roundtrip `hypothesis` tests to `test_binascii` (#107863) 2023-09-01 21:04:07 +00:00
Alex Waygood a1cbace91b
gh-105509: Simplify implementation of `typing.Annotated` (#105510) 2023-09-01 13:57:25 -07:00
Victor Stinner 23f54c1200
Make test_fcntl quiet (#108758)
Running test_fcntl logs two "struct.pack: ..." lines because
multiprocessing imports test_fcntl twice with
test.support.verbose=1. Move get_lockdata() inside TestFcntl test
case and only call it where it's needed, to stop logging these lines.
2023-09-01 14:59:09 +02:00
Irit Katriel 844f4c2e12
gh-108727: Fix segfault due to missing tp_dealloc definition for CounterOptimizer_Type (GH-108734) 2023-09-01 10:16:09 +01:00
Nikita Sobolev 044b8b3b6a
gh-107805: Fix signatures of module-level generated functions in `turtle` (#107807)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-09-01 01:18:15 -07:00
Victor Stinner ad73674283
gh-107603: Argument Clinic: Only include pycore_gc.h if needed (#108726)
Argument Clinic now only includes pycore_gc.h if PyGC_Head is needed,
and only includes pycore_runtime.h if _Py_ID() is needed.

* Add 'condition' optional argument to Clinic.add_include().
* deprecate_keyword_use() includes pycore_runtime.h when using
  the _PyID() function.
* Fix rendering of includes: comments start at the column 35.
* Mark PC/clinic/_wmimodule.cpp.h and
  "Objects/stringlib/clinic/*.h.h" header files as generated in
  .gitattributes.

Effects:

* 42 header files generated by AC no longer include the internal C
  API, instead of 4 header files before. For example,
  Modules/clinic/_abc.c.h no longer includes the internal C API.
* Fix _testclinic_depr.c.h: it now always includes pycore_runtime.h
  to get _Py_ID().
2023-08-31 23:42:34 +02:00
Victor Stinner 2bd960b579
gh-108337: Add pyatomic.h header (#108701)
This adds a new header that provides atomic operations on common data
types. The intention is that this will be exposed through Python.h,
although that is not the case yet. The only immediate use is in
the test file.

Co-authored-by: Sam Gross <colesbury@gmail.com>
2023-08-31 21:41:18 +00:00
Victor Stinner c6d56135e1
gh-108638: Fix tests when _stat extension is missing (#108689)
Fix test_inspect and test_pydoc when the _stat extension is missing.
Skip tests relying on _stat when _stat is missing.
2023-08-31 23:14:23 +02:00
Ethan Furman d48760b2f1
gh-108682: [Enum] raise TypeError if super().__new__ called in custom __new__ (GH-108704)
When overriding the `__new__` method of an enum, the underlying data type should be created directly; i.e. .

    member = object.__new__(cls)
    member = int.__new__(cls, value)
    member = str.__new__(cls, value)

Calling `super().__new__()` finds the lookup version of `Enum.__new__`, and will now raise an exception when detected.
2023-08-31 12:45:12 -07:00
Victor Stinner 13a00078b8
gh-108634: Py_TRACE_REFS uses a hash table (#108663)
Python built with "configure --with-trace-refs" (tracing references)
is now ABI compatible with Python release build and debug build.
Moreover, it now also supports the Limited API.

Change Py_TRACE_REFS build:

* Remove _PyObject_EXTRA_INIT macro.
* The PyObject structure no longer has two extra members (_ob_prev
  and _ob_next).
* Use a hash table (_Py_hashtable_t) to trace references (all
  objects): PyInterpreterState.object_state.refchain.
* Py_TRACE_REFS build is now ABI compatible with release build and
  debug build.
* Limited C API extensions can now be built with Py_TRACE_REFS:
  xxlimited, xxlimited_35, _testclinic_limited.
* No longer rename PyModule_Create2() and PyModule_FromDefAndSpec2()
  functions to PyModule_Create2TraceRefs() and
  PyModule_FromDefAndSpec2TraceRefs().
* _Py_PrintReferenceAddresses() is now called before
  finalize_interp_delete() which deletes the refchain hash table.
* test_tracemalloc find_trace() now also filters by size to ignore
  the memory allocated by _PyRefchain_Trace().

Test changes for Py_TRACE_REFS:

* Add test.support.Py_TRACE_REFS constant.
* Add test_sys.test_getobjects() to test sys.getobjects() function.
* test_exceptions skips test_recursion_normalizing_with_no_memory()
  and test_memory_error_in_PyErr_PrintEx() if Python is built with
  Py_TRACE_REFS.
* test_repl skips test_no_memory().
* test_capi skisp test_set_nomemory().
2023-08-31 18:33:34 +02:00