* Add --fast-ci and --slow-ci options to libregrtest:
* --fast-ci uses a default timeout of 10 minutes and "-u all,-cpu"
(skip slowest tests).
* --slow-ci uses a default timeout of 20 minues and "-u all" (run
all tests).
* regrtest header now lists test resources.
* Makefile changes:
* "make test", "make hostrunnertest" and "make coverage-report" now
use --fast-ci option and TESTTIMEOUT variable.
* "make buildbottest" now uses "--slow-ci". Remove options which
became redundant with "--slow-ci".
* "make testall" and "make testuniversal" now use --slow-ci option
and TESTTIMEOUT variable.
* "make testall" now uses "find -exec rm ..." instead of
"find ... -print|xargs rm ...", same as "make clean".
* GitHub Actions workflow:
* Ubuntu and Address Sanitizer jobs now use "make test". Remove
options which became redundant with "--fast-ci".
* Windows jobs now use --fast-ci option.
* Use -j0 to detect the number of CPUs.
* Set Makefile TESTTIMEOUT default to an empty string, since
--slow-ci and --fast-ci use different default timeout. It's now
accepted to pass "--timeout=" to regrtest: treated as not timeout.
* Tools/scripts/run_tests.py now uses --fast-ci option.
* Tools/buildbot/test.bat now uses --slow-ci option. Remove
--timeout=1200 option, redundant with --slow-ci.
* Only show GitHub check annotations on changed doc paragraphs
* Improve check-warnings script arg parsing following Hugo's suggestions
* Factor filtering warnings by modified diffs into helper function
* Build docs on unmerged branch so warning lines match & avoid deep clone
---------
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
Co-authored-by: Ezio Melotti <ezio.melotti@gmail.com>
Remove the "cpython/pytime.h" header file: it only contained private
functions. Move functions to the internal pycore_time.h header file.
Move tests from _testcapi to _testinternalcapi. Rename also test
methods to have the same name than tested C functions.
No longer export these functions:
* _PyTime_Add()
* _PyTime_As100Nanoseconds()
* _PyTime_FromMicrosecondsClamp()
* _PyTime_FromTimespec()
* _PyTime_FromTimeval()
* _PyTime_GetPerfCounterWithInfo()
* _PyTime_MulDiv()
Upgrade builds to OpenSSL 1.1.1u.
This OpenSSL version addresses a pile if less-urgent CVEs since 1.1.1t.
The Mac/BuildScript/build-installer.py was already updated.
Also updates _ssl_data_111.h from OpenSSL 1.1.1u, _ssl_data_300.h from 3.0.9, and adds a new _ssl_data_31.h file from 3.1.1 along with the ssl.c code to use it.
Manual edits to the _ssl_data_300.h file prevent it from removing any existing definitions in case those exist in some peoples builds and were important (avoiding regressions during backporting).
backports of this prior to 3.12 will not include the openssl 3.1 header.
* CI: Temporarily skip paths with spaces to avoid "Error: One of your files includes a space"
* Dummy NEWS file to test the action. Will be deleted before merge.
* Revert "Dummy NEWS file to test the action. Will be deleted before merge."
This reverts commit 05cd028fd4.
* Add basic mypy workflow to CI
* Make the type check pass
---------
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
These are stubs to be used for adding hypothesis (https://hypothesis.readthedocs.io/en/latest/) tests to the standard library.
When the tests are run in an environment where `hypothesis` and its various dependencies are not installed, the stubs will turn any tests with examples into simple parameterized tests and any tests without examples are skipped.
It also adds hypothesis tests for the `zoneinfo` module, and a Github Actions workflow to run the hypothesis tests as a non-required CI job.
The full hypothesis interface is not stubbed out — missing stubs can be added as necessary.
Co-authored-by: Zac Hatfield-Dodds <zac.hatfield.dodds@gmail.com>
<!--
Thanks for your contribution!
Please read this comment in its entirety. It's quite important.
# Pull Request title
It should be in the following format:
```
gh-NNNNN: Summary of the changes made
```
Where: gh-NNNNN refers to the GitHub issue number.
Most PRs will require an issue number. Trivial changes, like fixing a
typo, do not need an issue.
# Backport Pull Request title
If this is a backport PR (PR made against branches other than `main`),
please ensure that the PR title is in the following format:
```
[X.Y] <title from the original PR> (GH-NNNN)
```
Where: [X.Y] is the branch name, e.g. [3.6].
GH-NNNN refers to the PR number from `main`.
-->
<!-- gh-issue-number: gh-103726 -->
* Issue: gh-103726
<!-- /gh-issue-number -->
"awaiting changes" means somebody put a review that requested changes.
"awaiting change review" means that the PR author published changes
after a red review and then requested a re-review.
Remove the bundled setuptools wheel from ensurepip, and stop installing setuptools in environments created by venv.
Co-Authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
Co-authored-by: C.A.M. Gerlach <CAM.Gerlach@Gerlach.CAM>
Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net>
This will keep us from adding new unsupported (i.e. non-const) C global variables, which would break interpreter isolation.
FYI, historically it is very uncommon for new global variables to get added. Furthermore, it is rare for new code to break the c-analyzer. So the check should almost always pass unnoticed.
Note that I've removed test_check_c_globals. A test wasn't a great fit conceptually and was super slow on debug builds. A CI check is a better fit.
This also resolves gh-100237.
https://github.com/python/cpython/issues/81057
Our legacy mailgun account is associated with a parent rackspace account that I am trying to decomission.
The necessary secret has been added to the GitHub Actions Secrets already, so this is ready to go on approval.
* gh-98098: Move zipfile into a package.
* Moved test_zipfile to a package
* Extracted module for test_path.
* Add blurb
* Add jaraco as owner of zipfile.Path.
* Synchronize with minor changes found at jaraco/zipp@d9e7f4352d.
* Auto-cancel old builds when new commit pushed to branch
* Add a fallback
Co-authored-by: Ezio Melotti <ezio.melotti@gmail.com>
* Use the same group for all workflows.
Co-authored-by: Ezio Melotti <ezio.melotti@gmail.com>
* Update project-updater.yml
Signed-off-by: sashashura <93376818+sashashura@users.noreply.github.com>
* Update project-updater.yml
repository-projects: write is not needed because a separate secrets.ADD_TO_PROJECT_PAT is used
Signed-off-by: sashashura <93376818+sashashura@users.noreply.github.com>
* Automatically update the `asyncio` GitHub project.
* Use a matrix to add issues to projects.
* Remove trailing whitespace.
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
If an HTTP link is redirected to a same looking HTTPS link, the latter can
be used directly without changes in readability and behavior.
It protects from a men-in-the-middle attack.
This change does not affect Python examples.
* gh-95218: Move tests for importlib.resources into test_importlib.resources.
* Also update makefile
* Include test_importlib/resources in code ownership rule.
The Issue templates are using the markup to make text bold.
We should be using proper text headers instead.
I replaced the **bold** text markup with L1 headers.
* Add minimal issue templates
* Wording tweaks
* Apply suggestions from code review
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
* Improve words in `security` template
* Update bug.md
* Update crash.md
* Add link to security vulnerability website from first page
* Never edit on your phone
* Apply suggestions from code review
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
* Update .github/ISSUE_TEMPLATE/config.yml
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Ezio Melotti <ezio.melotti@gmail.com>
Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@innova.no>
* There might not be a traceback if there's a crash
Co-authored-by: Ezio Melotti <ezio.melotti@gmail.com>
* Update .github/ISSUE_TEMPLATE/config.yml
Co-authored-by: Ezio Melotti <ezio.melotti@gmail.com>
* Reorder `config.yml` file
* Fix Erlend's nits
* version -> architecture
* Apply suggestions from code review
Co-authored-by: Ezio Melotti <ezio.melotti@gmail.com>
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
Co-authored-by: Ezio Melotti <ezio.melotti@gmail.com>
Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@innova.no>
This effectively reverts the Makefile change in gh-31637. I've added some notes so it is more clear what is going on.
We also update the "Check if generated files are up to date" job to run "make regen-deepfreeze" to ensure "make regen-global-objects" catches deepfreeze.c.
https://bugs.python.org/issue47146
This adds a new standard library module, `tomllib`, for parsing TOML.
The implementation is based on Tomli (https://github.com/hukkin/tomli).
## Steps taken (converting `tomli` to `tomllib`)
- Move everything in `tomli:src/tomli` to `Lib/tomllib`. Exclude `py.typed`.
- Remove `__version__ = ...` line from `Lib/tomllib/__init__.py`
- Move everything in `tomli:tests` to `Lib/test/test_tomllib`. Exclude the following test data dirs recursively:
- `tomli:tests/data/invalid/_external/`
- `tomli:tests/data/valid/_external/`
- Create `Lib/test/test_tomllib/__main__.py`:
```python
import unittest
from . import load_tests
unittest.main()
```
- Add the following to `Lib/test/test_tomllib/__init__.py`:
```python
import os
from test.support import load_package_tests
def load_tests(*args):
return load_package_tests(os.path.dirname(__file__), *args)
```
Also change `import tomli as tomllib` to `import tomllib`.
- In `cpython/Lib/tomllib/_parser.py` replace `__fp` with `fp` and `__s` with
`s`. Add the `/` to `load` and `loads` function signatures.
- Run `make regen-stdlib-module-names`
- Create `Doc/library/tomllib.rst` and reference it in `Doc/library/fileformats.rst`
Skip tests on ASAN and/or MSAN builds:
* multiprocessing tests
* test___all__
* test_concurrent_futures
* test_decimal
* test_peg_generator
* test_tools
The ASAN job of GitHub Actions no longer excludes these tests.
We're no longer using _Py_IDENTIFIER() (or _Py_static_string()) in any core CPython code. It is still used in a number of non-builtin stdlib modules.
The replacement is: PyUnicodeObject (not pointer) fields under _PyRuntimeState, statically initialized as part of _PyRuntime. A new _Py_GET_GLOBAL_IDENTIFIER() macro facilitates lookup of the fields (along with _Py_GET_GLOBAL_STRING() for non-identifier strings).
https://bugs.python.org/issue46541#msg411799 explains the rationale for this change.
The core of the change is in:
* (new) Include/internal/pycore_global_strings.h - the declarations for the global strings, along with the macros
* Include/internal/pycore_runtime_init.h - added the static initializers for the global strings
* Include/internal/pycore_global_objects.h - where the struct in pycore_global_strings.h is hooked into _PyRuntimeState
* Tools/scripts/generate_global_objects.py - added generation of the global string declarations and static initializers
I've also added a --check flag to generate_global_objects.py (along with make check-global-objects) to check for unused global strings. That check is added to the PR CI config.
The remainder of this change updates the core code to use _Py_GET_GLOBAL_IDENTIFIER() instead of _Py_IDENTIFIER() and the related _Py*Id functions (likewise for _Py_GET_GLOBAL_STRING() instead of _Py_static_string()). This includes adding a few functions where there wasn't already an alternative to _Py*Id(), replacing the _Py_Identifier * parameter with PyObject *.
The following are not changed (yet):
* stop using _Py_IDENTIFIER() in the stdlib modules
* (maybe) get rid of _Py_IDENTIFIER(), etc. entirely -- this may not be doable as at least one package on PyPI using this (private) API
* (maybe) intern the strings during runtime init
https://bugs.python.org/issue46541
Skip the 3 slowest tests of the Address Sanitizer CI of GitHub
Actions:
* test_tools
* test_peg_generator
* test_concurrent_futures
These tests take between 5 and 20 minutes on this CI which makes this
CI job the slowest. Making this CI job faster makes the whole Python
workflow faster. These tests are run on all others CIs.
Example of Address Sanitizer output:
10 slowest tests:
- test_peg_generator: 17 min 33 sec
- test_tools: 8 min 27 sec
- test_concurrent_futures: 5 min 24 sec
- test_zipfile: 2 min 41 sec
- test_compileall: 2 min 21 sec
- test_asyncio: 2 min 17 sec
- test_gdb: 1 min 43 sec
- test_weakref: 1 min 35 sec
- test_pickle: 1 min 18 sec
- test_subprocess: 1 min 12 sec
Moreover, test_concurrent_futures also seems to be affected by
bpo-45200 bug: libasan dead lock in pthread_create().
Check that users don't push changes with outdated or patched autoconf.
The presence of runstatedir option and aclocal 1.16.3 are good markers.
Use my container image to regenerate autoconf files. "Check for changes"
will fail later when any file is regenerated.
Use ccache in check_generated_files to speed up testing.
Remove the asyncore and asynchat modules, deprecated in Python
3.6: use the asyncio module instead.
Remove the smtpd module, deprecated in Python 3.6: the aiosmtpd
module can be used instead, it is based on asyncio.
* Remove asyncore, asynchat and smtpd documentation
* Remove test_asyncore, test_asynchat and test_smtpd
* Rename Lib/asynchat.py to Lib/test/support/_asynchat.py
* Rename Lib/asyncore.py to Lib/test/support/_asyncore.py
* Rename Lib/smtpd.py to Lib/test/support/_smtpd.py
* Remove DeprecationWarning from private _asyncore, _asynchat and
_smtpd modules
* _smtpd: remove deprecated properties
Add Modules subdirs to SRCDIRS to generate directories for out-of-tree
object files.
Debian wants ncurses lib. Works on Fedora, too.
Debian also needs pkg-config to detect correct flags.
Remove more outdated comments. Makefile now tracks header dependencies
-lintl is injected by configure when needed. Build _dbm with
gdbm-compat.
Group some modules by purpose. socket, select, and mmap work on Windows,
too.
Frozen modules must be added to several files in order to work properly. Before this change this had to be done manually. Here we add a tool to generate the relevant lines in those files instead. This helps us avoid mistakes and omissions.
https://bugs.python.org/issue45019
Victor Stinner
Adam Turner
Hugo van Kemenade
Nikita Sobolev
Mariatta
Brett Cannon
Alex Waygood
C.A.M. Gerlach
Ned Deily
Dong-hee Na
Kumar Aditya
Erlend E. Aasland
Jason R. Coombs
Sviatoslav Sydorenko
dependabot[bot]
Sviatoslav Sydorenko
Erlend E. Aasland
Petr Viktorin
Gregory P. Smith
Itamar Ostricher
Steve Dower
Pablo Galindo Salgado
T. Wouters
Zac Hatfield-Dodds
Jelle Zijlstra
Paul Ganssle
Sebastian Pipping
Irit Katriel
Carol Willing
Łukasz Langa
Pradyun Gedam
Ezio Melotti
Eric Snow
Ee Durbin
Barney Gale
Shantanu
Illia Volochii
Kumar Aditya
Filipe Laíns
Zachary Ware
Noam Cohen
Michael Droettboom
Jacob Walls
Alex
Christian Heimes
Guido van Rossum
Yury Selivanov
Serhiy Storchaka
Oleg Iarygin
Varun Sharma
Taneli Hukkinen
Brandt Bucher
Erlend Egeberg Aasland
Mark Dickinson
Ryan Mast
Ken Jin
Julien Palard
Marcono1234