Commit Graph

169 Commits

Author SHA1 Message Date
Victor Stinner 859618c8cd
gh-109566, regrtest: Add --fast-ci and --slow-ci options (#109570)
* Add --fast-ci and --slow-ci options to libregrtest:

  * --fast-ci uses a default timeout of 10 minutes and "-u all,-cpu"
    (skip slowest tests).
  * --slow-ci uses a default timeout of 20 minues and "-u all" (run
    all tests).

* regrtest header now lists test resources.
* Makefile changes:

  * "make test", "make hostrunnertest" and "make coverage-report" now
    use --fast-ci option and TESTTIMEOUT variable.
  * "make buildbottest" now uses "--slow-ci". Remove options which
    became redundant with "--slow-ci".
  * "make testall" and "make testuniversal" now use --slow-ci option
    and TESTTIMEOUT variable.
  * "make testall" now uses "find -exec rm ..." instead of
    "find ... -print|xargs rm ...", same as "make clean".

* GitHub Actions workflow:

  * Ubuntu and Address Sanitizer jobs now use "make test". Remove
    options which became redundant with "--fast-ci".
  * Windows jobs now use --fast-ci option.
  * Use -j0 to detect the number of CPUs.

* Set Makefile TESTTIMEOUT default to an empty string, since
  --slow-ci and --fast-ci use different default timeout. It's now
  accepted to pass "--timeout=" to regrtest: treated as not timeout.
* Tools/scripts/run_tests.py now uses --fast-ci option.
* Tools/buildbot/test.bat now uses --slow-ci option. Remove
  --timeout=1200 option, redundant with --slow-ci.
2023-09-26 17:22:50 +02:00
Victor Stinner 67d9363372
gh-109566: Run GHA and buildbot tests with --fail-rerun (#109567) 2023-09-19 15:50:27 +02:00
Steve Dower 10772ec150
bpo-41173: Copy test results file from ARM worker before uploading (GH-21305) 2020-07-08 00:24:39 +01:00
Paul Monson 55d12ce8b8 bpo-36511: clean up python process before deploy on ARM Windows buildbots (GH-14431) 2019-09-09 22:11:17 +01:00
Paul Monson ed70a344b5 bpo-36511: Fix failures in Windows ARM32 buildbot (GH-15181) 2019-08-08 16:12:33 -07:00
Paul Monson f8dd77d360 bpo-36511: Fix -u parameters for ARM32 tests (GH-14280) 2019-06-21 09:40:05 -07:00
Paul Monson a1952122a3 bpo-36511: Improve ARM32 buildbot scripts (GH-14251) 2019-06-20 09:33:32 -07:00
Paul Monson f355069a33 bpo-36511: Add buildbot scripts and fix tests for Windows ARM32 buildbot (GH-13454) 2019-06-19 13:09:54 -07:00
Paul Monson e7e5039d69 bpo-37181: Fix test_regrtest failures on Windows arm64 (GH-13872) 2019-06-07 10:58:41 -07:00
Paul Monson 51394b8c3d bpo-36511: Ensure error code propagates out of batch files (GH-13529) 2019-05-24 09:15:39 -07:00
Paul Monson 4f820723c8 bpo-36511: Windows arm32 buildbot changes (remove extra space) (GH-13351)
@zooba
I just realized that this whitespace fix didn't get pushed.


https://bugs.python.org/issue36511
2019-05-15 16:10:39 -07:00
Paul Monson 67ff6a103a bpo-36511: Windows ARM32 buildbot changes (GH-12917) 2019-05-15 15:42:29 -07:00
Victor Stinner 00561189fb buildbot: pass --fail-env-changed to regrtest (#2509)
Make tests fail if a test altered the environment.
2017-06-30 16:44:08 +02:00
Victor Stinner 2d98c53542 Revert "bpo-30673: test.bat: add -t option (timeout) (#2211)" (#2245)
This reverts commit 258bfc462b.
2017-06-16 14:38:34 +02:00
Victor Stinner 258bfc462b bpo-30673: test.bat: add -t option (timeout) (#2211) 2017-06-15 20:19:45 +02:00
Zachary Ware 6b6e687766 bpo-27425: Be more explicit in .gitattributes (GH-840)
Updates checked-in line endings on several files.
2017-06-10 14:58:42 -05:00
Victor Stinner 2f462a68f9 Buildbot: give 20 minute per test file
It seems like at least 2 buildbots need more than 15 minutes per test file.
Example with "AMD64 Snow Leop 3.x":

    10 slowest tests:
    - test_tools: 14 min 40 sec
    - test_tokenize: 11 min 57 sec
    - test_datetime: 11 min 25 sec
    - ...
2016-09-12 13:04:17 +02:00
Zachary Ware d07b66b817 Backed out changeset 491bbba73bca
This change didn't have the intended effect.
2016-09-10 08:55:15 -05:00
Victor Stinner e103aaca67 Show regrtest env changed warn on Windows buildbot
Issue #27829: don't pass --quiet option to regrtest to see
"Warning -- xxx was modified by ..." warnings.
2016-09-10 04:07:38 -04:00
Steve Dower 5daba1a59a Remove buildbot diagnostic code. 2016-09-09 15:45:47 -07:00
Zachary Ware 664cc5071a Remove another useless buildbot script 2016-09-09 15:42:06 -07:00
Steve Dower 37df068e86 Expands buildbot validation code 2016-09-09 15:39:11 -07:00
Zachary Ware 8f35675853 Remove outdated buildbot scripts 2016-09-09 15:35:38 -07:00
Steve Dower 5510d821d3 Adds temporary validation code to buildbot script 2016-09-09 15:33:42 -07:00
Steve Dower b230cc4b5c Ensures buildbots don't have zip files in build directory. 2016-09-09 15:24:11 -07:00
Victor Stinner f7457001a6 Tests: add --slowest option to buildbots
Display the top 10 slowest tests.
2016-08-17 11:27:40 +02:00
Victor Stinner 6d81a2136d regrtest doesn't ignore -j1 anymore
* regrtest now uses subprocesses when the -j1 command line option
  is used: each test file runs in a fresh child process. Before, the -j1 option
  was ignored.
* Tools/buildbot/test.bat script now uses -j1 by default to run
  each test file in fresh child process.
2016-05-20 13:15:55 +02:00
Victor Stinner 96f6e7a1ed Buildbots: change also Windows timeout from 1 hour to 15 min 2016-03-23 12:38:01 +01:00
Steve Dower 4a6f4825f4 Backs out buildbot clean trigger. 2016-03-08 13:10:29 -08:00
Steve Dower 79993a90e2 Adds warning to prepare_ssl when nasm is not available.
Force clean of externals on buildbots.
2016-03-08 12:50:57 -08:00
Steve Dower 3cc6da2b07 Removes deprecated -n option from buildbot script. 2015-10-08 09:06:17 -07:00
Zachary Ware e74fe18ebb Allow PCbuild\rt.bat to accept unlimited arguments for regrtest.
This makes it possible to pass more than 4 tests by name through
Tools\buildbot\test.bat
2015-09-03 23:43:37 -05:00
Steve Dower 807404921a Removes --clean-only option from buildbots. 2015-08-18 20:02:07 -07:00
Steve Dower e7e69391e3 Issue #24847: Fixes Tix revision number and reverts fix for the wrong revision.
Triggers clean on buildbots.
2015-08-18 19:29:51 -07:00
Zachary Ware 4ab4ac8e03 Merge 3.4 2015-06-17 10:08:44 -05:00
Zachary Ware 9fe164364a Deprecate unused scripts in Tools/buildbot.
I would just outright delete them, but the readme in PCbuild recommended
their use, so I figure it would be nice to leave them there for a while.
2015-06-16 10:56:14 -05:00
Zachary Ware 785273cd84 Merge forward extras beyond #21907 backport. 2015-06-10 00:43:20 -05:00
Zachary Ware 6250df81bf Clean up/refactor the batch scripts used for building on Windows.
This is mostly a backport of issue #21907, but also includes a few
extras necessary to make the bulidbot scripts as thin as possible.
2015-06-09 23:16:52 -05:00
Zachary Ware 7dc9dea778 Issue #20035: Reimplement tkinter._fix module as a C function.
The new private C function makes no permanent changes to the environment
and is #ifdef'd out on non-Windows platforms.
2015-05-22 11:36:53 -05:00
Zachary Ware 774ac377da Closes #17202: Merge with 3.4 2015-04-13 12:11:40 -05:00
Zachary Ware 4c9c848159 Issue #17202: Add .bat to .hgeol to force them to CRLF.
Using LF can a script to fail if it tries to use a label that is
split across 512 byte blocks.  Who knows why.
2015-04-13 11:59:54 -05:00
Steve Dower bb24087a2c Issue #23260: Update Windows installer 2015-02-05 22:08:48 -08:00
Zachary Ware 494b09ca5f Backed out changeset ceaac3d483a1 (buildbots refreshed) 2014-12-15 15:44:33 -06:00
Zachary Ware d0b07d5d5e Fetch new external sources on the buildbots 2014-12-15 15:42:37 -06:00
Steve Dower 10e65856a3 Removes use of ValueOrDefault function that is not always available on the buildbots. 2014-12-12 12:18:11 -08:00
Steve Dower 65e4cb10d9 Issue #22919: Windows build updated to support VC 14.0 (Visual Studio 2015), which will be used for the official 3.5 release. 2014-11-22 12:54:57 -08:00
Zachary Ware 740e1dcdc2 Issue #17896: Move Windows external lib sources from .. to externals. 2014-11-01 22:48:24 -05:00
Zachary Ware 4b2b1de0bd Issue #17896: Move Windows external lib sources from .. to externals. 2014-11-01 22:39:21 -05:00
Zachary Ware aa3ea7ee78 Issue #17717: Pull NASM from svn.python.org for OpenSSL build. 2014-11-01 17:11:08 -05:00
Zachary Ware 77770bdaa8 Turn externals cleaning back off 2014-10-17 17:40:04 -05:00