Commit Graph

5 Commits

Author SHA1 Message Date
Victor Stinner 8fba9523cf
bpo-42398: Fix "make regen-all" race condition (GH-23362)
Fix a race condition in "make regen-all" when make -jN option is used
to run jobs in parallel. The clinic.py script now only use atomic
write to write files. Moveover, generated files are now left
unchanged if the content does not change, to not change the file
modification time.

The "make regen-all" command runs "make clinic" and "make
regen-importlib" targets:

* "make regen-importlib" builds object files (ex: Modules/_weakref.o)
  from source files (ex: Modules/_weakref.c) and clinic files (ex:
  Modules/clinic/_weakref.c.h)
* "make clinic" always rewrites all clinic files
  (ex: Modules/clinic/_weakref.c.h)

Since there is no dependency between "clinic" and "regen-importlib"
Makefile targets, these two targets can be run in parallel. Moreover,
half of clinic.py file writes are not atomic and so there is a race
condition when "make regen-all" runs jobs in parallel using make -jN
option (which can be passed in MAKEFLAGS environment variable).

Fix clinic.py to make all file writes atomic:

* Add write_file() function to ensure that all file writes are
  atomic: write into a temporary file and then use os.replace().
* Moreover, write_file() doesn't recreate or modify the file if the
  content does not change to avoid modifying the file modification
  file.
* Update test_clinic to verify these assertions with a functional
  test.
* Remove Clinic.force attribute which was no longer used, whereas
  Clinic.verify remains useful.
2020-11-18 15:36:27 +01:00
Hai Shi 4660597b51
bpo-40275: Use new test.support helper submodules in tests (GH-21448) 2020-08-03 18:49:18 +02:00
Eric Snow ee536b2020
bpo-36876: Add a tool that identifies unsupported global C variables. (#15877) 2019-09-11 19:49:45 +01:00
Serhiy Storchaka 837c7dc1ed
bpo-35578: Add an example file for testing Argument Clinic converters. (GH-11306) 2018-12-25 10:17:28 +02:00
Victor Stinner 65fc98e7b1
bpo-26901: Fix the Argument Clinic test suite (GH-8879)
* Fix Tools/clinic/clinic_test.py: add missing
  FakeClinic.destination_buffers attribute and pass a file argument
  to Clinic().
* Rename Tools/clinic/clinic_test.py to Lib/test/test_clinic.py:
  add temporary Tools/clinic/ to sys.path to import the clinic
  module.

Co-Authored-By: Pablo Galindo <pablogsal@gmail.com>
2018-09-03 23:17:20 +02:00