cpython/Lib
Victor Stinner 8fba9523cf
bpo-42398: Fix "make regen-all" race condition (GH-23362)
Fix a race condition in "make regen-all" when make -jN option is used
to run jobs in parallel. The clinic.py script now only use atomic
write to write files. Moveover, generated files are now left
unchanged if the content does not change, to not change the file
modification time.

The "make regen-all" command runs "make clinic" and "make
regen-importlib" targets:

* "make regen-importlib" builds object files (ex: Modules/_weakref.o)
  from source files (ex: Modules/_weakref.c) and clinic files (ex:
  Modules/clinic/_weakref.c.h)
* "make clinic" always rewrites all clinic files
  (ex: Modules/clinic/_weakref.c.h)

Since there is no dependency between "clinic" and "regen-importlib"
Makefile targets, these two targets can be run in parallel. Moreover,
half of clinic.py file writes are not atomic and so there is a race
condition when "make regen-all" runs jobs in parallel using make -jN
option (which can be passed in MAKEFLAGS environment variable).

Fix clinic.py to make all file writes atomic:

* Add write_file() function to ensure that all file writes are
  atomic: write into a temporary file and then use os.replace().
* Moreover, write_file() doesn't recreate or modify the file if the
  content does not change to avoid modifying the file modification
  file.
* Update test_clinic to verify these assertions with a functional
  test.
* Remove Clinic.force attribute which was no longer used, whereas
  Clinic.verify remains useful.
2020-11-18 15:36:27 +01:00
..
asyncio bpo-42140: Improve asyncio.wait function (GH-22938) 2020-11-10 14:12:52 -08:00
collections Revert "Fix all Python Cookbook links (#22205)" (GH-22424) 2020-09-27 01:47:25 +01:00
concurrent bpo-39349: Add cancel_futures to Executor.shutdown base class (GH-22023) 2020-09-01 17:18:07 -04:00
ctypes bpo-41100: Support macOS 11 and Apple Silicon (GH-22855) 2020-11-08 10:05:27 +01:00
curses
dbm
distutils bpo-41100: Support macOS 11 and Apple Silicon (GH-22855) 2020-11-08 10:05:27 +01:00
email bpo-30681: Support invalid date format or value in email Date header (GH-22090) 2020-10-26 17:31:06 -07:00
encodings bpo-39337: encodings.normalize_encoding() now ignores non-ASCII characters (GH-22219) 2020-10-14 17:43:31 +02:00
ensurepip bpo-41490: Bump vendored pip to version 20.2.3 (#22527) 2020-10-04 18:45:31 +02:00
html
http bpo-40968: Send http/1.1 ALPN extension (#20959) 2020-11-13 16:37:52 +01:00
idlelib bpo-40511: Stop unwanted flashing of IDLE calltips (GH-20910) 2020-11-01 22:59:52 -05:00
importlib Bump magic number. (GH-23245) 2020-11-12 10:42:44 +00:00
json
lib2to3
logging bpo-26789: Fix logging.FileHandler._open() at exit (GH-23053) 2020-11-02 23:17:46 +01:00
msilib
multiprocessing bpo-41344: Raise ValueError when creating shared memory of size 0 (GH-21556) 2020-08-30 20:03:11 +01:00
pydoc_data Python 3.10.0a2 2020-11-03 00:02:08 +00:00
site-packages
sqlite3 bpo-42264: Deprecate sqlite3.OptimizedUnicode (GH-23163) 2020-11-17 17:55:12 +02:00
test bpo-42398: Fix "make regen-all" race condition (GH-23362) 2020-11-18 15:36:27 +01:00
tkinter bpo-41876: Overload __repr__ for tkinter Font objects (GH-22450) 2020-10-14 13:02:51 +03:00
turtledemo
unittest bpo-41877 Check for asert, aseert, assrt in mocks (GH-23165) 2020-11-05 09:04:38 -08:00
urllib bpo-40968: Send http/1.1 ALPN extension (#20959) 2020-11-13 16:37:52 +01:00
venv bpo-35328: Set VIRTUAL_ENV_PROMPT at venv activation (GH-21587) 2020-07-28 09:41:57 +01:00
wsgiref
xml bpo-40624: Add support for the XPath != operator in xml.etree (GH-22147) 2020-11-09 08:02:39 +01:00
xmlrpc
zoneinfo bpo-41065: Use zip-strict in zoneinfo (GH-21031) 2020-06-23 10:21:26 -04:00
__future__.py bpo-41314: fixed annotations __future__ version (GH-21616) 2020-07-25 14:42:49 -07:00
__phello__.foo.py
_aix_support.py
_bootsubprocess.py
_collections_abc.py Fix: Docstrings hidden by slots. (GH-23352) 2020-11-17 13:50:23 -08:00
_compat_pickle.py
_compression.py
_markupbase.py bpo-31844: Remove _markupbase.ParserBase.error() (GH-8562) 2020-07-16 09:13:05 +03:00
_osx_support.py bpo-41100: Support macOS 11 and Apple Silicon (GH-22855) 2020-11-08 10:05:27 +01:00
_py_abc.py
_pydecimal.py Revert "bpo-26680: Incorporate is_integer in all built-in and standard library numeric types (GH-6121)" (GH-22584) 2020-10-07 16:43:44 -07:00
_pyio.py
_sitebuiltins.py
_strptime.py
_threading_local.py
_weakrefset.py
abc.py bpo-41905: Add abc.update_abstractmethods() (GH-22485) 2020-10-06 10:40:50 -07:00
aifc.py
antigravity.py
argparse.py bpo-40862: Raise TypeError when const is given to argparse.BooleanOptionalAction (GH-20623) 2020-06-05 15:00:42 -07:00
ast.py bpo-41887: omit leading spaces/tabs on ast.literal_eval (#22469) 2020-10-03 17:46:44 -07:00
asynchat.py
asyncore.py
base64.py bpo-16995: add support for base32 extended hex (base32hex) (GH-20441) 2020-08-10 07:48:20 -07:00
bdb.py
binhex.py bpo-29566: binhex.binhex now consitently writes MacOS 9 line endings. (GH-23059) 2020-11-01 01:08:48 -08:00
bisect.py bpo-4356: Add key function support to the bisect module (GH-20556) 2020-10-19 22:04:01 -07:00
bz2.py
cProfile.py bpo-40492: Fix --outfile with relative path when the program changes it working dir (GH-19910) 2020-10-18 23:48:31 +03:00
calendar.py bpo-35078: Allow customization of CSS class name of a month in calendar module (gh-10137) 2020-06-02 13:33:09 +02:00
cgi.py bpo-34226: fix cgi.parse_multipart without content_length (GH-8530) 2020-06-15 07:58:54 -07:00
cgitb.py
chunk.py
cmd.py
code.py
codecs.py bpo-41521: Replace whitelist/blacklist with allowlist/denylist (GH-21822) 2020-08-11 06:26:59 -07:00
codeop.py bpo-41520: Fix second codeop regression (GH-21848) 2020-08-13 13:18:49 -04:00
colorsys.py
compileall.py
configparser.py
contextlib.py bpo-42395: Add aclosing to __all__ (GH-23356) 2020-11-17 15:18:05 -08:00
contextvars.py
copy.py bpo-40792: Make the result of PyNumber_Index() always having exact type int. (GH-20443) 2020-05-28 10:33:45 +03:00
copyreg.py bpo-41052: Fix pickling heap types implemented in C with protocols 0 and 1 (GH-22870) 2020-10-24 21:14:23 +03:00
crypt.py
csv.py
dataclasses.py bpo-41747: Ensure all dataclass methods uses their parents' qualname (GH-22155) 2020-10-21 09:49:22 -04:00
datetime.py bpo-41966: Fix pickling pure datetime.time subclasses (GH-22731) 2020-10-18 17:49:48 +03:00
decimal.py
difflib.py
dis.py bpo-42246: Partial implementation of PEP 626. (GH-23113) 2020-11-12 09:43:29 +00:00
doctest.py bpo-42133: update parts of the stdlib to fall back to `__spec__.loader` when `__loader__` is missing (#22929) 2020-11-06 18:45:56 -08:00
enum.py bpo-41816: `StrEnum.__str__` is `str.__str__` (GH-22362) 2020-09-22 13:00:07 -07:00
filecmp.py
fileinput.py
fnmatch.py
formatter.py
fractions.py
ftplib.py
functools.py bpo-31082: Use "iterable" in the docstring for functools.reduce() (GH-20796) 2020-06-28 15:40:54 +09:00
genericpath.py
getopt.py
getpass.py
gettext.py
glob.py bpo-38144: Re-add accidentally removed audition for glob. (GH-22805) 2020-10-20 19:45:38 +03:00
graphlib.py [doc] Fix typo in the graphlib docs (GH-22661) 2020-10-12 07:33:34 -07:00
gzip.py
hashlib.py Improve blake2 comment for Victor (GH-20981) 2020-06-19 07:11:02 -07:00
heapq.py Revert "Fix all Python Cookbook links (#22205)" (GH-22424) 2020-09-27 01:47:25 +01:00
hmac.py bpo-40791: Use CRYPTO_memcmp() for compare_digest (#20456) 2020-05-27 21:50:06 +02:00
imaplib.py bpo-26543: Fix IMAP4.noop when debug mode is enabled (GH-15206) 2020-06-02 03:17:45 +02:00
imghdr.py bpo-41043: Escape literal part of the path for glob(). (GH-20994) 2020-06-20 11:10:31 +03:00
imp.py
inspect.py bpo-42133: update parts of the stdlib to fall back to `__spec__.loader` when `__loader__` is missing (#22929) 2020-11-06 18:45:56 -08:00
io.py
ipaddress.py bpo-41521: Replace whitelist/blacklist with allowlist/denylist (GH-21822) 2020-08-11 06:26:59 -07:00
keyword.py Delete remaining references to Grammar/Grammar from docs (#21624) 2020-07-26 08:27:52 -07:00
linecache.py bpo-42133: update parts of the stdlib to fall back to `__spec__.loader` when `__loader__` is missing (#22929) 2020-11-06 18:45:56 -08:00
locale.py bpo-42208: Add _locale._get_locale_encoding() (GH-23052) 2020-10-31 01:32:11 +01:00
lzma.py
mailbox.py
mailcap.py
mimetypes.py bpo-41048: mimetypes should read the rule file using UTF-8, not the locale encoding (GH-20998) 2020-06-29 11:36:48 +03:00
modulefinder.py
netrc.py
nntplib.py
ntpath.py
nturl2path.py
numbers.py Revert "bpo-26680: Incorporate is_integer in all built-in and standard library numeric types (GH-6121)" (GH-22584) 2020-10-07 16:43:44 -07:00
opcode.py
operator.py
optparse.py
os.py
pathlib.py bpo-23706: Add newline parameter to pathlib.Path.write_text (GH-22420) (GH-22420) 2020-10-20 19:08:19 -07:00
pdb.py bpo-41609: Fix output of pdb's whatis command for instance methods (GH-21935) 2020-08-27 02:51:12 +02:00
pickle.py
pickletools.py
pipes.py
pkgutil.py bpo-41006: pkgutil imports lazily re (GH-20939) 2020-06-17 19:11:50 +02:00
platform.py bpo-42029: Remove IRIX code (GH-23023) 2020-10-29 15:16:23 +01:00
plistlib.py bpo-42249: Fix writing binary Plist files larger than 4 GiB. (GH-23121) 2020-11-03 16:15:56 +02:00
poplib.py
posixpath.py
pprint.py bpo-39994: Fix pprint handling of dict subclasses that override __repr__ (GH-21892) 2020-08-30 20:29:53 +03:00
profile.py bpo-40492: Fix --outfile with relative path when the program changes it working dir (GH-19910) 2020-10-18 23:48:31 +03:00
pstats.py bpo-41811: create SortKey members using first given value (GH-22316) 2020-09-19 11:12:57 -07:00
pty.py
py_compile.py bpo-38731: Add --quiet option to py_compile CLI (GH-17134) 2020-07-25 22:58:45 +03:00
pyclbr.py bpo-39411: pyclbr rewrite on AST (#18103) 2020-11-11 10:14:12 +03:00
pydoc.py bpo-40939: Remove PEG parser easter egg (__new_parser__) (#20802) 2020-06-11 09:09:21 -07:00
queue.py
quopri.py
random.py bpo-41773: Raise exception for non-finite weights in random.choices(). (GH-22441) 2020-09-28 18:32:10 -07:00
re.py
reprlib.py
rlcompleter.py bpo-39314: Closes parenthesis when autocompleting for functions that take no arguments (GH-20562) 2020-06-30 22:48:15 +09:00
runpy.py
sched.py bpo-19270: Fixed sched.scheduler.cancel to cancel correct event (GH-22729) 2020-10-19 10:33:43 +03:00
secrets.py
selectors.py bpo-41182 selector: use DefaultSelector based upon implementation (GH-21257) 2020-07-22 20:13:37 -07:00
shelve.py bpo-34204: Use pickle.DEFAULT_PROTOCOL in shelve (GH-19639) 2020-10-29 02:44:35 -07:00
shlex.py
shutil.py bpo-42014: shutil.rmtree: call onerror with correct function (GH-22585) 2020-11-10 08:06:02 -08:00
signal.py
site.py bpo-42133: update parts of the stdlib to fall back to `__spec__.loader` when `__loader__` is missing (#22929) 2020-11-06 18:45:56 -08:00
smtpd.py
smtplib.py Cosmetic smtplib changes (GH-8718) 2020-05-22 17:50:58 -07:00
sndhdr.py bpo-41043: Escape literal part of the path for glob(). (GH-20994) 2020-06-20 11:10:31 +03:00
socket.py
socketserver.py Revert "bpo-37193: remove thread objects which finished process its request (GH-13893)" (GH-23107) 2020-11-02 16:48:56 +00:00
sre_compile.py
sre_constants.py
sre_parse.py
ssl.py
stat.py
statistics.py Some reformatting (suggested by Black) and minor factoring. (GH-20865) 2020-06-13 19:17:28 -07:00
string.py
stringprep.py
struct.py
subprocess.py bpo-41586: Add pipesize parameter to subprocess & F_GETPIPE_SZ and F_SETPIPE_SZ to fcntl. (GH-21921) 2020-10-19 16:30:02 -07:00
sunau.py
symtable.py bpo-41840: Report module-level globals as both local and global in the symtable module (GH-22391) 2020-10-03 20:45:55 +01:00
sysconfig.py bpo-41627: Distinguish 32 and 64-bit user site packages on Windows (GH-22098) 2020-09-05 00:45:54 +01:00
tabnanny.py
tarfile.py bpo-39693: mention KeyError in tarfile extractfile documentation (GH-18639) 2020-10-20 10:05:01 -04:00
telnetlib.py
tempfile.py tempfile: Use random.choises() instead of choise() (GH-23068) 2020-11-01 20:02:03 +09:00
textwrap.py bpo-28660: Make TextWrapper break long words on hyphens (GH-22721) 2020-10-18 20:01:15 +03:00
this.py
threading.py bpo-42350: Fix Thread._reset_internal_locks() (GH-23268) 2020-11-16 15:20:34 +01:00
timeit.py bpo-40670: More reliable validation of statements in timeit.Timer. (GH-22358) 2020-09-22 16:16:46 +03:00
token.py
tokenize.py
trace.py bpo-41138: Fix trace CLI for non-UTF-8 files. (GH-21177) 2020-06-28 13:34:22 +03:00
traceback.py bpo-26389: Allow passing an exception object in the traceback module (GH-22610) 2020-11-05 22:18:44 +00:00
tracemalloc.py
tty.py
turtle.py bpo-41720: Add "return NotImplemented" in turtle.Vec2D.__rmul__(). (GH-22092) 2020-09-07 18:55:22 +03:00
types.py bpo-41810: Reintroduce `types.EllipsisType`, `.NoneType` & `.NotImplementedType` (GH-22336) 2020-09-22 08:55:34 -07:00
typing.py bpo-42345: Fix three issues with typing.Literal parameters (GH-23294) 2020-11-16 18:23:19 -08:00
uu.py
uuid.py bpo-41364: Reduce import overhead of uuid module (GH-21586) 2020-07-22 00:15:47 +01:00
warnings.py
wave.py
weakref.py
webbrowser.py bpo-41754: Ignore NotADirectoryError in invocation of xdg-settings (GH-23075) 2020-11-08 10:46:55 +01:00
xdrlib.py
zipapp.py
zipfile.py bpo-42043: Add support for zipfile.Path subclasses (#22716) 2020-10-25 14:45:05 -04:00
zipimport.py bpo-42131: Add PEP 451-related methods to zipimport (GH-23187) 2020-11-13 15:14:58 -08:00