cpython/Lib
Victor Stinner 8fba9523cf
bpo-42398: Fix "make regen-all" race condition (GH-23362)
Fix a race condition in "make regen-all" when make -jN option is used
to run jobs in parallel. The clinic.py script now only use atomic
write to write files. Moveover, generated files are now left
unchanged if the content does not change, to not change the file
modification time.

The "make regen-all" command runs "make clinic" and "make
regen-importlib" targets:

* "make regen-importlib" builds object files (ex: Modules/_weakref.o)
  from source files (ex: Modules/_weakref.c) and clinic files (ex:
  Modules/clinic/_weakref.c.h)
* "make clinic" always rewrites all clinic files
  (ex: Modules/clinic/_weakref.c.h)

Since there is no dependency between "clinic" and "regen-importlib"
Makefile targets, these two targets can be run in parallel. Moreover,
half of clinic.py file writes are not atomic and so there is a race
condition when "make regen-all" runs jobs in parallel using make -jN
option (which can be passed in MAKEFLAGS environment variable).

Fix clinic.py to make all file writes atomic:

* Add write_file() function to ensure that all file writes are
  atomic: write into a temporary file and then use os.replace().
* Moreover, write_file() doesn't recreate or modify the file if the
  content does not change to avoid modifying the file modification
  file.
* Update test_clinic to verify these assertions with a functional
  test.
* Remove Clinic.force attribute which was no longer used, whereas
  Clinic.verify remains useful.
2020-11-18 15:36:27 +01:00
..
asyncio bpo-42140: Improve asyncio.wait function (GH-22938) 2020-11-10 14:12:52 -08:00
collections Revert "Fix all Python Cookbook links (#22205)" (GH-22424) 2020-09-27 01:47:25 +01:00
concurrent bpo-39349: Add cancel_futures to Executor.shutdown base class (GH-22023) 2020-09-01 17:18:07 -04:00
ctypes bpo-41100: Support macOS 11 and Apple Silicon (GH-22855) 2020-11-08 10:05:27 +01:00
curses
dbm
distutils bpo-41100: Support macOS 11 and Apple Silicon (GH-22855) 2020-11-08 10:05:27 +01:00
email bpo-30681: Support invalid date format or value in email Date header (GH-22090) 2020-10-26 17:31:06 -07:00
encodings bpo-39337: encodings.normalize_encoding() now ignores non-ASCII characters (GH-22219) 2020-10-14 17:43:31 +02:00
ensurepip bpo-41490: Bump vendored pip to version 20.2.3 (#22527) 2020-10-04 18:45:31 +02:00
html
http bpo-40968: Send http/1.1 ALPN extension (#20959) 2020-11-13 16:37:52 +01:00
idlelib bpo-40511: Stop unwanted flashing of IDLE calltips (GH-20910) 2020-11-01 22:59:52 -05:00
importlib Bump magic number. (GH-23245) 2020-11-12 10:42:44 +00:00
json
lib2to3 bpo-40360: Deprecate lib2to3 module in light of PEP 617 (GH-19663) 2020-04-24 11:19:46 -07:00
logging bpo-26789: Fix logging.FileHandler._open() at exit (GH-23053) 2020-11-02 23:17:46 +01:00
msilib
multiprocessing bpo-41344: Raise ValueError when creating shared memory of size 0 (GH-21556) 2020-08-30 20:03:11 +01:00
pydoc_data Python 3.10.0a2 2020-11-03 00:02:08 +00:00
site-packages
sqlite3 bpo-42264: Deprecate sqlite3.OptimizedUnicode (GH-23163) 2020-11-17 17:55:12 +02:00
test bpo-42398: Fix "make regen-all" race condition (GH-23362) 2020-11-18 15:36:27 +01:00
tkinter bpo-41876: Overload __repr__ for tkinter Font objects (GH-22450) 2020-10-14 13:02:51 +03:00
turtledemo bpo-40431: Fix syntax typo in turtledemo (GH-19777) 2020-04-29 02:00:07 +02:00
unittest bpo-41877 Check for asert, aseert, assrt in mocks (GH-23165) 2020-11-05 09:04:38 -08:00
urllib bpo-40968: Send http/1.1 ALPN extension (#20959) 2020-11-13 16:37:52 +01:00
venv bpo-35328: Set VIRTUAL_ENV_PROMPT at venv activation (GH-21587) 2020-07-28 09:41:57 +01:00
wsgiref
xml bpo-40624: Add support for the XPath != operator in xml.etree (GH-22147) 2020-11-09 08:02:39 +01:00
xmlrpc
zoneinfo bpo-41065: Use zip-strict in zoneinfo (GH-21031) 2020-06-23 10:21:26 -04:00
__future__.py bpo-41314: fixed annotations __future__ version (GH-21616) 2020-07-25 14:42:49 -07:00
__phello__.foo.py
_aix_support.py
_bootsubprocess.py bpo-40094: Add os.waitstatus_to_exitcode() (GH-19201) 2020-04-01 18:49:29 +02:00
_collections_abc.py Fix: Docstrings hidden by slots. (GH-23352) 2020-11-17 13:50:23 -08:00
_compat_pickle.py
_compression.py
_markupbase.py bpo-31844: Remove _markupbase.ParserBase.error() (GH-8562) 2020-07-16 09:13:05 +03:00
_osx_support.py bpo-41100: Support macOS 11 and Apple Silicon (GH-22855) 2020-11-08 10:05:27 +01:00
_py_abc.py
_pydecimal.py Revert "bpo-26680: Incorporate is_integer in all built-in and standard library numeric types (GH-6121)" (GH-22584) 2020-10-07 16:43:44 -07:00
_pyio.py
_sitebuiltins.py
_strptime.py
_threading_local.py
_weakrefset.py bpo-39481: Make weakref and WeakSet generic (GH-19497) 2020-04-13 21:54:40 -07:00
abc.py bpo-41905: Add abc.update_abstractmethods() (GH-22485) 2020-10-06 10:40:50 -07:00
aifc.py
antigravity.py bpo-9216: Nobody expects the geohashing FIPS inquisition (GH-19520) 2020-04-14 12:49:11 -07:00
argparse.py bpo-40862: Raise TypeError when const is given to argparse.BooleanOptionalAction (GH-20623) 2020-06-05 15:00:42 -07:00
ast.py bpo-41887: omit leading spaces/tabs on ast.literal_eval (#22469) 2020-10-03 17:46:44 -07:00
asynchat.py
asyncore.py
base64.py bpo-16995: add support for base32 extended hex (base32hex) (GH-20441) 2020-08-10 07:48:20 -07:00
bdb.py
binhex.py bpo-29566: binhex.binhex now consitently writes MacOS 9 line endings. (GH-23059) 2020-11-01 01:08:48 -08:00
bisect.py bpo-4356: Add key function support to the bisect module (GH-20556) 2020-10-19 22:04:01 -07:00
bz2.py bpo-40443: Remove unused imports in the stdlib (GH-19803) 2020-04-30 11:26:33 +02:00
cProfile.py bpo-40492: Fix --outfile with relative path when the program changes it working dir (GH-19910) 2020-10-18 23:48:31 +03:00
calendar.py bpo-35078: Allow customization of CSS class name of a month in calendar module (gh-10137) 2020-06-02 13:33:09 +02:00
cgi.py bpo-34226: fix cgi.parse_multipart without content_length (GH-8530) 2020-06-15 07:58:54 -07:00
cgitb.py
chunk.py
cmd.py
code.py
codecs.py bpo-41521: Replace whitelist/blacklist with allowlist/denylist (GH-21822) 2020-08-11 06:26:59 -07:00
codeop.py bpo-41520: Fix second codeop regression (GH-21848) 2020-08-13 13:18:49 -04:00
colorsys.py
compileall.py bpo-40495: compileall option to hardlink duplicate pyc files (GH-19901) 2020-05-14 16:17:22 +02:00
configparser.py
contextlib.py bpo-42395: Add aclosing to __all__ (GH-23356) 2020-11-17 15:18:05 -08:00
contextvars.py
copy.py bpo-40792: Make the result of PyNumber_Index() always having exact type int. (GH-20443) 2020-05-28 10:33:45 +03:00
copyreg.py bpo-41052: Fix pickling heap types implemented in C with protocols 0 and 1 (GH-22870) 2020-10-24 21:14:23 +03:00
crypt.py
csv.py
dataclasses.py bpo-41747: Ensure all dataclass methods uses their parents' qualname (GH-22155) 2020-10-21 09:49:22 -04:00
datetime.py bpo-41966: Fix pickling pure datetime.time subclasses (GH-22731) 2020-10-18 17:49:48 +03:00
decimal.py
difflib.py bpo-40394 - difflib.SequenceMatched.find_longest_match default args (GH-19742) 2020-04-29 22:42:45 -05:00
dis.py bpo-42246: Partial implementation of PEP 626. (GH-23113) 2020-11-12 09:43:29 +00:00
doctest.py bpo-42133: update parts of the stdlib to fall back to `__spec__.loader` when `__loader__` is missing (#22929) 2020-11-06 18:45:56 -08:00
enum.py bpo-41816: `StrEnum.__str__` is `str.__str__` (GH-22362) 2020-09-22 13:00:07 -07:00
filecmp.py bpo-39481: PEP 585 for difflib, filecmp, fileinput (#19422) 2020-04-09 21:47:31 -07:00
fileinput.py bpo-39481: PEP 585 for difflib, filecmp, fileinput (#19422) 2020-04-09 21:47:31 -07:00
fnmatch.py bpo-40480: restore ability to join fnmatch.translate() results (GH-20049) 2020-05-11 21:19:20 -05:00
formatter.py
fractions.py
ftplib.py bpo-39380: Change ftplib encoding from latin-1 to utf-8 (GH-18048) 2020-04-14 01:07:56 +02:00
functools.py bpo-31082: Use "iterable" in the docstring for functools.reduce() (GH-20796) 2020-06-28 15:40:54 +09:00
genericpath.py
getopt.py
getpass.py
gettext.py bpo-40275: Import locale module lazily in gettext (GH-19905) 2020-05-14 03:22:30 +02:00
glob.py bpo-38144: Re-add accidentally removed audition for glob. (GH-22805) 2020-10-20 19:45:38 +03:00
graphlib.py [doc] Fix typo in the graphlib docs (GH-22661) 2020-10-12 07:33:34 -07:00
gzip.py
hashlib.py Improve blake2 comment for Victor (GH-20981) 2020-06-19 07:11:02 -07:00
heapq.py Revert "Fix all Python Cookbook links (#22205)" (GH-22424) 2020-09-27 01:47:25 +01:00
hmac.py bpo-40791: Use CRYPTO_memcmp() for compare_digest (#20456) 2020-05-27 21:50:06 +02:00
imaplib.py bpo-26543: Fix IMAP4.noop when debug mode is enabled (GH-15206) 2020-06-02 03:17:45 +02:00
imghdr.py bpo-41043: Escape literal part of the path for glob(). (GH-20994) 2020-06-20 11:10:31 +03:00
imp.py
inspect.py bpo-42133: update parts of the stdlib to fall back to `__spec__.loader` when `__loader__` is missing (#22929) 2020-11-06 18:45:56 -08:00
io.py
ipaddress.py bpo-41521: Replace whitelist/blacklist with allowlist/denylist (GH-21822) 2020-08-11 06:26:59 -07:00
keyword.py Delete remaining references to Grammar/Grammar from docs (#21624) 2020-07-26 08:27:52 -07:00
linecache.py bpo-42133: update parts of the stdlib to fall back to `__spec__.loader` when `__loader__` is missing (#22929) 2020-11-06 18:45:56 -08:00
locale.py bpo-42208: Add _locale._get_locale_encoding() (GH-23052) 2020-10-31 01:32:11 +01:00
lzma.py
mailbox.py bpo-39481: PEP 585 for dataclasses, mailbox, contextvars (GH-19425) 2020-04-14 16:14:15 -07:00
mailcap.py bpo-40094: mailcap.test() uses waitstatus_to_exitcode() (GH-19287) 2020-04-02 02:00:06 +02:00
mimetypes.py bpo-41048: mimetypes should read the rule file using UTF-8, not the locale encoding (GH-20998) 2020-06-29 11:36:48 +03:00
modulefinder.py bpo-40443: Remove unused imports in the stdlib (GH-19803) 2020-04-30 11:26:33 +02:00
netrc.py
nntplib.py bpo-39305: Update nntplib to merge nntplib.NNTP and nntplib._NNTPBase (GH-19817) 2020-05-16 19:31:54 +09:00
ntpath.py
nturl2path.py
numbers.py Revert "bpo-26680: Incorporate is_integer in all built-in and standard library numeric types (GH-6121)" (GH-22584) 2020-10-07 16:43:44 -07:00
opcode.py
operator.py
optparse.py
os.py bpo-39481: Implementation for PEP 585 (#18239) 2020-04-07 09:50:06 -07:00
pathlib.py bpo-23706: Add newline parameter to pathlib.Path.write_text (GH-22420) (GH-22420) 2020-10-20 19:08:19 -07:00
pdb.py bpo-41609: Fix output of pdb's whatis command for instance methods (GH-21935) 2020-08-27 02:51:12 +02:00
pickle.py bpo-39435: Make the first argument of pickle.loads() positional-only. (GH-19846) 2020-05-02 09:38:01 +03:00
pickletools.py
pipes.py
pkgutil.py bpo-41006: pkgutil imports lazily re (GH-20939) 2020-06-17 19:11:50 +02:00
platform.py bpo-42029: Remove IRIX code (GH-23023) 2020-10-29 15:16:23 +01:00
plistlib.py bpo-42249: Fix writing binary Plist files larger than 4 GiB. (GH-23121) 2020-11-03 16:15:56 +02:00
poplib.py
posixpath.py
pprint.py bpo-39994: Fix pprint handling of dict subclasses that override __repr__ (GH-21892) 2020-08-30 20:29:53 +03:00
profile.py bpo-40492: Fix --outfile with relative path when the program changes it working dir (GH-19910) 2020-10-18 23:48:31 +03:00
pstats.py bpo-41811: create SortKey members using first given value (GH-22316) 2020-09-19 11:12:57 -07:00
pty.py
py_compile.py bpo-38731: Add --quiet option to py_compile CLI (GH-17134) 2020-07-25 22:58:45 +03:00
pyclbr.py bpo-39411: pyclbr rewrite on AST (#18103) 2020-11-11 10:14:12 +03:00
pydoc.py bpo-40939: Remove PEG parser easter egg (__new_parser__) (#20802) 2020-06-11 09:09:21 -07:00
queue.py bpo-39481: PEP 585 for a variety of modules (GH-19423) 2020-04-10 07:46:36 -07:00
quopri.py
random.py bpo-41773: Raise exception for non-finite weights in random.choices(). (GH-22441) 2020-09-28 18:32:10 -07:00
re.py bpo-40016: re docstring: Clarify relationship of inline and argument flags (#19078) 2020-03-25 14:44:47 -04:00
reprlib.py
rlcompleter.py bpo-39314: Closes parenthesis when autocompleting for functions that take no arguments (GH-20562) 2020-06-30 22:48:15 +09:00
runpy.py bpo-40108: Improve the error message in runpy when importing a module that includes the extension (GH-19239) 2020-03-31 12:23:55 +01:00
sched.py bpo-19270: Fixed sched.scheduler.cancel to cancel correct event (GH-22729) 2020-10-19 10:33:43 +03:00
secrets.py bpo-40286: Add randbytes() method to random.Random (GH-19527) 2020-04-17 19:05:35 +02:00
selectors.py bpo-41182 selector: use DefaultSelector based upon implementation (GH-21257) 2020-07-22 20:13:37 -07:00
shelve.py bpo-34204: Use pickle.DEFAULT_PROTOCOL in shelve (GH-19639) 2020-10-29 02:44:35 -07:00
shlex.py bpo-33262: Deprecate passing None for `s` to shlex.split() (GH-6514) 2020-04-01 09:58:55 -04:00
shutil.py bpo-42014: shutil.rmtree: call onerror with correct function (GH-22585) 2020-11-10 08:06:02 -08:00
signal.py
site.py bpo-42133: update parts of the stdlib to fall back to `__spec__.loader` when `__loader__` is missing (#22929) 2020-11-06 18:45:56 -08:00
smtpd.py
smtplib.py Cosmetic smtplib changes (GH-8718) 2020-05-22 17:50:58 -07:00
sndhdr.py bpo-41043: Escape literal part of the path for glob(). (GH-20994) 2020-06-20 11:10:31 +03:00
socket.py
socketserver.py Revert "bpo-37193: remove thread objects which finished process its request (GH-13893)" (GH-23107) 2020-11-02 16:48:56 +00:00
sre_compile.py
sre_constants.py
sre_parse.py
ssl.py bpo-40443: Remove unused imports in stdlib (GH-19815) 2020-05-01 02:38:00 +02:00
stat.py
statistics.py Some reformatting (suggested by Black) and minor factoring. (GH-20865) 2020-06-13 19:17:28 -07:00
string.py
stringprep.py
struct.py
subprocess.py bpo-41586: Add pipesize parameter to subprocess & F_GETPIPE_SZ and F_SETPIPE_SZ to fcntl. (GH-21921) 2020-10-19 16:30:02 -07:00
sunau.py
symtable.py bpo-41840: Report module-level globals as both local and global in the symtable module (GH-22391) 2020-10-03 20:45:55 +01:00
sysconfig.py bpo-41627: Distinguish 32 and 64-bit user site packages on Windows (GH-22098) 2020-09-05 00:45:54 +01:00
tabnanny.py
tarfile.py bpo-39693: mention KeyError in tarfile extractfile documentation (GH-18639) 2020-10-20 10:05:01 -04:00
telnetlib.py
tempfile.py tempfile: Use random.choises() instead of choise() (GH-23068) 2020-11-01 20:02:03 +09:00
textwrap.py bpo-28660: Make TextWrapper break long words on hyphens (GH-22721) 2020-10-18 20:01:15 +03:00
this.py
threading.py bpo-42350: Fix Thread._reset_internal_locks() (GH-23268) 2020-11-16 15:20:34 +01:00
timeit.py bpo-40670: More reliable validation of statements in timeit.Timer. (GH-22358) 2020-09-22 16:16:46 +03:00
token.py
tokenize.py
trace.py bpo-41138: Fix trace CLI for non-UTF-8 files. (GH-21177) 2020-06-28 13:34:22 +03:00
traceback.py bpo-26389: Allow passing an exception object in the traceback module (GH-22610) 2020-11-05 22:18:44 +00:00
tracemalloc.py
tty.py
turtle.py bpo-41720: Add "return NotImplemented" in turtle.Vec2D.__rmul__(). (GH-22092) 2020-09-07 18:55:22 +03:00
types.py bpo-41810: Reintroduce `types.EllipsisType`, `.NoneType` & `.NotImplementedType` (GH-22336) 2020-09-22 08:55:34 -07:00
typing.py bpo-42345: Fix three issues with typing.Literal parameters (GH-23294) 2020-11-16 18:23:19 -08:00
uu.py
uuid.py bpo-41364: Reduce import overhead of uuid module (GH-21586) 2020-07-22 00:15:47 +01:00
warnings.py
wave.py
weakref.py bpo-36144: Add union operators to WeakValueDictionary584 (#19127) 2020-03-24 18:51:29 -07:00
webbrowser.py bpo-41754: Ignore NotADirectoryError in invocation of xdg-settings (GH-23075) 2020-11-08 10:46:55 +01:00
xdrlib.py
zipapp.py
zipfile.py bpo-42043: Add support for zipfile.Path subclasses (#22716) 2020-10-25 14:45:05 -04:00
zipimport.py bpo-42131: Add PEP 451-related methods to zipimport (GH-23187) 2020-11-13 15:14:58 -08:00