cpython/Lib
Victor Stinner 8fba9523cf
bpo-42398: Fix "make regen-all" race condition (GH-23362)
Fix a race condition in "make regen-all" when make -jN option is used
to run jobs in parallel. The clinic.py script now only use atomic
write to write files. Moveover, generated files are now left
unchanged if the content does not change, to not change the file
modification time.

The "make regen-all" command runs "make clinic" and "make
regen-importlib" targets:

* "make regen-importlib" builds object files (ex: Modules/_weakref.o)
  from source files (ex: Modules/_weakref.c) and clinic files (ex:
  Modules/clinic/_weakref.c.h)
* "make clinic" always rewrites all clinic files
  (ex: Modules/clinic/_weakref.c.h)

Since there is no dependency between "clinic" and "regen-importlib"
Makefile targets, these two targets can be run in parallel. Moreover,
half of clinic.py file writes are not atomic and so there is a race
condition when "make regen-all" runs jobs in parallel using make -jN
option (which can be passed in MAKEFLAGS environment variable).

Fix clinic.py to make all file writes atomic:

* Add write_file() function to ensure that all file writes are
  atomic: write into a temporary file and then use os.replace().
* Moreover, write_file() doesn't recreate or modify the file if the
  content does not change to avoid modifying the file modification
  file.
* Update test_clinic to verify these assertions with a functional
  test.
* Remove Clinic.force attribute which was no longer used, whereas
  Clinic.verify remains useful.
2020-11-18 15:36:27 +01:00
..
asyncio bpo-42140: Improve asyncio.wait function (GH-22938) 2020-11-10 14:12:52 -08:00
collections Revert "Fix all Python Cookbook links (#22205)" (GH-22424) 2020-09-27 01:47:25 +01:00
concurrent bpo-39349: Add cancel_futures to Executor.shutdown base class (GH-22023) 2020-09-01 17:18:07 -04:00
ctypes bpo-41100: Support macOS 11 and Apple Silicon (GH-22855) 2020-11-08 10:05:27 +01:00
curses
dbm
distutils bpo-41100: Support macOS 11 and Apple Silicon (GH-22855) 2020-11-08 10:05:27 +01:00
email bpo-30681: Support invalid date format or value in email Date header (GH-22090) 2020-10-26 17:31:06 -07:00
encodings bpo-39337: encodings.normalize_encoding() now ignores non-ASCII characters (GH-22219) 2020-10-14 17:43:31 +02:00
ensurepip bpo-41490: Bump vendored pip to version 20.2.3 (#22527) 2020-10-04 18:45:31 +02:00
html bpo-37328: remove deprecated HTMLParser.unescape (GH-14186) 2019-08-27 11:48:06 +09:00
http bpo-40968: Send http/1.1 ALPN extension (#20959) 2020-11-13 16:37:52 +01:00
idlelib bpo-40511: Stop unwanted flashing of IDLE calltips (GH-20910) 2020-11-01 22:59:52 -05:00
importlib Bump magic number. (GH-23245) 2020-11-12 10:42:44 +00:00
json bpo-39828: Fix json.tool to catch BrokenPipeError (GH-18779) 2020-03-10 08:41:44 +01:00
lib2to3 bpo-40360: Deprecate lib2to3 module in light of PEP 617 (GH-19663) 2020-04-24 11:19:46 -07:00
logging bpo-26789: Fix logging.FileHandler._open() at exit (GH-23053) 2020-11-02 23:17:46 +01:00
msilib Remove binding of captured exceptions when not used to reduce the chances of creating cycles (GH-17246) 2019-11-19 21:34:03 +00:00
multiprocessing bpo-41344: Raise ValueError when creating shared memory of size 0 (GH-21556) 2020-08-30 20:03:11 +01:00
pydoc_data Python 3.10.0a2 2020-11-03 00:02:08 +00:00
site-packages
sqlite3 bpo-42264: Deprecate sqlite3.OptimizedUnicode (GH-23163) 2020-11-17 17:55:12 +02:00
test bpo-42398: Fix "make regen-all" race condition (GH-23362) 2020-11-18 15:36:27 +01:00
tkinter bpo-41876: Overload __repr__ for tkinter Font objects (GH-22450) 2020-10-14 13:02:51 +03:00
turtledemo bpo-40431: Fix syntax typo in turtledemo (GH-19777) 2020-04-29 02:00:07 +02:00
unittest bpo-41877 Check for asert, aseert, assrt in mocks (GH-23165) 2020-11-05 09:04:38 -08:00
urllib bpo-40968: Send http/1.1 ALPN extension (#20959) 2020-11-13 16:37:52 +01:00
venv bpo-35328: Set VIRTUAL_ENV_PROMPT at venv activation (GH-21587) 2020-07-28 09:41:57 +01:00
wsgiref
xml bpo-40624: Add support for the XPath != operator in xml.etree (GH-22147) 2020-11-09 08:02:39 +01:00
xmlrpc bpo-38786: Add parsing of https links to pydoc (GH-17143) 2019-11-13 18:13:52 +02:00
zoneinfo bpo-41065: Use zip-strict in zoneinfo (GH-21031) 2020-06-23 10:21:26 -04:00
__future__.py bpo-41314: fixed annotations __future__ version (GH-21616) 2020-07-25 14:42:49 -07:00
__phello__.foo.py
_aix_support.py bpo-39936: _aix_support uses _bootsubprocess (GH-18970) 2020-03-12 23:15:34 +01:00
_bootsubprocess.py bpo-40094: Add os.waitstatus_to_exitcode() (GH-19201) 2020-04-01 18:49:29 +02:00
_collections_abc.py Fix: Docstrings hidden by slots. (GH-23352) 2020-11-17 13:50:23 -08:00
_compat_pickle.py bpo-37757: Disallow PEP 572 cases that expose implementation details (GH-15131) 2019-08-25 23:45:40 +10:00
_compression.py
_markupbase.py bpo-31844: Remove _markupbase.ParserBase.error() (GH-8562) 2020-07-16 09:13:05 +03:00
_osx_support.py bpo-41100: Support macOS 11 and Apple Silicon (GH-22855) 2020-11-08 10:05:27 +01:00
_py_abc.py
_pydecimal.py Revert "bpo-26680: Incorporate is_integer in all built-in and standard library numeric types (GH-6121)" (GH-22584) 2020-10-07 16:43:44 -07:00
_pyio.py bpo-39674: Revert "bpo-37330: open() no longer accept 'U' in file mode (GH-16959)" (GH-18767) 2020-03-04 18:50:22 +01:00
_sitebuiltins.py
_strptime.py Fix outdated comment in _strptime.py (GH-17929) 2020-01-12 12:53:00 -08:00
_threading_local.py
_weakrefset.py bpo-39481: Make weakref and WeakSet generic (GH-19497) 2020-04-13 21:54:40 -07:00
abc.py bpo-41905: Add abc.update_abstractmethods() (GH-22485) 2020-10-06 10:40:50 -07:00
aifc.py
antigravity.py bpo-9216: Nobody expects the geohashing FIPS inquisition (GH-19520) 2020-04-14 12:49:11 -07:00
argparse.py bpo-40862: Raise TypeError when const is given to argparse.BooleanOptionalAction (GH-20623) 2020-06-05 15:00:42 -07:00
ast.py bpo-41887: omit leading spaces/tabs on ast.literal_eval (#22469) 2020-10-03 17:46:44 -07:00
asynchat.py Remove binding of captured exceptions when not used to reduce the chances of creating cycles (GH-17246) 2019-11-19 21:34:03 +00:00
asyncore.py bpo-15999: Always pass bool instead of int to socket.setblocking(). (GH-15621) 2019-09-01 12:12:52 +03:00
base64.py bpo-16995: add support for base32 extended hex (base32hex) (GH-20441) 2020-08-10 07:48:20 -07:00
bdb.py bpo-39649: Remove obsolete check for `__args__` in bdb.Bdb.format_stack_entry (GH-18531) 2020-02-23 22:14:53 -05:00
binhex.py bpo-29566: binhex.binhex now consitently writes MacOS 9 line endings. (GH-23059) 2020-11-01 01:08:48 -08:00
bisect.py bpo-4356: Add key function support to the bisect module (GH-20556) 2020-10-19 22:04:01 -07:00
bz2.py bpo-40443: Remove unused imports in the stdlib (GH-19803) 2020-04-30 11:26:33 +02:00
cProfile.py bpo-40492: Fix --outfile with relative path when the program changes it working dir (GH-19910) 2020-10-18 23:48:31 +03:00
calendar.py bpo-35078: Allow customization of CSS class name of a month in calendar module (gh-10137) 2020-06-02 13:33:09 +02:00
cgi.py bpo-34226: fix cgi.parse_multipart without content_length (GH-8530) 2020-06-15 07:58:54 -07:00
cgitb.py
chunk.py
cmd.py
code.py Fix documentation in code.py (GH-17988) 2020-01-15 01:17:25 +05:30
codecs.py bpo-41521: Replace whitelist/blacklist with allowlist/denylist (GH-21822) 2020-08-11 06:26:59 -07:00
codeop.py bpo-41520: Fix second codeop regression (GH-21848) 2020-08-13 13:18:49 -04:00
colorsys.py
compileall.py bpo-40495: compileall option to hardlink duplicate pyc files (GH-19901) 2020-05-14 16:17:22 +02:00
configparser.py
contextlib.py bpo-42395: Add aclosing to __all__ (GH-23356) 2020-11-17 15:18:05 -08:00
contextvars.py
copy.py bpo-40792: Make the result of PyNumber_Index() always having exact type int. (GH-20443) 2020-05-28 10:33:45 +03:00
copyreg.py bpo-41052: Fix pickling heap types implemented in C with protocols 0 and 1 (GH-22870) 2020-10-24 21:14:23 +03:00
crypt.py closes bpo-38402: Check error of primitive crypt/crypt_r. (GH-16599) 2019-10-07 21:22:17 -07:00
csv.py
dataclasses.py bpo-41747: Ensure all dataclass methods uses their parents' qualname (GH-22155) 2020-10-21 09:49:22 -04:00
datetime.py bpo-41966: Fix pickling pure datetime.time subclasses (GH-22731) 2020-10-18 17:49:48 +03:00
decimal.py
difflib.py bpo-40394 - difflib.SequenceMatched.find_longest_match default args (GH-19742) 2020-04-29 22:42:45 -05:00
dis.py bpo-42246: Partial implementation of PEP 626. (GH-23113) 2020-11-12 09:43:29 +00:00
doctest.py bpo-42133: update parts of the stdlib to fall back to `__spec__.loader` when `__loader__` is missing (#22929) 2020-11-06 18:45:56 -08:00
enum.py bpo-41816: `StrEnum.__str__` is `str.__str__` (GH-22362) 2020-09-22 13:00:07 -07:00
filecmp.py bpo-39481: PEP 585 for difflib, filecmp, fileinput (#19422) 2020-04-09 21:47:31 -07:00
fileinput.py bpo-39481: PEP 585 for difflib, filecmp, fileinput (#19422) 2020-04-09 21:47:31 -07:00
fnmatch.py bpo-40480: restore ability to join fnmatch.translate() results (GH-20049) 2020-05-11 21:19:20 -05:00
formatter.py
fractions.py bpo-39350: Fix fractions for int subclasses (GH-18375) 2020-02-07 23:42:51 +01:00
ftplib.py bpo-39380: Change ftplib encoding from latin-1 to utf-8 (GH-18048) 2020-04-14 01:07:56 +02:00
functools.py bpo-31082: Use "iterable" in the docstring for functools.reduce() (GH-20796) 2020-06-28 15:40:54 +09:00
genericpath.py bpo-38807: Add os.PathLike to exception message raised by _check_arg_types (#17160) 2019-11-18 21:54:00 -08:00
getopt.py
getpass.py Remove binding of captured exceptions when not used to reduce the chances of creating cycles (GH-17246) 2019-11-19 21:34:03 +00:00
gettext.py bpo-40275: Import locale module lazily in gettext (GH-19905) 2020-05-14 03:22:30 +02:00
glob.py bpo-38144: Re-add accidentally removed audition for glob. (GH-22805) 2020-10-20 19:45:38 +03:00
graphlib.py [doc] Fix typo in the graphlib docs (GH-22661) 2020-10-12 07:33:34 -07:00
gzip.py bpo-39389: gzip: fix compression level metadata (GH-18077) 2020-01-21 13:25:24 +02:00
hashlib.py Improve blake2 comment for Victor (GH-20981) 2020-06-19 07:11:02 -07:00
heapq.py Revert "Fix all Python Cookbook links (#22205)" (GH-22424) 2020-09-27 01:47:25 +01:00
hmac.py bpo-40791: Use CRYPTO_memcmp() for compare_digest (#20456) 2020-05-27 21:50:06 +02:00
imaplib.py bpo-26543: Fix IMAP4.noop when debug mode is enabled (GH-15206) 2020-06-02 03:17:45 +02:00
imghdr.py bpo-41043: Escape literal part of the path for glob(). (GH-20994) 2020-06-20 11:10:31 +03:00
imp.py bpo-39674: Revert "bpo-37330: open() no longer accept 'U' in file mode (GH-16959)" (GH-18767) 2020-03-04 18:50:22 +01:00
inspect.py bpo-42133: update parts of the stdlib to fall back to `__spec__.loader` when `__loader__` is missing (#22929) 2020-11-06 18:45:56 -08:00
io.py
ipaddress.py bpo-41521: Replace whitelist/blacklist with allowlist/denylist (GH-21822) 2020-08-11 06:26:59 -07:00
keyword.py Delete remaining references to Grammar/Grammar from docs (#21624) 2020-07-26 08:27:52 -07:00
linecache.py bpo-42133: update parts of the stdlib to fall back to `__spec__.loader` when `__loader__` is missing (#22929) 2020-11-06 18:45:56 -08:00
locale.py bpo-42208: Add _locale._get_locale_encoding() (GH-23052) 2020-10-31 01:32:11 +01:00
lzma.py
mailbox.py bpo-39481: PEP 585 for dataclasses, mailbox, contextvars (GH-19425) 2020-04-14 16:14:15 -07:00
mailcap.py bpo-40094: mailcap.test() uses waitstatus_to_exitcode() (GH-19287) 2020-04-02 02:00:06 +02:00
mimetypes.py bpo-41048: mimetypes should read the rule file using UTF-8, not the locale encoding (GH-20998) 2020-06-29 11:36:48 +03:00
modulefinder.py bpo-40443: Remove unused imports in the stdlib (GH-19803) 2020-04-30 11:26:33 +02:00
netrc.py
nntplib.py bpo-39305: Update nntplib to merge nntplib.NNTP and nntplib._NNTPBase (GH-19817) 2020-05-16 19:31:54 +09:00
ntpath.py bpo-38453: Ensure ntpath.realpath correctly resolves relative paths (GH-16967) 2019-11-15 09:49:21 -08:00
nturl2path.py
numbers.py Revert "bpo-26680: Incorporate is_integer in all built-in and standard library numeric types (GH-6121)" (GH-22584) 2020-10-07 16:43:44 -07:00
opcode.py bpo-39320: Handle unpacking of **values in compiler (GH-18141) 2020-01-27 09:57:45 +00:00
operator.py
optparse.py
os.py bpo-39481: Implementation for PEP 585 (#18239) 2020-04-07 09:50:06 -07:00
pathlib.py bpo-23706: Add newline parameter to pathlib.Path.write_text (GH-22420) (GH-22420) 2020-10-20 19:08:19 -07:00
pdb.py bpo-41609: Fix output of pdb's whatis command for instance methods (GH-21935) 2020-08-27 02:51:12 +02:00
pickle.py bpo-39435: Make the first argument of pickle.loads() positional-only. (GH-19846) 2020-05-02 09:38:01 +03:00
pickletools.py
pipes.py
pkgutil.py bpo-41006: pkgutil imports lazily re (GH-20939) 2020-06-17 19:11:50 +02:00
platform.py bpo-42029: Remove IRIX code (GH-23023) 2020-10-29 15:16:23 +01:00
plistlib.py bpo-42249: Fix writing binary Plist files larger than 4 GiB. (GH-23121) 2020-11-03 16:15:56 +02:00
poplib.py bpo-39259: poplib now rejects timeout = 0 (GH-17912) 2020-01-10 15:34:05 +01:00
posixpath.py
pprint.py bpo-39994: Fix pprint handling of dict subclasses that override __repr__ (GH-21892) 2020-08-30 20:29:53 +03:00
profile.py bpo-40492: Fix --outfile with relative path when the program changes it working dir (GH-19910) 2020-10-18 23:48:31 +03:00
pstats.py bpo-41811: create SortKey members using first given value (GH-22316) 2020-09-19 11:12:57 -07:00
pty.py bpo-39184: Add audit events to command execution functions in os and pty modules (GH-17824) 2020-02-05 11:15:00 +11:00
py_compile.py bpo-38731: Add --quiet option to py_compile CLI (GH-17134) 2020-07-25 22:58:45 +03:00
pyclbr.py bpo-39411: pyclbr rewrite on AST (#18103) 2020-11-11 10:14:12 +03:00
pydoc.py bpo-40939: Remove PEG parser easter egg (__new_parser__) (#20802) 2020-06-11 09:09:21 -07:00
queue.py bpo-39481: PEP 585 for a variety of modules (GH-19423) 2020-04-10 07:46:36 -07:00
quopri.py bpo-15999: Clean up of handling boolean arguments. (GH-15610) 2019-09-01 12:16:51 +03:00
random.py bpo-41773: Raise exception for non-finite weights in random.choices(). (GH-22441) 2020-09-28 18:32:10 -07:00
re.py bpo-40016: re docstring: Clarify relationship of inline and argument flags (#19078) 2020-03-25 14:44:47 -04:00
reprlib.py
rlcompleter.py bpo-39314: Closes parenthesis when autocompleting for functions that take no arguments (GH-20562) 2020-06-30 22:48:15 +09:00
runpy.py bpo-40108: Improve the error message in runpy when importing a module that includes the extension (GH-19239) 2020-03-31 12:23:55 +01:00
sched.py bpo-19270: Fixed sched.scheduler.cancel to cancel correct event (GH-22729) 2020-10-19 10:33:43 +03:00
secrets.py bpo-40286: Add randbytes() method to random.Random (GH-19527) 2020-04-17 19:05:35 +02:00
selectors.py bpo-41182 selector: use DefaultSelector based upon implementation (GH-21257) 2020-07-22 20:13:37 -07:00
shelve.py bpo-34204: Use pickle.DEFAULT_PROTOCOL in shelve (GH-19639) 2020-10-29 02:44:35 -07:00
shlex.py bpo-33262: Deprecate passing None for `s` to shlex.split() (GH-6514) 2020-04-01 09:58:55 -04:00
shutil.py bpo-42014: shutil.rmtree: call onerror with correct function (GH-22585) 2020-11-10 08:06:02 -08:00
signal.py
site.py bpo-42133: update parts of the stdlib to fall back to `__spec__.loader` when `__loader__` is missing (#22929) 2020-11-06 18:45:56 -08:00
smtpd.py bpo-35800: Deprecate smtpd.MailmanProxy (GH-11675) 2019-10-12 10:24:26 -07:00
smtplib.py Cosmetic smtplib changes (GH-8718) 2020-05-22 17:50:58 -07:00
sndhdr.py bpo-41043: Escape literal part of the path for glob(). (GH-20994) 2020-06-20 11:10:31 +03:00
socket.py Fix host in address of socket.create_server example. (GH-17706) 2020-01-11 10:46:30 +05:30
socketserver.py Revert "bpo-37193: remove thread objects which finished process its request (GH-13893)" (GH-23107) 2020-11-02 16:48:56 +00:00
sre_compile.py
sre_constants.py
sre_parse.py
ssl.py bpo-40443: Remove unused imports in stdlib (GH-19815) 2020-05-01 02:38:00 +02:00
stat.py bpo-38109: Add missing constants to Lib/stat.py (GH-16665) 2019-10-10 09:34:46 +02:00
statistics.py Some reformatting (suggested by Black) and minor factoring. (GH-20865) 2020-06-13 19:17:28 -07:00
string.py bpo-38208: Simplify string.Template by using __init_subclass__(). (GH-16256) 2019-10-21 09:36:21 +03:00
stringprep.py
struct.py
subprocess.py bpo-41586: Add pipesize parameter to subprocess & F_GETPIPE_SZ and F_SETPIPE_SZ to fcntl. (GH-21921) 2020-10-19 16:30:02 -07:00
sunau.py
symtable.py bpo-41840: Report module-level globals as both local and global in the symtable module (GH-22391) 2020-10-03 20:45:55 +01:00
sysconfig.py bpo-41627: Distinguish 32 and 64-bit user site packages on Windows (GH-22098) 2020-09-05 00:45:54 +01:00
tabnanny.py
tarfile.py bpo-39693: mention KeyError in tarfile extractfile documentation (GH-18639) 2020-10-20 10:05:01 -04:00
telnetlib.py
tempfile.py tempfile: Use random.choises() instead of choise() (GH-23068) 2020-11-01 20:02:03 +09:00
textwrap.py bpo-28660: Make TextWrapper break long words on hyphens (GH-22721) 2020-10-18 20:01:15 +03:00
this.py
threading.py bpo-42350: Fix Thread._reset_internal_locks() (GH-23268) 2020-11-16 15:20:34 +01:00
timeit.py bpo-40670: More reliable validation of statements in timeit.Timer. (GH-22358) 2020-09-22 16:16:46 +03:00
token.py
tokenize.py
trace.py bpo-41138: Fix trace CLI for non-UTF-8 files. (GH-21177) 2020-06-28 13:34:22 +03:00
traceback.py bpo-26389: Allow passing an exception object in the traceback module (GH-22610) 2020-11-05 22:18:44 +00:00
tracemalloc.py bpo-37961, tracemalloc: add Traceback.total_nframe (GH-15545) 2019-10-15 14:00:16 +02:00
tty.py
turtle.py bpo-41720: Add "return NotImplemented" in turtle.Vec2D.__rmul__(). (GH-22092) 2020-09-07 18:55:22 +03:00
types.py bpo-41810: Reintroduce `types.EllipsisType`, `.NoneType` & `.NotImplementedType` (GH-22336) 2020-09-22 08:55:34 -07:00
typing.py bpo-42345: Fix three issues with typing.Literal parameters (GH-23294) 2020-11-16 18:23:19 -08:00
uu.py bpo-38945: UU Encoding: Don't let newline in filename corrupt the output format (#17418) 2019-12-02 14:25:21 -08:00
uuid.py bpo-41364: Reduce import overhead of uuid module (GH-21586) 2020-07-22 00:15:47 +01:00
warnings.py bpo-39056: Fix handling invalid warning category in the -W option. (GH-17618) 2020-01-05 14:15:27 +02:00
wave.py Fix a typo in wave module docstring (GH-17009) 2019-11-04 22:32:10 -06:00
weakref.py bpo-36144: Add union operators to WeakValueDictionary584 (#19127) 2020-03-24 18:51:29 -07:00
webbrowser.py bpo-41754: Ignore NotADirectoryError in invocation of xdg-settings (GH-23075) 2020-11-08 10:46:55 +01:00
xdrlib.py
zipapp.py
zipfile.py bpo-42043: Add support for zipfile.Path subclasses (#22716) 2020-10-25 14:45:05 -04:00
zipimport.py bpo-42131: Add PEP 451-related methods to zipimport (GH-23187) 2020-11-13 15:14:58 -08:00