Commit Graph

3812 Commits

Author SHA1 Message Date
Victor Stinner 0e2ac21dd4
bpo-39573: Convert Py_TYPE() and Py_SIZE() back to macros (GH-23366)
This change partically reverts
commit ad3252bad9
and the commit fe2978b3b9.

Many third party C extension modules rely on the ability of using
Py_TYPE() to set an object type: "Py_TYPE(obj) = type;" or to set an
object type using: "Py_SIZE(obj) = size;".
2020-11-18 18:48:06 +01:00
Christian Heimes 07f2adedf0
bpo-40998: Address compiler warnings found by ubsan (GH-20929)
Signed-off-by: Christian Heimes <christian@python.org>

Automerge-Triggered-By: GH:tiran
2020-11-18 07:38:53 -08:00
Christian Heimes 46f59ebd01
bpo-1635741: Port _hashlib to multiphase initialization (GH-23358)
Signed-off-by: Christian Heimes <christian@python.org>
2020-11-18 16:12:13 +01:00
Christian Heimes c7011012fa
bpo-1635741: Port symtable module to multiphase initialization (GH-23361)
Signed-off-by: Christian Heimes <christian@python.org>
2020-11-18 07:03:51 -08:00
Victor Stinner 8fba9523cf
bpo-42398: Fix "make regen-all" race condition (GH-23362)
Fix a race condition in "make regen-all" when make -jN option is used
to run jobs in parallel. The clinic.py script now only use atomic
write to write files. Moveover, generated files are now left
unchanged if the content does not change, to not change the file
modification time.

The "make regen-all" command runs "make clinic" and "make
regen-importlib" targets:

* "make regen-importlib" builds object files (ex: Modules/_weakref.o)
  from source files (ex: Modules/_weakref.c) and clinic files (ex:
  Modules/clinic/_weakref.c.h)
* "make clinic" always rewrites all clinic files
  (ex: Modules/clinic/_weakref.c.h)

Since there is no dependency between "clinic" and "regen-importlib"
Makefile targets, these two targets can be run in parallel. Moreover,
half of clinic.py file writes are not atomic and so there is a race
condition when "make regen-all" runs jobs in parallel using make -jN
option (which can be passed in MAKEFLAGS environment variable).

Fix clinic.py to make all file writes atomic:

* Add write_file() function to ensure that all file writes are
  atomic: write into a temporary file and then use os.replace().
* Moreover, write_file() doesn't recreate or modify the file if the
  content does not change to avoid modifying the file modification
  file.
* Update test_clinic to verify these assertions with a functional
  test.
* Remove Clinic.force attribute which was no longer used, whereas
  Clinic.verify remains useful.
2020-11-18 15:36:27 +01:00
Christian Heimes ce04e7105b
bpo-41561: skip test_min_max_version_mismatch (GH-22308)
skip test_min_max_version_mismatch when TLS 1.0 is not available

Signed-off-by: Christian Heimes <christian@python.org>
2020-11-18 13:10:53 +01:00
Victor Stinner 7f9b25a21a
bpo-41713: Port _signal module to multi-phase init (GH-23355)
Port the _signal extension module to the multi-phase initialization
API (PEP 489).

Co-Authored-By: Mohamed Koubaa <koubaa.m@gmail.com>
2020-11-17 23:28:25 +01:00
Mark Shannon 266b462238
bpo-42349: Compiler clean up. More yak-shaving for PEP 626. (GH-23267)
Make sure that CFG from compiler front-end is correct. Be a bit more aggressive in the compiler back-end.
2020-11-17 19:30:14 +00:00
Victor Stinner 0ae323b87f
bpo-41686: Always create the SIGINT event on Windows (GH-23344)
bpo-41686, bpo-41713: On Windows, the SIGINT event,
_PyOS_SigintEvent(), is now created even if Python is configured to
not install signal handlers (PyConfig.install_signal_handlers=0 or
Py_InitializeEx(0)).

Changes:

* Move global variables initialization from signal_exec() to
  _PySignal_Init() to clarify that they are global variables cleared
  by _PySignal_Fini().
* _PySignal_Fini() now closes sigint_event.
* IntHandler is no longer a global variable.
2020-11-17 18:15:20 +01:00
Erlend Egeberg Aasland a1f401a58b
bpo-42264: Deprecate sqlite3.OptimizedUnicode (GH-23163) 2020-11-17 17:55:12 +02:00
Victor Stinner 296a796951
bpo-41713: Remove PyOS_InitInterrupts() function (GH-23342)
Remove the undocumented PyOS_InitInterrupts() C function.

* Rename PyOS_InitInterrupts() to _PySignal_Init(). It now installs
  other signal handlers, not only SIGINT.
* Rename PyOS_FiniInterrupts() to _PySignal_Fini()
2020-11-17 16:22:23 +01:00
Erlend Egeberg Aasland 2ffba2a102
bpo-41861, _sqlite3 : Add NEWS entry and rename variables (GH-23337) 2020-11-17 13:52:54 +01:00
Yurii Karabas f03d318ca4
bpo-42345: Fix three issues with typing.Literal parameters (GH-23294)
Literal equality no longer depends on the order of arguments.

Fix issue related to `typing.Literal` caching by adding `typed` parameter to `typing._tp_cache` function.

Add deduplication of `typing.Literal` arguments.
2020-11-16 18:23:19 -08:00
Pablo Galindo b0aba1fcdc
bpo-42381: Allow walrus in set literals and set comprehensions (GH-23332)
Currently walruses are not allowerd in set literals and set comprehensions:

>>> {y := 4, 4**2, 3**3}
  File "<stdin>", line 1
    {y := 4, 4**2, 3**3}
       ^
SyntaxError: invalid syntax

but they should be allowed as well per PEP 572
2020-11-17 01:17:12 +00:00
Pablo Galindo a57b3d30f6
bpo-41625: Expose the splice() system call in the os module (GH-21947) 2020-11-17 00:00:38 +00:00
Steve Dower 9cc9e27725
bpo-42120: Remove macro defining copysign to _copysign on Windows (GH-23326) 2020-11-16 23:22:42 +00:00
Lysandros Nikolaou cae60187cf
bpo-42316: Allow unparenthesized walrus operator in indexes (GH-23317) 2020-11-17 01:09:35 +02:00
Lysandros Nikolaou cb3e5ed071
bpo-42374: Allow unparenthesized walrus in genexps (GH-23319)
This fixes a regression that was introduced by the new parser.

Automerge-Triggered-By: GH:lysnikolaou
2020-11-16 15:08:35 -08:00
Zackery Spytz f62dad16b8
bpo-38506: Fix the Windows py.exe launcher's misordering of 3.10 (GH-18307) 2020-11-16 21:32:35 +00:00
Kevin Adler c79667ff79
bpo-42087: Remove support for AIX 5.3 and below (GH-22830)
As AIX 5.3 and below do not support thread_cputime, it was decided in
https://bugs.python.org/issue40680 to require AIX 6.1 and above. This
commit removes workarounds for — and references to — older, unsupported
AIX versions.
2020-11-16 16:16:10 +01:00
Victor Stinner ae6cd7cfda
bpo-37205: time.time() cannot fail with fatal error (GH-23314)
time.time(), time.perf_counter() and time.monotonic() functions can
no longer fail with a Python fatal error, instead raise a regular
Python exception on failure.

Remove _PyTime_Init(): don't check system, monotonic and perf counter
clocks at startup anymore.

On error, _PyTime_GetSystemClock(), _PyTime_GetMonotonicClock() and
_PyTime_GetPerfCounter() now silently ignore the error and return 0.
They cannot fail with a Python fatal error anymore.

Add py_mach_timebase_info() and win_perf_counter_frequency()
sub-functions.
2020-11-16 16:08:05 +01:00
Victor Stinner 5909a494cd
bpo-42350: Fix Thread._reset_internal_locks() (GH-23268)
Fix the threading.Thread class at fork: do nothing if the thread is
already stopped (ex: fork called at Python exit). Previously, an
error was logged in the child process.
2020-11-16 15:20:34 +01:00
Victor Stinner 3df5c68487
bpo-37205: time.perf_counter() and time.monotonic() are system-wide (GH-23284)
time.perf_counter() on Windows and time.monotonic() on macOS are now
system-wide. Previously, they used an offset computed at startup to
reduce the precision loss caused by the float type. Use
time.perf_counter_ns() and time.monotonic_ns() added in Python 3.7 to
avoid this precision loss.
2020-11-16 13:21:45 +01:00
Yash Shete aa01011003
bpo-42153 Fix link to IMAP documents in imaplib.rst (GH-23297)
The University of Washington stopped hosting the IMAP documents. Link to a rescued copy on GitHub.
2020-11-15 21:02:35 -08:00
kj 384b7a4bd9
bpo-42332: Add weakref slot to types.GenericAlias (GH-23250)
Automerge-Triggered-By: GH:gvanrossum
2020-11-15 19:27:23 -08:00
Serhiy Storchaka a26215db11
bpo-42318: Fix support of non-BMP characters in Tkinter on macOS (GH-23281) 2020-11-15 18:16:59 +02:00
Brett Cannon d2e94bb084
bpo-42131: Add PEP 451-related methods to zipimport (GH-23187)
Specifically, find_spec(), create_module(), and exec_module().

Co-authored-by: Nick Coghlan <ncoghlan@gmail.com>
2020-11-13 15:14:58 -08:00
Steve Dower 9b6934230c
bpo-40754: Adds _testinternalcapi to Windows installer for test suite (GH-23271) 2020-11-13 22:24:36 +00:00
Christian Heimes cd9fed6afb
bpo-41001: Add os.eventfd() (#20930)
Co-authored-by: Kyle Stanley <aeros167@gmail.com>
2020-11-13 19:48:52 +01:00
Christian Heimes f97406be4c
bpo-40968: Send http/1.1 ALPN extension (#20959)
Signed-off-by: Christian Heimes <christian@python.org>
2020-11-13 16:37:52 +01:00
Victor Stinner b3b98082c5
bpo-41617: Add _Py__has_builtin() macro (GH-23260)
Fix building pycore_bitutils.h internal header on old clang version
without __builtin_bswap16() (ex: Xcode 4.6.3 on Mac OS X 10.7).

Add a new private _Py__has_builtin() macro to check for availability
of a preprocessor builtin function.

Co-Authored-By: Joshua Root <jmr@macports.org>

Co-authored-by: Joshua Root <jmr@macports.org>
2020-11-13 15:38:17 +01:00
Victor Stinner d96a7a8313
bpo-42296: On Windows, fix CTRL+C regression (GH-23257)
On Windows, fix a regression in signal handling which prevented to
interrupt a program using CTRL+C. The signal handler can be run in a
thread different than the Python thread, in which case the test
deciding if the thread can handle signals is wrong.

On Windows, _PyEval_SignalReceived() now always sets eval_breaker to
1 since it cannot test _Py_ThreadCanHandleSignals(), and
  eval_frame_handle_pending() always calls
  _Py_ThreadCanHandleSignals() to recompute eval_breaker.
2020-11-13 14:44:42 +01:00
Mario Corchero 750c5abf43
bpo-42308: Add threading.__excepthook__ (GH-23218)
Add threading.__excepthook__ to allow retrieving the original value
of threading.excepthook in case it is set to a broken or a different
value.
2020-11-12 18:27:44 +01:00
Victor Stinner b5cc05bbe6
bpo-38823: Always build _ctypes with wchar_t (GH-23248)
It is no longer possible to build the _ctypes extension module
without wchar_t type: remove CTYPES_UNICODE macro. Anyway, the
wchar_t type is required to build Python.
2020-11-12 15:38:13 +01:00
Jakub Stasiak fd4ed57674
bpo-42237: Fix os.sendfile() on illumos (GH-23154) 2020-11-12 11:49:30 +02:00
Mark Shannon 877df851c3
bpo-42246: Partial implementation of PEP 626. (GH-23113)
* Implement new line number table format, as defined in PEP 626.
2020-11-12 09:43:29 +00:00
Batuhan Taskaya fa476fe132
bpo-39411: pyclbr rewrite on AST (#18103)
- Rewrite pyclbr using an AST processor
- Add is_async to the pyclbr.Function
2020-11-11 10:14:12 +03:00
Victor Stinner 95ce7cd0a6
bpo-1635741: Fix typo in PyModule_AddObjectRef() doc (GH-23234)
It is similar to PyModule_AddObject(), not to itself.
2020-11-11 01:52:26 +01:00
Diogo Dutra 7e5ef0a571
bpo-42140: Improve asyncio.wait function (GH-22938)
# Improve asyncio.wait function

The original code creates the futures set two times.
We can create this set before, avoiding the second creation.

This new behaviour [breaks the aiokafka library](https://github.com/aio-libs/aiokafka/pull/672), because it gives an iterator to that function, so the second iteration become empty.

Automerge-Triggered-By: GH:1st1
2020-11-10 14:12:52 -08:00
Hai Shi a13b26cac1
bpo-41073: PyType_GetSlot() can now accept static types. (GH-21931)
PyType_GetSlot() can now accept static types.

Co-Authored-By: Petr Viktorin <encukou@gmail.com>

Automerge-Triggered-By: GH:encukou
2020-11-10 12:53:46 -08:00
Victor Stinner ace3f9a0ce
bpo-42260: Fix _PyConfig_Read() if compute_path_config=0 (GH-23220)
Fix _PyConfig_Read() if compute_path_config=0: use values set by
Py_SetPath(), Py_SetPythonHome() and Py_SetProgramName(). Add
compute_path_config parameter to _PyConfig_InitPathConfig().

The following functions now return NULL if called before
Py_Initialize():

* Py_GetExecPrefix()
* Py_GetPath()
* Py_GetPrefix()
* Py_GetProgramFullPath()
* Py_GetProgramName()
* Py_GetPythonHome()

These functions no longer automatically computes the Python Path
Configuration. Moreover, Py_SetPath() no longer computes
program_full_path.
2020-11-10 21:10:22 +01:00
Vladimir Matveev 1e996c3a3b
bpo-42085: Introduce dedicated entry in PyAsyncMethods for sending values (#22780) 2020-11-10 12:09:55 -08:00
Michal Čihař e59b2deffd
bpo-42014: shutil.rmtree: call onerror with correct function (GH-22585)
The onerror is supposed to be called with failed function, but in this case lstat is wrongly used instead of open.

Not sure if this needs bug or not...

Automerge-Triggered-By: GH:hynek
2020-11-10 08:06:02 -08:00
Andrew Svetlov 42d873c63a
bpo-42183: Fix a stack overflow error for asyncio Task or Future repr() (GH-23020)
The overflow occurs under some circumstances when a task or future
recursively returns itself.

Co-authored-by: Kyle Stanley <aeros167@gmail.com>
2020-11-10 15:58:31 +02:00
Petr Viktorin 0b9c4c6fcf
bpo-42171: Add PEP573-related items to the limited API (GH-23009) 2020-11-10 05:47:31 -08:00
jack1142 bfc6b63102
bpo-36310: Allow pygettext.py to detect calls to gettext in f-strings. (GH-19875)
Adds support to Tools/i18n/pygettext.py for gettext calls in f-strings. This process is done by parsing the f-strings, processing each value, and flagging the ones which contain a gettext call.

Co-authored-by: Batuhan Taskaya <batuhanosmantaskaya@gmail.com>
2020-11-10 01:50:45 +03:00
Tom Gringauz a117167d8d
bpo-41543: contextlib.nullcontext can fill in for an async context manager (GH-21870)
Co-authored-by: Andrew Svetlov <andrew.svetlov@gmail.com>
2020-11-09 14:34:07 +02:00
Ammar Askar 97e8b1eaea
bpo-40624: Add support for the XPath != operator in xml.etree (GH-22147) 2020-11-09 08:02:39 +01:00
kj 4eb41d055e
bpo-42233: Add union type expression support for GenericAlias and fix de-duplicating of GenericAlias (GH-23077) 2020-11-08 20:00:13 -08:00
Ronald Oussoren 23831a7a90
bpo-41754: Ignore NotADirectoryError in invocation of xdg-settings (GH-23075)
It is not clear why this can happen, but several users have mentioned
getting this exception on macOS.
2020-11-08 10:46:55 +01:00