Commit Graph

46308 Commits

Author SHA1 Message Date
Victor Stinner 5909a494cd
bpo-42350: Fix Thread._reset_internal_locks() (GH-23268)
Fix the threading.Thread class at fork: do nothing if the thread is
already stopped (ex: fork called at Python exit). Previously, an
error was logged in the child process.
2020-11-16 15:20:34 +01:00
kj 384b7a4bd9
bpo-42332: Add weakref slot to types.GenericAlias (GH-23250)
Automerge-Triggered-By: GH:gvanrossum
2020-11-15 19:27:23 -08:00
Serhiy Storchaka a26215db11
bpo-42318: Fix support of non-BMP characters in Tkinter on macOS (GH-23281) 2020-11-15 18:16:59 +02:00
Brett Cannon d2e94bb084
bpo-42131: Add PEP 451-related methods to zipimport (GH-23187)
Specifically, find_spec(), create_module(), and exec_module().

Co-authored-by: Nick Coghlan <ncoghlan@gmail.com>
2020-11-13 15:14:58 -08:00
Christian Heimes cd9fed6afb
bpo-41001: Add os.eventfd() (#20930)
Co-authored-by: Kyle Stanley <aeros167@gmail.com>
2020-11-13 19:48:52 +01:00
Christian Heimes f97406be4c
bpo-40968: Send http/1.1 ALPN extension (#20959)
Signed-off-by: Christian Heimes <christian@python.org>
2020-11-13 16:37:52 +01:00
Mark Shannon cc75ab791d
bpo-42246: Eliminate jumps to exit blocks by copying those blocks. (#23251)
* Compiler: eliminate jumps to short exit blocks by copying.
2020-11-12 19:49:33 +00:00
Mario Corchero 750c5abf43
bpo-42308: Add threading.__excepthook__ (GH-23218)
Add threading.__excepthook__ to allow retrieving the original value
of threading.excepthook in case it is set to a broken or a different
value.
2020-11-12 18:27:44 +01:00
Mark Shannon c6409156c4
Bump magic number. (GH-23245) 2020-11-12 10:42:44 +00:00
Mark Shannon 877df851c3
bpo-42246: Partial implementation of PEP 626. (GH-23113)
* Implement new line number table format, as defined in PEP 626.
2020-11-12 09:43:29 +00:00
Batuhan Taskaya fa476fe132
bpo-39411: pyclbr rewrite on AST (#18103)
- Rewrite pyclbr using an AST processor
- Add is_async to the pyclbr.Function
2020-11-11 10:14:12 +03:00
Diogo Dutra 7e5ef0a571
bpo-42140: Improve asyncio.wait function (GH-22938)
# Improve asyncio.wait function

The original code creates the futures set two times.
We can create this set before, avoiding the second creation.

This new behaviour [breaks the aiokafka library](https://github.com/aio-libs/aiokafka/pull/672), because it gives an iterator to that function, so the second iteration become empty.

Automerge-Triggered-By: GH:1st1
2020-11-10 14:12:52 -08:00
Vladimir Matveev 1e996c3a3b
bpo-42085: Introduce dedicated entry in PyAsyncMethods for sending values (#22780) 2020-11-10 12:09:55 -08:00
Michal Čihař e59b2deffd
bpo-42014: shutil.rmtree: call onerror with correct function (GH-22585)
The onerror is supposed to be called with failed function, but in this case lstat is wrongly used instead of open.

Not sure if this needs bug or not...

Automerge-Triggered-By: GH:hynek
2020-11-10 08:06:02 -08:00
Andrew Svetlov 42d873c63a
bpo-42183: Fix a stack overflow error for asyncio Task or Future repr() (GH-23020)
The overflow occurs under some circumstances when a task or future
recursively returns itself.

Co-authored-by: Kyle Stanley <aeros167@gmail.com>
2020-11-10 15:58:31 +02:00
Ikko Ashimine 9303b9ecc3
Fix typo in test_array.py (GH-23189) 2020-11-10 18:12:36 +05:30
Victor Stinner 9e1b828265
bpo-42260: Compute the path config in the main init (GH-23211)
The path configuration is now computed in the "main" initialization.
The core initialization no longer computes it.

* Add _PyConfig_Read() function to read the configuration without
  computing the path configuration.
* pyinit_core() no longer computes the path configuration: it is now
  computed by init_interp_main().
* The path configuration output members of PyConfig are now optional:

  * executable
  * base_executable
  * prefix
  * base_prefix
  * exec_prefix
  * base_exec_prefix

* _PySys_UpdateConfig() now skips NULL strings in PyConfig.
* _testembed: Rename test_set_config() to test_init_set_config() for
  consistency with other tests.
2020-11-10 13:21:52 +01:00
jack1142 bfc6b63102
bpo-36310: Allow pygettext.py to detect calls to gettext in f-strings. (GH-19875)
Adds support to Tools/i18n/pygettext.py for gettext calls in f-strings. This process is done by parsing the f-strings, processing each value, and flagging the ones which contain a gettext call.

Co-authored-by: Batuhan Taskaya <batuhanosmantaskaya@gmail.com>
2020-11-10 01:50:45 +03:00
Tom Gringauz a117167d8d
bpo-41543: contextlib.nullcontext can fill in for an async context manager (GH-21870)
Co-authored-by: Andrew Svetlov <andrew.svetlov@gmail.com>
2020-11-09 14:34:07 +02:00
Ammar Askar 97e8b1eaea
bpo-40624: Add support for the XPath != operator in xml.etree (GH-22147) 2020-11-09 08:02:39 +01:00
kj 4eb41d055e
bpo-42233: Add union type expression support for GenericAlias and fix de-duplicating of GenericAlias (GH-23077) 2020-11-08 20:00:13 -08:00
Ronald Oussoren 23831a7a90
bpo-41754: Ignore NotADirectoryError in invocation of xdg-settings (GH-23075)
It is not clear why this can happen, but several users have mentioned
getting this exception on macOS.
2020-11-08 10:46:55 +01:00
Ronald Oussoren 41761933c1
bpo-41100: Support macOS 11 and Apple Silicon (GH-22855)
Co-authored-by:  Lawrence D’Anna <lawrence_danna@apple.com>

* Add support for macOS 11 and Apple Silicon (aka arm64)
   
  As a side effect of this work use the system copy of libffi on macOS, and remove the vendored copy

* Support building on recent versions of macOS while deploying to older versions

  This allows building installers on macOS 11 while still supporting macOS 10.9.
2020-11-08 10:05:27 +01:00
Brett Cannon 825ac38332
bpo-42133: update parts of the stdlib to fall back to `__spec__.loader` when `__loader__` is missing (#22929) 2020-11-06 18:45:56 -08:00
Hai Shi 88c2cfd9ff
bpo-41832: PyType_FromModuleAndSpec() now accepts NULL tp_doc (GH-23123) 2020-11-06 17:04:47 +01:00
Zackery Spytz 91e93794d5
bpo-26389: Allow passing an exception object in the traceback module (GH-22610)
The format_exception(), format_exception_only(), and
print_exception() functions can now take an exception object as a positional-only argument.

Co-Authored-By: Matthias Bussonnier <bussonniermatthias@gmail.com>
2020-11-05 22:18:44 +00:00
Victor Stinner dc42af8fd1
bpo-42260: PyConfig_Read() only parses argv once (GH-23168)
The PyConfig_Read() function now only parses PyConfig.argv arguments
once: PyConfig.parse_argv is set to 2 after arguments are parsed.
Since Python arguments are strippped from PyConfig.argv, parsing
arguments twice would parse the application options as Python
options.

* Rework the PyConfig documentation.
* Fix _testinternalcapi.set_config() error handling.
* SetConfigTests no longer needs parse_argv=0 when restoring the old
  configuration.
2020-11-05 18:58:07 +01:00
Victor Stinner f3cb814315
bpo-42260: Add _PyConfig_FromDict() (GH-23167)
* Rename config_as_dict() to _PyConfig_AsDict().
* Add 'module_search_paths_set' to _PyConfig_AsDict().
* Add _PyConfig_FromDict().
* Add get_config() and set_config() to _testinternalcapi.
* Add config_check_consistency().
2020-11-05 18:12:33 +01:00
vabr-g 4662fa9bfe
bpo-41877 Check for asert, aseert, assrt in mocks (GH-23165)
Currently, a Mock object which is not unsafe will raise an
AttributeError if an attribute with the prefix assert or assret is
accessed on it. This protects against misspellings of real assert
method calls, which lead to tests passing silently even if the tested
code does not satisfy the intended assertion.

Recently a check was done in a large code base (Google) and three
more frequent ways of misspelling assert were found causing harm:
asert, aseert, assrt. These are now added to the existing check.
2020-11-05 09:04:38 -08:00
Pablo Galindo 80449f243b
bpo-42266: Handle monkey-patching descriptors in LOAD_ATTR cache (GH-23157) 2020-11-05 09:23:15 +00:00
Kazantcev Andrey 178695b7ae
bpo-40816 Add AsyncContextDecorator class (GH-20516)
Co-authored-by: Yury Selivanov <yury@edgedb.com>
2020-11-05 10:52:24 +02:00
Victor Stinner 048a35659a
bpo-42260: Add _PyInterpreterState_SetConfig() (GH-23158)
* Inline _PyInterpreterState_SetConfig(): replace it with
  _PyConfig_Copy().
* Add _PyErr_SetFromPyStatus()
* Add _PyInterpreterState_GetConfigCopy()
* Add a new _PyInterpreterState_SetConfig() function.
* Add an unit which gets, modifies, and sets the config.
2020-11-05 00:45:56 +01:00
serge-sans-paille 100964e031
Disable peg generator tests when building with PGO (GH-23141)
Otherwise, when running the testsuite, test_peg_generator tries to compile C
code using the optimized flags and fails because it cannot find the profile
data.
2020-11-04 23:01:08 +00:00
Victor Stinner 3529718925
bpo-42236: os.device_encoding() respects UTF-8 Mode (GH-23119)
On Unix, the os.device_encoding() function now returns 'UTF-8' rather
than the device encoding if the Python UTF-8 Mode is enabled.
2020-11-04 11:20:10 +01:00
Mario Corchero 0001a1b69e
bpo-42251: Add gettrace and getprofile to threading (GH-23125)
This allows to retrieve the functions that were set in these two, which might differ from sys.gettrace and sys.getprofile within a thread.
2020-11-04 09:27:43 +00:00
Pablo Galindo a9abbda618 Python 3.10.0a2
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEEz9yiRbEEPPKl+Xhl/+h0BBaL2EcFAl+gngEACgkQ/+h0BBaL
 2EfDABAAnQ/bHY40QjoQ+us2vufDj3uvm2S7w0LMM+kgrehhqEoG/FmVtfb8XIvD
 S8FFtSa/VkrVUXvxI/mMpWL78se+Qk9rVQJ8dUZ4SvFJVe1NJdCMNV2Ij263QgJU
 r6bNu42EODDrFAgG9vq5ImND1qnGkF+hKlMHYV3HZhTP4drBEY+TMaB55Vmsp9TH
 eBnfY50y+k8w0hJmsZA1LaVLAXU9WHaWVB90/D22Z9MJmmQF5ofsIFXYj+1IcEkE
 P5CLSCRSzVlpdeNiZWrcBIEK8ap4ZkHvwD8RI6Fu3E/h2TXeiDbp/nCHZDPZMVVk
 OVx3Z4BFTNXE4nmRDIiO5+GqIbgG10LFQKOrOvHQr6inookUaFGwJKwkaEheNr2T
 vcgZMy61OEOeiLqCe6AJ/PuNlQ267zslYucUpNqpQpCu8j691OGEHzv+NNcSRrEq
 T/S4spifDVpFPGLRDkFjJe1OQ33bVZFdJx7guY6WJXVkZILS8vghqQYng55CNzd/
 cdMqb5T6BBRb2SDqJ/qvUhyr7D8YML0pP5pywUUGKTdo84+mW5F/N2Q4g28ik9zv
 pgaehbAwS8jgoPbqH0Tz7qpXiWga9fTbFU47Az07xw5tQFD2+C5fdx+b1h2NBz2q
 exccvIw3Fn8zWikrKbeLgaSAWa2saw25dw0DMes8gG9uHD4+1JA=
 =XAGl
 -----END PGP SIGNATURE-----
gpgsig -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEEz9yiRbEEPPKl+Xhl/+h0BBaL2EcFAl+hk9UACgkQ/+h0BBaL
 2EdkYxAAs5ujyihMfyoRpU9jV7pNIIbpCHMgV2s74ZBh5jnDPYRfIrGtaE/9HlTp
 PPAScsJvnPwBGNdg8Bs1cLAk/VgJC696ccJXmNugfpxs9CSzhEKQ0oF8PPm+oyqH
 LgU6Dv8dP3+ebOko/vnRPTXbL5gdbvVJvNiV5KUjE6FL+IRl2PHlvUdcEplkPXZ4
 179kgP+8hMCB1Ai1KbMDpO2XIgdkJO+iXgXg9QNDCv2zcyr6OegGYI4cPyB3m/YK
 ZiBTnYK48x/l28Gdi+LISOGa4NNQ7R0j+Qao7YDoqe1yy8Ubj3NrLub+nNnSqnqb
 8PYaFz0sWpJ4I/vjrMaUOJ+hKSwdRZqHF/LqfQBFyBCndmhi6tJNqAzcByqi03HL
 YnXlAYJI9T3rFzBOutfjvbFWkiM2DKAL6dWBIsMeuNEO6MQWTwQHKfidXJyx9zCo
 Vv5QLG5daLFDIgI4qBjJ4POxgl+7eR+5sJBNIVz43KhemiRKLdmFKWmv6moloVB+
 1xoNGmwqz4dzmlAMUj2vJiPuUYl6+P38q1dpAChOXx4hOQZ1ybsmUVUD4S+bLGVQ
 f/VYg6B2I/Fr7Yz9HVYe309abKUSzSeYnW9FkAuTHTmZPjgZHdIJl7euTE0FtNOd
 HQ0paNeC44AoqEnghH35FhFXs+wbedszyc++TltTWn32v0nSVk0=
 =qIu/
 -----END PGP SIGNATURE-----

Merge tag 'v3.10.0a2' into master

Python 3.10.0a2
2020-11-03 17:30:57 +00:00
Serhiy Storchaka 212d32f45c
bpo-42249: Fix writing binary Plist files larger than 4 GiB. (GH-23121) 2020-11-03 16:15:56 +02:00
Pablo Galindo 114ee5dec0
Python 3.10.0a2 2020-11-03 00:02:08 +00:00
Victor Stinner 45df61fd2d
bpo-26789: Fix logging.FileHandler._open() at exit (GH-23053)
The logging.FileHandler class now keeps a reference to the builtin
open() function to be able to open or reopen the file during Python
finalization.

Fix errors like:

    Exception ignored in: (...)
    Traceback (most recent call last):
      (...)
      File ".../logging/__init__.py", line 1463, in error
      File ".../logging/__init__.py", line 1577, in _log
      File ".../logging/__init__.py", line 1587, in handle
      File ".../logging/__init__.py", line 1649, in callHandlers
      File ".../logging/__init__.py", line 948, in handle
      File ".../logging/__init__.py", line 1182, in emit
      File ".../logging/__init__.py", line 1171, in _open
    NameError: name 'open' is not defined
2020-11-02 23:17:46 +01:00
Serhiy Storchaka 34637a0ce2
bpo-42103: Improve validation of Plist files. (GH-22882)
* Prevent some possible DoS attacks via providing invalid Plist files
  with extremely large number of objects or collection sizes.
* Raise InvalidFileException for too large bytes and string size instead of returning garbage.
* Raise InvalidFileException instead of ValueError for specific invalid datetime (NaN).
* Raise InvalidFileException instead of TypeError for non-hashable dict keys.
* Add more tests for invalid Plist files.
2020-11-02 23:01:40 +02:00
Irit Katriel 6fdfcec5b1
bpo-41943: Fix bug where assertLogs doesn't correctly filter messages… (GH-22565)
… by level

@vsajip , @pitrou

Automerge-Triggered-By: GH:vsajip
2020-11-02 11:25:29 -08:00
Jason R. Coombs aca67da4fe
Revert "bpo-37193: remove thread objects which finished process its request (GH-13893)" (GH-23107)
This reverts commit c415590212.
2020-11-02 16:48:56 +00:00
Lysandros Nikolaou 301822859b
bpo-42224: Fix test_format when locale does not expect number grouping (GH-23067) 2020-11-02 17:27:30 +02:00
Julien Danjou 64366fa9b3
bpo-41435: Add sys._current_exceptions() function (GH-21689)
This adds a new function named sys._current_exceptions() which is equivalent ot
sys._current_frames() except that it returns the exceptions currently handled
by other threads. It is equivalent to calling sys.exc_info() for each running
thread.
2020-11-02 16:16:25 +02:00
Jakub Stasiak 3d86d090dc
bpo-42230: Improve asyncio documentation regarding accepting sets vs iterables (GH-23073)
People call wait() and as_completed() with various non-set iterables,
a list should be the most common but there are others as well[1].

Considering typeshed also documents wait()[2] and as_completed()[3]
as accepting arbitrary iterables I think it's a good idea to document
the status quo better.

[1] https://github.com/aio-libs/aiokafka/pull/672
[2] 620989bac5/stdlib/3/asyncio/tasks.pyi (L161)
[3] 620989bac5/stdlib/3/asyncio/tasks.pyi (L40)
2020-11-02 12:56:35 +02:00
Joongi Kim 6e8dcdaaa4
bpo-41229: Update docs for explicit aclose()-required cases and add contextlib.aclosing() method (GH-21545)
This is a PR to:

 * Add `contextlib.aclosing` which ia analogous to `contextlib.closing` but for async-generators with an explicit test case for [bpo-41229]()
 * Update the docs to describe when we need explicit `aclose()` invocation.

which are motivated by the following issues, articles, and examples:

 * [bpo-41229]()
 * https://github.com/njsmith/async_generator
 * https://vorpus.org/blog/some-thoughts-on-asynchronous-api-design-in-a-post-asyncawait-world/#cleanup-in-generators-and-async-generators
 * https://www.python.org/dev/peps/pep-0533/
 * https://github.com/achimnol/aiotools/blob/ef7bf0cea7af/src/aiotools/context.py#L152

Particuarly regarding [PEP-533](https://www.python.org/dev/peps/pep-0533/), its acceptance (`__aiterclose__()`) would make this little addition of `contextlib.aclosing()` unnecessary for most use cases, but until then this could serve as a good counterpart and analogy to `contextlib.closing()`. The same applies for `contextlib.closing` with `__iterclose__()`.
Also, still there are other use cases, e.g., when working with non-generator objects with `aclose()` methods.
2020-11-02 00:02:48 -08:00
Tal Einat da7bb7b4d7
bpo-40511: Stop unwanted flashing of IDLE calltips (GH-20910)
They were occurring with both repeated 'force-calltip' invocations and by typing parentheses
 in expressions, strings, and comments in the argument code.

Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
2020-11-01 22:59:52 -05:00
MARUYAMA Norihiro c415590212
bpo-37193: remove thread objects which finished process its request (GH-13893)
* bpo-37193: remove the thread which finished process request from threads list

* rename variable t to thread.

* don't remove thread from list if it is daemon.

* use lock to protect self._threads.

* use finally block in case of exception from shutdown_request().

* check "not thread.daemon" before lock to avoid holding the lock if it's unnecessary.

* fix the place of _threads_lock.

* separate code to remove a current thread into a function.

* check ValueError when removing thread.

* fix wrong code which all instance shared same lock.

* Extract thread management into a _Threads class to encapsulate atomic operations and separate concerns.

* Replace multiple references of 'block_on_close' with one, avoiding the possibility that 'block_on_close' could change during the course of processing requests. Now, there's exactly one _threads object with behavior fixed for the duration.

* Add docstrings to private classes.

* Add test to ensure that a ThreadingTCPServer can be closed without serving any requests.

* Use _NoThreads as the default value. Fixes AttributeError when server is closed without serving any requests.

* Add blurb

* Add test capturing failure.

Co-authored-by: Jason R. Coombs <jaraco@jaraco.com>
2020-11-01 18:51:04 -05:00
kj 1f7dfb277e
bpo-42233: Correctly repr GenericAlias when used with typing module (GH-23081)
Noticed by @serhiy-storchaka in the bpo.  `typing`'s types were not showing the parameterized generic. 
Eg. previously:
```python
>>> typing.Union[dict[str, float], list[int]]
'typing.Union[dict, list]'
```
Now:
```python
>>> typing.Union[dict[str, float], list[int]]
'typing.Union[dict[str, float], list[int]]'
```

Automerge-Triggered-By: GH:gvanrossum
2020-11-01 10:13:38 -08:00
Inada Naoki d2810054c7
tempfile: Use random.choises() instead of choise() (GH-23068) 2020-11-01 20:02:03 +09:00