Commit Graph

23051 Commits

Author SHA1 Message Date
Sebastian Berg 427c84f13f
bpo-39274: Ensure Fraction.__bool__() returns a bool (GH-18017)
Some numerator types used (specifically NumPy) decides to not
return a Python boolean for the "a != b" operation. Using the equivalent
call to bool() guarantees a bool return also for such types.
2020-02-06 15:54:05 +01:00
Petr Viktorin 3f563cea56
bpo-39245: Make Vectorcall C API public (GH-17893)
* Add backcompat defines and move non-limited API declaration to cpython/

This partially reverts commit 2ff58a24e8
which added PyObject_CallNoArgs to the 3.9+ stable ABI. This should not
be done; there are enough other call APIs in the stable ABI to choose from.

* Adjust documentation

Mark all newly public functions as added in 3.9.
Add a note about the 3.8 provisional names.
Add notes on public API.

* Put PyObject_CallNoArgs back in the limited API

* Rename PyObject_FastCallDict to PyObject_VectorcallDict
2020-02-06 15:48:27 +01:00
Serhiy Storchaka 54b4f14712
bpo-38149: Call sys.audit() only once per call for glob.glob(). (GH-18360) 2020-02-06 10:26:37 +02:00
Shantanu 8b6f6526f8
bpo-39559: Remove unused, undocumented argument from uuid.getnode (GH-18369) 2020-02-05 22:43:09 +02:00
Victor Stinner 509dd90f46
bpo-39542: Convert PyType_Check() to static inline function (GH-18364)
Convert PyType_HasFeature(), PyType_Check() and PyType_CheckExact()
macros to static inline functions.
2020-02-05 14:24:17 +01:00
Victor Stinner f58bd7c169
bpo-39542: Make PyObject_INIT() opaque in limited C API (GH-18363)
In the limited C API, PyObject_INIT() and PyObject_INIT_VAR() are now
defined as aliases to PyObject_Init() and PyObject_InitVar() to make
their implementation opaque. It avoids to leak implementation details
in the limited C API.

Exclude the following functions from the limited C API, move them
from object.h to cpython/object.h:

* _Py_NewReference()
* _Py_ForgetReference()
* _PyTraceMalloc_NewReference()
* _Py_GetRefTotal()
2020-02-05 13:12:19 +01:00
Victor Stinner 0fa4f43db0
bpo-39542: Exclude trashcan from the limited C API (GH-18362)
Exclude trashcan mechanism from the limited C API: it requires access to
PyTypeObject and PyThreadState structure fields, whereas these structures
are opaque in the limited C API.

The trashcan mechanism never worked with the limited C API. Move it
from object.h to cpython/object.h.
2020-02-05 12:23:27 +01:00
Zackery Spytz b439a715cb
bpo-39553: Delete HAVE_SXS protected code (GH-18356)
https://bugs.python.org/issue39553



Automerge-Triggered-By: @zooba
2020-02-04 19:13:00 -08:00
Jakub Stasiak cf5b109dbb
bpo-39491: Merge PEP 593 (typing.Annotated) support (#18260)
* bpo-39491: Merge PEP 593 (typing.Annotated) support

PEP 593 has been accepted some time ago. I got a green light for merging
this from Till, so I went ahead and combined the code contributed to
typing_extensions[1] and the documentation from the PEP 593 text[2].

My changes were limited to:

* removing code designed for typing_extensions to run on older Python
  versions
* removing some irrelevant parts of the PEP text when copying it over as
  documentation and otherwise changing few small bits to better serve
  the purpose
* changing the get_type_hints signature to match reality (parameter
  names)

I wasn't entirely sure how to go about crediting the authors but I used
my best judgment, let me know if something needs changing in this
regard.

[1] 8280de241f/typing_extensions/src_py3/typing_extensions.py
[2] 17710b8798/pep-0593.rst
2020-02-04 17:10:19 -08:00
Anthony Shaw 89ae20b30e
bpo-39185 Add the d[etailed] and q[uiet] verbosity levels for msbuild (GH-17791) 2020-02-05 11:30:19 +11:00
Saiyang Gou 95f6001021
bpo-39184: Add audit events to command execution functions in os and pty modules (GH-17824) 2020-02-05 11:15:00 +11:00
Philipp Gesang cb1c0746f2
closes bpo-39510: Fix use-after-free in BufferedReader.readinto() (GH-18295)
When called on a closed object, readinto() segfaults on account
of a write to a freed buffer:

    ==220553== Process terminating with default action of signal 11 (SIGSEGV): dumping core
    ==220553==  Access not within mapped region at address 0x2A
    ==220553==    at 0x48408A0: memmove (vg_replace_strmem.c:1272)
    ==220553==    by 0x58DB0C: _buffered_readinto_generic (bufferedio.c:972)
    ==220553==    by 0x58DCBA: _io__Buffered_readinto_impl (bufferedio.c:1053)
    ==220553==    by 0x58DCBA: _io__Buffered_readinto (bufferedio.c.h:253)

Reproducer:

    reader = open ("/dev/zero", "rb")
    _void  = reader.read (42)
    reader.close ()
    reader.readinto (bytearray (42)) ### BANG!

The problem exists since 2012 when commit dc469454ec added code
to free the read buffer on close().

Signed-off-by: Philipp Gesang <philipp.gesang@intra2net.com>
2020-02-04 13:25:16 -08:00
Stefan Behnel 9538bc9185
bpo-39432: Implement PEP-489 algorithm for non-ascii "PyInit_*" symbol names in distutils (GH-18150)
Make it export the correct init symbol also on Windows.



https://bugs.python.org/issue39432
2020-02-04 07:24:30 -08:00
Eddie Elizondo 4590f72259
bpo-38076 Clear the interpreter state only after clearing module globals (GH-18039)
Currently, during runtime destruction, `_PyImport_Cleanup` is clearing the interpreter state before clearing out the modules themselves. This leads to a segfault on modules that rely on the module state to clear themselves up.

For example, let's take the small snippet added in the issue by @DinoV :
```
import _struct

class C:
    def __init__(self):
        self.pack = _struct.pack
    def __del__(self):
        self.pack('I', -42)

_struct.x = C()
```

The module `_struct` uses the module state to run `pack`. Therefore, the module state has to be alive until after the module has been cleared out to successfully run `C.__del__`. This happens at line 606, when `_PyImport_Cleanup` calls `_PyModule_Clear`. In fact, the loop that calls `_PyModule_Clear` has in its comments: 

> Now, if there are any modules left alive, clear their globals to minimize potential leaks.  All C extension modules actually end up here, since they are kept alive in the interpreter state.

That means that we can't clear the module state (which is used by C Extensions) before we run that loop.

Moving `_PyInterpreterState_ClearModules` until after it, fixes the segfault in the code snippet.

Finally, this updates a test in `io` to correctly assert the error that it now throws (since it now finds the io module state). The test that uses this is: `test_create_at_shutdown_without_encoding`. Given this test is now working is a proof that the module state now stays alive even when `__del__` is called at module destruction time. Thus, I didn't add a new tests for this.


https://bugs.python.org/issue38076
2020-02-04 02:29:25 -08:00
Victor Stinner c6e5c1123b
bpo-39489: Remove COUNT_ALLOCS special build (GH-18259)
Remove:

* COUNT_ALLOCS macro
* sys.getcounts() function
* SHOW_ALLOC_COUNT code in listobject.c
* SHOW_TRACK_COUNT code in tupleobject.c
* PyConfig.show_alloc_count field
* -X showalloccount command line option
* @test.support.requires_type_collecting decorator
2020-02-03 15:17:15 +01:00
Steve Cirelli 032de7324e
bpo-39450 Stripped whitespace before parsing the docstring in TestCase.shortDescription (GH-18175) 2020-02-03 07:06:50 +00:00
Pierre Glaser 0f2f35e15f
bpo-39492: Fix a reference cycle between reducer_override and a Pickler instance (GH-18266)
This also needs a backport to 3.8


https://bugs.python.org/issue39492



Automerge-Triggered-By: @pitrou
2020-02-02 10:55:21 -08:00
Kyle Stanley 339fd46cb7
bpo-39349: Add *cancel_futures* to Executor.shutdown() (GH-18057) 2020-02-02 13:49:00 +01:00
Andrew Svetlov 90d9ba6ef1
bpo-34793: Drop old-style context managers in asyncio.locks (GH-17533) 2020-02-01 13:12:52 +02:00
Victor Stinner 4d96b4635a
bpo-39511: PyThreadState_Clear() calls on_delete (GH-18296)
PyThreadState.on_delete is a callback used to notify Python when a
thread completes. _thread._set_sentinel() function creates a lock
which is released when the thread completes. It sets on_delete
callback to the internal release_sentinel() function. This lock is
known as Threading._tstate_lock in the threading module.

The release_sentinel() function uses the Python C API. The problem is
that on_delete is called late in the Python finalization, when the C
API is no longer fully working.

The PyThreadState_Clear() function now calls the
PyThreadState.on_delete callback. Previously, that happened in
PyThreadState_Delete().

The release_sentinel() function is now called when the C API is still
fully working.
2020-02-01 02:30:25 +01:00
Zackery Spytz bfdeaa37b3
bpo-38792: Remove IDLE shell calltip before new prompt. (#17150)
Previously, a calltip might be left after SyntaxError, KeyboardInterrupt, or Shell Restart.

Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
Co-authored-by: Tal Einat <taleinat+github@gmail.com>
2020-01-30 20:55:42 -05:00
Victor Stinner c232c9110c
bpo-39502: Skip test_zipfile.test_add_file_after_2107() on AIX (GH-18282)
Skip test_zipfile.test_add_file_after_2107() if time.localtime()
fails with OverflowError. It is the case on AIX 6.1 for example.
2020-01-30 15:47:53 +01:00
Dong-hee Na 8d49f7ceb4
bpo-39434: Improve float __floordiv__ performance and error message (GH-18147) 2020-01-30 13:23:15 +00:00
Victor Stinner c38fd0df2b
bpo-39353: binascii.crc_hqx() is no longer deprecated (GH-18276)
The binascii.crc_hqx() function is no longer deprecated.
2020-01-30 09:56:40 +01:00
Shantanu 2e6569b669
bpo-39493: Fix definition of IO.closed in typing.py (#18265) 2020-01-29 18:52:36 -08:00
Carl Friedrich Bolz-Tereick a327677905
bpo-39485: fix corner-case in method-detection of mock (GH-18252)
Replace check for whether something is a method in the mock module. The
previous version fails on PyPy, because there no method wrappers exist
(everything looks like a regular Python-defined function). Thus the
isinstance(getattr(result, '__get__', None), MethodWrapperTypes) check
returns True for any descriptor, not just methods.

This condition could also return erroneously True in CPython for
C-defined descriptors.

Instead to decide whether something is a method, just check directly
whether it's a function defined on the class. This passes all tests on
CPython and fixes the bug on PyPy.
2020-01-29 15:43:37 +00:00
Joannah Nanjekye e1e80002e2
bpo-39153: Clarify C API *SetItem refcounting semantics (GH-18220)
Some of the *SetItem methods in the C API steal a reference to the
given value. This annotates the better behaved ones to assure the
reader that these are not the ones with the inconsistent behaviour.

* 📜🤖 Added by blurb_it.

* make docs consistent with signature

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2020-01-29 21:20:53 +10:00
Bruce Merry d07d9f4c43
bpo-36051: Drop GIL during large bytes.join() (GH-17757)
Improve multi-threaded performance by dropping the GIL in the fast path
of bytes.join. To avoid increasing overhead for small joins, it is only
done if the output size exceeds a threshold.
2020-01-29 16:09:24 +09:00
Steve Dower 6a65eba44b
bpo-39401: Avoid unsafe DLL load on Windows 7 and earlier (GH-18231)
As Windows 7 is not supported by Python 3.9, we just replace the dynamic load with a static import. Backports will have a different fix to ensure they continue to behave the same.
2020-01-29 13:46:33 +11:00
David Carlier aabdeb766b bpo-38960: DTrace build fix for FreeBSD. (GH-17451)
DTrace build fix for FreeBSD.

- allowing passing an extra flag as it need to define the arch size.
- casting some probe's arguments.
2020-01-28 13:53:32 +01:00
Rémi Lapeyre 2cca8efe46 bpo-36350: inspect: Replace OrderedDict with dict. (GH-12412) 2020-01-28 21:47:03 +09:00
Adam Meily 0be3246d4f bpo-39439: Fix multiprocessing spawn path in a venv on Windows (GH-18158) 2020-01-28 21:34:23 +11:00
Zackery Spytz 13c1c3556f bpo-39393: Misleading error message on dependent DLL resolution failure (GH-18093) 2020-01-28 20:42:43 +11:00
Christoph Reiter c45a2aa9e2 bpo-38883: Don't use POSIX `$HOME` in `pathlib.Path.home/expanduser` on Windows (GH-17961)
In bpo-36264 os.path.expanduser was changed to ignore HOME on Windows.

Path.expanduser/home still honored HOME despite being documented as behaving the same
as os.path.expanduser. This makes them also ignore HOME so that both implementations
behave the same way again.
2020-01-28 20:41:50 +11:00
Terry Jan Reedy 2824c45a0a
bpo-39392: Turtle overlap fill depends on OS (#18223)
Whether or not overlap regions for self-intersecting polygons
or multiple shapes are filled depends on the operating system graphics,
typeof overlap, and number of overlaps.
2020-01-27 18:41:18 -05:00
Cheryl Sabella dd023ad161 bpo-30780: Add IDLE configdialog tests (#3592)
Expose dialog buttons to test code and complete their test coverage.
Complete test coverage for highlights and keys tabs.

Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
2020-01-27 17:15:56 -05:00
Dong-hee Na 4dbf2d8c67 bpo-39453: Make list.__contains__ hold strong references to avoid crashes (GH-18181) 2020-01-27 15:02:23 +00:00
Mark Shannon 8a4cd700a7
bpo-39320: Handle unpacking of **values in compiler (GH-18141)
* Add DICT_UPDATE and DICT_MERGE bytecodes. Use them for ** unpacking.

* Remove BUILD_MAP_UNPACK and BUILD_MAP_UNPACK_WITH_CALL, as they are now unused.

* Update magic number for ** unpacking opcodes.

* Update dis.rst to incorporate new bytecodes.

* Add blurb entry.
2020-01-27 09:57:45 +00:00
Karthikeyan Singaravelan 72b1004657 bpo-25597: Ensure wraps' return value is used for magic methods in MagicMock (#16029) 2020-01-27 06:48:15 +00:00
加和 4515a590a4 Fix linecache.py add lazycache to __all__ and use dict.clear to clear the cache (GH-4641) 2020-01-25 21:07:40 -05:00
Vegard Stikbakke aef7dc8987 bpo-38932: Mock fully resets child objects on reset_mock(). (GH-17409) 2020-01-25 15:44:46 +00:00
Łukasz Langa b07ead3411 Python 3.9.0a3
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE4/8oOcBIslwITevpsmmV4xAlBWgFAl4rXBoACgkQsmmV4xAl
 BWid+xAAmRi6rMnpTrJG6V4herWLg4tQg4LaghxXPHtAfcZCgYkbA3kI4nq7SoKy
 7XnJRX6Uk3FhT2rzM2+sg+sPDMNz5ki+G6L0rYrmouZSy+y7/KDBTzh7u8nnAZD1
 t72Zl3LyWFgeg58MVld5LWoCt7+ywAfIAEbeYAFonH4yY1MqK8Q5+kbNtuMBar81
 S44i/UIjVD1GJTzxxDqWU4B0BmCGizenBiyssAwTbM556dzjR7tDqWhDfiXcMLgW
 7Wq5sj96GjNOrNTbtPikiu3sl97hHlJgZEel/vBq2RNArEspRa0F3uqNYyaRedHr
 F9XDlsXLdj+18w1tG1HejZiGquW9y55vexEtDp2e2FXl9bHm5n7D8PUX8rCuw6li
 lR2W+oB0i8/Jd1R8GymJ4RfHrh7jHoTCskcLp8BrBlFBU0d3DAoYFT9rA5GSj3dx
 MIQgA2KbDJEYU7i0/uLLqK6L37PD8NaAPhdVjUN1B2qJ47RrC3yc5tPHIuyiBMYu
 M2PW8LNhg/+ZIQjznSam3xiLDzEv/TeHA0YgfdIaXgdtybFSGIUWMxCGQjNtzrHS
 u4WjsQy9yFAtF+mgPKqXBI+coBtLUhvRfOGHwhnbjkao7i65ztZC2VX/e2CfquO/
 DDJZoiGS0wGbDJfeApNaT7/Kac2uGgvYfx1foGReK3fqRqLl8aU=
 =xsba
 -----END PGP SIGNATURE-----
gpgsig -----BEGIN PGP SIGNATURE-----
 
 iQJEBAABCAAuFiEE4/8oOcBIslwITevpsmmV4xAlBWgFAl4sSAgQHGx1a2FzekBs
 YW5nYS5wbAAKCRCyaZXjECUFaGJkD/9ktOT+LKyW8+9nKRkVNHy4+PRGdKA/7O7y
 q8qieymxaJoq2sFKQmgBvDTEh/S6Z75BlsHiTucpDnFe+WVwLJ79g0w9ehbzWx8g
 XjNn1ONvsd/sDgUxZ3YiYy6uMbrCMPmcb0VPdlVW1l55uCDccNkh94Sa0lDEYnJo
 +Xr6kDd+dyrF6XswyfDSSYriUNHODfT4aezFbS/EAaD3uX+hqtEW+kl1C3unbN5H
 xxqe986NYGX5Yd7sIr9tqVBz4m0gQLDXX1i10pZFi+4pqog/ZV6XQbW+RSjr3SuU
 FktleKz5erwY2fEz9KHRjonpPDPwEnWOnnn+MNPDECuIVyHswkAidUS69DkzfUpf
 us1RrUoOmzi3cBxhrPCLVMnFFKpdxigiKWVa1CVNEhr7nhLLT3JPdfjXY/XEezW3
 i9xJkCaxJUw7xrt808O1MHhkKAC9VFk/s2YDpmhCcmwfF8IR2K4L0XpQRG4+dfwz
 WFPA8xQC2W+BHsHQoXW4XFZJQaCDmXZX2c0SOjl2fwZUyi2h3vvqJPfDxtjIRltN
 TqAZJuaZgJNhtvK9R2BIY6Xr72owczdM8z35Je44K6oiUgFYs4UN9U+6D4sdRkSc
 YBDgRmnGHTb07VWTWxyuzkXmUDiQ458KpdA9xDE47463v+NqkD9scR3EgX/e6OTR
 0+EgUU6dYg==
 =MyW2
 -----END PGP SIGNATURE-----

Merge tag 'v3.9.0a3'

Python 3.9.0a3
2020-01-25 14:52:06 +01:00
Matthew Kokotovich 62865f4532 bpo-39082: Allow AsyncMock to correctly patch static/class methods (GH-18116) 2020-01-25 10:17:47 +00:00
Cheryl Sabella d0d9fa8c5e bpo-39388: IDLE: Fix bug when cancelling out of configdialog (GH-18068)
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
2020-01-25 04:00:54 -05:00
Łukasz Langa c33378df39
Python 3.9.0a3 2020-01-24 22:05:07 +01:00
Serhiy Storchaka 9017e0bd5e bpo-39430: Fix race condition in lazy imports in tarfile. (GH-18161)
Use `from ... import ...` to ensure module is fully loaded before accessing its attributes.
2020-01-24 09:55:52 -08:00
mbarkhau 88704334e5 bpo-39390 shutil: fix argument types for ignore callback (GH-18122) 2020-01-24 15:51:16 +01:00
Karthikeyan Singaravelan 66b00a9d3a bpo-38473: Handle autospecced functions and methods used with attach_mock (GH-16784) 2020-01-24 13:14:29 +00:00
Victor Stinner b8d1262e8a
bpo-39395: putenv() and unsetenv() always available (GH-18135)
The os.putenv() and os.unsetenv() functions are now always available.

On non-Windows platforms, Python now requires setenv() and unsetenv()
functions to build.

Remove putenv_dict from posixmodule.c: it's not longer needed.
2020-01-24 14:05:48 +01:00
Victor Stinner 161e7b36b1
bpo-39413: Implement os.unsetenv() on Windows (GH-18163)
The os.unsetenv() function is now also available on Windows.
2020-01-24 11:53:44 +01:00