Commit Graph

22963 Commits

Author SHA1 Message Date
Steve Dower 00923c6399
bpo-38622: Add missing audit events for ctypes module (GH-17158) 2019-11-18 11:32:46 -08:00
jsnklln e243bae999 bpo-38722: Runpy use io.open_code() (GH-17234)
https://bugs.python.org/issue38722



Automerge-Triggered-By: @taleinat
2019-11-18 11:11:13 -08:00
Tal Einat ee703cbb41 bpo-38809: Windows build scripts use python.exe from virtual envs (GH-17164)
https://bugs.python.org/issue38809
2019-11-18 09:32:25 -08:00
Victor Stinner 04394df74b
bpo-38631: Avoid Py_FatalError() in float.__getformat__() (GH-17232)
Replace Py_FatalError() with a regular RuntimeError exception in
float.__getformat__().
2019-11-18 17:39:48 +01:00
Victor Stinner 59c80889ff
Revert "bpo-38811: Check for presence of os.link method in pathlib. (GH-17170)" (#17219)
This reverts commit 111772fc27.
2019-11-18 12:26:37 +01:00
Zackery Spytz 2bc343417a bpo-36589: Fix the error handling in curses.update_lines_cols(). (GH-12766)
Return None instead of 1.
2019-11-17 19:10:13 +02:00
Toke Høiland-Jørgensen 111772fc27 bpo-38811: Check for presence of os.link method in pathlib. (GH-17170)
Fix also the Path.symplink() method implementation for the case when
symlinks are not supported.
2019-11-17 19:06:38 +02:00
Andrey Doroschenko 645005e947 bpo-38724: Implement subprocess.Popen.__repr__ (GH-17151) 2019-11-17 16:08:31 +02:00
Serhiy Storchaka a0652328a2
bpo-28286: Deprecate opening GzipFile for writing implicitly. (GH-16417)
Always specify the mode argument for writing.
2019-11-16 18:56:57 +02:00
Serhiy Storchaka bd44a7ead9
bpo-38650: Constify PyStructSequence_UnnamedField. (GH-17005)
Make it a constant and referring to a constant string.
2019-11-16 18:55:29 +02:00
Serhiy Storchaka 5fd5cb8d85
bpo-38639: Optimize floor(), ceil() and trunc() for floats. (GH-16991) 2019-11-16 18:00:57 +02:00
Eric Snow 73cdb0c6b2 bpo-38816: Add notes in the C-API docs about fork in subinterpreters. (GH-17176)
The C-API docs are a bit sparse on the interplay between C `fork()` and the CPython runtime.  This change adds some more information on the subject.


https://bugs.python.org/issue38816
2019-11-15 13:28:54 -08:00
Steve Dower abde52cd8e
bpo-38453: Ensure ntpath.realpath correctly resolves relative paths (GH-16967)
Ensure isabs() is always True for \\?\ prefixed paths
Avoid unnecessary usage of readlink() to avoid resolving broken links incorrectly
Ensure shutil tests run in test directory
2019-11-15 09:49:21 -08:00
Phil Connell b22030073b bpo-38778: Document that os.fork is not allowed in subinterpreters (GH-17123)
Small docs update for [bpo-34651](https://bugs.python.org/issue34651).

Other references to fork (e.g. the PyOS.*Fork functions or discussions of fork() when embedding Python) point back to os.fork, so I don't think any other updates are needed.





https://bugs.python.org/issue38778



Automerge-Triggered-By: @ericsnowcurrently
2019-11-15 08:56:03 -08:00
Andrey Doroschenko e8acc865a3 bpo-38351: Modernize email examples from %-formatting to f-strings (GH-17162) 2019-11-15 11:03:46 +02:00
Benjamin Peterson 3ccdd9b180
closes bpo-38692: Add a pidfd child process watcher to asyncio. (GH-17069) 2019-11-13 19:08:50 -08:00
Andrew Svetlov dad6be5ffe bpo-38785: Prevent asyncio from crashing (GH-17144)
if parent `__init__` is not called from a constructor of object derived from `asyncio.Future`



https://bugs.python.org/issue38785
2019-11-13 13:36:46 -08:00
Kirill 61289d4366 bpo-38786: Add parsing of https links to pydoc (GH-17143) 2019-11-13 18:13:52 +02:00
Zackery Spytz 9c2844927d bpo-4630: Add cursor no-blink option for IDLE (GH-16960)
This immediately toggles shell, editor, and output windows, but does not affect other input widgets.
2019-11-13 02:13:33 -05:00
Ilya Kulakov d6d6e2aa02 Add Ilya Kulakov to Misc/ACKS. (GH-17130)
Contributions on bpo-26467 and bpo-29302.
2019-11-12 18:33:04 -08:00
Benjamin Peterson 74fa9f723f
closes bpo-27805: Ignore ESPIPE in initializing seek of append-mode files. (GH-17112)
This change, which follows the behavior of C stdio's fdopen and Python 2's file object, allows pipes to be opened in append mode.
2019-11-12 14:51:34 -08:00
jsnklln d593881505 bpo-38723: Pdb._runscript should use io.open_code() instead of open() (GH-17127)
Co-Authored-By: Brandt Bucher <brandtbucher@gmail.com>
2019-11-12 14:42:47 -08:00
Zackery Spytz c8b53dc3d8 bpo-26353: IDLE adds an unneeded newline when saving a shell window (GH-17103) 2019-11-12 05:54:10 -05:00
Manjusaka 051ff526b5 bpo-38565: add new cache_parameters method for lru_cache (GH-16916) 2019-11-11 23:30:18 -08:00
Brandt Bucher a0ed99bca8 bpo-38438: Simplify argparse "star nargs" usage. (GH-17106) 2019-11-11 12:47:48 -08:00
Raymond Hettinger 84ac437658
bpo-38761: Register WeakSet as a MutableSet (GH-17104) 2019-11-10 20:12:04 -08:00
Inada Naoki 6cbc84fb99
bpo-38613: Optimize set operations of dict keys. (GH-16961) 2019-11-08 00:59:04 +09:00
l0rb 519cb8772a bpo-38716: stop rotating handlers from setting inherited namer and rotator to None (GH-17072) 2019-11-06 21:21:40 +00:00
Benjamin Peterson 5c0c325453 closes bpo-38713: Expose P_PIDFD in os if it's defined. (GH-17071)
https://bugs.python.org/issue38713
2019-11-05 21:58:31 -08:00
Benjamin Peterson 6c4c45efae
bpo-38692: Add os.pidfd_open. (GH-17063) 2019-11-05 19:21:29 -08:00
Jeroen Demeyer bf17d41826 bpo-37645: add new function _PyObject_FunctionStr() (GH-14890)
Additional note: the `method_check_args` function in `Objects/descrobject.c` is written in such a way that it applies to all kinds of descriptors. In particular, a future re-implementation of `wrapper_descriptor` could use that code.

CC @vstinner @encukou 


https://bugs.python.org/issue37645



Automerge-Triggered-By: @encukou
2019-11-05 07:48:04 -08:00
Eddie Elizondo b3966639d2 bpo-35381 Remove all static state from posixmodule (GH-15892)
After #9665, this moves the remaining types in posixmodule to be heap-allocated to make it compatible with PEP384 as well as modifying all the type accessors to fully make the type opaque.

The original PR that got messed up a rebase: https://github.com/python/cpython/pull/10854. All the issues in that commit have now been addressed since https://github.com/python/cpython/pull/11661 got committed.

This change also removes any state from the data segment and onto the module state itself.


https://bugs.python.org/issue35381



Automerge-Triggered-By: @encukou
2019-11-05 07:16:14 -08:00
Benjamin Peterson 62161ce989
closes bpo-37633: Reëxport some function compatibility wrappers for macros in ``pythonrun.h``. (GH-17056) 2019-11-04 21:34:14 -08:00
Victor Stinner f4b1e3d7c6
bpo-38644: Add Py_EnterRecursiveCall() to the limited API (GH-17046)
Provide Py_EnterRecursiveCall() and Py_LeaveRecursiveCall() as
regular functions for the limited API. Previously, there were defined
as macros, but these macros didn't work with the limited API which
cannot access PyThreadState.recursion_depth field.

Remove _Py_CheckRecursionLimit from the stable ABI.

Add Include/cpython/ceval.h header file.
2019-11-04 19:48:34 +01:00
Alexandru Ardelean 6552563b3d bpo-38684: haslib: fix build when Blake2 not enabled in OpenSSL (#17043) 2019-11-04 15:55:56 +01:00
Ram Rachum 8d4fef4ee2 bpo-38422: Clarify docstrings of pathlib suffix(es) (GH-16679)
Whenever I use `path.suffix` I have to check again whether it includes the dot or not. I decided to add it to the docstring so I won't have to keep checking. 


https://bugs.python.org/issue38422



Automerge-Triggered-By: @pitrou
2019-11-02 09:46:24 -07:00
Anthony Sottile b32cb97bce bpo-38312: Add curses.{get,set}_escdelay and curses.{get,set}_tabsize. (GH-16938) 2019-10-31 11:13:48 +02:00
Lucas Cimon b15100fe7d bpo-38586: setting logging.Handler .name property in fileConfig (GH-16918) 2019-10-31 08:06:25 +00:00
Victor Stinner a4ed6ed9f3
bpo-38614: Increase asyncio test_communicate() timeout (GH-16995)
Fix test_communicate() of test_asyncio.test_subprocess: use
support.LONG_TIMEOUT (5 minutes), instead of 1 minute.
2019-10-30 16:00:44 +01:00
Pablo Galindo 6c3e66a34b
bpo-38640: Allow break and continue in always false while loops (GH-16992) 2019-10-30 11:53:26 +00:00
Victor Stinner 24c6258269
bpo-38614: Add timeout constants to test.support (GH-16964)
Add timeout constants to test.support:

* LOOPBACK_TIMEOUT
* INTERNET_TIMEOUT
* SHORT_TIMEOUT
* LONG_TIMEOUT
2019-10-30 12:41:43 +01:00
Serhiy Storchaka 865c3b257f
bpo-28029: Make "".replace("", s, n) returning s for any n != 0. (GH-16981) 2019-10-30 12:03:53 +02:00
Daniel Hillier da6ce58dd5 bpo-36993: Improve error reporting for zipfiles with bad zip64 extra data. (GH-14656) 2019-10-29 09:24:18 +02:00
Steve Dower 0ac6137dd3
bpo-38589: Fixes HTML Help shortcut when Windows is not installed to C drive (GH-16968) 2019-10-28 14:34:25 -07:00
Zackery Spytz edb172a872 bpo-38519: Internal include files missing on Windows (GH-16921) 2019-10-28 10:03:27 -07:00
Victor Stinner e471e72977
bpo-37330: open() no longer accept 'U' in file mode (GH-16959)
open(), io.open(), codecs.open() and fileinput.FileInput no longer
accept "U" ("universal newline") in the file mode. This flag was
deprecated since Python 3.3.
2019-10-28 15:40:08 +01:00
Dong-hee Na 3bfc8e0fcc bpo-38602: Add fcntl.F_OFD_XXXX for fcntlmodule (GH-16956) 2019-10-28 09:31:15 +02:00
Marco Rougeth 85c6f8c65c bpo-38592 Add pt-br switcher to Python Docs website (GH-16924)
* 📜🤖 Added by blurb_it.
2019-10-27 09:30:12 +01:00
Serhiy Storchaka 5c32af7522
bpo-38334: Fix seeking backward on an encrypted zipfile.ZipExtFile. (GH-16937)
Test by Daniel Hillier.
2019-10-27 10:22:14 +02:00
Terry Jan Reedy e3f90b217a
bpo-38598: Do not try to compile IDLE shell or output windows (GH-16939) 2019-10-26 21:15:10 -04:00
Serhiy Storchaka 26ae9f6d3d
bpo-38535: Fix positions for AST nodes for calls without arguments in decorators. (GH-16861) 2019-10-26 16:46:05 +03:00
Zsolt Dollenstein 96b06aefe2 bpo-33348: parse expressions after * and ** in lib2to3 (GH-6586)
These are valid even in python 2.7


https://bugs.python.org/issue33348



Automerge-Triggered-By: @gpshead
2019-10-23 23:19:07 -07:00
Girts a01ba333af bpo-30618: add readlink to pathlib.Path (GH-8285)
This adds a "readlink" method to pathlib.Path objects that calls through
to os.readlink.


https://bugs.python.org/issue30618



Automerge-Triggered-By: @gpshead
2019-10-23 14:18:40 -07:00
Victor Stinner 1b53a24fb4
bpo-34679: ProactorEventLoop only uses set_wakeup_fd() in main thread (GH-16901)
bpo-34679, bpo-38563: asyncio.ProactorEventLoop.close() now only calls
signal.set_wakeup_fd() in the main thread.
2019-10-23 17:25:29 +02:00
Victor Stinner 028f7349a0
bpo-37415: Fix stdatomic.h header check for ICC compiler (GH-16717)
Fix stdatomic.h header check for ICC compiler: the ICC implementation
lacks atomic_uintptr_t type which is needed by Python.

Test:

* atomic_int and atomic_uintptr_t types
* atomic_load_explicit() and atomic_store_explicit()
* memory_order_relaxed and memory_order_seq_cst constants

But don't test ATOMIC_VAR_INIT(): it's not used in Python.
2019-10-22 21:53:50 +02:00
Tyler Kieft f548a3e4a2 bpo-38303: Make audioop extension module PEP-384 compatible (GH-16497)
https://bugs.python.org/issue38303



Automerge-Triggered-By: @encukou
2019-10-22 06:07:03 -07:00
Serhiy Storchaka 10ecbadb79
bpo-31202: Preserve case of literal parts in Path.glob() on Windows. (GH-16860) 2019-10-21 20:37:15 +03:00
Serhiy Storchaka 5bc6a7c06e
bpo-38540: Fix possible leak in PyArg_Parse for "es#" and "et#". (GH-16869) 2019-10-21 11:35:07 +03:00
Dong-hee Na 2eba6ad7bf bpo-38493: Add os.CLD_KILLED and os.CLD_STOPPED. (GH-16821) 2019-10-21 10:01:05 +03:00
Hai Shi 06cd5b6acd bpo-38465: Convert the type of exports counters to Py_ssize_t. (GH-16746) 2019-10-21 09:31:46 +03:00
Raymond Hettinger 58ccd201fa
bpo-36321: Fix misspelled attribute name in namedtuple() (GH-16858) 2019-10-20 10:19:47 -07:00
Dong-hee Na 24dc2f8c56 bpo-38525: Fix a segmentation fault when using reverse iterators of empty dict (GH-16846)
The reverse iterator for empty dictionaries was not handling correctly shared-key dictionaries.
2019-10-19 21:01:08 +01:00
Raymond Hettinger 5eabec022b
bpo-38521: Fix error in NormalDist.__eq__() (GH-16840) 2019-10-18 14:20:35 -07:00
Victor Stinner ecb035cd14
bpo-38502: regrtest uses process groups if available (GH-16829)
test.regrtest now uses process groups in the multiprocessing mode
(-jN command line option) if process groups are available: if
os.setsid() and os.killpg() functions are available.
2019-10-18 15:49:08 +02:00
Tim Graham 5a88d50ff0 bpo-27657: Fix urlparse() with numeric paths (#661)
* bpo-27657: Fix urlparse() with numeric paths

Revert parsing decision from bpo-754016 in favor of the documented
consensus in bpo-16932 of how to treat strings without a // to
designate the netloc.

* bpo-22891: Remove urlsplit() optimization for 'http' prefixed inputs.
2019-10-18 06:07:20 -07:00
Steve Dower fbe3c76c7c bpo-38418: Fixes audit event for os.system to be named 'os.system' (GH-16670)
https://bugs.python.org/issue38418



Automerge-Triggered-By: @zooba
2019-10-18 00:52:15 -07:00
Gregory P. Smith f33c57d5c7
bpo-33604: Raise TypeError on missing hmac arg. (GH-16805)
Also updates the documentation to clarify the situation surrounding
the digestmod parameter that is required despite its position in the
argument list as of 3.8.0 as well as removing old python2 era
references to "binary strings".

We indavertently had this raise ValueError in 3.8.0 for the missing
arg.  This is not considered an API change as no reasonable code would
be catching this missing argument error in order to handle it.
2019-10-17 20:30:42 -07:00
Inada Naoki 9c11029bb4
bpo-37587: json: Use _PyUnicodeWriter when scanning string. (GH-15591) 2019-10-17 16:12:41 +09:00
Steve Dower 7aebbd1182
bpo-38492: Remove pythonw.exe dependency on the Microsoft C++ runtime (GH-16824) 2019-10-16 10:27:17 -07:00
Victor Stinner fab4ef2df0
bpo-35998: Fix test_asyncio.test_start_tls_server_1() (GH-16815)
main() is now responsible to send the ANSWER, rather than
ServerProto. main() now waits until it got the HELLO before sending
the ANSWER over the new transport.

Previously, there was a race condition between main() replacing the
protocol and the protocol sending the ANSWER once it gets the HELLO.

TLSv1.3 was disabled for the test: reenable it.
2019-10-16 02:36:42 +02:00
Joannah Nanjekye 3cd21aa6a1
bpo-38468 : Refactor python-config (#16749)
* refactor python-config

* 📜🤖 Added by blurb_it.

* Update Misc/NEWS.d/next/Build/2019-10-13-16-18-24.bpo-38468.buCO84.rst

Co-Authored-By: Pablo Galindo <Pablogsal@gmail.com>

* Use getvar for pyver

* Update news entry with affected file.

* Update Misc/NEWS.d/next/Build/2019-10-13-16-18-24.bpo-38468.buCO84.rst

Co-Authored-By: Victor Stinner <vstinner@python.org>

* Update 2019-10-13-16-18-24.bpo-38468.buCO84.rst
2019-10-15 11:18:47 -03:00
Julien Danjou 8d59eb1b66 bpo-37961, tracemalloc: add Traceback.total_nframe (GH-15545)
Add a total_nframe field to the traces collected by the tracemalloc module.
This field indicates the original number of frames before it was truncated.
2019-10-15 14:00:16 +02:00
Pablo Galindo f3ef06a7cb
bpo-38478: Correctly handle keyword argument with same name as positional-only parameter (GH-16800) 2019-10-15 12:40:02 +01:00
Victor Stinner eb1dda2b56 bpo-38470: Fix test_compileall.test_compile_dir_maxlevels() (GH-16789)
Fix test_compile_dir_maxlevels() on Windows without long path
support: only create 3 subdirectories instead of between 20 and 100
subdirectories.

Fix also compile_dir() to use the current sys.getrecursionlimit()
value as the default maxlevels value, rather than using
sys.getrecursionlimit() value read at startup.
2019-10-15 11:26:13 +02:00
Vinay Sajip 0b60f64e43
bpo-11410: Standardize and use symbol visibility attributes across POSIX and Windows. (GH-16347) 2019-10-15 08:26:12 +01:00
Stein Karlsen aad2ee0156 bpo-32498: urllib.parse.unquote also accepts bytes (GH-7768) 2019-10-14 13:36:29 +03:00
Pablo Galindo fd5c414880
bpo-38469: Handle named expression scope with global/nonlocal keywords (GH-16755)
The symbol table handing of PEP572's assignment expressions is not resolving correctly the scope of some variables in presence of global/nonlocal keywords in conjunction with comprehensions.
2019-10-14 05:18:05 +01:00
Pablo Galindo 466326dcdf
bpo-38379: Don't block collection of unreachable objects when some objects resurrect (GH-16687)
Currently if any finalizer invoked during garbage collection resurrects any object, the gc gives up and aborts the collection. Although finalizers are assured to only run once per object, this behaviour of the gc can lead to an ever-increasing memory situation if new resurrecting objects are allocated in every new gc collection.

To avoid this, recompute what objects among the unreachable set need to be resurrected and what objects can be safely collected. In this way, resurrecting objects will not block the collection of other objects in the unreachable set.
2019-10-13 16:48:59 +01:00
Samuel Colvin 793cb85437 bpo-38431: Fix __repr__ method of InitVar to work with typing objects. (GH-16702) 2019-10-13 14:45:36 +03:00
Serhiy Storchaka 140a7d1f35
bpo-38378: Rename parameters "out" and "in" of os.sendfile(). (GH-16742)
They conflicted with keyword "in".

Also rename positional-only parameters of private os._fcopyfile()
for consistency.
2019-10-13 11:59:31 +03:00
Gregory P. Smith f3751efb5c
bpo-38417: Add umask support to subprocess (GH-16726)
On POSIX systems, allow the umask to be set in the child process before we exec.
2019-10-12 13:24:56 -07:00
Samuel Colvin 822922af90 bpo-35800: Deprecate smtpd.MailmanProxy (GH-11675)
Since `smtpd.MailmanProxy` is already broken, it is not formally deprecated in 3.9. It will be removed in 3.10.


https://bugs.python.org/issue35800
2019-10-12 10:24:26 -07:00
Abhilash Raj 19a3d87300 bpo-38449: Revert "bpo-22347: Update mimetypes.guess_type to allow oper parsing of URLs (GH-15522)" (GH-16724)
This reverts commit 87bd2071c7.



https://bugs.python.org/issue38449
2019-10-11 22:41:35 -07:00
Ruediger Pluem 2b7dc40b2a bpo-38347: find pathfix for Python scripts whose name contain a '-' (GH-16536)
pathfix.py: Assume all files that end on '.py' are Python scripts when working recursively.
2019-10-11 15:36:50 +02:00
Pablo Galindo 320dd504dd
bpo-38437: Activate GC_DEBUG when PY_DEBUG is set (GH-16707) 2019-10-10 22:45:17 +01:00
Ronan Lamy 7bb14316b8 bpo-38109: Add missing constants to Lib/stat.py (GH-16665)
Add missing stat.S_IFDOOR, stat.S_IFPORT, stat.S_IFWHT,
stat.S_ISDOOR, stat.S_ISPORT, and stat.S_ISWHT values to
the Python implementation of the stat module.
2019-10-10 09:34:46 +02:00
Tim Peters ecbf35f933
bpo-38379: don't claim objects are collected when they aren't (#16658)
* bpo-38379:  when a finalizer resurrects an object,
nothing is actually collected in this run of gc.
Change the stats to relect that truth.
2019-10-09 12:37:30 -05:00
Antonio Gutierrez 594e2edfb5 closes bpo-36161: Use thread-safe ttyname_r instead of ttyname. (GH-14868)
Signed-off-by: Antonio Gutierrez <chibby0ne@gmail.com>
2019-10-08 19:19:48 -07:00
Victor Stinner 0ec618af98
bpo-37531: regrtest ignores output on timeout (GH-16659)
bpo-37531, bpo-38207: On timeout, regrtest no longer attempts to call
`popen.communicate() again: it can hang until all child processes
using stdout and stderr pipes completes. Kill the worker process and
ignores its output.

Reenable test_regrtest.test_multiprocessing_timeout().

bpo-37531: Change also the faulthandler timeout of the main process
from 1 minute to 5 minutes, for Python slowest buildbots.
2019-10-08 18:45:43 +02:00
Pablo Galindo 10cd00a9e3
bpo-38395: Fix ownership in weakref.proxy methods (GH-16632)
The implementation of weakref.proxy's methods call back into the Python
API using a borrowed references of the weakly referenced object
(acquired via PyWeakref_GET_OBJECT). This API call may delete the last
reference to the object (either directly or via GC), leaving a dangling
pointer, which can be subsequently dereferenced.

To fix this, claim a temporary ownership of the referenced object when
calling the appropriate method. Some functions because at the moment they
do not need to access the borrowed referent, but to protect against
future changes to these functions, ownership need to be fixed in
all potentially affected methods.
2019-10-08 16:30:50 +01:00
Victor Stinner 03ab6b4fc6
bpo-38118: Ignore Valgrind false alarm in PyUnicode_Decode() (GH-16651)
Valgrind emits "Conditional jump or move depends on uninitialised
value(s)" false alarms on GCC builtin strcmp() function. The GCC code
is correct.

Valgrind bug: https://bugs.kde.org/show_bug.cgi?id=264936
2019-10-08 15:42:17 +02:00
Serhiy Storchaka 13abda4100
bpo-38405: Make nested subclasses of typing.NamedTuple pickleable. (GH-16641) 2019-10-08 16:29:52 +03:00
Serhiy Storchaka b690a2759e
bpo-36698: IDLE no longer fails when write non-encodable characters to stderr. (GH-16583)
It now escapes them with a backslash, as the regular Python interpreter.
Added the "errors" field to the standard streams.
2019-10-08 14:32:25 +03:00
Serhiy Storchaka d05b000c6b
bpo-38371: Tkinter: deprecate the split() method. (GH-16584) 2019-10-08 14:31:35 +03:00
Antonio Gutierrez 0d3fe8ae49 closes bpo-38402: Check error of primitive crypt/crypt_r. (GH-16599)
Checks also for encryption algorithms methods not supported in different
OSs.

Signed-off-by: Antonio Gutierrez <chibby0ne@gmail.com>
2019-10-07 21:22:17 -07:00
Victor Stinner 1b18455695
bpo-38392: PyObject_GC_Track() validates object in debug mode (GH-16615)
In debug mode, PyObject_GC_Track() now calls tp_traverse() of the
object type to ensure that the object is valid: test that objects
visited by tp_traverse() are valid.

Fix pyexpat.c: only track the parser in the GC once the parser is
fully initialized.
2019-10-08 00:09:31 +02:00
James Abel e310af9e29 bpo-38344: Fix syntax in activate.bat (GH-16533) 2019-10-07 14:07:19 -07:00
Ricardo Bánffy 15ae75d660 bpo-38294: Add list of no-longer-escaped chars to re.escape documentation. (GH-16442)
Prior to 3.7, re.escape escaped many characters that don't have
special meaning in Python, but that use to require escaping in other
tools and languages. This commit aims to make it clear which characters
were, but are no longer escaped.
2019-10-07 23:54:35 +03:00
Victor Stinner 6876257eaa
bpo-36389: _PyObject_CheckConsistency() available in release mode (GH-16612)
bpo-36389, bpo-38376: The _PyObject_CheckConsistency() function is
now also available in release mode. For example, it can be used to
debug a crash in the visit_decref() function of the GC.

Modify the following functions to also work in release mode:

* _PyDict_CheckConsistency()
* _PyObject_CheckConsistency()
* _PyType_CheckConsistency()
* _PyUnicode_CheckConsistency()

Other changes:

* _PyMem_IsPtrFreed(ptr) now also returns 1 if ptr is NULL
  (equals to 0).
* _PyBytesWriter_CheckConsistency() now returns 1 and is only used
  with assert().
* Reorder _PyObject_Dump() to write safe fields first, and only
  attempt to render repr() at the end.
2019-10-07 18:42:01 +02:00
Serhiy Storchaka ef092fe990
bpo-25988: Do not expose abstract collection classes in the collections module. (GH-10596) 2019-10-07 12:10:15 +03:00
Dong-hee Na c38e725d17 bpo-38210: Fix intersection operation with dict view and iterator. (GH-16602) 2019-10-06 14:28:33 +03:00
Andrei Troie 65dcc8a8dc bpo-38332: Catch KeyError from unknown cte in encoded-word. (GH-16503)
KeyError should cause a failure in parsing the encoded word and should be caught and raised as a _InvalidEWError instead.
2019-10-05 09:19:15 -07:00
nde 3faf826e58 bpo-38341: Add SMTPNotSupportedError in the exports of smtplib (#16525)
Add SMTPNotSupportedError in the exports of smtplib

Co-Authored-By: Brandt Bucher <brandtbucher@gmail.com>
2019-10-04 17:30:58 -07:00
Joannah Nanjekye 8855e47d09 bpo-38266: Revert bpo-37878: Make PyThreadState_DeleteCurrent() Internal (GH-16558)
Revert the removal of PyThreadState_DeleteCurrent() with documentation.
2019-10-04 13:35:42 +02:00
Serhiy Storchaka 06cb94bc84
bpo-13153: Use OS native encoding for converting between Python and Tcl. (GH-16545)
On Windows use UTF-16 (or UTF-32 for 32-bit Tcl_UniChar) with the
"surrogatepass" error handler for converting to/from Tcl Unicode objects.

On Linux use UTF-8 with the "surrogateescape" error handler for converting
to/from Tcl String objects.

Converting strings from Tcl to Python and back now never fails
(except MemoryError).
2019-10-04 13:09:52 +03:00
Steve Dower 353fb1ecbf
bpo-38359: Ensures pyw.exe launcher reads correct registry key (GH-16561) 2019-10-03 08:31:21 -07:00
Steve Dower a0e3d27e4e
bpo-38355: Fix ntpath.realpath failing on sys.executable (GH-16551) 2019-10-03 08:31:03 -07:00
Victor Stinner 5e0ea7540f
bpo-37474: Don't call fedisableexcept() on FreeBSD (GH-16515)
On FreeBSD, Python no longer calls fedisableexcept() at startup to
control the floating point control mode. The call became useless
since FreeBSD 6: it became the default mode.
2019-10-01 13:12:29 +02:00
Victor Stinner 3c30a76f3d
bpo-38304: Remove PyConfig.struct_size (GH-16500) (GH-16508)
For now, we'll rely on the fact that the config structures aren't covered by the stable ABI.

We may revisit this in the future if we further explore the idea of offering a stable embedding API.

(cherry picked from commit bdace21b76)
2019-10-01 10:56:37 +02:00
Giampaolo Rodola 94e165096f
bpo-38319: Fix shutil._fastcopy_sendfile(): set sendfile() max block size (GH-16491) 2019-10-01 11:40:54 +08:00
Maxwell A McKinnon cf57cabef8 bpo-32689: Updates shutil.move to allow for Path objects to be used as source arg (GH-15326)
Important work originally done by @emilyemorehouse two years ago and nearly ready to go in.

This bug has affected many people and in some cases has been a dealbreaker to the adoption of the otherwise wonderful pathlib and PEP519. https://stackoverflow.com/questions/33625931/copy-file-with-pathlib-in-python.

This adds the outstanding test request from that PR @vstinner (https://github.com/python/cpython/pull/5393).

Test fails without the change, passes with it, along with every other test in test_shutil.

Some variants were experimented with to make the one line change and the most performant one was picked.


# Added Test for PathLike directory destination, the current fail case

```
Lib/test/test_shutil.py::TestMove::test_move_file_pathlike FAILED                                                               [100%]

============================================================== FAILURES ===============================================================
__________________________________________________ TestMove.test_move_file_pathlike ___________________________________________________

self = <test.test_shutil.TestMove testMethod=test_move_file_pathlike>

    def test_move_file_pathlike(self):
        # Move a file to another location on the same filesystem.
        src = pathlib.Path(self.src_file)
>       self._check_move_file(src, self.dst_dir, self.dst_file)

Lib/test/test_shutil.py:1563:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Lib/test/test_shutil.py:1545: in _check_move_file
    shutil.move(src, dst)
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/shutil.py:562: in move
    real_dst = os.path.join(dst, _basename(src))
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

path = PosixPath('/var/folders/r2/psq74t5x3nbfzlph8bh2pvdw0000gn/T/tmp9ie0wh9_/foo')

    def _basename(path):
        # A basename() variant which first strips the trailing slash, if present.
        # Thus we always get the last component of the path, even for directories.
        sep = os.path.sep + (os.path.altsep or '')
>       return os.path.basename(path.rstrip(sep))
E       AttributeError: 'PosixPath' object has no attribute 'rstrip'

/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/shutil.py:526: AttributeError
============================================== 1 failed, 102 deselected in 0.30 seconds ===============================================
```

After change:

```
========================================================= test session starts =========================================================
platform darwin -- Python 3.7.4, pytest-5.0.1, py-1.8.0, pluggy-0.12.0 -- /Users/maxwellmckinnon/.venvs/TA3.7/bin/python3.7
cachedir: .pytest_cache
rootdir: /Users/maxwellmckinnon/dev/cpython
plugins: cov-2.7.1, mock-1.10.4
collected 103 items / 102 deselected / 1 selected

Lib/test/test_shutil.py::TestMove::test_move_file_pathlike PASSED                                                               [100%]

============================================== 1 passed, 102 deselected in 0.06 seconds ===============================================
```

Running all the tests in test_shutil.py
```
╰─ pytest Lib/test/test_shutil.py -v
========================================================= test session starts =========================================================
platform darwin -- Python 3.7.4, pytest-5.0.1, py-1.8.0, pluggy-0.12.0 -- /Users/maxwellmckinnon/.venvs/TA3.7/bin/python3.7
cachedir: .pytest_cache
rootdir: /Users/maxwellmckinnon/dev/cpython
plugins: cov-2.7.1, mock-1.10.4
collected 103 items

Lib/test/test_shutil.py::TestShutil::test_chown PASSED                                                                          [  0%]
Lib/test/test_shutil.py::TestShutil::test_copy PASSED                                                                           [  1%]
...
Lib/test/test_shutil.py::TermsizeTests::test_stty_match SKIPPED                                                                 [ 99%]
Lib/test/test_shutil.py::PublicAPITests::test_module_all_attribute PASSED                                                       [100%]

================================================ 96 passed, 7 skipped in 1.25 seconds =================================================
```

# Performance Considerations
Is it considered poor form to get rid of _basename altogether and make use of pathlib in the move function? I'm not sure if the idea is for all these modules to strictly avoid circular dependencies. They are already using os.path which is just as much a citizen in 3.8 as pathlib right?

e.g.

`real_dst = os.path.join(dst, _basename(src))`
becomes
`real_dst = Path(dst) / Path(src).name`

I've looked around and familiarized myself, and I now think importing pathlib here is fine. My only remaining concern is that of performance.

Here's the performance difference for this step. 

```
In [46]: %timeit real_dst = os.path.join("a/b/c", _basename('b/'))
2.71 µs ± 62.6 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

In [47]: %timeit real_dst = Path("a/b/c") / Path('b/').name
12.4 µs ± 65.3 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
```

Is 10us significant or insignificant compared to the least expensive operation this function will do? I don't know. Let's find out.

```
In [55]: %timeit os.rename('/tmp/a/a.txt', '/tmp/a/b.txt'); os.rename('/tmp/a/b.txt', '/tmp/a/a.txt')
124 µs ± 2.18 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)
```
62us to rename. 10us seems significant enough that we wouldn't want to favor the Path sugar suggestion. 16% speed decrease from adding the 10us.

What do people think? I was hoping to get to use pathlib.Path here, but I suspect for this low level move, it should be as fast as possible, and 16% is not worth one line of sugary code to me.



https://bugs.python.org/issue32689



Automerge-Triggered-By: @gvanrossum
2019-09-30 19:41:16 -07:00
Neil Schemenauer bcda460baf Clear weakrefs in garbage found by the GC (#16495)
Fix a bug due to the interaction of weakrefs and the cyclic garbage
collector. We must clear any weakrefs in garbage in order to prevent
their callbacks from executing and causing a crash.
2019-09-30 19:06:45 +02:00
Yury Selivanov fc4a044a3c
bpo-30773: Fix ag_running; prohibit running athrow/asend/aclose in parallel (#7468) 2019-09-29 22:59:11 -07:00
Yury Selivanov 6758e6e12a
bpo-38242: Revert "bpo-36889: Merge asyncio streams (GH-13251)" (#16482)
See https://bugs.python.org/issue38242 for more details
2019-09-29 21:59:55 -07:00
Lisa Roach 3667e1ee6c
bpo-38163: Child mocks detect their type as sync or async (GH-16471) 2019-09-29 21:56:47 -07:00
Lisa Roach 25e115ec00
bpo-38161: Removes _AwaitEvent from AsyncMock. (GH-16443) 2019-09-29 21:01:28 -07:00
Victor Stinner fb4ae152a9
bpo-38317: Fix PyConfig.warnoptions priority (GH-16478)
Fix warnings options priority: PyConfig.warnoptions has the highest
priority, as stated in the PEP 587.

* Document options order in PyConfig.warnoptions documentation.
* Make PyWideStringList_INIT macro private: replace "Py" prefix
  with "_Py".
* test_embed: add test_init_warnoptions().
2019-09-30 01:40:17 +02:00
Andrew Svetlov 58498bc717
bpo-38019: correctly handle pause/resume reading of closed asyncio unix pipe (GH-16472) 2019-09-29 15:00:35 +03:00
Lisa Roach 9a7d951950
bpo-38108: Makes mock objects inherit from Base (GH-16060) 2019-09-28 18:42:44 -07:00
Brandt Bucher f185a73249 bpo-38310: Predict BUILD_MAP_UNPACK_WITH_CALL -> CALL_FUNCTION_EX. (GH-16467) 2019-09-28 17:12:49 -07:00
T. Wouters c8165036f3 bpo-38115: Deal with invalid bytecode offsets in lnotab (GH-16079)
Document that lnotab can contain invalid bytecode offsets (because of
terrible reasons that are difficult to fix). Make dis.findlinestarts()
ignore invalid offsets in lnotab. All other uses of lnotab in CPython
(various reimplementations of addr2line or line2addr in Python, C and gdb)
already ignore this, because they take an address to look for, instead.

Add tests for the result of dis.findlinestarts() on wacky constructs in
test_peepholer.py, because it's the easiest place to add them.
2019-09-28 07:49:15 -07:00
Jason R. Coombs 7774d7831e
bpo-38216, bpo-36274: Allow subclasses to separately override validation and encoding behavior (GH-16448)
* bpo-38216: Allow bypassing input validation

* bpo-36274: Also allow the URL encoding to be overridden.

* bpo-38216, bpo-36274: Add tests demonstrating a hook for overriding validation, test demonstrating override encoding, and a test to capture expectation of the interface for the URL.

* Call with skip_host to avoid tripping on the host checking in the URL.

* Remove obsolete comment.

* Make _prepare_path_encoding its own attr.

This makes overriding just that simpler.

Also, don't use the := operator to make backporting easier.

* Add a news entry.

* _prepare_path_encoding -> _encode_prepared_path()

* Once again separate the path validation and request encoding, drastically simplifying the behavior. Drop the guarantee that all processing happens in _prepare_path.
2019-09-28 08:32:01 -04:00
Victor Stinner 441b10cf28
bpo-38304: Add PyConfig.struct_size (GH-16451)
Add a new struct_size field to PyPreConfig and PyConfig structures to
allow to modify these structures in the future without breaking the
backward compatibility.

* Replace private _config_version field with public struct_size field
  in PyPreConfig and PyConfig.
* Public PyPreConfig_InitIsolatedConfig() and
  PyPreConfig_InitPythonConfig()
  return type becomes PyStatus, instead of void.
* Internal _PyConfig_InitCompatConfig(),
  _PyPreConfig_InitCompatConfig(), _PyPreConfig_InitFromConfig(),
  _PyPreConfig_InitFromPreConfig() return type becomes PyStatus,
  instead of void.
* Remove _Py_CONFIG_VERSION
* Update the Initialization Configuration documentation.
2019-09-28 04:28:35 +02:00
Jesús Cea 52d1b86bde
bpo-38301: In Solaris family, we must be sure to use '-D_REENTRANT' (#16446) 2019-09-28 03:44:32 +02:00
Dong-hee Na e8650a4f8c bpo-38243, xmlrpc.server: Escape the server_title (GH-16373)
Escape the server title of xmlrpc.server.DocXMLRPCServer
when rendering the document page as HTML.
2019-09-27 21:59:37 +02:00
Michael Felt 0bcbfa43d5 bpo-28009: Fix uuid.uuid1() and uuid.get_node() on AIX (GH-8672) 2019-09-26 22:43:15 +03:00
Christian Heimes df6ac7e2b8 bpo-38275: Skip ssl tests for disabled versions (GH-16386)
test_ssl now handles disabled TLS/SSL versions better. OpenSSL's crypto
policy and run-time settings are recognized and tests for disabled versions
are skipped.

Signed-off-by: Christian Heimes <christian@python.org>



https://bugs.python.org/issue38275
2019-09-26 08:02:59 -07:00
Victor Stinner 64b4a3a2de
bpo-38239: Fix test_gdb for Link Time Optimization (LTO) (GH-16422) 2019-09-26 16:54:13 +02:00
Lumír 'Frenzy' Balhar 8e7bb991de bpo-38112: Compileall improvements (GH-16012)
* Raise the limit of maximum path depth to actual  recursion limit

* Add posibilities to adjust a path compiled in .pyc  file.

Now, you can:
- Strip a part of path from a beggining of path into compiled file
   example "-s /test /test/build/real/test.py" → "build/real/test.py"
- Append some new path to a beggining of path into compiled file
   example "-p /boo real/test.py" → "/boo/real/test.py"

You can also use both options in the same time. In that case,
striping is done before appending.

* Add a possibility to specify multiple optimization levels

Each optimization level then leads to separated compiled file.
Use `action='append'` instead of `nargs='+'` for the -o option.
Instead of `-o 0 1 2`, specify `-o 0 -o 1 -o 2`. It's more to type,
but much more explicit.

* Add a symlinks limitation feature

This feature allows us to limit byte-compilation of symbolic
links if they are pointing outside specified dir (build root
for example).
2019-09-26 08:28:26 +02:00
Benjamin Peterson 52b9408038
closes bpo-38174: Update vendored expat library to 2.2.8. (GH-16346)
Fixes CVE-2019-15903. See full changelog at https://github.com/libexpat/libexpat/blob/R_2_2_8/expat/Changes.
2019-09-25 21:33:58 -07:00
Christian Heimes df69e75edc
bpo-38142: Updated _hashopenssl.c to be PEP 384 compliant (#16071)
* Updated _hashopenssl.c to be PEP 384 compliant
* Remove refleak test from test_hashlib. The updated type no longer accepts random arguments to __init__.
2019-09-25 23:03:30 +02:00
Christian Heimes bfd0c963d8 bpo-38271: encrypt private key test files with AES256 (GH-16385)
The private keys for test_ssl were encrypted with 3DES in traditional
PKCS#5 format. 3DES and the digest algorithm of PKCS#5 are blocked by
some strict crypto policies. Use PKCS#8 format with AES256 encryption
instead.

Signed-off-by: Christian Heimes <christian@python.org>



https://bugs.python.org/issue38271



Automerge-Triggered-By: @tiran
2019-09-25 08:55:02 -07:00
Christian Heimes c64a1a61e6 bpo-38270: Check for hash digest algorithms and avoid MD5 (GH-16382)
Make it easier to run and test Python on systems with restrict crypto policies:

* add requires_hashdigest to test.support to check if a hash digest algorithm is available and working
* avoid MD5 in test_hmac
* replace MD5 with SHA256 in test_tarfile
* mark network tests that require MD5 for MD5-based digest auth or CRAM-MD5


https://bugs.python.org/issue38270
2019-09-25 07:30:20 -07:00
PatrikKopkan 1dc1acbd73 bpo-37064: Add option -a to pathfix.py tool (GH-15717)
Add option -a to Tools/Scripts/pathfix.py script: add flags.
2019-09-25 14:26:28 +02:00
Yury Selivanov edad4d89e3 bpo-38248: Fix inconsistent immediate asyncio.Task cancellation (GH-16330) 2019-09-25 03:32:08 -07:00
Serhiy Storchaka f163aeaa8c
bpo-38219: Optimize dict creating and updating by a dict. (GH-16268) 2019-09-25 09:47:00 +03:00
Dong-hee Na ad7736faf5 bpo-38265: Update os.pread to accept the length type as Py_ssize_t. (GH-16359) 2019-09-25 08:47:04 +03:00
Samuel Freilich b5a7a4f0c2 bpo-36871: Handle spec errors in assert_has_calls (GH-16005)
The fix in PR 13261 handled the underlying issue about the spec for specific methods not being applied correctly, but it didn't fix the issue that was causing the misleading error message.

The code currently grabs a list of responses from _call_matcher (which may include exceptions). But it doesn't reach inside the list when checking if the result is an exception. This results in a misleading error message when one of the provided calls does not match the spec.


https://bugs.python.org/issue36871



Automerge-Triggered-By: @gpshead
2019-09-24 12:08:31 -07:00
Victor Stinner 1ce152a42e
bpo-38234: Py_SetPath() uses the program full path (GH-16357)
Py_SetPath() now sets sys.executable to the program full path
(Py_GetProgramFullPath()), rather than to the program name
(Py_GetProgramName()).

Fix also memory leaks in pathconfig_set_from_config().
2019-09-24 17:44:15 +02:00
Victor Stinner b0e1ae5f54
bpo-37123: multiprocessing test_mymanager() accepts SIGTERM (GH-16349)
Multiprocessing test test_mymanager() now also expects -SIGTERM, not
only exitcode 0.

bpo-30356: BaseManager._finalize_manager() sends SIGTERM to the
manager process if it takes longer than 1 second to stop, which
happens on slow buildbots.
2019-09-24 14:19:48 +02:00
Victor Stinner 99799c7220
bpo-38212: Increase MP test_queue_feeder_donot_stop_onexc() timeout (GH-16348)
Multiprocessing tests: increase test_queue_feeder_donot_stop_onexc()
timeout from 1 to 60 seconds.
2019-09-24 12:47:49 +02:00
Lisa Roach ef04851775
bpo-38136: Updates await_count and call_count to be different things (GH-16192) 2019-09-23 20:49:40 -07:00
Victor Stinner 9c42f8cda5
bpo-38234: Fix _PyConfig_InitPathConfig() (GH-16335)
* _PyConfig_InitPathConfig() now starts by copying the global path
  configuration, and then override values set in PyConfig.
* _PyPathConfig_Calculate() implementations no longer override
  _PyPathConfig fields which are already computed. For example,
  if _PyPathConfig.prefix is not NULL, leave it unchanged.
* If Py_SetPath() has been called, _PyConfig_InitPathConfig() doesn't
  call _PyPathConfig_Calculate() anymore.
* _PyPathConfig_Calculate() no longer uses PyConfig,
  except to initialize PyCalculatePath structure.
* pathconfig_calculate(): remove useless temporary
  "_PyPathConfig new_config" variable.
* calculate_module_search_path(): remove hack to workaround memory
  allocation failure, call Py_FatalError() instead.
* Fix get_program_full_path(): handle memory allocation failure.
2019-09-23 18:47:29 +02:00
Victor Stinner fcdb027234
bpo-38236: Dump path config at first import error (GH-16300)
Python now dumps path configuration if it fails to import the Python
codecs of the filesystem and stdio encodings.
2019-09-23 14:45:47 +02:00
Ammar Askar 87d6cd3604 bpo-38237: Make pow's arguments have more descriptive names and be keyword passable (GH-16302)
Edit: `math.pow` changes removed on Mark's request.


https://bugs.python.org/issue38237



Automerge-Triggered-By: @rhettinger
2019-09-20 21:28:49 -07:00
Victor Stinner e267793aa4
bpo-38234: Fix PyConfig_Read() when Py_SetPath() was called (GH-16298)
* If Py_SetPath() has been called, _PyConfig_InitPathConfig() now
  uses its value.
* Py_Initialize() now longer copies path configuration from PyConfig
  to the global path configuration (_Py_path_config).
2019-09-21 01:50:16 +02:00
Victor Stinner b1542583be
bpo-38205: Py_UNREACHABLE() calls Py_FatalError() (GH-16290) 2019-09-20 23:10:16 +02:00
Ram Rachum 9c2682efc6 bpo-37937: Mention frame.f_trace in sys.settrace docs (GH-15439)
Mention frame.f_trace in sys.settrace docs, as well as the fact you still
need to call `sys.settrace` to enable the tracing machinery before setting
`frame.f_trace` will have any effect.
2019-09-21 01:19:02 +10:00
Lisa Roach 8b03f943c3
bpo-38093: Correctly returns AsyncMock for async subclasses. (GH-15947) 2019-09-19 21:04:18 -07:00
Michael Selik 2702638eab bpo-34002: Minor efficiency and clarity improvements in email package. (GH-7999)
* Check intersection of two sets explicitly

Comparing ``len(a) > ``len(a - b)`` is essentially looking for an
intersection between the two sets. If set ``b`` does not intersect ``a``
then ``len(a - b)`` will be equal to ``len(a)``. This logic is more
clearly expressed as ``a & b``.

* Change while/pop to a for-loop

Copying the list, then repeatedly popping the first element was
unnecessarily slow. I also cleaned up a couple other inefficiencies.
There's no need to unpack a tuple, then re-pack and append it. The list
can be created with the first element instead of empty. Secondly, the
``endswith`` method returns a bool, so there's no need for an if-
statement to set ``encoding`` to True or False.

* Use set.intersection to check for intersections

``a.intersection(b)`` method is more clear of purpose than ``not
a.isdisjoint(b)`` and avoids an unnecessary set construction that ``a &
set(b)`` performs.

* Use not isdisjoint instead of intersection

While it reads slightly worse, the isdisjoint method will stop when it
finds a counterexample and returns a bool, rather than looping over the
entire iterable and constructing a new set.
2019-09-19 20:25:55 -07:00
Eddie Elizondo 3368f3c6ae bpo-38140: Make dict and weakref offsets opaque for C heap types (#16076)
* Make dict and weakref offsets opaque for C heap types

* Add news
2019-09-19 17:29:05 +01:00
t k 96b1c59c71 bpo-38155: Add __all__ to datetime module (GH-16203)
https://bugs.python.org/issue38155
2019-09-19 14:34:41 +01:00
Kyle Stanley 9fdc64cf12 bpo-34037: Fix test_asyncio failure and add loop.shutdown_default_executor() (GH-15735) 2019-09-19 15:47:22 +03:00
Serhiy Storchaka 8fc5839a9d
bpo-38191: Turn warnings into errors in NamedTuple() and TypedDict(). (GH-16238) 2019-09-17 22:41:55 +03:00
Serhiy Storchaka 2bf31ccab3
bpo-38191: Accept arbitrary keyword names in NamedTuple() and TypedDict(). (GH-16222)
This includes such names as "cls", "self", "typename", "_typename",
"fields" and "_fields".
Passing positional arguments by keyword is deprecated.
2019-09-17 21:22:00 +03:00
Andrew Svetlov c275312a62 bpo-38013: make async_generator_athrow object tolerant to throwing exceptions (GH-16070)
Even when the helper is not started yet.

This behavior follows conventional generator one.
There is no reason for `async_generator_athrow` to handle `gen.throw()` differently.



https://bugs.python.org/issue38013
2019-09-17 05:59:49 -07:00
Abraham Toriz Cruz 5f5f11faf9 bpo-37828: Fix default mock_name in unittest.mock.assert_called error (GH-16166)
In the format string for assert_called the evaluation order is incorrect and hence for mock's without name, 'None' is printed whereas it should be 'mock' like for other messages. The error message is ("Expected '%s' to have been called." % self._mock_name or 'mock').
2019-09-17 12:16:08 +01:00
Serhiy Storchaka f669581a95
bpo-38185: Fixed case-insensitive string comparison in sqlite3.Row indexing. (GH-16190) 2019-09-17 09:20:56 +03:00
Terry Jan Reedy dfd34a9cd5
bpo-35379: When exiting IDLE, catch any AttributeError. (GH-16212)
One happens when EditorWindow.close is called twice.
Printing a traceback, when IDLE is run from a terminal,
is useless and annoying.
2019-09-17 02:05:04 -04:00
Terry Jan Reedy 0048afc16a
bpo-38183: Test_idle ignores user config directory GH-16198)
It no longer tries to create or access .idlerc or any files within.
Users must run IDLE to discover problems with saving settings.
2019-09-16 19:04:21 -04:00
Christian Heimes 724f1a5723 bpo-33936: Don't call obsolete init methods with OpenSSL 1.1.0+ (GH-16140)
``OPENSSL_VERSION_1_1`` was never defined in ``_hashopenssl.c``. 


https://bugs.python.org/issue33936
2019-09-16 12:10:05 -07:00
Serhiy Storchaka 8debfa5040
bpo-38175: Fix a memory leak in comparison of sqlite3.Row objects. (GH-16155) 2019-09-16 20:15:18 +03:00
Steve Dower 3ab73f6bbf
bpo-38117: Update bundled Windows OpenSSL to 1.1.1d (GH-16184) 2019-09-16 15:21:16 +01:00
Ned Deily 24d1597e43
bpo-38117: Updated OpenSSL to 1.1.1d in macOS installer. (GH-16170) 2019-09-16 03:38:27 +01:00
Dino Viehland f919054e53
bpo-38116: Convert select module to PEP-384 (#15971)
* Convert select module to PEP-384

Summary: Do the necessary versions to be Pyro-compatible, including migrating `PyType_Ready` to `PyType_FromSpec` and moving static data into a new `_selectstate` struct.

* 📜🤖 Added by blurb_it.

* Fixup Mac OS/X build
2019-09-14 15:20:27 +01:00
Serhiy Storchaka 279f44678c
bpo-37206: Unrepresentable default values no longer represented as None. (GH-13933)
In ArgumentClinic, value "NULL" should now be used only for unrepresentable default values
(like in the optional third parameter of getattr). "None" should be used if None is accepted
as argument and passing None has the same effect as not passing the argument at all.
2019-09-14 12:24:05 +03:00
Serhiy Storchaka bf169915ec
bpo-38005: Fixed comparing and creating of InterpreterID and ChannelID. (GH-15652)
* Fix a crash in comparing with float (and maybe other crashes).
* They are now never equal to strings and non-integer numbers.
* Comparison with a large number no longer raises OverflowError.
* Arbitrary exceptions no longer silenced in constructors and comparisons.
* TypeError raised in the constructor contains now the name of the type.
* Accept only ChannelID and int-like objects in channel functions.
* Accept only InterpreterId, int-like objects and str in the InterpreterId constructor.
* Accept int-like objects, not just int in interpreter related functions.
2019-09-13 22:50:27 +03:00
plokmijnuhby e082e7cbe4 bpo-37953: Fix ForwardRef hash and equality checks (GH-15400)
Ideally if we stick a ForwardRef in a dictionary we would like to reliably be able to get it out again.

https://bugs.python.org/issue37953
2019-09-13 20:40:54 +01:00
Steve Dower f2b7556ef8 bpo-38092: Reduce overhead when using multiprocessing in a Windows virtual environment (GH-16098)
https://bugs.python.org/issue38092
2019-09-13 09:40:19 -07:00
Gregory P. Smith 5b9ff7a0dc
bpo-34706: Preserve subclassing in inspect.Signature.from_callable (GH-16108)
https://bugs.python.org/issue34706

Specifically in the case of a class that does not override its
constructor signature inherited from object.

These are Buck Evan @bukzor's changes cherrypicked from GH-9344.
2019-09-13 17:13:51 +01:00
Joannah Nanjekye 2ccb50cf77 bpo-37449: Move ensurepip off of pkgutil and to importlib.resources (GH-15109)
Move ensurepip off of pkgutil and to importlib.resources.


https://bugs.python.org/issue37449
2019-09-13 09:01:20 -07:00
Elizabeth Uselton d6a9d17d8b bpo-37555: Update _CallList.__contains__ to respect ANY (#14700)
* Flip equality to use mock calls' __eq__

* bpo-37555: Regression test demonstrating assert_has_calls not working with ANY and spec_set

Co-authored-by: Neal Finne <neal@nealfinne.com>

* Revert "Flip equality to use mock calls' __eq__"

This reverts commit 94ddf54c5a.

* bpo-37555: Add regression tests for mock ANY ordering issues

Add regression tests for whether __eq__ is order agnostic on _Call and _CallList, which is useful for comparisons involving ANY, especially if the ANY comparison is to a class not defaulting __eq__ to NotImplemented.

Co-authored-by: Neal Finne <neal@nealfinne.com>

* bpo-37555: Fix _CallList and _Call order sensitivity

_Call and _CallList depend on ordering to correctly process that an object being compared to ANY with __eq__ should return True. This fix updates the comparison to check both a == b and b == a and return True if either condition is met, fixing situations from the tests in the previous two commits where assertEqual would not be commutative if checking _Call or _CallList objects. This seems like a reasonable fix considering that the Python data model specifies that if an object doesn't know how to compare itself to another object it should return NotImplemented, and that on getting NotImplemented from a == b, it should try b == a, implying that good behavior for __eq__ is commutative. This also flips the order of comparison in _CallList's __contains__ method, guaranteeing ANY will be on the left and have it's __eq__ called for equality checking, fixing the interaction between assert_has_calls and ANY.

Co-author: Neal Finne <neal@neal.finne.com>

* bpo-37555: Ensure _call_matcher returns _Call object

* Adding ACK and news entry

* bpo-37555: Replacing __eq__ with == to sidestep NotImplemented

bool(NotImplemented) returns True, so it's necessary to use ==
instead of __eq__ in this comparison.

* bpo-37555: cleaning up changes unnecessary to the final product

* bpo-37555: Fixed call on bound arguments to respect args and kwargs

* Revert "bpo-37555: Add regression tests for mock ANY ordering issues"

This reverts commit 49c5310ad4.

* Revert "bpo-37555: cleaning up changes unnecessary to the final product"

This reverts commit 18e964ba01.

* Revert "bpo-37555: Replacing __eq__ with == to sidestep NotImplemented"

This reverts commit f295eaca5b.

* Revert "bpo-37555: Fix _CallList and _Call order sensitivity"

This reverts commit 874fb697b8.

* Updated NEWS.d

* bpo-37555: Add tests checking every function using _call_matcher both with and without spec

* bpo-37555: Ensure all assert methods using _call_matcher are actually passing calls

* Remove AnyCompare and use call objects everywhere.

* Revert "Remove AnyCompare and use call objects everywhere."

This reverts commit 24973c0b32.

* Check for exception in assert_any_await
2019-09-13 16:54:32 +01:00
Zackery Spytz b761e3aed1 bpo-25068: urllib.request.ProxyHandler now lowercases the dict keys (GH-13489) 2019-09-13 15:07:07 +01:00
Doyle Rowland 0519d497b0 closes bpo-36002: Use AC_PATH_TOOL to find llvm-profdata and llvm-ar. (GH-14998) 2019-09-13 14:38:07 +01:00
Christian Heimes 995b5d38e7 bpo-38153: Normalize hashlib algorithm names (GH-16083)
Signed-off-by: Christian Heimes <christian@python.org>
2019-09-13 14:31:19 +01:00
Andrew Svetlov 9eb35ab0d7
bpo-38148: Add slots to asyncio transports (GH-16077)
* bpo-38148: Add slots to asyncio transports

* Update Misc/NEWS.d/next/Library/2019-09-13-08-55-43.bpo-38148.Lnww6D.rst

Co-Authored-By: Kyle Stanley <aeros167@gmail.com>
2019-09-13 15:18:46 +03:00
Ashwin Ramaswami ff2e182865 bpo-12707: deprecate info(), geturl(), getcode() methods in favor of headers, url, and status properties for HTTPResponse and addinfourl (GH-11447)
Co-Authored-By: epicfaace <aramaswamis@gmail.com>
2019-09-13 12:40:07 +01:00
Xtreak bb41147eab bpo-12144: Handle cookies with expires attribute in CookieJar.make_cookies (GH-13921)
Handle time comparison for cookies with `expires` attribute when `CookieJar.make_cookies` is called.

Co-authored-by: Demian Brecht <demianbrecht@gmail.com>


https://bugs.python.org/issue12144



Automerge-Triggered-By: @asvetlov
2019-09-13 04:29:00 -07:00
Rémi Lapeyre 6a517c6749 bpo-8538: Add support for boolean actions to argparse (GH-11478)
Co-Authored-By: remilapeyre <remi.lapeyre@henki.fr>
2019-09-13 11:17:43 +01:00
Dino Viehland 04f0bbfbed bpo-38075: Port _randommodule.c to PEP-384 (GH-15798)
- Migrate `Random_Type` to `PyType_FromSpec`
- To simulate an old use of `PyLong_Type.tp_as_number->nb_absolute`, I added
  code to the module init function to stash `int.__abs__` for later
  use. Ideally we'd use `PyType_GetSlot()` instead, but it doesn't currently
  work for static types in CPython, and implementing it just for this case
  doesn't seem worth it.
- Do exact check for long and dispatch to PyNumber_Absolute, use vector call when not exact.
2019-09-13 03:12:27 -07:00
Jakub Kulík 42671aea2d bpo-37785: Fix xgettext warning in argparse (GH-15161) 2019-09-13 11:25:32 +02:00
Christian Heimes 7cad53e6b0 bpo-9216: Add usedforsecurity to hashlib constructors (GH-16044)
The usedforsecurity keyword only argument added to the hash constructors is useful for FIPS builds and similar restrictive environment with non-technical requirements that legacy algorithms be forbidden by their implementations without being explicitly annotated as not being used for any security related purposes.  Linux distros with FIPS support benefit from this being standard rather than making up their own way(s) to do it.

Contributed and Signed-off-by: Christian Heimes christian@python.org
2019-09-12 19:30:00 -05:00
Steve Dower ed93a8852d
bpo-38133: Allow py.exe launcher to locate installations from the Microsoft Store (GH-16025) 2019-09-12 18:16:50 +01:00
Patrick McLean 2b2ead7438 bpo-36046: Add user and group parameters to subprocess (GH-11950)
* subprocess: Add user, group and extra_groups paremeters to subprocess.Popen

This adds a `user` parameter to the Popen constructor that will call
setreuid() in the child before calling exec(). This allows processes
running as root to safely drop privileges before running the subprocess
without having to use a preexec_fn.

This also adds a `group` parameter that will call setregid() in
the child process before calling exec().

Finally an `extra_groups` parameter was added that will call
setgroups() to set the supplimental groups.
2019-09-12 18:15:44 +01:00
Daniel Hahler 855df7f273 closes bpo-37803: pdb: fix handling of options (--help / --version) (GH-15193)
The "--" should not be included with long options passed to
getopt.getopt.

Fixes https://bugs.python.org/issue37803
2019-09-12 16:46:37 +01:00
Sergey Fedoseev c6734ee7c5 bpo-37802: Slightly improve perfomance of PyLong_FromUnsigned*() (GH-15192) 2019-09-12 15:41:14 +01:00
animalize 4ffd05d7ec bpo-21872: fix lzma library decompresses data incompletely (GH-14048)
* 1. add test case with wrong behavior
* 2. fix bug when max_length == -1
* 3. allow b"" as valid input data for decompress_buf()
* 4. when max_length >= 0, let needs_input mechanism works
* add more asserts to test case
2019-09-12 15:20:37 +01:00
Berker Peksag 2f1b857562 bpo-36991: Fix incorrect exception escaping ZipFile.extract() (GH-13632) 2019-09-12 15:13:44 +01:00
Christian Heimes 5a4f82f457 bpo-38132: Simplify _hashopenssl code (GH-16023)
Signed-off-by: Christian Heimes <christian@python.org>
2019-09-12 13:42:07 +01:00
Brandt Bucher 224b8aaa7e bpo-26868: Fix example usage of PyModule_AddObject. (#15725)
* Add a note to the PyModule_AddObject docs.

* Correct example usages of PyModule_AddObject.

* Whitespace.

* Clean up wording.

* 📜🤖 Added by blurb_it.

* First code review.

* Add < 0 in the tests with PyModule_AddObject
2019-09-12 13:11:20 +01:00
Christian Heimes 64117e059b bpo-38134: Remove PKBDF2_HMAC_fast from _hashopenssl (GH-16028)
Signed-off-by: Christian Heimes <christian@python.org>
2019-09-12 12:46:37 +01:00
Hai Shi f545638b57 bpo-9938: Add optional keyword argument exit_on_error to argparse.ArgumentParser (GH-15362)
Co-Authored-by: Xuanji Li <xuanji@gmail.com>


https://bugs.python.org/issue9938



Automerge-Triggered-By: @matrixise
2019-09-12 03:56:05 -07:00
Divij Rajkumar 692a0dc915 bpo-38008: Move builtin protocol whitelist to mapping instead of list (GH-15647)
Fixes https://bugs.python.org/issue38008
2019-09-12 11:13:51 +01:00
bggardner 954900a3f9 closes bpo-37405: Make socket.getsockname() always return a tuple for AF_CAN. (GH-14392)
This fixes a regression from 3.5. In recent releases, `getsockname()` in the AF_CAN case has returned a string.
2019-09-12 11:02:48 +01:00
Jason R. Coombs 8ed6503eca bpo-38121: Sync importlib.metadata with 0.22 backport (GH-15993)
* bpo-38121: Sync importlib.metadata with 0.22 backport

* 📜🤖 Added by blurb_it.
2019-09-12 10:29:11 +01:00
Zackery Spytz 421a72af4d bpo-21120: Exclude Python-ast.h, ast.h and asdl.h from the limited API (#14634)
The PyArena type is not part of the limited API, so these headers
shouldn't be part of it either.
2019-09-12 10:27:14 +01:00
Greg Price a65678c5c9 bpo-37760: Convert from length-18 lists to a dataclass, in makeunicodedata. (GH-15265)
Now the fields have names!  Much easier to keep straight as a
reader than the elements of an 18-tuple.

Runs about 10-15% slower: from 10.8s to 12.3s, on my laptop.
Fortunately that's perfectly fine for this maintenance script.
2019-09-12 10:23:43 +01:00
ewosborne f9c95a4ba2 bpo-32820: __format__ method for ipaddress (#5627)
* bits method and test_bits

* Cleaned up assert string

* blurb

* added docstring

* Faster method, per Eric Smith

* redoing as __format__

* added ipv6 method

* test cases and cleanup

* updated news

* cleanup and NEWS.d

* cleaned up old NEWS

* removed cut and paste leftover

* one more cleanup

* moved to regexp, moved away from v4- and v6-specific versions of __format__

* More cleanup, added ipv6 test cases

* more cleanup

* more cleanup

* cleanup

* cleanup

* cleanup per review, part 1

* addressed review comments around help string and regexp matching

* wrapped v6 test strings. contiguous integers: break at 72char. with underscores: break so that it looks clean.

*  's' and '' tests for pv4 and ipv6

* whitespace cleanup

* Remove trailing whitespace

* Remove more trailing whitespace

* Remove an excess blank line
2019-09-12 10:03:31 +01:00
Joannah Nanjekye 92777d5e5a bpo-18578: Rename and document test.bytecode_helper as test.support.bytecode_helper (GH-15168)
Rename and document test.bytecode_helper as test.support.bytecode_helper
2019-09-12 10:02:59 +01:00
Benjamin Peterson 39de95b746
closes bpo-38124: Fix bounds check in PyState_AddModule. (GH-16007)
The >=, checking whether a module index was in already in the module-by-index list, needed to be strict.

Also, fold nested ifs into one and fix some bad spacing.
2019-09-12 00:43:22 +01:00