Commit Graph

22894 Commits

Author SHA1 Message Date
Victor Stinner d7c87d982d
bpo-37400: Fix test_os.test_chown() (GH-14374)
Use os.getgroups() rather than grp.getgrall() to get groups.
Rename also the test to test_chown_gid().
2019-06-25 17:06:24 +02:00
Victor Stinner 3939c321c9
bpo-20443: _PyConfig_Read() gets the absolute path of run_filename (GH-14053)
Python now gets the absolute path of the script filename specified on
the command line (ex: "python3 script.py"): the __file__ attribute of
the __main__ module, sys.argv[0] and sys.path[0] become an absolute
path, rather than a relative path.

* Add _Py_isabs() and _Py_abspath() functions.
* _PyConfig_Read() now tries to get the absolute path of
  run_filename, but keeps the relative path if _Py_abspath() fails.
* Reimplement os._getfullpathname() using _Py_abspath().
* Use _Py_isabs() in getpath.c.
2019-06-25 15:02:43 +02:00
Serhiy Storchaka 894263ba80
bpo-24214: Fixed the UTF-8 and UTF-16 incremental decoders. (GH-14304)
* The UTF-8 incremental decoders fails now fast if encounter
  a sequence that can't be handled by the error handler.
* The UTF-16 incremental decoders with the surrogatepass error
  handler decodes now a lone low surrogate with final=False.
2019-06-25 11:54:18 +03:00
Raymond Hettinger 1791128677
bpo-36546: Mark first argument as position only (GH-14363) 2019-06-25 04:39:22 +02:00
Pablo Galindo 3f5b9088b0
bpo-37394: Fix pure Python implementation of the queue module (GH-14351) 2019-06-25 02:53:30 +01:00
Victor Stinner 36456df138
bpo-37392: Remove sys.setcheckinterval() (GH-14355)
Remove sys.getcheckinterval() and sys.setcheckinterval() functions.
They were deprecated since Python 3.2. Use sys.getswitchinterval()
and sys.setswitchinterval() instead.

Remove also check_interval field of the PyInterpreterState structure.
2019-06-25 03:01:08 +02:00
David K. Hess 9fc720e5e4 bpo-4963: Fix for initialization and non-deterministic behavior issues in mimetypes (GH-3062) 2019-06-24 16:46:59 -07:00
animalize 8bd2872adb bpo-25361: Enable SSE2 instructions for Windows 32-bit build (GH-12438) 2019-06-24 16:43:26 -07:00
animalize 7fd2ba354e bpo-35360: Update Windows builds to use SQLite 3.28.0 (GH-14179) 2019-06-24 16:22:14 -07:00
ziheng 6ffd9b05df bpo-32627: Fix compile error when conflicting `_uuid` headers included (GH-11751) 2019-06-24 13:59:50 -04:00
Steve Dower 60419a7e96
bpo-37363: Add audit events for a range of modules (GH-14301) 2019-06-24 08:42:54 -07:00
Jeroen Demeyer a8b27e623d bpo-36974: inherit tp_vectorcall_offset unconditionally (GH-13858) 2019-06-24 12:41:05 +02:00
Victor Stinner 47fbc4e45b
bpo-37359: Add --cleanup option to python3 -m test (GH-14332)
* regrtest: Add --cleanup option to remove "test_python_*" directories
  of previous failed test jobs.
* Add "make cleantest" to run "python3 -m test --cleanup".
2019-06-24 12:03:00 +02:00
Gabe Appleton 2ac3bab2a6 bpo-37345: Add formal UDPLITE support (GH-14258)
At the moment you can definitely use UDPLITE sockets on Linux systems, but it would be good if this support were formalized such that you can detect support at runtime easily.

At the moment, to make and use a UDPLITE socket requires something like the following code:

```
>>> import socket
>>> a = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, 136)
>>> b = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, 136)
>>> a.bind(('localhost', 44444))
>>> b.sendto(b'test'*256, ('localhost', 44444))
>>> b.setsockopt(136, 10, 16)
>>> b.sendto(b'test'*256, ('localhost', 44444))
>>> b.setsockopt(136, 10, 32)
>>> b.sendto(b'test'*256, ('localhost', 44444))
>>> b.setsockopt(136, 10, 64)
>>> b.sendto(b'test'*256, ('localhost', 44444))
```

If you look at this through Wireshark, you can see that the packets are different in that the checksums and checksum coverages change.

With the pull request that I am submitting momentarily, you could do the following code instead:

```
>>> import socket
>>> a = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDPLITE)
>>> b = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDPLITE)
>>> a.bind(('localhost', 44444))
>>> b.sendto(b'test'*256, ('localhost', 44444))
>>> b.set_send_checksum_coverage(16)
>>> b.sendto(b'test'*256, ('localhost', 44444))
>>> b.set_send_checksum_coverage(32)
>>> b.sendto(b'test'*256, ('localhost', 44444))
>>> b.set_send_checksum_coverage(64)
>>> b.sendto(b'test'*256, ('localhost', 44444))
```

One can also detect support for UDPLITE just by checking

```
>>> hasattr(socket, 'IPPROTO_UDPLITE')
```


https://bugs.python.org/issue37345
2019-06-24 02:58:56 -07:00
Inada Naoki 770847a7db
bpo-37348: optimize decoding ASCII string (GH-14283)
`_PyUnicode_Writer` is a relatively complex structure.  Initializing it is significant overhead when decoding short ASCII string.
2019-06-24 12:30:24 +09:00
Jörn Heissler c8a35417db bpo-35224: Reverse evaluation order of key: value in dict comprehensions (GH-14139)
… as proposed in PEP 572; key is now evaluated before value.





https://bugs.python.org/issue35224
2019-06-22 07:40:55 -07:00
Steve Dower 184f3d4f39 bpo-37364: Use io.open_code() to read .pth files (GH-14299)
https://bugs.python.org/issue37364
2019-06-21 15:16:46 -07:00
Steve Dower f5690925df bpo-37351: Removes libpython38.a from standard Windows distribution (#14276) 2019-06-21 14:28:46 -07:00
Victor Stinner e56a123fd0
bpo-37362: test_gdb now ignores stderr (GH-14287)
test_gdb no longer fails if it gets an "unexpected" message on
stderr: it now ignores stderr. The purpose of test_gdb is to test
that python-gdb.py commands work as expected, not to test gdb.
2019-06-21 23:17:30 +02:00
Joannah Nanjekye a0d73a143a bpo-30202 : Update test.test_importlib.test_abc to test find_spec() (GH-12847) 2019-06-21 11:17:00 -07:00
Zackery Spytz 08286d52b2 bpo-37316: mmap.mmap() passes the wrong variable to PySys_Audit() (GH-14152)
Also, add a missing call to va_end() in PySys_Audit().
2019-06-21 08:31:59 -07:00
Michael Felt 08970cb03c bpo-36210: update optional extension handling for AIX (GH-12202)
* Switch to officially supported curses from 3rd-party ASIS supported ncurses
* stop saying optional modules osaudiodev and spwd are missing on AIX

Patch by M.Felt
2019-06-21 23:58:00 +10:00
Eric V. Smith 0c48618cc0
Fix bpo number in News file. (GH-14260) 2019-06-20 04:22:28 -04:00
Terry Jan Reedy 54cf2e0780
bpo-37325: Fix focus traversal for 2 IDLE dialogs (#14209)
Tab now moves focus across and down for Help Source and Custom Run.
2019-06-18 17:08:24 -04:00
Paul Ganssle d287215df5
Add pganssle to CODEOWNERS and ACKS (GH-14138)
Also adds abalkin to CODEOWNERS for date and time related files.
2019-06-18 19:57:45 +01:00
Mike Gleen 6b9c204ee7 bpo-34903: Document that some strptime formats only require 1 digit (GH-14149)
For datetime.datetime.strptime(), the leading zero for some two-digit formats is optional.

This adds a footnote to the strftime/strptime documentation to reflect this fact, and adds some tests to ensure that it is true.

bpo-34903
2019-06-18 19:14:57 +01:00
animalize d8f336fdc1 bpo-35360: Update macOS installer to use SQLite 3.28.0 (GH-14180) 2019-06-18 05:25:19 -04:00
Ned Deily f3fb8393e3
bpo-34631: Updated OpenSSL to 1.1.1c in macOS installer (GH-14187) 2019-06-18 04:17:33 -04:00
Cheryl Sabella 201bc2d18b bpo-5680: IDLE: Customize running a module (GH-13763)
The initialize options are 1) add command line options, which are appended to sys.argv as if passed on a real command line, and 2) skip the shell restart. The customization dialog is accessed by a new entry on the Run menu.
2019-06-17 22:24:10 -04:00
Victor Stinner 343ed0ffe0
bpo-37189: Export old PyRun_XXX() functions (#14142)
Many PyRun_XXX() functions like PyRun_String() were no longer
exported in libpython38.dll by mistake. Export them again to fix the
ABI compatibiliy.
2019-06-18 00:15:13 +02:00
Victor Stinner ac7b1a3f32
bpo-37320: Remove openfp() of aifc, sunau and wave (GH-14169)
aifc.openfp() alias to aifc.open(), sunau.openfp() alias to
sunau.open(), and wave.openfp() alias to wave.open() have been
removed. They were deprecated since Python 3.7.
2019-06-18 00:00:24 +02:00
Terry Jan Reedy 8fac122109
bpo-37321: Edit IDLE subprocess connection error messages. (#14170)
Mainly, add a doc reference to message in pyshell.
2019-06-17 17:23:28 -04:00
Tal Einat 5bff3c86ab bpo-37039: Make IDLE's Zoom Height adjust to users' screens (GH-13678)
Measure required height by quickly maximizing once per screen.
A search for a better method failed.
2019-06-17 15:41:00 -04:00
Steve Dower a268edd6a4
bpo-34631: Updated OpenSSL to 1.1.1c in Windows installer (GH-14163) 2019-06-17 11:36:08 -07:00
Cooper Lees 4acdbf11b1 bpo-34556: Add --upgrade-deps to venv module (#13100)
Add --upgrade-deps to venv module
- This allows for pip + setuptools to be automatically upgraded to the latest version on PyPI
- Update documentation to represent this change

bpo-34556: Add --upgrade to venv module
2019-06-17 19:18:13 +01:00
Vinay Sajip ca7b504a4d
bpo-37111: Add 'encoding' and 'errors' parameters to logging.basicCon… (GH-14008) 2019-06-17 17:40:52 +01:00
Serhiy Storchaka 231aad3849
bpo-37315: Deprecate accepting floats in math.factorial(). (GH-14147) 2019-06-17 16:57:27 +03:00
Victor Stinner 2ff58a24e8
bpo-37194: Add a new public PyObject_CallNoArgs() function (GH-13890)
Add a new public PyObject_CallNoArgs() function to the C API: call a
callable Python object without any arguments.

It is the most efficient way to call a callback without any argument.
On x86-64, for example, PyObject_CallFunctionObjArgs(func, NULL)
allocates 960 bytes on the stack per call, whereas
PyObject_CallNoArgs(func) only allocates 624 bytes per call.

It is excluded from stable ABI 3.8.

Replace private _PyObject_CallNoArg() with public
PyObject_CallNoArgs() in C extensions: _asyncio, _datetime,
_elementtree, _pickle, _tkinter and readline.
2019-06-17 14:27:23 +02:00
Victor Stinner 8bf08ee45b
bpo-37312: Remove _dummy_thread and dummy_threading modules (GH-14143)
Remove _dummy_thread and dummy_threading modules. These modules
were deprecated since Python 3.7 which requires threading support.
2019-06-17 14:18:43 +02:00
Jeroen Demeyer 2e9954d347 bpo-36922: use Py_TPFLAGS_METHOD_DESCRIPTOR in lookup_maybe_method() (GH-13865) 2019-06-17 20:53:20 +09:00
Zackery Spytz 28fca0c422 bpo-37267: Do not check for FILE_TYPE_CHAR in os.dup() on Windows (GH-14051)
On Windows, os.dup() no longer creates an inheritable fd when handling a
character file.
2019-06-17 09:17:14 +02:00
Hai Shi c83356cae2 closes bpo-37300: Remove unnecessary Py_XINCREF in classobject.c. (GH-14120) 2019-06-16 13:19:19 -07:00
Jeroen Demeyer 5600b5e1b2 bpo-28805: document METH_FASTCALL (GH-14079) 2019-06-17 02:03:22 +09:00
Rémi Lapeyre 8047e0e1c6 bpo-35922: Fix RobotFileParser when robots.txt has no relevant crawl delay or request rate (GH-11791)
Co-Authored-By: Tal Einat <taleinat+github@gmail.com>
2019-06-16 09:48:57 +03:00
Michael Felt 3a1d50e7e5 bpo-28009: Fix uuid SkipUnless logic to be based on platform programs capable of introspection (GH-12777)
uuid could try fallback methods that had no chance of working on a particular
platform, and this could cause spurious test failures, as well as degraded
performance as fallback options were tried and failed.

This fixes both the uuid module and its test's SkipUnless logic to use a
prefiltered list of techniques that may at least potentially work on that platform.

Patch by Michael Felt (aixtools).
2019-06-16 01:52:29 +10:00
Andrew Svetlov 0237265e82
Use threadpool for reading from file in sendfile fallback mode (#14076) 2019-06-15 14:05:35 +03:00
Andrew Svetlov ef2152354f
bpo-37279: Fix asyncio sendfile support when extra data are sent in fallback mode. (GH-14075) 2019-06-15 14:05:08 +03:00
Victor Stinner 066e5b1a91
bpo-37266: Daemon threads are now denied in subinterpreters (GH-14049)
In a subinterpreter, spawning a daemon thread now raises an
exception. Daemon threads were never supported in subinterpreters.
Previously, the subinterpreter finalization crashed with a Pyton
fatal error if a daemon thread was still running.

* Add _thread._is_main_interpreter()
* threading.Thread.start() now raises RuntimeError if the thread is a
  daemon thread and the method is called from a subinterpreter.
* The _thread module now uses Argument Clinic for the new function.
* Use textwrap.dedent() in test_threading.SubinterpThreadingTests
2019-06-14 18:55:22 +02:00
Zackery Spytz 9765efcb39 bpo-19865: ctypes.create_unicode_buffer() supports non-BMP strings on Windows (GH-14081) 2019-06-14 17:53:59 +02:00
Andrew Svetlov f0749da9a5 bpo-35998: Avoid TimeoutError in test_asyncio: test_start_tls_server_1() (GH-14080) 2019-06-14 17:26:24 +02:00
Victor Stinner 07559450b2
bpo-37278: Fix test_asyncio ProactorLoopCtrlC (GH-14074)
Join the thread to prevent leaking a running thread and leaking a
reference.

Cleanup also the test:

* asyncioWindowsProactorEventLoopPolicy became the default policy,
  there is no need to set it manually.
* Only start the thread once the loop is running.
* Use a shorter sleep in the thread (100 ms rather than 1 sec).
* Use close_loop(loop) rather than loop.close().
* Use longer variable names.
2019-06-14 13:02:51 +02:00
Pablo Galindo 05f8318655
bpo-37269: Correctly optimise conditionals with constant booleans (GH-14071)
Fix a regression introduced by af8646c805 that was causing code of the form:

if True and False:
   do_something()

to be optimized incorrectly, eliminating the block.
2019-06-14 06:54:53 +01:00
Michael Felt d0eeb936d8 bpo-37077: Add native thread ID (TID) for AIX (GH-13624)
This is the followup  for issue36084



https://bugs.python.org/issue37077
2019-06-13 15:34:46 -07:00
Pablo Galindo 3498c642f4
bpo-37213: Handle negative line deltas correctly in the peephole optimizer (GH-13969)
The peephole optimizer was not optimizing correctly bytecode after negative deltas were introduced. This is due to the fact that some special values (255) were being searched for in both instruction pointer delta and line number deltas.
2019-06-13 19:16:22 +01:00
Victor Stinner 6d22cc8e90
bpo-37261: Fix support.catch_unraisable_exception() (GH-14052)
The __exit__() method of test.support.catch_unraisable_exception
context manager now ignores unraisable exception raised when clearing
self.unraisable attribute.
2019-06-13 14:44:54 +02:00
Victor Stinner 63ab4ba07b
bpo-37210: Fix pure Python pickle when _pickle is unavailable (GH-14016)
Allow pure Python implementation of pickle to work
even when the C _pickle module is unavailable.

Fix test_pickle when _pickle is missing: declare PyPicklerHookTests
outside "if has_c_implementation:" block.
2019-06-13 13:58:51 +02:00
Jeroen Demeyer b4b814b398 bpo-37231: optimize calls of special methods (GH-13973) 2019-06-13 18:26:44 +09:00
Jeffrey Kintscher 8725c83ed5 bpo-35070: test_getgrouplist may fail on macOS if too many groups (GH-13071) 2019-06-13 03:01:29 -04:00
Tim Peters d1c85a27ea
bpo-37257: obmalloc: stop simple arena thrashing (#14039)
GH-14039:  allow (no more than) one wholly empty arena on the usable_arenas list.

This prevents thrashing in some easily-provoked simple cases that could end up creating and destroying an arena on each loop iteration in client code.   Intuitively, if the only arena on the list becomes empty, it makes scant sense to give it back to the system unless we know we'll never need another free pool again before another arena frees a pool.  If the latter obtains, then - yes - this will "waste" an arena.
2019-06-12 22:41:03 -05:00
Victor Stinner 468e5fec8a
bpo-36402: Fix threading._shutdown() race condition (GH-13948)
Fix a race condition at Python shutdown when waiting for threads.
Wait until the Python thread state of all non-daemon threads get
deleted (join all non-daemon threads), rather than just wait until
Python threads complete.

* Add threading._shutdown_locks: set of Thread._tstate_lock locks
  of non-daemon threads used by _shutdown() to wait until all Python
  thread states get deleted. See Thread._set_tstate_lock().
* Add also threading._shutdown_locks_lock to protect access to
  threading._shutdown_locks.
* Add test_finalization_shutdown() test.
2019-06-13 01:30:17 +02:00
Paul Monson b4c7defe58 bpo-36779: time.tzname returns empty string on Windows if default cod… (GH-13073)
Calling setlocale(LC_CTYPE, "") on a system where GetACP() returns CP_UTF8 results in empty strings in _tzname[].

This causes time.tzname to be an empty string.
I have reported the bug to the UCRT team and will follow up, but it will take some time get a fix into production.

In the meantime one possible workaround is to temporarily change the locale by calling setlocale(LC_CTYPE, "C") before calling _tzset and restore the current locale after if the GetACP() == CP_UTF8 or CP_UTF7

@zooba 


https://bugs.python.org/issue36779
2019-06-12 16:13:27 -07:00
Victor Stinner 95f61c8b16
bpo-37069: regrtest uses sys.unraisablehook (GH-13759)
regrtest now uses sys.unraisablehook() to mark a test as "environment
altered" (ENV_CHANGED) if it emits an "unraisable exception".
Moreover, regrtest logs a warning in this case.

Use "python3 -m test --fail-env-changed" to catch unraisable
exceptions in tests.
2019-06-13 01:09:04 +02:00
David Carlier 5287022eee bpo-37160: Thread native ID NetBSD support (GH-13835) 2019-06-12 17:37:56 +02:00
Guido van Rossum 10b55c1643 bpo-35766: Change format for feature_version to (major, minor) (GH-13992)
(A single int is still allowed, but undocumented.)



https://bugs.python.org/issue35766
2019-06-11 17:23:12 -07:00
Andrew Svetlov 65aa64fae8 bpo-36607: Eliminate RuntimeError raised by asyncio.all_tasks() (GH-13971)
If internal tasks weak set is changed by another thread during iteration.



https://bugs.python.org/issue36607
2019-06-11 08:27:30 -07:00
Raymond Hettinger 1f11cf9521
bpo-37219: Remove erroneous optimization for differencing an empty set (GH-13965) 2019-06-11 01:15:24 -07:00
Victor Stinner 4f6f7c5a61
bpo-18748: Fix _pyio.IOBase destructor (closed case) (GH-13952)
_pyio.IOBase destructor now does nothing if getting the closed
attribute fails to better mimick _io.IOBase finalizer.
2019-06-11 02:49:06 +02:00
Christian Heimes 8a8b59c979 bpo-37215: Fix dtrace issue introduce by bpo-36842 (GH-13940)
Signed-off-by: Christian Heimes <christian@python.org>


https://bugs.python.org/issue37215
2019-06-10 08:19:48 -07:00
Marcin Niemira 45a14942c9 bpo-11122: fix hardcoded path checking for rpmbuild in bdist_rpm.py (GH-10594) 2019-06-08 17:05:05 -04:00
Raymond Hettinger e119b3d136
bpo-37178: Allow a one argument form of math.perm() (GH-13905) 2019-06-08 08:58:11 -07:00
Rémi Lapeyre 8cc605acdd bpo-34886: Fix subprocess.run handling of exclusive arguments (GH-11727)
Fix an unintended ValueError from :func:`subprocess.run` when checking for
conflicting `input` and `stdin` or `capture_output` and `stdout` or `stderr` args
when they were explicitly provided but with `None` values within a passed in
`**kwargs` dict rather than as passed directly by name.
2019-06-08 07:56:24 -07:00
Philipp A d407d2a726 bpo-37173: Show passed class in inspect.getfile error (GH-13861)
Currently, inspect.getfile(str) will report nonsense:

```pytb
>>> inspect.getfile(str)
TypeError: <module 'builtins' (built-in)> is a built-in class
```

This fixes that


https://bugs.python.org/issue37173
2019-06-08 05:05:46 -07:00
zygocephalus 03d5831a2d bpo-37150: Throw ValueError if FileType class object was passed in add_argument (GH-13805)
There is a possibility that someone (like me) accidentally will omit parentheses with `FileType` arguments after `FileType`, and parser will contain wrong file until someone will try to use it. 

Example:
```python
parser = argparse.ArgumentParser()
parser.add_argument('-x', type=argparse.FileType)
```


https://bugs.python.org/issue37150
2019-06-07 13:08:36 -07:00
Victor Stinner 3bf0f3ad20
bpo-37169: Rewrite _PyObject_IsFreed() unit tests (GH-13888)
Replace two Python function calls with a single one to ensure that no
memory allocation is done between the invalid object is created and
when _PyObject_IsFreed() is called.
2019-06-07 16:22:21 +02:00
Tal Einat 554450fb4e
bpo-37177: make IDLE's search dialogs transient (GH-13869)
This avoids the search dialogs being hidden behind the editor window.
2019-06-07 08:54:40 +03:00
Zackery Spytz dc2476500d bpo-37170: Fix the cast on error in PyLong_AsUnsignedLongLongMask() (GH-13860) 2019-06-06 22:39:23 +02:00
websurfer5 f6713e84af bpo-36520: Email header folded incorrectly (#13608)
* bpo-36520: reset the encoded word offset when starting a new
line during an email header folding operation

* 📜🤖 Added by blurb_it.

* bpo-36520: add an additional test case, and provide descriptive
comments for the test_folding_with_utf8_encoding_* tests

* bpo-36520: fix whitespace issue

* bpo-36520: changes per reviewer request -- remove extraneous
backslashes; add whitespace between terminating quotes and
line-continuation backslashes; use "bpo-" instead of
"issue #" in comments
2019-06-06 12:53:27 -07:00
Raymond Hettinger e985804207
bpo-37165: Convert _count_elements to the argument clinic (GH-13848) 2019-06-05 16:05:25 -07:00
Ashwin Ramaswami c4c15ed7a2 bpo-35551: encodings update (GH-11446) 2019-06-05 18:18:06 -04:00
Abhilash Raj 66c4f3f38b bpo-21315: Fix parsing of encoded words with missing leading ws. (#13425)
* bpo-21315: Fix parsing of encoded words with missing leading ws.

Because of missing leading whitespace, encoded word would get parsed as
unstructured token. This patch fixes that by looking for encoded words when
splitting tokens with whitespace.

Missing trailing whitespace around encoded word now register a defect
instead.

Original patch suggestion by David R. Murray on bpo-21315.
2019-06-05 09:56:33 -07:00
Raymond Hettinger 6c01ebcc0d
bpo-37158: Simplify and speed-up statistics.fmean() (GH-13832) 2019-06-05 07:39:38 -07:00
Zackery Spytz 9aa78566fb bpo-34767: Do not always create a collections.deque() in asyncio.Lock() (GH-13834)
https://bugs.python.org/issue34767
2019-06-05 02:33:26 -07:00
Terry Jan Reedy 949fe976d5
bpo-35763: Make IDLE calltip note about '/' less obtrusive (GH-13791)
Add it to the end of the first line if there is room.  Tests were reworked.
2019-06-04 21:55:37 -04:00
Zachary Ware 19a1e1eb86 bpo-34282: Remove deprecated enum _convert method (GH-13823) 2019-06-04 23:03:10 +02:00
Łukasz Langa 23d7ce7471 Python 3.8.0b1
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE4/8oOcBIslwITevpsmmV4xAlBWgFAlz2r2cACgkQsmmV4xAl
 BWhasQ/9EDLL+KtFUdi3iMr7+C1UN/2u2v3OnOsZQA0WE7TfEebbf5GS4baN/pHk
 PslVY6kRqRfQIxjemXFpq52mn9+3SUE4m3avkgnUZMu8XEaUczM4IdcizGMO3qNr
 c1iz6BraJo8gZ1g+dNcRbEc89V9e1SzrBZOR9MXGjxxo0HHEPdjW9ptg1BtHq9To
 n4iivdQSnnxzP7D7SpItWPL/gFv90GBbXaTRerGUlmj/uwnBhzHB1fi4r+WDn8yR
 4AiFVSlUFpTXYOiH5A3MYQtLq8kH9AAmo6Bmj/73NvChxv7GV5v968+tlEWEK+/h
 UtTUEw8L/oSH74m8wd0q0tQdTe8VJgtwKyR6h5GLL8TLa73nN/jK4Koqirpc8mpK
 DcHVLpxHQ/yhPtAcQXp81qqJSbhnPOvbRjNQP5bHcNIl8p+OsF7N9z/tu6gmMorO
 k+f2OCOC0ZsG/EL7DPz6kG6+Cy5NhA+/IJ9I3Ayk6AsSjbzqtgai3bKKVF3s27r8
 OIZprMAVKsmsPpEfuSG8rshKxmLH5/n7YuwSuMmWXv06QspfG43zxp1VYq94zBoI
 wpBy8QxBwef7oDpMI+WgE/g8Prj1NmQuIy7co/pZDsOHeQWr96z90BcpSX8XKHKT
 9W829jFKoQEnGnuFgoUxacR8a4bQKVla5mVFvjUtGjb9GS9kJkk=
 =V2hy
 -----END PGP SIGNATURE-----
gpgsig -----BEGIN PGP SIGNATURE-----
 
 iQJEBAABCAAuFiEE4/8oOcBIslwITevpsmmV4xAlBWgFAlz2zD8QHGx1a2FzekBs
 YW5nYS5wbAAKCRCyaZXjECUFaIw/EACXkzdCOq5D/itTvFElnNWjlon8ot4toVEf
 zw4isSYiYS4B3dBI6+VyCZWNxR5F6fwfyhPLHf7CzPn+7Yp5+dX9//x0HqhNLQCY
 PmwuYG/4nWcww8UhUweGVI1u8Fjrw+LzLJPVyhLQia/TXUBfCFcggS2xo9S6mSsA
 isJr9RaCDaoit6ZjFIuo+sOn2YTx2eYBe6LlhlSx9zz0ayyy+ADqPeUq/hoEFDPD
 EXUOwsLJPo/vu7sFqKOhHBWtMCcnvkkbiBLrAmHS2vhKjl22zVez0lUiVxLUlhZT
 WcmfRkCtoBKu2rYbm85pt08fgF/CnHq9oO57yDBZ4j/Lv0HqQkp+/yZRzWqSSThw
 vw4z4Ur3l06ZMxsTH+Dd/xjqc41AlpedJNhuAgOf0URrbl4H48y5U6AG1vZWrpAk
 wUsTF1QGXL1SAam2pjI2cTpjRaCh3RK2wf0KPewxUpcvcRcGtGAbq+EFHHOFg2Lm
 AcxAuAEUXKcyJQnLtFiIOWJFttaRzAB6zq8sEL0kS1BfAEspqTk+ZdnDY1ajppqu
 7sb3/TGrxa+38Th5iVvUehHGQqrQYLORVn0w8PKUXunbUbv6AYoDJHKxiOUO4JsX
 L5jZEPjTeG0BjntTnvVYdgqPXAVbJr76LmlCPkPcmSruvXrfZP1OOi+ybepwbTpD
 8pw88nSTuA==
 =zOCJ
 -----END PGP SIGNATURE-----

Merge tag 'v3.8.0b1'

Python 3.8.0b1
2019-06-04 21:53:26 +02:00
Abhilash Raj aa79707262 bpo-30835: email: Fix AttributeError when parsing invalid CTE (GH-13598)
* bpo-30835: email: Fix AttributeError when parsing invalid Content-Transfer-Encoding

Parsing an email containing a multipart Content-Type, along with a
Content-Transfer-Encoding containing an invalid (non-ASCII-decodable) byte
will fail. email.feedparser.FeedParser._parsegen() gets the header and
attempts to convert it to lowercase before comparing it with the accepted
encodings, but as the header contains an invalid byte, it's returned as a
Header object rather than a str.

Cast the Content-Transfer-Encoding header to a str to avoid this.

Found using the AFL fuzzer.

Reported-by: Daniel Axtens <dja@axtens.net>
Signed-off-by: Andrew Donnellan <andrew@donnellan.id.au>

* Add email and NEWS entry for the bugfix.
2019-06-04 11:00:47 -07:00
Łukasz Langa 3b5deb0116
Python 3.8.0b1 2019-06-04 19:44:34 +02:00
Abhilash Raj 46d88a1131 bpo-35805: Add parser for Message-ID email header. (GH-13397)
* bpo-35805: Add parser for Message-ID header.

This parser is based on the definition of Identification Fields from RFC 5322
Sec 3.6.4.

This should also prevent folding of Message-ID header using RFC 2047 encoded
words and hence fix bpo-35805.

* Prevent folding of non-ascii message-id headers.
* Add fold method to MsgID token to prevent folding.
2019-06-04 10:41:34 -07:00
Victor Stinner bc6469f79c
bpo-37153: test_venv.test_mutiprocessing() calls pool.terminate() (GH-13816)
test_venv.test_mutiprocessing() now explicitly calls pool.terminate()
to wait until the pool completes.
2019-06-04 19:03:13 +02:00
Łukasz Langa 7f9a2ae780
Revert "bpo-34037, asyncio: add BaseEventLoop.wait_executor_on_close (GH-13786)" (#13802)
This reverts commit 0f0a30f4da.
2019-06-04 13:03:20 +02:00
Victor Stinner 0fd2c300c2
Revert "bpo-36818: Add PyInterpreterState.runtime field. (gh-13129)" (GH-13795)
This reverts commit 396e0a8d9d.
2019-06-04 03:15:09 +02:00
Andrew Svetlov 9535aff942 Revert "bpo-35621: Support running subprocesses in asyncio when loop is executed in non-main thread (#13630)" (GH-13793)
https://bugs.python.org/issue35621
2019-06-03 17:09:19 -07:00
Victor Stinner 0f0a30f4da
bpo-34037, asyncio: add BaseEventLoop.wait_executor_on_close (GH-13786)
Add BaseEventLoop.wait_executor_on_close attribute: true by default.

loop.close() now waits for the default executor to finish by default.
Set loop.wait_executor_on_close attribute to False to not wait for
the executor.
2019-06-03 23:31:04 +02:00
Christian Heimes 78c7d52779
bpo-37120: Add SSLContext.num_tickets (GH-13719)
Signed-off-by: Christian Heimes <christian@python.org>
2019-06-03 21:00:10 +02:00
Christian Heimes 47eb223406 bpo-36868: Fix what's new for SSLContext.hostname_checks_common_name (GH-13248)
What's new now mentions SSLContext.hostname_checks_common_name instead of SSLContext.host_flags.


https://bugs.python.org/issue36868
2019-06-03 11:51:27 -07:00
Christian Heimes 06651ee418
bpo-37081: Test with OpenSSL 1.1.1c (GH-13631)
Signed-off-by: Christian Heimes <christian@python.org>
2019-06-03 20:10:19 +02:00
Victor Stinner e225bebc14
Revert "bpo-33608: Factor out a private, per-interpreter _Py_AddPendingCall(). (gh-13714)" (GH-13780)
This reverts commit 6a150bcaeb.
2019-06-03 18:14:24 +02:00
David Carlier 0b9956e916 bpo-37087: Adding native ID support for OpenBSD (GH-13654) 2019-06-03 17:43:33 +02:00
Inada Naoki 91234a1636
bpo-26219: per opcode cache for LOAD_GLOBAL (GH-12884)
This patch implements per opcode cache mechanism, and use it in
only LOAD_GLOBAL opcode.

Based on Yury's opcache3.patch in bpo-26219.
2019-06-03 21:30:58 +09:00
Ned Deily 0288dd6a51
bpo-36231: Support building on macOS without /usr/include (GH-13773) 2019-06-03 06:34:48 -04:00
Pablo Galindo 8565f6b6db
bpo-35814: Allow unpacking in r.h.s of annotated assignment expressions (GH-13760) 2019-06-03 08:34:20 +01:00
Augusto Hack 01ee12ba35 bpo-33569 Preserve type information with dataclasses.InitVar (GH-8927) 2019-06-02 22:14:48 -04:00
Victor Stinner 0025350294
bpo-37069: tests use catch_unraisable_exception() (GH-13762)
Modify test_coroutines, test_cprofile, test_generators, test_raise,
test_ssl and test_yield_from to use
support.catch_unraisable_exception() rather than
support.captured_stderr().

test_thread: remove test_save_exception_state_on_error() which is now
updated. test_unraisable_exception() checks that sys.unraisablehook()
is called to handle _thread.start_new_thread() exception.

test_cprofile now rely on unittest for test discovery: replace
support.run_unittest() with unittest.main().
2019-06-03 03:51:43 +02:00
Matthias Bussonnier 13136e83a6 bpo-36896: Clarify that some types constructors are unstable (GH-13271) 2019-06-03 02:43:21 +02:00
Michele Angrisano aca273e240 bpo-37014: Update docstring and Documentation of fileinput.FileInput(). (GH-13545)
* bpo-37014: Update docstring and Documentation of fileinput.FileInput()

* Explain the behavior of fileinput.FileInput() when reading stdin.

* Update blurb.

* bpo-37014: Fix typo in the docstring and documentation.
2019-06-02 23:01:49 +02:00
Cheryl Sabella 6bdc4dee01 bpo-35610: IDLE - Replace .context_use_ps1 with .prompt_last_line (GH-11307)
Changes in bpo- 31858 made the less informative 'context_use_ps1' redundant.
2019-06-02 14:56:47 -04:00
Pablo Galindo 7ffcf848df
bpo-37126: Allow structseq objects to be tracked by the GC (GH-13729) 2019-06-02 15:45:13 +01:00
Andrew Svetlov 13ed07998a
bpo-35621: Support running subprocesses in asyncio when loop is executed in non-main thread (#13630) 2019-06-02 13:56:38 +03:00
Mark Dickinson c52996785a
bpo-36027: Extend three-argument pow to negative second argument (GH-13266) 2019-06-02 10:24:06 +01:00
Serhiy Storchaka 5ae299ac78
bpo-37128: Add math.perm(). (GH-13731) 2019-06-02 11:16:49 +03:00
Anthony Sottile b7daabd711 Improve version added references in `typing` module docs (GH-13457) 2019-06-02 01:13:25 +01:00
Eric Snow 6a150bcaeb
bpo-33608: Factor out a private, per-interpreter _Py_AddPendingCall(). (gh-13714) 2019-06-01 15:39:46 -06:00
Serhiy Storchaka bdbad71b9d
bpo-20092. Use __index__ in constructors of int, float and complex. (GH-13108) 2019-06-02 00:05:48 +03:00
Cheryl Sabella 1a4d9ffa1a bpo-32411: IDLE: Remove line number sort in browser.py (#5011)
Insertion in line order makes sorting keys by line order unneeded.
2019-06-01 17:03:22 -04:00
Sergey Fedoseev e5f6207ba6 bpo-34303: Micro-optimizations in functools.reduce() (GH-8598) 2019-06-01 13:32:17 -07:00
Pablo Galindo cd74e66a8c
bpo-37122: Make co->co_argcount represent the total number of positonal arguments in the code object (GH-13726) 2019-06-01 18:08:04 +01:00
Bar Harel 6b282e1887 bpo-36813: Fix QueueListener to call task_done() upon termination. (GH-13113)
Fixed QueueListener in order to avoid random deadlocks.
Unable to add regression tests atm due to time constraints, will add it in a bit.
Regarding implementation, although it's nested, it does not cause performance issues whatsoever, and does not call task_done() in case of an exception (which is the right thing to do IMHO).


https://bugs.python.org/issue36813
2019-06-01 02:19:09 -07:00
Yash Aggarwal 4a686504eb bpo-35431: Implemented math.comb (GH-11414) 2019-06-01 00:21:27 -07:00
Mario Corchero 354227a1e9 Add option to trace to run modules (GH-5134)
Adds a new option in trace that allows tracing runnable modules. It is
exposed as `--module module_name` as `-m` is already in use for another
argument.
2019-06-01 14:49:09 +10:00
Eric Snow 396e0a8d9d
bpo-36818: Add PyInterpreterState.runtime field. (gh-13129)
https://bugs.python.org/issue36818
2019-05-31 21:16:47 -06:00
Tim Peters 1c263e39c4
bpo-37029: keep usable_arenas in sorted order without searching (#13612)
This adds a vector of "search fingers" so that usable_arenas can be kept in sorted order (by number of free pools) via constant-time operations instead of linear search.

This should reduce worst-case time for reclaiming a great many objects from O(A**2) to O(A), where A is the number of arenas.  See bpo-37029.
2019-05-31 21:16:04 -05:00
Zackery Spytz 549e55a308 bpo-12202: Properly check MsiSummaryInfoGetProperty() calls in msilib (GH-13711) 2019-06-01 03:16:20 +03:00
Ammar Askar a6ec1ce1ac bpo-33361: Fix bug with seeking in StreamRecoders (GH-8278) 2019-05-31 22:44:00 +03:00
Pablo Galindo aac4d0342c
bpo-26826: Expose copy_file_range in the os module (GH-7255) 2019-05-31 19:39:47 +01:00
Zackery Spytz c8d5bf6c3f bpo-12639: msilib.Directory.start_component() fails if *keyfile* is not None (GH-13688)
msilib.Directory.start_component() was passing an extra argument to CAB.gen_id().
2019-05-31 09:43:13 -07:00
Akshay Sharma 4612671df2 bpo-25735: math.factorial doc should mention integer return type (GH-6420) 2019-05-31 12:41:17 -04:00
Christian Heimes 8cbb5b6625
bpo-26835: Add file sealing constants to fcntl (GH-13694)
Co-authored-by: nanjekyejoannah <nanjekyejoannah@gmail.com>
2019-05-31 18:32:33 +02:00
Zackery Spytz c7f803b08e bpo-36379: __ipow__ must be a ternaryfunc, not a binaryfunc (GH-13546)
If a type's __ipow__ method was implemented in C, attempting to use
the *modulo* parameter would cause crashes.


https://bugs.python.org/issue36379
2019-05-31 02:46:36 -07:00
Christian Heimes c7f7069e77
bpo-34271: Add ssl debugging helpers (GH-10031)
The ssl module now can dump key material to a keylog file and trace TLS
protocol messages with a tracing callback. The default and stdlib
contexts also support SSLKEYLOGFILE env var.

The msg_callback and related enums are private members. The feature
is designed for internal debugging and not for end users.

Signed-off-by: Christian Heimes <christian@python.org>
2019-05-31 11:44:05 +02:00
Serhiy Storchaka e9b51c0ad8
bpo-26660, bpo-35144: Fix permission errors in TemporaryDirectory cleanup. (GH-10320)
TemporaryDirectory.cleanup() failed when non-writeable or non-searchable
files or directories were created inside a temporary directory.
2019-05-31 11:30:37 +03:00
Serhiy Storchaka 38ab7d4721
bpo-31829: Make protocol 0 pickles be loadable in text mode in Python 2. (GH-11859)
Escape ``\r``, ``\0`` and ``\x1a`` (end-of-file on Windows) in Unicode strings.
2019-05-31 11:29:39 +03:00
Serhiy Storchaka 14a0e16c88
bpo-36548: Improve the repr of re flags. (GH-12715) 2019-05-31 10:39:47 +03:00
Ivan Levkivskyi 4c23aff065
bpo-29262: Add get_origin() and get_args() introspection helpers to typing (GH-13685)
This is an old feature request that appears from time to time. After a year of experimenting with various introspection capabilities in `typing_inspect` on PyPI, I propose to add these two most commonly used functions: `get_origin()` and `get_args()`. These are essentially thin public wrappers around private APIs: `__origin__` and `__args__`.

As discussed in the issue and on the typing tracker, exposing some public helpers instead of `__origin__` and `__args__` directly will give us more flexibility if we will decide to update the internal representation, while still maintaining backwards compatibility.

The implementation is very simple an is essentially a copy from `typing_inspect` with one exception: `ClassVar` was special-cased in `typing_inspect`, but I think this special-casing doesn't really help and only makes things more complicated.
2019-05-31 00:10:07 +01:00
Matthias Bussonnier eea47e0939 bpo-36953: Delay removal of ABCs from collections. (GH-13409)
Bump the removal to 3.9, indicate collections.abc available since 3.3,
replace version-changed directive to deprecated-removed.


https://bugs.python.org/issue36953
2019-05-30 14:45:48 -07:00
xdegaye 5437ccca14 bpo-36342: Fix test_multiprocessing in test_venv (GH-12513)
when platform lacks a functioning sem_open implementation



https://bugs.python.org/issue36342
2019-05-30 14:42:29 -07:00
Alex Grönholm 98ef92002e bpo-36999: Add asyncio.Task.get_coro() (GH-13680)
https://bugs.python.org/issue36999
2019-05-30 08:30:09 -07:00
Zackery Spytz 249b7d59d8 bpo-20602: Do not clear sys.flags and sys.float_info during shutdown (GH-8096)
There is no need to clear these immutable objects during shutdown.
2019-05-30 13:08:24 +02:00
Christian Heimes 6eb814b8ce
bpo-37098: Skip memfd_create test before Linux 3.17 (GH-13677) 2019-05-30 11:27:06 +02:00
Giampaolo Rodola 413d955f8e
bpo-36610: shutil.copyfile(): use sendfile() on Linux only (GH-13675)
...and avoid using it on Solaris as it can raise EINVAL if offset is equal or bigger than the size of the file
2019-05-30 14:05:41 +08:00
Ying Wang a16387ab2d bpo-24564: shutil.copystat(): ignore EINVAL on os.setxattr() (GH-13369) 2019-05-30 11:25:31 +08:00
Zackery Spytz 8f96c9f8ed bpo-37007: Implement socket.if_nametoindex(), if_indextoname() and if_nameindex() on Windows (GH-13522) 2019-05-29 14:02:37 -07:00
Antoine Pitrou ada319bb6d
bpo-32388: Remove cross-version binary compatibility requirement in tp_flags (GH-4944)
It is now allowed to add new fields at the end of the PyTypeObject struct without having to allocate a dedicated compatibility flag in tp_flags.

This will reduce the risk of running out of bits in the 32-bit tp_flags value.
2019-05-29 22:12:38 +02:00
Zackery Spytz 43fdbd2729 bpo-26836: Add os.memfd_create() (#13567)
* bpo-26836: Add os.memfd_create()

* Use the glibc wrapper for memfd_create()

Co-Authored-By: Christian Heimes <christian@python.org>

* Fix deletions caused by autoreconf.

* Use MFD_CLOEXEC as the default value for *flags*.

* Add memset_s to configure.ac.

* Revert memset_s changes.

* Apply the requested changes.

* Tweak the docs.
2019-05-29 21:57:03 +02:00
Gregory P. Smith 0c2f930564
bpo-22385: Support output separators in hex methods. (#13578)
* bpo-22385: Support output separators in hex methods.

Also in binascii.hexlify aka b2a_hex.

The underlying implementation behind all hex generation in CPython uses the
same pystrhex.c implementation.  This adds support to bytes, bytearray,
and memoryview objects.

The binascii module functions exist rather than being slated for deprecation
because they return bytes rather than requiring an intermediate step through a
str object.

This change was inspired by MicroPython which supports sep in its binascii
implementation (and does not yet support the .hex methods).

https://bugs.python.org/issue22385
2019-05-29 11:46:58 -07:00
Jeroen Demeyer aacc77fbd7 bpo-36974: implement PEP 590 (GH-13185)
Co-authored-by: Jeroen Demeyer <J.Demeyer@UGent.be>
Co-authored-by: Mark Shannon <mark@hotpy.org>
2019-05-29 20:31:52 +02:00
Anthony Sottile d30da5dd9a bpo-36983: Fix typing.__all__ and add test for exported names (GH-13456)
https://bugs.python.org/issue36983
2019-05-29 11:19:37 -07:00
Andrew Svetlov 4dd3e3f9bb bpo-32972: Async test case (GH-13386)
Add explicit `asyncSetUp` and `asyncTearDown` methods.
The rest is the same as for #13228

`AsyncTestCase` create a loop instance for every test for the sake of test isolation.
Sometimes a loop shared between all tests can speed up tests execution time a lot but it requires control of closed resources after every test finish. Basically, it requires nested supervisors support that was discussed with @1st1 many times. Sorry, asyncio supervisors have no chance to land on Python 3.8.

The PR intentionally does not provide API for changing the used event loop or getting the test loop: use `asyncio.set_event_loop_policy()` and `asyncio.get_event_loop()` instead.

The PR adds four overridable methods to base `unittest.TestCase` class:
```
    def _callSetUp(self):
        self.setUp()

    def _callTestMethod(self, method):
        method()

    def _callTearDown(self):
        self.tearDown()

    def _callCleanup(self, function, /, *args, **kwargs):
        function(*args, **kwargs)
```
It allows using asyncio facilities with minimal influence on the unittest code.

The last but not least: the PR respects contextvars. The context variable installed by `asyncSetUp` is available on test, `tearDown` and a coroutine scheduled by `addCleanup`.


https://bugs.python.org/issue32972
2019-05-29 02:33:59 -07:00
Bo Bayles ca80495592 bpo-22454: Add shlex.join() (the opposite of shlex.split()) (GH-7605) 2019-05-29 09:06:11 +01:00
依云 744c08a9c7 bpo-35246: fix support for path-like args in asyncio subprocess (GH-13628)
Drop isinstance checks from create_subprocess_exec function and let
subprocess module do them.

https://bugs.python.org/issue35246


https://bugs.python.org/issue35246
2019-05-28 23:50:59 -07:00
Victor Stinner 8b09500345
bpo-37076: _thread.start_new_thread() calls _PyErr_WriteUnraisableMsg() (GH-13617)
_thread.start_new_thread() now logs uncaught exception raised by the
function using sys.unraisablehook(), rather than sys.excepthook(), so
the hook gets access to the function which raised the exception.
2019-05-29 02:57:56 +02:00
Guido van Rossum 77f0ed7a42
bpo-37072: Fix crash in PyAST_FromNodeObject() when flags is NULL (#13634)
I'm confident that this fixes the reported crash. flags=NULL is treated as using the latest minor version.

https://bugs.python.org/issue37072
2019-05-28 16:44:58 -07:00
Dino Viehland 415406999d
bpo-37001: Makes symtable.symtable have parity with compile for input (#13483)
* Makes symtable.symtable have parity for accepted datatypes
for source code as compile()

* Add NEWS blurb
2019-05-28 16:21:17 -07:00
Francisco Facioni ab0716ed1e bpo-22102: Fixes zip files with disks set to 0 (GH-5985) 2019-05-28 19:15:11 -04:00
Serhiy Storchaka 9e3c452639
bpo-31961: Fix support of path-like executables in subprocess. (GH-5914) 2019-05-28 22:49:35 +03:00
Joannah Nanjekye 2e33ecd7c9 bpo-22640: Add silent mode to py_compile.compile() (GH-12976) 2019-05-28 19:29:04 +03:00
Zackery Spytz 3c8724fc60 bpo-33407: Implement Py_DEPRECATED() on MSVC (GH-8980) 2019-05-28 17:16:33 +02:00
Victor Stinner 17a5588740
bpo-33725: multiprocessing uses spawn by default on macOS (GH-13603)
On macOS, the multiprocessing module now uses the "spawn" start
method by default.
2019-05-28 16:02:50 +02:00
Zackery Spytz 05f16416d9 bpo-26423: Fix possible overflow in wrap_lenfunc() (GH-13606)
Fix possible overflow in wrap_lenfunc() when
sizeof(long) < sizeof(Py_ssize_t) (e.g., 64-bit Windows).
2019-05-28 14:55:28 +02:00
Mario Corchero 04530812e9 bpo-32299: Return patched dict when using patch.dict as a context manager (GH-11062) 2019-05-28 08:53:30 -04:00
Jeroen Demeyer eb65e2443a bpo-36922: implement PEP-590 Py_TPFLAGS_METHOD_DESCRIPTOR (GH-13338)
Co-authored-by: Mark Shannon <mark@hotpy.org>
2019-05-28 14:42:53 +02:00
Inada Naoki 9a7e5b1b42
bpo-35279: reduce default max_workers of ThreadPoolExecutor (GH-13618) 2019-05-28 21:02:52 +09:00
Andrew Svetlov bafd4b5ac8 bpo-29883: Asyncio proactor udp (GH-13440)
Follow-up for #1067


https://bugs.python.org/issue29883
2019-05-28 02:52:15 -07:00
Ivan Levkivskyi 74d7f76e2c
bpo-37058: PEP 544: Add Protocol to typing module (GH-13585)
I tried to get rid of the `_ProtocolMeta`, but unfortunately it didn'y work. My idea to return a generic alias from `@runtime_checkable` made runtime protocols unpickleable. I am not sure what is worse (a custom metaclass or having some classes unpickleable), so I decided to stick with the status quo (since there were no complains so far). So essentially this is a copy of the implementation in `typing_extensions` with two modifications:
* Rename `@runtime` to `@runtime_checkable` (plus corresponding updates).
* Allow protocols that extend `collections.abc.Iterable` etc.
2019-05-28 08:40:15 +01:00
Matthias Bussonnier 3880f263d2 bpo-36933: Remove sys.set_coroutine_wrapper (marked for removal in 3.8) (GH-13577)
It has been documented as deprecated and to be removed in 3.8; 

From a comment on another thread – which I can't find ; leave get_coro_wrapper() for now, but always return `None`.


https://bugs.python.org/issue36933
2019-05-28 00:10:59 -07:00
Xtreak 436c2b0d67 bpo-36996: Handle async functions when mock.patch is used as a decorator (GH-13562)
Return a coroutine while patching async functions with a decorator. 

Co-authored-by: Andrew Svetlov <andrew.svetlov@gmail.com>


https://bugs.python.org/issue36996
2019-05-28 00:07:38 -07:00
sbstp f0d4c64019 bpo-36686: Improve the documentation of the std* params in loop.subprocess_exec (GH-13586)
https://bugs.python.org/issue36686
2019-05-27 16:51:19 -07:00
Victor Stinner a3568417c4
bpo-37054, _pyio: Fix BytesIO and TextIOWrapper __del__() (GH-13601)
Fix destructor _pyio.BytesIO and _pyio.TextIOWrapper: initialize
their _buffer attribute as soon as possible (in the class body),
because it's used by __del__() which calls close().
2019-05-28 01:44:21 +02:00
Victor Stinner cd590a7ced
bpo-1230540: Add threading.excepthook() (GH-13515)
Add a new threading.excepthook() function which handles uncaught
Thread.run() exception. It can be overridden to control how uncaught
exceptions are handled.

threading.ExceptHookArgs is not documented on purpose: it should not
be used directly.

* threading.excepthook() and threading.ExceptHookArgs.
* Add _PyErr_Display(): similar to PyErr_Display(), but accept a
  'file' parameter.
* Add _thread._excepthook(): C implementation of the exception hook
  calling _PyErr_Display().
* Add _thread._ExceptHookArgs: structseq type.
* Add threading._invoke_excepthook_wrapper() which handles the gory
  details to ensure that everything remains alive during Python
  shutdown.
* Add unit tests.
2019-05-28 00:39:52 +02:00
Andrew Svetlov 23b4b697e5 bpo-36889: Merge asyncio streams (GH-13251)
https://bugs.python.org/issue36889
2019-05-27 12:56:22 -07:00
Eric V. Smith 6f6ff8a565
bpo-37050: Remove expr_text from FormattedValue ast node, use Constant node instead (GH-13597)
When using the "=" debug functionality of f-strings, use another Constant node (or a merged constant node) instead of adding expr_text to the FormattedValue node.
2019-05-27 15:31:52 -04:00
Zackery Spytz 02db696732 bpo-32941: Add madvise() for mmap objects (GH-6172)
Allow mmap objects to access the madvise() system call.
2019-05-27 18:48:16 +02:00
Victor Stinner 331a6a56e9
bpo-36763: Implement the PEP 587 (GH-13592)
* Add a whole new documentation page:
  "Python Initialization Configuration"
* PyWideStringList_Append() return type is now PyStatus,
  instead of int
* PyInterpreterState_New() now calls PyConfig_Clear() if
  PyConfig_InitPythonConfig() fails.
* Rename files:

  * Python/coreconfig.c => Python/initconfig.c
  * Include/cpython/coreconfig.h => Include/cpython/initconfig.h
  * Include/internal/: pycore_coreconfig.h => pycore_initconfig.h

* Rename structures

  * _PyCoreConfig => PyConfig
  * _PyPreConfig => PyPreConfig
  * _PyInitError => PyStatus
  * _PyWstrList => PyWideStringList

* Rename PyConfig fields:

  * use_module_search_paths => module_search_paths_set
  * module_search_path_env => pythonpath_env

* Rename PyStatus field: _func => func
* PyInterpreterState: rename core_config field to config
* Rename macros and functions:

  * _PyCoreConfig_SetArgv() => PyConfig_SetBytesArgv()
  * _PyCoreConfig_SetWideArgv() => PyConfig_SetArgv()
  * _PyCoreConfig_DecodeLocale() => PyConfig_SetBytesString()
  * _PyInitError_Failed() => PyStatus_Exception()
  * _Py_INIT_ERROR_TYPE_xxx enums => _PyStatus_TYPE_xxx
  * _Py_UnixMain() => Py_BytesMain()
  * _Py_ExitInitError() => Py_ExitStatusException()
  * _Py_PreInitializeFromArgs() => Py_PreInitializeFromBytesArgs()
  * _Py_PreInitializeFromWideArgs() => Py_PreInitializeFromArgs()
  * _Py_PreInitialize() => Py_PreInitialize()
  * _Py_RunMain() => Py_RunMain()
  * _Py_InitializeFromConfig() => Py_InitializeFromConfig()
  * _Py_INIT_XXX() => _PyStatus_XXX()
  * _Py_INIT_FAILED() => _PyStatus_EXCEPTION()

* Rename 'err' PyStatus variables to 'status'
* Convert RUN_CODE() macro to config_run_code() static inline function
* Remove functions:

  * _Py_InitializeFromArgs()
  * _Py_InitializeFromWideArgs()
  * _PyInterpreterState_GetCoreConfig()
2019-05-27 16:39:22 +02:00
Yury Selivanov 8cd5165ba0 bpo-37027: Return a proxy socket object from transp.get_extra_info('socket') (GH-13530)
Return a safe to use proxy socket object from `transport.get_extra_info('socket')`




https://bugs.python.org/issue37027
2019-05-27 06:57:19 -07:00
Rémi Lapeyre 674ee12600 bpo-35397: Remove deprecation and document urllib.parse.unwrap (GH-11481) 2019-05-27 09:43:45 -04:00
Andrew Svetlov 1f39c28e48 bpo-37035: Don't log OSError (GH-13548)
https://bugs.python.org/issue37035
2019-05-27 06:28:34 -07:00
Xtreak ff6b2e66b1 bpo-37047: Refactor AsyncMock setup logic for autospeccing (GH-13574)
Handle late binding and attribute access in unittest.mock.AsyncMock
setup for autospeccing.
2019-05-27 14:56:23 +02:00
Yury Selivanov 431b540bf7
bpo-32528: Make asyncio.CancelledError a BaseException. (GH-13528)
This will address the common mistake many asyncio users make:
an "except Exception" clause breaking Tasks cancellation.

In addition to this change, we stop inheriting asyncio.TimeoutError
and asyncio.InvalidStateError from their concurrent.futures.*
counterparts.  There's no point for these exceptions to share the
inheritance chain.

In 3.9 we'll focus on implementing supervisors and cancel scopes,
which should allow better handling of all exceptions, including
SystemExit and KeyboardInterrupt
2019-05-27 14:45:12 +02:00
Yury Selivanov 16cefb0bc7
bpo-37028: asyncio REPL; activated via 'python -m asyncio'. (GH-13472)
This makes it easy to play with asyncio APIs with simply
using async/await in the REPL.
2019-05-27 13:42:29 +02:00
Raymond Hettinger b821868e6d
bpo-36772 Allow lru_cache to be used as decorator without making a function call (GH-13048) 2019-05-26 11:27:35 -07:00
Chih-Hsuan Yen aaf47caf35 bpo-37053: handle strings like u"bar" correctly in Tools/parser/unparse.py (GH-13583)
Constant.kind is added in https://bugs.python.org/issue36280.
Current possible values for Constant.kind are "u" or None.

For r'bar' and b'bar', Constant.kind value is None, so there's no need
for special handling.


https://bugs.python.org/issue37053
2019-05-26 10:08:19 -07:00
Antoine Pitrou 91f4380ced
bpo-36785: PEP 574 implementation (GH-7076) 2019-05-26 17:10:09 +02:00
Julien Palard 180dc1b0f4 bpo-28866: No type cache for types with specialized mro, invalidation is hard. (#13157)
* No type cache for types with specialized mro, invalidation is hard.

* FIX: Don't disable method cache custom types that do not implement mro().

* fixing implem.

* Avoid storing error flags, also decref.

* news entry

* Clear as soon as we're getting an error.

* FIX: Reference leak.
2019-05-26 15:25:47 +01:00
Ivan Levkivskyi 135c6a56e5
bpo-37049: PEP 589: Add TypedDict to typing module (GH-13573)
The implementation is straightforward and essentially is just copied from `typing_extensions`.
2019-05-26 09:39:24 +01:00
Ivan Levkivskyi b891c465bb
bpo-37046: PEP 586: Add Literal to typing module (#13572)
The implementation is straightforward and essentially is just copied from `typing_extensions`.
2019-05-26 09:37:48 +01:00
Ivan Levkivskyi f367242d10
bpo-37045: PEP 591: Add final qualifiers to typing module (GH-13571)
The implementation is straightforward, it just mimics `ClassVar` (since the latter is also a name/access qualifier, not really a type). Also it is essentially copied from `typing_extensions`.
2019-05-26 09:37:07 +01:00
Michael J. Sullivan 47dd2f9fd8 bpo-37017: PyObject_CallMethodObjArgs uses LOAD_METHOD optimization (GH-13516)
Update PyObject_CallMethodObjArgs and _PyObject_CallMethodIdObjArgs
to use _PyObject_GetMethod to avoid creating a bound method object
in many cases.

On a microbenchmark of PyObject_CallMethodObjArgs calling a method on
an interpreted Python class, this optimization resulted in a 1.7x
speedup.
2019-05-26 16:23:33 +09:00
Terry Jan Reedy 81bb97df61
bpo-37038: Make idlelib.run runnable; add test clause (GH-13560) 2019-05-24 21:59:53 -04:00
Jason R. Coombs 1bbf7b661f bpo-34632: Add importlib.metadata (GH-12547)
Add importlib.metadata module as forward port of the standalone importlib_metadata.
2019-05-24 16:59:01 -07:00
Victor Stinner a9f05d69cc
bpo-37032: Add CodeType.replace() method (GH-13542) 2019-05-24 23:57:23 +02:00
Victor Stinner 1c9debd236
bpo-35907: Fix typo in the NEWS entry (GH-13559) 2019-05-24 23:06:25 +02:00
Rémi Lapeyre 458560347f bpo-36969: Make PDB args command display positional only arguments (GH-13459) 2019-05-24 21:44:31 +01:00
Victor Stinner deffee5774
bpo-35907: Clarify the NEWS entry (GH-13523) 2019-05-24 22:06:32 +02:00
Berker Peksag 14738ff83d
bpo-8138: Initialize wsgiref's SimpleServer as single-threaded (GH-12977) 2019-05-24 20:24:42 +03:00
E. M. Bray c994c8fc19 bpo-21536: On Cygwin, C extensions must be linked with libpython (GH-13549)
It is also possible to link against a library or executable with a
statically linked libpython, but not both with the same DLL.  In fact
building a statically linked python is currently broken on Cygwin
for other (related) reasons.

The same problem applies to other POSIX-like layers over Windows
(MinGW, MSYS) but Python's build system does not seem to attempt
to support those platforms at the moment.
2019-05-24 17:33:47 +02:00
Dan Rose 2a37f8f55b bpo-36045: builtins.help() now prefixes `async` for async functions (GH-12010)
Previously, it was hard to tell whether a function should be awaited. It was also incorrect (per PEP 484) to put this in the type hint for coroutine functions. Added this info to the output of builtins.help and pydoc.


https://bugs.python.org/issue36045
2019-05-24 04:38:01 -07:00
Cheryl Sabella c95c93d4eb
bpo-20285: Improve help docs for object (GH-4759) 2019-05-24 06:43:29 -04:00
Matěj Cepl 608876b6b1 bpo-23395: Fix PyErr_SetInterrupt if the SIGINT signal is ignored or not handled (GH-7778)
``_thread.interrupt_main()`` now avoids setting the Python error status if the ``SIGINT`` signal is ignored or not handled by Python.
2019-05-23 22:30:00 +02:00
Steve Dower b82e17e626
bpo-36842: Implement PEP 578 (GH-12613)
Adds sys.audit, sys.addaudithook, io.open_code, and associated C APIs.
2019-05-23 08:45:22 -07:00
Damien Nadé 394119afc6 bpo-37008: make mock_open handle able to honor next() (GH-13492)
I've reported the issue on https://bugs.python.org/issue37008 and now I'm trying to bring a solution to this minor issue.

I think it could be trivially backported to 3.7 branch.


https://bugs.python.org/issue37008
2019-05-23 03:03:25 -07:00
David Carlier 51aa35e9e1 bpo-33164: update blake2 implementation (GH-6286) 2019-05-23 13:32:44 +09:00
Victor Stinner 0a8e57248b
bpo-36721: Add --embed option to python-config (GH-13500)
To embed Python into an application, a new --embed option must be
passed to "python3-config --libs --embed" to get "-lpython3.8" (link
the application to libpython). To support both 3.8 and older, try
"python3-config --libs --embed" first and fallback to "python3-config
--libs" (without --embed) if the previous command fails.

Add a pkg-config "python-3.8-embed" module to embed Python into an
application: "pkg-config python-3.8-embed --libs" includes
"-lpython3.8".  To support both 3.8 and older, try "pkg-config
python-X.Y-embed --libs" first and fallback to "pkg-config python-X.Y
--libs" (without --embed) if the previous command fails (replace
"X.Y" with the Python version).

On the other hand, "pkg-config python3.8 --libs" no longer contains
"-lpython3.8". C extensions must not be linked to libpython (except
on Android, case handled by the script); this change is backward
incompatible on purpose.

"make install" now also installs "python-3.8-embed.pc".
2019-05-23 03:30:23 +02:00
Batuhan Taşkaya ef5bb25e2d bpo-27737: Allow whitespace only headers encoding (#13478) 2019-05-22 21:13:16 -04:00
Victor Stinner df22c03b93
bpo-36829: PyErr_WriteUnraisable() normalizes exception (GH-13507)
PyErr_WriteUnraisable() now creates a traceback object if there is no
current traceback. Moreover, call PyErr_NormalizeException() and
PyException_SetTraceback() to normalize the exception value. Ignore
silently any error.
2019-05-23 01:00:58 +02:00
Victor Stinner e4d300e07c
bpo-36829: Add test.support.catch_unraisable_exception() (GH-13490)
* Copy test_exceptions.test_unraisable() to
  test_sys.UnraisableHookTest().
* Use catch_unraisable_exception() in test_coroutines,
  test_exceptions, test_generators.
2019-05-22 23:44:02 +02:00
Sean 904e34d4e6 bpo-24882: Let ThreadPoolExecutor reuse idle threads before creating new thread (#6375)
* Fixes issue 24882

* Add news file entry for change.

* Change test_concurrent_futures.ThreadPoolShutdownTest

Adjust the shutdown test so that, after submitting three jobs
to the executor, the test checks for less than three threads,
instead of looking for exactly three threads.

If idle threads are being recycled properly, then we should have
less than three threads.

* Switched idle count to semaphor, Updated tests

As suggested by reviewer tomMoral, swapped lock-protected counter
with a semaphore to track the number of unused threads.

Adjusted test_threads_terminate to wait for completiton of the
previous future before submitting a new one (and checking the
number of threads used).

Also added a new test to confirm the thread pool can be saturated.

* Updates tests as requested by pitrou.

* Correct minor whitespace error.

* Make test_saturation faster
2019-05-22 23:29:58 +02:00
Sam Martin 2a3a2ece50 bpo-33110: Catch errors raised when running add_done_callback on already completed futures (GH-13141)
Wrap the callback call within the `add_done_callback` function within concurrent.futures, in order to behave in an identical manner to callbacks added to a running future are triggered once it has completed.
2019-05-22 23:29:02 +02:00
Michael J. Sullivan d8a82e2897 bpo-36878: Only allow text after `# type: ignore` if first character ASCII (GH-13504)
This disallows things like `# type: ignoreé`, which seems wrong.

Also switch to using Py_ISALNUM for the alnum check, for consistency
with other code (and maybe correctness re: locale issues?).


https://bugs.python.org/issue36878
2019-05-22 13:43:36 -07:00
Victor Stinner 0c2b6a3943
bpo-35907, CVE-2019-9948: urllib rejects local_file:// scheme (GH-13474)
CVE-2019-9948: Avoid file reading as disallowing the unnecessary URL
scheme in URLopener().open() and URLopener().retrieve()
of urllib.request.

Co-Authored-By: SH <push0ebp@gmail.com>
2019-05-22 22:15:01 +02:00
Jake Tesler b121f63155 bpo-36084: Add native thread ID (TID) to threading.Thread (GH-13463)
Add native thread ID (TID) to threading.Thread objects
(supported platforms: Windows, FreeBSD, Linux, macOS).
2019-05-22 17:43:16 +02:00
Jelle Zijlstra b3be407288 bpo-33482: fix codecs.StreamRecoder.writelines (GH-6779)
A very simple fix. I found this while writing typeshed stubs for StreamRecoder.


https://bugs.python.org/issue33482
2019-05-22 08:18:26 -07:00
Michael J. Sullivan 933e1509ec bpo-36878: Track extra text added to 'type: ignore' in the AST (GH-13479)
GH-13238 made extra text after a # type: ignore accepted by the parser.
This finishes the job and actually plumbs the extra text through the
parser and makes it available in the AST.
2019-05-22 15:54:20 +01:00
Paul Dagnelie 4c7a46eb3c bpo-36972: Add SupportsIndex (GH-13448)
In order to support typing checks calling hex(), oct() and bin() on user-defined classes, a SupportIndex protocol is required. The ability to check these at runtime would be good to add for completeness sake. This is pretty much just a copy of SupportsInt with the names tweaked.
2019-05-22 15:23:01 +01:00
Marcel Plch 33e71e01e9 bpo-31862: Port binascii to PEP 489 multiphase initialization (GH-4108) 2019-05-22 13:51:25 +02:00
Jeroen Demeyer 77aa396bb9 bpo-36907: fix refcount bug in _PyStack_UnpackDict() (GH-13381) 2019-05-22 13:09:35 +02:00
Victor Stinner ef9d9b6312
bpo-36829: Add sys.unraisablehook() (GH-13187)
Add new sys.unraisablehook() function which can be overridden to
control how "unraisable exceptions" are handled. It is called when an
exception has occurred but there is no way for Python to handle it.
For example, when a destructor raises an exception or during garbage
collection (gc.collect()).

Changes:

* Add an internal UnraisableHookArgs type used to pass arguments to
  sys.unraisablehook.
* Add _PyErr_WriteUnraisableDefaultHook().
* The default hook now ignores exception on writing the traceback.
* test_sys now uses unittest.main() to automatically discover tests:
  remove test_main().
* Add _PyErr_Init().
* Fix PyErr_WriteUnraisable(): hold a strong reference to sys.stderr
  while using it
2019-05-22 11:28:22 +02:00
Batuhan Taşkaya 7abf8c6081 bpo-25652: Fix __rmod__ of UserString (GH-13326)
The ``__rmod__`` method of ``collections.UserString`` class had a bug that made it unusable. 


https://bugs.python.org/issue25652
2019-05-21 13:27:36 -07:00
Matthias Bussonnier 565b4f1ac7 bpo-34616: Add PyCF_ALLOW_TOP_LEVEL_AWAIT to allow top-level await (GH-13148)
Co-Authored-By: Yury Selivanov <yury@magic.io>
2019-05-21 16:12:02 -04:00
Batuhan Taşkaya aa32a7e111 bpo-23378: Add an extend action to argparse (GH-13305)
Add an extend action to argparse


https://bugs.python.org/issue23378
2019-05-21 10:47:42 -07:00
Jörg Stucke d5c120f7eb bpo-36035: fix Path.rglob for broken links (GH-11988)
Links creating an infinite symlink loop would raise an exception.
2019-05-21 19:44:40 +02:00
Max Bernstein ccb7ca728e bpo-36929: Modify io/re tests to allow for missing mod name (#13392)
* bpo-36929: Modify io/re tests to allow for missing mod name

For a vanishingly small number of internal types, CPython sets the
tp_name slot to mod_name.type_name, either in the PyTypeObject or the
PyType_Spec. There are a few minor places where this surfaces:

* Custom repr functions for those types (some of which ignore the
  tp_name in favor of using a string literal, such as _io.TextIOWrapper)
* Pickling error messages

The test suite only tests the former. This commit modifies the test
suite to allow Python implementations to omit the module prefix.

https://bugs.python.org/issue36929
2019-05-21 10:09:21 -07:00
Lihua Zhao 4fb1502189 bpo-36648: fix mmap issue for VxWorks (GH-12394)
The mmap module set MAP_SHARED flag when map anonymous memory, however VxWorks
only support MAP_PRIVATE when map anonymous memory, this commit clear MAP_SHARED
and set MAP_PRIVATE.
2019-05-21 12:50:14 +02:00
pxinwr f2d7ac7e5b bpo-31904: Add posix module support for VxWorks (GH-12118) 2019-05-21 12:46:37 +02:00
Victor Stinner d12e75734d
Revert "bpo-36084: Add native thread ID to threading.Thread objects (GH-11993)" (GH-13458)
This reverts commit 4959c33d25.
2019-05-21 12:44:57 +02:00
Erik Janssens 925af1d99b bpo-36965: Fix includes in main.c on Windows with non-MSC compilers (GH-13421)
Include windows.h rather than crtdbg.h to get STATUS_CONTROL_C_EXIT constant.
Moreover, include windows.h on Windows, not only when MSC is used.
2019-05-21 12:11:11 +02:00
Rémi Lapeyre bf457c7d82 bpo-36969: Make PDB args command display keyword only arguments (GH-13452) 2019-05-20 23:17:30 +01:00
Matthias Bussonnier 1a3faf9d97 bpo-36952: Remove the bufsize parameter in fileinput.input(). (GH-13400)
This parameter is marked as deprecated since 3.6 and for removal in 3.8.
It already had no effects.
2019-05-20 23:44:11 +03:00
Batuhan Taşkaya 4011d865d0 bpo-23896: Add a grammar where exec isn't a stmt (#13272)
https://bugs.python.org/issue23896
2019-05-20 13:27:10 -07:00
Thomas Moreau c09a9f56c0 bpo-36888: Add multiprocessing.parent_process() (GH-13247) 2019-05-20 21:37:05 +02:00
Batuhan Taşkaya 5ae1c84bcd bpo-36949: Implement __repr__ on WeakSet (GH-13415) 2019-05-20 10:01:07 -07:00
Lisa Roach 77b3b7701a
bpo-26467: Adds AsyncMock for asyncio Mock library support (GH-9296) 2019-05-20 09:19:53 -07:00
Geoff Shannon 522ccef869 bpo-22865: Expand on documentation for the pty.spawn function (GH-11980) 2019-05-20 17:06:16 +02:00
Niklas Fiekas 9932fd91e8 bpo-35721: Close socket pair if Popen in _UnixSubprocessTransport fails (GH-11553)
This slightly expands an existing test case `test_popen_error` to trigger a `ResourceWarning` and fixes it.


https://bugs.python.org/issue35721
2019-05-20 05:02:16 -07:00
Terry Jan Reedy 6d965b39b7
bpo-36958: In IDLE, print exit message (GH-13435)
Print any argument other than None or int passed to SystemExit
or sys.exit().
2019-05-19 22:52:22 -04:00
Lysandros Nikolaou d673810b9d bpo-35252: Remove FIXME from test_functools (GH-10551) 2019-05-19 15:11:20 -07:00
Berker Peksag 7c59362a15
bpo-29183: Fix double exceptions in wsgiref.handlers.BaseHandler (GH-12914) 2019-05-19 18:56:15 +03:00
Bar Harel f4e1babf44 bpo-27141: Fix collections.UserList and UserDict shallow copy. (GH-4094) 2019-05-19 16:57:13 +03:00
Xtreak c661b30f89 bpo-36948: Fix NameError in urllib.request.URLopener.retrieve (GH-13389) 2019-05-19 16:40:05 +03:00
Jelle Zijlstra 9892f454d1 bpo-33519: clarify that .copy() is not part of the MutableSequence ABC (GH-6965) 2019-05-18 20:17:56 -04:00
Batuhan Taşkaya fa19a25c23 Add support for PEP572 in ast_unparse.c (GH-13337) 2019-05-18 23:10:20 +01:00
Anthony Sottile abea73bf4a bpo-2180: Treat line continuation at EOF as a `SyntaxError` (GH-13401)
This makes the parser consistent with the tokenize module (already the case
in `pypy`).

sample
------

```python
x = 5\
```

before
------

```console
$ python3 t.py
$ python3 -mtokenize t.py
t.py:2:0: error: EOF in multi-line statement
```

after
-----

```console
$ ./python t.py
  File "t.py", line 3
    x = 5\

         ^
SyntaxError: unexpected EOF while parsing
$ ./python -m tokenize t.py
t.py:2:0: error: EOF in multi-line statement
```



https://bugs.python.org/issue2180
2019-05-18 11:27:16 -07:00
Mark Dickinson 73934b9da0
bpo-36887: add math.isqrt (GH-13244)
* Add math.isqrt function computing the integer square root.

* Code cleanup: remove redundant comments, rename some variables.

* Tighten up code a bit more; use Py_XDECREF to simplify error handling.

* Update Modules/mathmodule.c

Co-Authored-By: Serhiy Storchaka <storchaka@gmail.com>

* Update Modules/mathmodule.c

Use real argument clinic type instead of an alias

Co-Authored-By: Serhiy Storchaka <storchaka@gmail.com>

* Add proof sketch

* Updates from review.

* Correct and expand documentation.

* Fix bad reference handling on error; make some variables block-local; other tidying.

* Style and consistency fixes.

* Add missing error check; don't try to DECREF a NULL a

* Simplify some error returns.

* Another two test cases:

- clarify that floats are rejected even if they happen to be
  squares of small integers
- TypeError beats ValueError for a negative float

* Documentation and markup improvements; thanks Serhiy for the suggestions!

* Cleaner Misc/NEWS entry wording.

* Clean up (with one fix) to the algorithm explanation and proof.
2019-05-18 12:29:50 +01:00
Toshio Kuratomi e7b1136ec3 Document a workaround for a curses bug (GH-13209) 2019-05-17 15:54:02 -07:00
Victor Stinner 12083284c5
bpo-36763: _Py_RunMain() doesn't call Py_Exit() anymore (GH-13390)
Py_Main() and _Py_RunMain() now return the exitcode rather than
calling Py_Exit(exitcode) when calling PyErr_Print() if the current
exception type is SystemExit.

* Add _Py_HandleSystemExit().
* Add pymain_exit_err_print().
* Add pymain_exit_print().
2019-05-17 23:05:29 +02:00
Edison A 98ff4d5fb6 bpo-36782: Created C API wrappers and added missing tests for functions in the PyDateTimeAPI. (#13088)
* created a c API wrapper for pyDate_FromDate and added the test

* 📜🤖 Added by blurb_it.

* fixed auto-alignment by vscode

* made changes as per PEP7

* Update 2019-05-04-21-25-19.bpo-36782.h3oPIb.rst

* Refactored code as per requested changes

* Remove Whitespace to Fix failed travis build

* Update 2019-05-04-21-25-19.bpo-36782.h3oPIb.rst

* Add a new line at end of ACKS

* Added C API function for PyDateTime_FromDateAndTime

* Added a test for the C API wrapper of PyDateTime_FromDateAndTime

* Added C API function for PyDateTime_FromDateAndTime

* Added a test for the C API wrapper of PyDateTime_FromDateAndTimeAndFold

* Remove Whitespace using patchcheck

* Added a C API function for PyTime_FromTime

* Added a test for the C API wrapper of PyTime_FromTime

* Added a C API function for PyTime_FromTimeAndFold

* Added a test for the C API wrapper of PyTime_FromTimeAndFold

* Added a C API function for PyDelta_FromDSU

* Added a test for the C API wrapper of PyDelta_FromDSU

* Refactor code, re-edit lines longer than 80 chars

* Fix Whitespace issues in DatetimeTester

* List all tests that were added in this PR

* Update 2019-05-04-21-25-19.bpo-36782.h3oPIb.rst

* Reformat code as per PEP7 guidelines

* Remove unused varibles from another function

* Added specific tests for the Fold Attribute

* Update 2019-05-04-21-25-19.bpo-36782.h3oPIb.rst

* Reformat code according to requested changes

* Reformat code to PEP7 Guidelines

* Reformat code to PEP7 Guidelines

* Re-add name to blurb

* Added a backtick to blurb file

* Update 2019-05-04-21-25-19.bpo-36782.h3oPIb.rst

* Remove the need to initialize mandatory parameters

* Make the macro parameter mandatory

* Re-arrange the order of unit-test args

* Removed the need to initialize macro

change all the int macro = 0 to int macro; now that macro is required

Co-Authored-By: Paul Ganssle <pganssle@users.noreply.github.com>

* Removed the need to initialize macro

change all the `int macro = 0` to `int macro`; now that macro is required

Co-Authored-By: Paul Ganssle <pganssle@users.noreply.github.com>

* Removed the need to initialize macro

change all the `int macro = 0` to `int macro`; now that macro is required

Co-Authored-By: Paul Ganssle <pganssle@users.noreply.github.com>

* Removed the need to initialize macro

change all the `int macro = 0` to `int macro`; now that macro is required

Co-Authored-By: Paul Ganssle <pganssle@users.noreply.github.com>

* Removed the need to initialize macro

change all the `int macro = 0` to `int macro`; now that macro is required

Co-Authored-By: Paul Ganssle <pganssle@users.noreply.github.com>

* Removed the need to initialize macro

change all the `int macro = 0` to `int macro`; now that macro is required

Co-Authored-By: Paul Ganssle <pganssle@users.noreply.github.com>
2019-05-17 22:28:42 +02:00
Abhilash Raj feac6cd775 bpo-33524: Fix the folding of email header when max_line_length is 0 or None (#13391)
and there are non-ascii characters in the header.
2019-05-17 15:28:44 -04:00
Pablo Galindo af8646c805
bpo-1875: Raise SyntaxError in invalid blocks that will be optimised away (GH-13332)
Move the check for dead conditionals (if 0) to the peephole optimizer
and make sure that the code block is still compiled to report any
existing syntax errors within.
2019-05-17 11:37:08 +01:00
Erwan Le Pape ac8eb8f36b bpo-35545: Fix asyncio discarding IPv6 scopes (GH-11271)
This PR proposes a solution to [bpo-35545](https://bugs.python.org/issue35545) by adding an optional `flowinfo` and `scopeid` to `asyncio.base_events._ipaddr_info` to carry the full address information into `_ipaddr_info` and avoid discarding IPv6 specific information.

Changelog entry & regression tests to come.


https://bugs.python.org/issue35545
2019-05-17 01:28:39 -07:00
Zackery Spytz 14514d9084 bpo-36946: Fix possible signed integer overflow when handling slices. (GH-13375)
The final addition (cur += step) may overflow, so use size_t for "cur".
"cur" is always positive (even for negative steps), so it is safe to use
size_t here.

Co-Authored-By: Martin Panter <vadmium+py@gmail.com>
2019-05-17 10:13:03 +03:00
Pablo Galindo aee19f54f6
bpo-36751: Undeprecate getfullargspec (GH-13245) 2019-05-16 21:08:15 +01:00
Andrew Svetlov 68b34a7204 bpo-36921: Deprecate @coroutine for sake of async def (GH-13346)
The second attempt. Now deprecate `@coroutine` only, keep `yield from fut` as is.


https://bugs.python.org/issue36921
2019-05-16 07:52:10 -07:00
Inada Naoki bfba8c373e
bpo-36748: optimize TextIOWrapper.write() for ASCII string (GH-13002) 2019-05-16 15:03:20 +09:00
Paul Monson fb7e7505ed bpo-35926: Add support for OpenSSL 1.1.1b on Windows (GH-11779) 2019-05-15 15:38:55 -07:00
‮zlohhcuB treboR d9e006bcef bpo-33123: pathlib: Add missing_ok parameter to Path.unlink (GH-6191)
Similarly to how several pathlib file creation functions have an "exists_ok" parameter, we should introduce "missing_ok" that makes removal functions not raise an exception when a file or directory is already absent.  IMHO, this should cover Path.unlink and Path.rmdir.  Note, Path.resolve() has a "strict" parameter since 3.6 that does the same thing. Naming this of this new parameter tries to be consistent with the "exists_ok" parameter as that is more explicit about what it does (as opposed to "strict").


https://bugs.python.org/issue33123
2019-05-15 15:02:11 -07:00
Antoine Pitrou 1a2dd82f56
bpo-36786: Run compileall in parallel during "make install" (GH-13078) 2019-05-15 23:45:18 +02:00
Jon Janzen c981ad16b0 bpo-26707: Enable plistlib to read UID keys. (GH-12153)
Plistlib currently throws an exception when asked to decode a valid
.plist file that was generated by Apple's NSKeyedArchiver. Specifically,
this is caused by a byte 0x80 (signifying a UID) not being understood.

This fixes the problem by enabling the binary plist reader and writer
to read and write plistlib.UID objects.
2019-05-15 23:14:38 +03:00
Krzysztof Wojcik c1f5667be1 bpo-33529, email: Fix infinite loop in email header encoding (GH-12020) 2019-05-14 18:55:23 +02:00
Andrew Svetlov f12ba7cd0a bpo-36916: asyncio: Swallow unhandled write() exception (GH-13313) 2019-05-14 18:09:44 +02:00
Victor Stinner 3c93153f7d
bpo-36915: regrtest always remove tempdir of worker processes (GH-13312)
When using multiprocessing (-jN option), worker processes now create
their temporary directory inside the temporary directory of the
main process. So the main process is able to remove temporary
directories of worker processes even if they crash or when they are
killed by regrtest on KeyboardInterrupt (CTRL+c).

Rework also how multiprocessing arguments are parsed in main.py.
2019-05-14 15:49:16 +02:00
Vinodhini Balusamy c0a1a07c7e bpo-32995 - Added context variable in glossary (GH-9741) 2019-05-14 08:11:41 -04:00
Nick Coghlan dae1229729
bpo-36797: Prune more legacy distutils documentation (GH-13092)
Removes more legacy distutils documentation, and more clearly
marks what is left as potentially outdated, with references to
setuptools as a replacement.
2019-05-14 22:04:30 +10:00
Nicolai Moore 5e48e3db6f bpo-36845: validate integer network prefix when constructing IP networks (GH-13298) 2019-05-14 19:32:59 +09:00
Inada Naoki f0be4bbb9b
bpo-27987: pymalloc: align by 16bytes on 64bit platform (GH-12850) 2019-05-14 18:51:15 +09:00
Jens Troeger 45b2f8893c bpo-34424: Handle different policy.linesep lengths correctly. (#8803) 2019-05-13 21:07:39 -04:00
Pierre Glaser b1dfcad6f0 bpo-36867: Create the resource_tracker before launching SharedMemoryManagers (GH-13276) 2019-05-13 21:15:32 +02:00
Victor Stinner b0917df329
bpo-36719: regrtest -jN no longer stops on crash (GH-13231)
"python3 -m test -jN ..." now continues the execution of next tests
when a worker process crash (CHILD_ERROR state). Previously, the test
suite stopped immediately. Use --failfast to stop at the first error.

Moreover, --forever now also implies --failfast.
2019-05-13 19:17:54 +02:00
Guido van Rossum 85c69d5c4c Fix typo in NEWS item about IDLE (os.flush() should be os.fsync()) (#13284) 2019-05-13 12:00:53 -04:00
Guido van Rossum 4f098b35f5 bpo-36807: When saving a file in IDLE, call flush and fsync (#13102) 2019-05-13 08:31:29 -04:00
Victor Stinner d5d9e81ce9
bpo-36728: Remove PyEval_ReInitThreads() from C API (GH-13241)
Remove the PyEval_ReInitThreads() function from the Python C API.
It should not be called explicitly: use PyOS_AfterFork_Child()
instead.

Rename PyEval_ReInitThreads() to _PyEval_ReInitThreads() and add a
'runtime' parameter.
2019-05-13 12:35:37 +02:00
Zackery Spytz cf599f6f6f bpo-6584: Add a BadGzipFile exception to the gzip module. (GH-13022)
Co-Authored-By: Filip Gruszczyński <gruszczy@gmail.com>
Co-Authored-By: Michele Orrù <maker@tumbolandia.net>
2019-05-13 10:50:52 +03:00
Matthias Bussonnier e2500610c6 bpo-36895: remove time.clock() as per removal notice. (GH-13270)
`time.clock()` was deprecated in 3.3, and marked for removal removal in
3.8; this thus remove it from the time module.
2019-05-12 18:34:44 -07:00
Jake Tesler 4959c33d25 bpo-36084: Add native thread ID to threading.Thread objects (GH-11993) 2019-05-12 19:08:24 +02:00
Michael J. Sullivan d8320ecb86 bpo-36878: Allow extra text after `# type: ignore` comments (GH-13238)
In the parser, when using the type_comments=True option, recognize
a TYPE_IGNORE as anything containing `# type: ignore` followed by
a non-alphanumeric character. This is to allow ignores such as
`# type: ignore[E1000]`.
2019-05-11 19:17:24 +01:00
Pierre Glaser f22cc69b01 bpo-36867: Make semaphore_tracker track other system resources (GH-13222)
The multiprocessing.resource_tracker replaces the multiprocessing.semaphore_tracker module. Other than semaphores, resource_tracker also tracks shared_memory segments. Patch by Pierre Glaser.
2019-05-10 22:59:08 +02:00
Pierre Glaser d0d64ad1f5 bpo-36368: Ignore SIGINT in SharedMemoryManager servers. (GH-12483)
Fix a bug crashing SharedMemoryManager instances in interactive sessions after
a Ctrl-C (KeyboardInterrupt) was sent.
2019-05-10 20:42:35 +02:00
Eric Snow 86ea58149c
bpo-36737: Use the module state C-API for warnings. (gh-13159) 2019-05-10 13:29:55 -04:00
Jeroen Demeyer 351c67416b bpo-35983: skip trashcan for subclasses (GH-11841)
Add new trashcan macros to deal with a double deallocation that could occur when the `tp_dealloc` of a subclass calls the `tp_dealloc` of a base class and that base class uses the trashcan mechanism.

Patch by Jeroen Demeyer.
2019-05-10 19:21:10 +02:00
Olexa Bilaniuk 79efbb7193 bpo-24538: Fix bug in shutil involving the copying of xattrs to read-only files. (PR-13212)
Extended attributes can only be set on user-writeable files, but shutil previously
first chmod()ed the destination file to the source's permissions and then tried to
copy xattrs. This will cause failures if attempting to copy read-only files with
xattrs, as occurs with Git clones on Lustre FS.
2019-05-10 11:22:06 +08:00
Anthony Shaw 948ed8c96b bpo-36814: ensure os.posix_spawn() handles None (GH-13144)
Fix an issue where os.posix_spawn() would incorrectly raise a TypeError
when file_actions is None.
2019-05-10 04:00:06 +02:00
Rémi Lapeyre fce5ff1e18 bpo-27497: Add return value to csv.DictWriter.writeheader (GH-12306)
csv.DictWriter.writeheader() now returns the return value of the
underlying csv.Writer.writerow() method.

Patch contributed by Ashish Nitin Patil.
2019-05-10 03:50:11 +02:00
Jeroen Demeyer d237b3f0f6 bpo-36601: clarify signal handler comment and remove unnecessary pid check. (GH-12784)
https://bugs.python.org/issue36601
2019-05-09 18:28:56 -07:00
Victor Stinner d267ac20c3
bpo-36778: cp65001 encoding becomes an alias to utf_8 (GH-13230) 2019-05-10 03:19:54 +02:00
Julien Palard 137be34180
bpo-32523: Simplifying news entries with multiple paragraphs. (GH-8154)
Having multiple paragraphs in a few news entry lead to inconsistent
spacing while rendered in HTML by mixing "visually compact lists"
(when no entry of the whole list contains multiple paragraphs) and
"sparse lists" (when at least one do).
2019-05-09 21:52:32 +02:00
Andrew Svetlov a076e4f5e4
bpo-36802: Drop awrite()/aclose(), support await write() and await close() instead (#13099) 2019-05-09 15:14:58 -04:00
Aviv Palivoda e6576248e5 bpo-30262: Don't expose private objects in sqlite3 (GH-1440)
The Cache and Statement objects are undocumented and implementation
details of the sqlite3 module.

They aren't usable from pure Python code.
2019-05-09 21:05:45 +03:00
Pablo Galindo f00828a742
bpo-36851: Clean the frame stack if the execution ends with a return and the stack is not empty (GH-13191) 2019-05-09 16:52:02 +01:00
Julien Palard afd1e6d2f0
bpo-36239: Skip comments in gettext infos (GH-12255) 2019-05-09 16:22:15 +02:00
Benjamin Peterson 3aca40d3cb
closes bpo-36861: Update Unicode database to 12.1.0. (GH-13214)
Adds ㋿.
2019-05-08 20:59:35 -07:00
Pierre Glaser 289f1f80ee bpo-35900: Enable custom reduction callback registration in _pickle (GH-12499)
Enable custom reduction callback registration for functions and classes in
_pickle.c, using the new Pickler's attribute ``reducer_override``.
2019-05-08 23:08:25 +02:00
Eric V. Smith 9a4135e939
bpo-36817: Add f-string debugging using '='. (GH-13123)
If a "=" is specified a the end of an f-string expression, the f-string will evaluate to the text of the expression, followed by '=', followed by the repr of the value of the expression.
2019-05-08 16:28:48 -04:00
Pierre Glaser 65d98d0f53 bpo-35900: Add a state_setter arg to save_reduce (GH-12588)
Allow reduction methods to return a 6-item tuple where the 6th item specifies a
custom state-setting method that's called instead of the regular
``__setstate__`` method.
2019-05-08 21:40:25 +02:00
Brian Quinlan 39889864c0 bpo-26903: Limit ProcessPoolExecutor to 61 workers on Windows (GH-13132)
Co-Authored-By: brianquinlan <brian@sweetapp.com>
2019-05-08 14:04:53 -04:00
Gregory P. Smith 6bd81734de bpo-36816: Update the self-signed.pythontest.net cert (GH-13192)
We updated the server, our testsuite must match.


https://bugs.python.org/issue36816


✈️ CLE -> DEN ✈️ #pycon2019
2019-05-08 09:35:09 -07:00
Zackery Spytz 94a64e9cd4 bpo-24048: Save the live exception during import.c's remove_module() (GH-13005)
Save the live exception during the course of remove_module().
2019-05-08 12:31:23 -04:00
Jason R. Coombs b2758ff955 bpo-36832: add zipfile.Path (#13153)
* bpo-36832: add zipfile.Path

* bpo-36832: add documentation for zipfile.Path

* 📜🤖 Added by blurb_it.

* Remove module reference from blurb.

* Sort the imports

* Update docstrings and docs per recommendations.

* Rely on test.support.temp_dir

* Signal that 'root' is the parameter.

* Correct spelling of 'mod'

* Convert docstring to comment for brevity.

* Fix more errors in the docs
2019-05-08 09:45:05 -04:00
Michael Blahay b1c3167c23 bpo-27639: Correct return type for UserList slicing operation (#13169)
* BPO-27639: Correct return type for UserList slicing operation

Added logic to __getitem__ magic method for UserList to ensure that the return
type matches that of self.
2019-05-07 17:41:06 -04:00
Andrew Svetlov 1cc0ee7d9f bpo-36801: Fix waiting in StreamWriter.drain for closing SSL transport (GH-13098)
https://bugs.python.org/issue36801
2019-05-07 13:53:19 -07:00
Romain Picard b35acc5b3a bpo-35125: remove inner callback on outer cancellation in asyncio shield (GH-10340)
When the future returned by shield is cancelled, its completion callback of the
inner future is not removed. This makes the callback list of inner inner future
grow each time a shield is created and cancelled.

This change unregisters the callback from the inner future when the outer
future is cancelled.



https://bugs.python.org/issue35125
2019-05-07 11:58:24 -07:00
Vincent Michel 63deaa5b70 bpo-31922: Do not connect UDP sockets when broadcast is allowed (GH-423)
*Moved from python/asyncio#493.*

This PR fixes issue python/asyncio#480, as explained in [this comment](https://github.com/python/asyncio/issues/480#issuecomment-278703828).

The `_SelectorDatagramTransport.sendto` method has to be modified ~~so `_sock.sendto` is used in all cases (because it is tricky to reliably tell if the socket is connected or not). Could that be an issue for connected sockets?~~ *EDIT* ... so `_sock.send` is used only if `_sock` is connected.

It also protects `socket.getsockname` against `OSError` in `_SelectorTransport`. This might happen on Windows if the socket is not connected (e.g. for UDP broadcasting).


https://bugs.python.org/issue31922
2019-05-07 10:18:49 -07:00
Gregory P. Smith 64aa6d2000
bpo-36533: Reinit logging.Handler locks on fork(). (GH-12704)
Instead of attempting to acquire and release them all across fork
which was leading to deadlocks in some applications that had chained
their own handlers while holding multiple locks.
2019-05-07 12:18:20 -04:00
Edison A 5765ecf79f bpo-36783: Added C API Documentation for Time_FromTimeAndFold and PyDateTime_FromDateAndTimeAndFold (GH-13147) 2019-05-07 11:00:21 -04:00
Łukasz Langa bbbdc2bb65 Python 3.8.0a4
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE4/8oOcBIslwITevpsmmV4xAlBWgFAlzQfXsACgkQsmmV4xAl
 BWhB8g/+P+lbqKLZKb9X1W1deSMAoEt9fdHwRK4SXdSYiVAypMURR5qsexAuXILn
 VunaphcHniUq/LTMfyA6LAX8HjKPqwKPfHi7hKfwaoYKw/SCKi50tXgbuCGj/ZKK
 ix5HO76Jh4YFf6KV64c8Mkxas8Lk59ybLqAaI1D2maL9olYxvmmsEGpaXliEexfI
 heItUk5sGaz5ARXPaC5PlQEL3lgRE16dC1t1/vHCU1sseN2tf2XkIszsb9wF1ei/
 Wlg1gkDL6JetdUwxAyH4J/nZunSEeZ0MwJjzBGfWxc1fVhGzXDvyeWg2VQ3/OB46
 mH68RKOuVWLy+jQKbBuPDMOgyuTtM2L+bOmuGxKYkotL5EOIz6BmaRvH9VTJNoJ6
 xoUVHOSaOZDSVwA4EUBPNbYIVHj7E/11SJQKzBxDO15Pn/T9Zw8yz689yRDoFj42
 zCQK/a+2xvjm+WpvNZO4pwyoTOtvT7phirk7C9lV4KM3xH6jkGr6tHYkTFFRXfc0
 uXgIeq70jgb/qESojuBNQziVqhhHFhS6tZ/485jDYCkLytImZKRLhzOe9Xd+7WiI
 Vi1tUHbwgRmvimqNHFA7sFME3LI5ryDcDV7DLUL7sLK13m0iWQ4tSOT4TrERhGBG
 TL4KyAzspL5bC56Vt8DFuI/N4Byy5dN7tnFj7+5N7qgK1+Oshk0=
 =1r7h
 -----END PGP SIGNATURE-----
gpgsig -----BEGIN PGP SIGNATURE-----
 
 iQJEBAABCAAuFiEE4/8oOcBIslwITevpsmmV4xAlBWgFAlzRnIkQHGx1a2FzekBs
 YW5nYS5wbAAKCRCyaZXjECUFaGzeD/9im9jmefZiVy+5GU4wSkfD4YEOCoh+Mrln
 KvQPO6h0d8Py/u2UBmwaWCpg4yeB2lc39oTyU6EPvbe3e10fsb0pfdInXqPNomt0
 PiMWviLe31TWBb4uw1664rhcHSpmfDpKEtAmWO/fOnl2gQqWCYVY+GHdpYI9h65o
 XYcm8PUZ2Knl/x0XZnnmTSZGkyf8GdbBpNFfe3V3twH7uirkxXV7KTxjyH93D8DO
 z8tCOtBncCMWsiHPCJ+Al5Q/c5Ymu6/D6MzmGeQuNWAl5saaudUJVXz0lKiixkVP
 zJDdCSBavMijmAK4sjr0PGT+fQCTVAJOvO5gWW79PfIbZJCIEV+QriCpIa0wXmQ3
 OXBQgdAAK1M+BE2TLUm5LUw5eWhjzbUbrNU0vchluy+UGbgw0hT10J0hATAlZaIM
 Y0BCnZl7qkBVrbrX+Dm+qoMRawgqt7DmK5OxBgH9LEErm8ryTEg0wtOANr5DDwMF
 TretjFwAnMhuz1kguvCAePd3JF0LRoEaf5dW/h4hz/sX1ldGMK1zm2B4nU6Yst8v
 RaEzDvBbxYAvRtWQO1i8wDOv8/VTpAGXfTskwofmxUZ5g0bqhiCgEw0PpMyN82Rw
 E2nsdrzV06zeSQrOPUGuy2vypuDMiqdguLsjKJQb6igHl1ThhXU1Nr3UHit1HCVL
 p0awZnaRyw==
 =i7by
 -----END PGP SIGNATURE-----

Merge tag 'v3.8.0a4'

Python 3.8.0a4
2019-05-07 16:56:06 +02:00
Rémi Lapeyre 11a8832c98 bpo-31855: unittest.mock.mock_open() results now respects the argument of read([size]) (GH-11521)
unittest.mock.mock_open() results now respects the argument of read([size])

Co-Authored-By: remilapeyre <remi.lapeyre@henki.fr>
2019-05-07 11:48:36 +01:00
Andrew Svetlov ad4ed87241
Forbid creating of stream objects outside of asyncio (#13101) 2019-05-06 22:52:11 -04:00
Gregory P. Smith 2cc0223f43
bpo-35925: Skip SSL tests that fail due to weak external certs. (GH-13124)
Modern Linux distros such as Debian Buster have default OpenSSL system
configurations that reject connections to servers with weak certificates
by default.  This causes our test suite run with external networking
resources enabled to skip these tests when they encounter such a failure.

Fixing the network servers is a separate issue.
2019-05-06 17:54:06 -04:00
Serhiy Storchaka d53cf99dca
bpo-36542: Allow to overwrite the signature for Python functions. (GH-12705) 2019-05-06 22:40:27 +03:00
Serhiy Storchaka 96aeaec647
bpo-36793: Remove unneeded __str__ definitions. (GH-13081)
Classes that define __str__ the same as __repr__ can
just inherit it from object.
2019-05-06 22:29:40 +03:00
Łukasz Langa c1004b8546
Python 3.8.0a4 2019-05-06 20:30:25 +02:00
Zackery Spytz 1a2252ed39 bpo-36594: Fix incorrect use of %p in format strings (GH-12769)
In addition, fix some other minor violations of C99.
2019-05-06 12:56:50 -04:00
twisteroid ambassador 88f07a804a bpo-33530: Implement Happy Eyeballs in asyncio, v2 (GH-7237)
Added two keyword arguments, `delay` and `interleave`, to
`BaseEventLoop.create_connection`. Happy eyeballs is activated if
`delay` is specified.

We now have documentation for the new arguments. `staggered_race()` is in its own module, but not exported to the main asyncio package.


https://bugs.python.org/issue33530
2019-05-05 04:14:35 -07:00
Joannah Nanjekye 6b5b013bcc bpo-26978: Implement pathlib.Path.link_to (Using os.link) (GH-12990) 2019-05-04 17:27:10 +02:00
Stefan Behnel 47541689cc
bpo-28238: Implement "{*}tag" and "{ns}*" wildcard tag selection support for ElementPath, and extend the surrounding tests and docs. (GH-12997) 2019-05-03 20:58:16 +02:00
gescheit c1964e9e21 bpo-36613: call remove_done_callback if exception (GH-12800)
Call remove_done_callback() in finally block.


https://bugs.python.org/issue36613
2019-05-03 08:18:02 -07:00
xdegaye 4461d704e2 bpo-36341: Fix tests calling bind() on AF_UNIX sockets (GH-12399)
Those tests may fail with PermissionError.



https://bugs.python.org/issue36341
2019-05-03 08:09:17 -07:00
Zackery Spytz c4e78b116f bpo-14546: Fix the argument handling in Tools/scripts/lll.py (GH-13026) 2019-05-02 18:03:43 +03:00
Gregory P. Smith b7378d7728
bpo-30458: Use InvalidURL instead of ValueError. (GH-13044)
Use http.client.InvalidURL instead of ValueError as the new error case's exception.
2019-05-01 16:39:21 -04:00
Stefan Behnel e1d5dd645d
bpo-13611: C14N 2.0 implementation for ElementTree (GH-12966)
* Implement C14N 2.0 as a new canonicalize() function in ElementTree.

Missing features:
- prefix renaming in XPath expressions (tag and attribute text is supported)
- preservation of original prefixes given redundant namespace declarations
2019-05-01 22:34:13 +02:00
Stefan Behnel dde3eebdaa
bpo-36676: Namespace prefix aware parsing support for the ET.XMLParser target (GH-12885)
* bpo-36676: Implement namespace prefix aware parsing support for the XMLParser target in ElementTree.
2019-05-01 21:49:58 +02:00
Stefan Behnel 43851a202c
bpo-36673: Implement comment/PI parsing support for the TreeBuilder in ElementTree. (#12883)
* bpo-36673: Implement comment/PI parsing support for the TreeBuilder in ElementTree.

* bpo-36673: Rewrite the comment/PI factory handling for the TreeBuilder in "_elementtree" to make it use the same factories as the ElementTree module, and to make it explicit when the comments/PIs are inserted into the tree and when they are not (which is the default).
2019-05-01 21:20:38 +02:00
Petter Strandmark 3d37ea25dc bpo-27682: Handle client connection terminations in wsgiref (GH-9713) 2019-05-01 20:32:15 +03:00
Gregory P. Smith c4e671eec2
bpo-30458: Disallow control chars in http URLs. (GH-12755)
Disallow control chars in http URLs in urllib.urlopen.  This addresses a potential security problem for applications that do not sanity check their URLs where http request headers could be injected.
2019-04-30 19:12:21 -07:00
Victor Stinner 463b82a3ef
bpo-36763: Fix Py_SetStandardStreamEncoding() (GH-13028)
Fix memory leak in Py_SetStandardStreamEncoding(): release memory
if the function is called twice.
2019-05-01 01:36:13 +02:00
Mario Corchero 0df635c7f8 Don't report deleted attributes in __dir__ (GH#10148)
When an attribute is deleted from a Mock, a sentinel is added rather
than just deleting the attribute. This commit checks for such sentinels
when returning the child mocks in the __dir__ method as users won't
expect deleted attributes to appear when performing dir(mock).
2019-04-30 19:56:36 +01:00
Steve Dower d537ab0ff9
bpo-36742: Fixes handling of pre-normalization characters in urlsplit() (GH-13017) 2019-04-30 12:03:02 +00:00
Victor Stinner b84cb70880
bpo-36734: Fix compilation of faulthandler.c on HP-UX (GH-12970)
Initialize "stack_t current_stack" to zero using memset().
2019-04-30 12:19:34 +02:00
gescheit 3bbcc92577 bpo-25430: improve performance of IPNetwork.__contains__ (GH-1785)
make a compare in bit-operation manner.
2019-04-30 16:54:30 +09:00
Pablo Galindo d5d2b45469
bpo-36751: Deprecate getfullargspec and report positional-only args as regular args (GH-13016)
* bpo-36751: Deprecate getfullargspec and report positional-only args as regular args

* Use inspect.signature in testhelpers
2019-04-30 02:01:14 +01:00
Marco Rougeth 81c5a90595 bpo-36232: Improve error message on dbm.open() when the db doesn't exist (GH-12060) 2019-04-29 16:23:28 -07:00
Ned Deily 883dfc668f
bpo-34602: Avoid failures setting macOS stack resource limit (GH-13011)
Under some conditions the earlier fix for bpo-18075, "Infinite recursion
tests triggering a segfault on Mac OS X", now causes failures on macOS
when attempting to change stack limit with resource.setrlimit
resource.RLIMIT_STACK, like regrtest does when running the test suite.
The reverted change had specified a non-default stack size when linking
the python executable on macOS.  As of macOS 10.14.4, the previous
code causes a hard failure when running tests, although similar
failures had been seen under some conditions under some earlier
systems.  For now, revert the original change and resume using
the default stack size when linking the interpreter.
2019-04-29 15:07:39 -04:00
Berker Peksag be6dbfb43b
bpo-1613500: Don't hardcode output file mode in fileinput.FileInput (GH-12986) 2019-04-29 17:55:39 +03:00
Paul Ganssle 88c0937056 bpo-36004: Add date.fromisocalendar (GH-11888)
This commit implements the first version of date.fromisocalendar, the
inverse function for date.isocalendar.
2019-04-29 15:22:03 +02:00
xdegaye a86e06433a bpo-35952: Fix test.pythoninfo when the compiler is missing (GH-13007) 2019-04-29 14:53:30 +02:00
Pablo Galindo 8c77b8cb91
bpo-36540: PEP 570 -- Implementation (GH-12701)
This commit contains the implementation of PEP570: Python positional-only parameters.

* Update Grammar/Grammar with new typedarglist and varargslist

* Regenerate grammar files

* Update and regenerate AST related files

* Update code object

* Update marshal.c

* Update compiler and symtable

* Regenerate importlib files

* Update callable objects

* Implement positional-only args logic in ceval.c

* Regenerate frozen data

* Update standard library to account for positional-only args

* Add test file for positional-only args

* Update other test files to account for positional-only args

* Add News entry

* Update inspect module and related tests
2019-04-29 13:36:57 +01:00
xdegaye 7a5a1cfe04 bpo-36747: Remove the stale scriptsinstall Makefile target (GH-13003) 2019-04-29 11:01:42 +02:00
Joannah Nanjekye f781d202a2 bpo-36475: Finalize PyEval_AcquireLock() and PyEval_AcquireThread() properly (GH-12667)
PyEval_AcquireLock() and PyEval_AcquireThread() now
terminate the current thread if called while the interpreter is
finalizing, making them consistent with PyEval_RestoreThread(),
Py_END_ALLOW_THREADS, and PyGILState_Ensure().
2019-04-29 10:38:45 +02:00
xdegaye 254b309c80 bpo-21536: On Android, C extensions are linked to libpython (GH-12989) 2019-04-29 09:27:40 +02:00
Gordon P. Hemsley 50fed0b64f bpo-32424: Improve test coverage for xml.etree.ElementTree (GH-12891)
* Fix typo in test_cyclic_gc subtest
* Improve test coverage for xml.etree.ElementTree
2019-04-28 06:41:43 +02:00
Paul Ganssle 4d8c8c0ad6 bpo-36025: Fix PyDate_FromTimestamp API (GH-11922)
In the process of converting the date.fromtimestamp function to use
argument clinic in GH-8535, the C API for PyDate_FromTimestamp was
inadvertently changed to expect a timestamp object rather than an
argument tuple.

This PR fixes this backwards-incompatible change by adding a new wrapper
function for the C API function that unwraps the argument tuple and
passes it to the underlying function.

This PR also adds tests for both PyDate_FromTimestamp and
PyDateTime_FromTimestamp to prevent any further regressions.
2019-04-27 22:39:40 +03:00
Paul Ganssle 5c403b2035 bpo-36722: Style and grammar edits for ABI news entries (GH-12979) 2019-04-27 20:14:35 +02:00
Victor Stinner 75120d2205
bpo-36719: regrtest always detect uncollectable objects (GH-12951)
regrtest now always detects uncollectable objects. Previously, the
check was only enabled by --findleaks. The check now also works with
-jN/--multiprocess N.

--findleaks becomes a deprecated alias to --fail-env-changed.
2019-04-26 09:28:53 +02:00
Mark Dickinson 7abb6c05af
bpo-36669: add matmul support to weakref.proxy (GH-12932) 2019-04-26 15:56:15 +09:00
Victor Stinner 3cde440f20
bpo-36725: Refactor regrtest multiprocessing code (GH-12961)
Rewrite run_tests_multiprocess() function as a new MultiprocessRunner
class with multiple methods to better report errors and stop
immediately when needed.

Changes:

* Worker processes are now killed immediately if tests are
  interrupted or if a test does crash (CHILD_ERROR): worker
  processes are killed.
* Rewrite how errors in a worker thread are reported to
  the main thread. No longer ignore BaseException or parsing errors
  silently.
* Remove 'finished' variable: use worker.is_alive() instead
* Always compute omitted tests. Add Regrtest.get_executed() method.
2019-04-26 08:40:25 +02:00
Victor Stinner 5422e3cfb7
bpo-36722: Debug build loads libraries built in release mode (GH-12952)
In debug build, import now also looks for C extensions compiled in
release mode and for C extensions compiled in the stable ABI.
2019-04-26 01:40:00 +02:00
Paul Monson 62dfd7d6fe bpo-35920: Windows 10 ARM32 platform support (GH-11774) 2019-04-25 18:36:45 +00:00
Victor Stinner 8c3ecc6bac
bpo-21536: C extensions are no longer linked to libpython (GH-12946)
On Unix, C extensions are no longer linked to libpython.

It is now possible to load a C extension built using a shared library
Python with a statically linked Python.

When Python is embedded, libpython must not be loaded with
RTLD_LOCAL, but RTLD_GLOBAL instead. Previously, using RTLD_LOCAL, it
was already not possible to load C extensions which were not linked
to libpython, like C extensions of the standard library built by the
"*shared*" section of Modules/Setup.

distutils, python-config and python-config.py have been modified.
2019-04-25 20:13:10 +02:00
Victor Stinner 0ef8c157e9
bpo-28552: Fix distutils.sysconfig for empty sys.executable (GH-12875)
bpo-28552, bpo-7774: Fix distutils.sysconfig if sys.executable is
None or an empty string: use os.getcwd() to initialize project_base.

Fix also the distutils build command: don't use sys.executable if
it's evaluated as false (None or empty string).
2019-04-25 11:59:34 +02:00
Victor Stinner 235e7b2b0d
bpo-36465: Fix test_regrtest on Windows (GH-12945)
Fix Py_DEBUG constant: check for sys.gettotalrefcount attribute
rather than sys.getobjects.

Update also SpecialBuilds.txt documentation.
2019-04-25 01:46:31 +02:00
Victor Stinner f4e4703e74
bpo-36465: Make release and debug ABI compatible (GH-12615)
Release build and debug build are now ABI compatible: the Py_DEBUG
define no longer implies Py_TRACE_REFS define which introduces the
only ABI incompatibility.

A new "./configure --with-trace-refs" build option is now required to
get Py_TRACE_REFS define which adds sys.getobjects() function and
PYTHONDUMPREFS environment variable.

Changes:

* Add ./configure --with-trace-refs
* Py_DEBUG no longer implies Py_TRACE_REFS
2019-04-25 00:56:28 +02:00
Thomas Moreau 004b93ea89 bpo-36668: FIX reuse semaphore tracker for child processes (#5172)
Fix the multiprocessing.semaphore_tracker so it is reused by child processes.
2019-04-24 21:45:52 +02:00
Joannah Nanjekye 70bf713617 bpo-30840: Document relative imports (#12831)
* document relative imports

* 📜🤖 Added by blurb_it.

* fix indentation error

* remove indentation

* Document relative imports

* Document relative imports

* remove from ...package

* Document relative imports

* remove trailing space

* Document relative imports

* Document relative imports
2019-04-25 01:14:44 +10:00
Victor Stinner 6c44fde3e0
bpo-36707: Remove the "m" flag (pymalloc) from SOABI (GH-12931)
"./configure --with-pymalloc" no longer adds the "m" flag to SOABI
(sys.implementation.cache_tag).

Enabling or disabling pymalloc has no impact on the ABI.
2019-04-24 16:10:09 +02:00
Victor Stinner d246a6766b
bpo-36454: Fix test_time.test_monotonic() (GH-12929)
Change test_time.test_monotonic() to test only the lower bound of elapsed time
after a sleep command rather than the upper bound. This prevents unnecessary
test failures on slow buildbots. Patch by Victor Stinner.
2019-04-24 00:15:12 +02:00
Zackery Spytz 359bd4f61b bpo-18372: Add missing PyObject_GC_Track() calls in the pickle module (GH-8505) 2019-04-23 20:56:08 +09:00
Raymond Hettinger 9013ccf6d8
bpo-36546: Add statistics.quantiles() (#12710) 2019-04-23 00:06:35 -07:00
Marcin Niemira ab86521a9d bpo-36523: Add docstring to io.IOBase.writelines (GH-12683) 2019-04-22 20:13:51 +09:00
Xtreak 9b21856b0f bpo-23078: Add support for {class,static}method to mock.create_autospec() (GH-11613)
Co-authored-by: Felipe <felipe.nospam.ochoa@gmail.com>
2019-04-22 05:30:23 +03:00
Raymond Hettinger 14adbd4598
bpo-36650: Fix handling of empty keyword args in C version of lru_cache. (GH-12881) 2019-04-20 07:20:44 -10:00
Stefan Behnel e8113f51a8
bpo-30485: Change the prefix for defining the default namespace in ElementPath from None to '' since there is existing code that uses that and it's more convenient to have an all-string-keys dict (e.g. when sorting items etc.). (#12860) 2019-04-18 19:05:03 +02:00
Victor Stinner 23bace26ec
bpo-36635: Add _testinternalcapi module (GH-12841)
Add a new _testinternalcapi module to test the internal C API.

Move _Py_GetConfigsAsDict() function to the internal C API:
_testembed now uses _testinternalcapi to access the function.
2019-04-18 11:37:26 +02:00
josh a6de52c74d bpo-32913: Added re.Match.groupdict example to regex HOWTO (GH-5821) 2019-04-17 15:43:30 -07:00
Steve Dower 4c3efd9cd0
bpo-36649: Remove trailing spaces for registry keys when installed via the Store (GH-12865) 2019-04-17 14:31:32 -07:00
Victor Stinner 5c75f37d47
bpo-36635: Change pyport.h for Py_BUILD_CORE_MODULE define (GH-12853)
Change PyAPI_FUNC(type), PyAPI_DATA(type) and PyMODINIT_FUNC macros
of pyport.h when Py_BUILD_CORE_MODULE is defined.

The Py_BUILD_CORE_MODULE define must be now be used to build a C
extension as a dynamic library accessing Python internals: export the
PyInit_xxx() function in DLL exports on Windows.

Changes:

* Py_BUILD_CORE_BUILTIN and Py_BUILD_CORE_MODULE now imply
  Py_BUILD_CORE directy in pyport.h.
* ceval.c compilation now fails with an error if Py_BUILD_CORE is not
  defined, just to ensure that Python is build with the correct
  defines.
* setup.py now compiles _pickle.c with Py_BUILD_CORE_MODULE define.
* setup.py compiles _json.c with Py_BUILD_CORE_MODULE define, rather
  than Py_BUILD_CORE_BUILTIN define
* PCbuild/pythoncore.vcxproj: Add Py_BUILD_CORE_BUILTIN define.
2019-04-17 23:02:26 +02:00
Victor Stinner 3092d6b263
bpo-32849: Fix is_valid_fd() on FreeBSD (GH-12852)
Fix Python Initialization code on FreeBSD to detect properly when
stdin file descriptor (fd 0) is invalid.

On FreeBSD, fstat() must be used to check if stdin (fd 0) is valid.
dup(0) doesn't fail if stdin is invalid in some cases.
2019-04-17 18:09:12 +02:00
Victor Stinner 197f0447e3
bpo-35755: Don't say "to mimick Unix which command behavior" (GH-12861) 2019-04-17 17:44:06 +02:00
Lihua Zhao 693c104ae7 bpo-31904: Port test_resource to VxWorks (GH-12719)
Skip tests cases setting RLIMIT_FSIZE and RLIMIT_CPU on VxWorks.
2019-04-17 17:41:33 +02:00
Victor Stinner 2c4c02f8a8
bpo-35755: Remove current directory from posixpath.defpath (GH-11586)
Document the change in a NEWS entry of the Security category.
2019-04-17 17:05:30 +02:00
Victor Stinner 228a3c99bd
bpo-35755: shutil.which() uses os.confstr("CS_PATH") (GH-12858)
shutil.which() and distutils.spawn.find_executable() now use
os.confstr("CS_PATH") if available instead of os.defpath, if the PATH
environment variable is not set.

Don't use os.confstr("CS_PATH") nor os.defpath if the PATH
environment variable is set to an empty string to mimick Unix 'which'
command behavior.

Changes:

* find_executable() now starts by checking for the executable in the
  current working directly case. Add an explicit
  "if not path: return None".
* Add tests for PATH='' (empty string), PATH=':' and for PATHEXT.
2019-04-17 16:26:36 +02:00
Lihua Zhao 36c41bc201 bpo-31904: Fix test_tabnanny on VxWorks (GH-12646)
Fix test_tabnanny on VxWorks: adjust ENOENT error message,
use os.strerror().
2019-04-17 11:46:50 +02:00
Inada Naoki 926b0cb5f6
bpo-36641: Add "const" to PyDoc_VAR macro (GH-12854)
It reduces "data" segment in python about 200KB.
2019-04-17 08:39:46 +09:00
Stéphane Wirtel 2b7f93b99a
bpo-36345: Update wsgiref example (GH-12562)
Use literalinclude markup to include Tools/scripts/serve.py code.
Tools/scripts/serve.py first argument on the command line is now optional.
2019-04-16 14:52:54 +02:00
Victor Stinner 74125a60b7
bpo-36348: IMAP4.logout() doesn't ignore exc (GH-12411)
The imap.IMAP4.logout() method no longer ignores silently arbitrary
exceptions.

Changes:

* The IMAP4.logout() method now expects a "BYE" untagged response,
  rather than relying on _check_bye() which raises a self.abort()
  exception.
* IMAP4.__exit__() now does nothing if the client already logged out.
* Add more debug info if test_logout() tests fail.
2019-04-15 18:23:20 +02:00
Jakub Molinski a9a28808e5 bpo-36625: Remove obsolete comments from docstrings in fractions module (GH-12822)
Remove left-over references to Python 3.0 as the future in Fraction class docstrings.
2019-04-15 13:37:04 +01:00
Victor Stinner 3c7931e514
bpo-36629: Add support.get_socket_conn_refused_errs() (GH-12834)
Fix test_imap4_host_default_value() of test_imaplib: catch also
errno.ENETUNREACH error.
2019-04-15 12:34:53 +02:00
pxinwr f1464f4d2e bpo-31904: Port the time module on VxWorks (GH-12305)
time.clock() is not available on VxWorks.
2019-04-15 11:06:21 +02:00
pxinwr 236d0b75c4 bpo-31904: Don't build the _crypt extension on VxWorks (GH-12833) 2019-04-15 11:02:20 +02:00
Mickaël Schoentgen 929b704738 bpo-31658: Make xml.sax.parse accepting Path objects (GH-8564) 2019-04-14 11:16:54 +02:00
Stefan Behnel e9927e1820
bpo-30485: support a default prefix mapping in ElementPath by passing None as prefix (#1823) 2019-04-14 10:09:09 +02:00
Bernt Røskar Brenna ffca16e25a bpo-36227: ElementTree.tostring() default_namespace and xml_declaration arguments (GH-12225)
Add new keyword arguments "default_namespace" and "xml_declaration" to functions ET.tostring() and ET.tostringlist(), as known from ElementTree.write().
2019-04-14 10:07:02 +02:00
Xtreak 830b43d03c bpo-36593: Fix isinstance check for Mock objects with spec executed under tracing (GH-12790)
In Python having a trace function in effect while mock is imported causes isinstance to be wrong for MagicMocks. This is due to the usage of super() in some class methods, as this sets the __class__ attribute. To avoid this, as a workaround, alias the usage of super .
2019-04-13 20:12:33 +01:00
Pablo Galindo f2cf1e3e28
bpo-36623: Clean parser headers and include files (GH-12253)
After the removal of pgen, multiple header and function prototypes that lack implementation or are unused are still lying around.
2019-04-13 17:05:14 +01:00
Victor Stinner a304b136ad
bpo-36618: Don't add -fmax-type-align flag to old clang (GH-12811) 2019-04-13 00:51:07 +02:00
Sebastian Rittau 1e8295402b bpo-35581: Document @typing.type_check_only (GH-11312) 2019-04-12 15:33:40 -07:00
mrh1997 1e2ad6c275 bpo-33922: Adding documentation for new "-64" suffix of Python launcher (GH-7849)
Since bpo-30291 it is possible to specify the architecture of Python when using the launcher
2019-04-12 15:26:47 -07:00
Victor Stinner e8f9acf034
bpo-36611: Disable serialno field of debug memory allocators (#12796)
Omit serialno field from debug hooks on Python memory allocators to
reduce the memory footprint by 5%.

Enable tracemalloc to get the traceback where a memory block has been
allocated when a fatal memory error is logged to decide where to put
a breakpoint.

Compile Python with PYMEM_DEBUG_SERIALNO defined to get back the
field.
2019-04-12 21:54:06 +02:00
Victor Stinner 23a683adf8
bpo-36618: Add -fmax-type-align=8 flag for clang (GH-12809)
Add -fmax-type-align=8 to CFLAGS when clang compiler is detected.

The pymalloc memory allocator aligns memory on 8 bytes. On x86-64,
clang expects alignment on 16 bytes by default and so uses MOVAPS
instruction which can lead to segmentation fault. Instruct clang that
Python is limited to alignemnt on 8 bytes to use MOVUPS instruction
instead: slower but don't trigger a SIGSEGV if the memory is not
aligned on 16 bytes.

Sadly, the flag must be expected to CFLAGS and not just
CFLAGS_NODIST, since third party C extensions can have the same
issue.
2019-04-12 21:27:37 +02:00
Paul Monson f4e5661e85 bpo-36509: Add iot layout for Windows IoT containers (GH-12663)
This enables using the `--preset-iot` option with the PC/layout script, but does not enable IoT builds as part of any normal release.
2019-04-12 09:55:57 -07:00
Eric Snow b75b1a3504
bpo-33608: Revert "Factor out a private, per-interpreter _Py_AddPendingCall()." (gh-12806)
This reverts commit f13c5c8b94 (gh-12360).
2019-04-12 10:20:10 -06:00
Kingsley M b015fc86f7 bpo-36549: str.capitalize now titlecases the first character instead of uppercasing it (GH-12804) 2019-04-12 08:35:39 -07:00
Eric Snow f13c5c8b94
bpo-33608: Factor out a private, per-interpreter _Py_AddPendingCall(). (gh-12360)
This is effectively an un-revert of #11617 and #12024 (reverted in #12159). Portions of those were merged in other PRs (with lower risk) and this represents the remainder. Note that I found 3 different bugs in the original PRs and have fixed them here.
2019-04-12 09:18:16 -06:00
Victor Stinner 44235041f3
bpo-18748: io.IOBase destructor now logs close() errors in dev mode (GH-12786)
In development mode (-X dev) and in debug build, the io.IOBase
destructor now logs close() exceptions. These exceptions are silent
by default in release mode.
2019-04-12 17:06:47 +02:00
Michael Felt 9d949f7796 bpo-36588: On AIX, remove major version from sys.platform (GH-12787)
On AIX, sys.platform doesn't contain the major version anymore.
Always return 'aix', instead of 'aix3' .. 'aix7'.  Since
older Python versions include the version number, it is recommended to
always use sys.platform.startswith('aix').
2019-04-12 16:15:32 +02:00
Victor Stinner 9b8314cfe2
bpo-36611: Fix test_sys.test_getallocatedblocks() (GH-12797)
Fix test_sys.test_getallocatedblocks() when tracemalloc is enabled.
If the name of Python memory allocators cannot get read, consider
that pymalloc is disabled.

Fix the following error:

./python -X tracemalloc -m test test_sys -v -m test_getallocatedblocks

ERROR: test_getallocatedblocks (test.test_sys.SysModuleTest)
------------------------------------------------------------
Traceback (most recent call last):
  File "Lib/test/test_sys.py", line 770, in test_getallocatedblocks
    alloc_name = _testcapi.pymem_getallocatorsname()
RuntimeError: cannot get allocators name
2019-04-12 15:15:04 +02:00
Inada Naoki 9e4f2f3a6b
bpo-20180: Use argument clinic for dict.pop() and dict.popitem() (GH-12792) 2019-04-12 16:11:28 +09:00
Victor Stinner 21a74a9d77
bpo-36605: make tags: parse Modules/_io directory (GH-12789)
"make tags" and "make TAGS" now also parse Modules/_io/*.c
and Modules/_io/*.h.
2019-04-11 22:28:12 +02:00
Victor Stinner 4c409beb4c
bpo-36389: Change PyMem_SetupDebugHooks() constants (GH-12782)
Modify CLEANBYTE, DEADDYTE and FORBIDDENBYTE constants: use 0xCD,
0xDD and 0xFD, rather than 0xCB, 0xBB and 0xFB, to use the same byte
patterns than Windows CRT debug malloc() and free().
2019-04-11 13:01:15 +02:00
Inada Naoki 536a35b3f1
bpo-36575: lsprof: Use _PyTime_GetPerfCounter() (GH-8378) 2019-04-11 19:11:46 +09:00
Christian Heimes d914596a67 bpo-36559: random module: optimize sha512 import (GH-12742)
The random module now prefers the lean internal _sha512 module over hashlib
for seed(version=2) to optimize import time.

Signed-off-by: Christian Heimes <christian@python.org>
2019-04-10 13:18:02 -07:00
Lorenz Mende 6955d44b41 bpo-34144: Fix of venv acvtivate.bat for win 10 (GH-8321)
The script needs to be updated to support win 10/ 1803 chcp.com command (output has trailing dot)





https://bugs.python.org/issue34144
2019-04-10 10:19:16 -07:00
Vidar Tonaas Fauske 0e10766574 bpo-31512: Add non-elevated symlink support for Windows (GH-3652) 2019-04-09 11:19:46 -07:00
Victor Stinner 8709490f48
bpo-34373: Fix time.mktime() on AIX (GH-12726)
Fix time.mktime() error handling on AIX for year before 1970.

Other changes:

* mktime(): rename variable 'buf' to 'tm'.
* _PyTime_localtime():

  * Use "localtime" rather than "ctime" in the error message
    (specific to AIX).
  * Always initialize errno to 0 just in case if localtime_r()
    doesn't set errno on error.
  * On AIX, avoid abs() which is limited to int type.
  * EINVAL constant is now always available.
2019-04-09 19:12:26 +02:00
Christian Heimes 8abc3f4f91 bpo-36577: setup.py reports missing OpenSSL again (GH-12746)
[bpo-36146](https://bugs.python.org/issue36146) introduced another regression. In case of missing OpenSSL
libraries or headers, setup.py no longer reported _hashlib and _ssl to
be missing.

Signed-off-by: Christian Heimes <christian@python.org>



https://bugs.python.org/issue36577
2019-04-09 09:40:12 -07:00
Victor Stinner e65f01f78d
bpo-36508: python-config don't export LINKFORSHARED (GH-12661)
python-config --ldflags no longer includes flags of the LINKFORSHARED
variable. The LINKFORSHARED variable must only be used to build
executables.
2019-04-09 18:12:44 +02:00
xdegaye 2ee077f795 bpo-36544 : Fix regression introduced in bpo-36146 (GH-12708)
https://bugs.python.org/issue36544
2019-04-09 08:20:08 -07:00
Quentin Dawans 56065d4c8a bpo-34139: Remove unix datagram socket from FS before binding (GH-8323)
https://bugs.python.org/issue34139
2019-04-09 06:40:59 -07:00
Victor Stinner 5aaac94eeb
bpo-36560: Fix reference leak hunting in regrtest (GH-12744)
Fix reference leak hunting in regrtest: compute also deltas (of
reference count, allocated memory blocks, file descriptor count)
during warmup, to ensure that everything is initialized before
starting to hunt reference leaks.

Other changes:

* Replace gc.collect() with support.gc_collect()
* Move calls to read memory statistics from dash_R_cleanup() to
  dash_R()
* Pass regrtest 'ns' to dash_R()
* dash_R() is now more quiet with --quiet option (don't display
  progress).
* Precompute the full range for "for it in range(repcount):" to
  ensure that the iteration doesn't allocate anything new.
* dash_R() now is responsible to call warm_caches().
2019-04-09 14:23:47 +02:00
Ammar Askar e16467af0b bpo-34060: Report system load when running test suite for Windows (GH-8357)
While Windows exposes the system processor queue length, the raw value
used for load calculations on Unix systems, it does not provide an API
to access the averaged value. Hence to calculate the load we must track
and average it ourselves. We can't use multiprocessing or a thread to
read it in the background while the tests run since using those would
conflict with test_multiprocessing and test_xxsubprocess.

Thus, we use Window's asynchronous IO API to run the tracker in the
background with it sampling at the correct rate. When we wish to access
the load we check to see if there's new data on the stream, if there is,
we update our load values.
2019-04-09 08:20:41 -04:00
Matthias Bussonnier a8abe097c1 bpo-33461: emit DeprecationWarning when json.loads(encoding=...) is used (GH-6762) 2019-04-09 16:17:25 +09:00
Giampaolo Rodola 8702b67dad
BPO-17561: set create_server backlog default to None (GH-12735)
It turns out doing socket.listen(0) does not equal to "choose a
reasonable default". It actually means "set backlog to 0".
As such set backlog=None as the default for socket.create_server.
Fixes the following BB failures:
https://github.com/python/cpython/pull/11784#issuecomment-481036369
Ref. BPO-1756, GH-11784.
2019-04-09 04:42:06 +02:00
Victor Stinner 79b5d29041
bpo-36565: Fix libregrtest for Python without builtin _abc (GH-12733)
Fix reference hunting (``python3 -m test -R 3:3``) when Python has no
built-in abc module: fix _get_dump() reimplementation of libregrtest.
2019-04-09 01:36:34 +02:00
Giampaolo Rodola eb7e29f2a9
bpo-35934: Add socket.create_server() utility function (GH-11784) 2019-04-09 00:34:02 +02:00
Mickaël Schoentgen 58721a9030 bpo-35416: fix potential resource warnings in distutils (GH-10918) 2019-04-08 22:08:48 +09:00
Michael Felt b7eec94c0e bpo-36503: remove references to 'aix3' and 'aix4' (GH-12658) 2019-04-08 09:51:33 +09:00
Raymond Hettinger 6463ba3061
bpo-27181: Add statistics.geometric_mean() (GH-12638) 2019-04-07 09:20:03 -07:00
Brandt Bucher 9d7b2c0909 bpo-35936: Updates to modulefinder (GH-11787)
* Properly handle SyntaxErrors in Python source files.

SyntaxErrors in the target module will rise normally, while SyntaxErrors in dependencies will be added to badmodules. This includes a new regression test.

* Fix name collision bug.

This fixes an issue where a "fromlist" import with the same name as a previously failed import would be incorrectly added to badmodules. This includes a new regression test.

* Replace mutable default values.

Bound empty lists have been replaced with the "if param is None" idiom.

* Replace deprecated imp usage.

Constants imported from imp have been moved to private module-level constants, and ModuleFinder.find_module has been refactored to use importlib. Other than an improvement on how frozen builtin imports are reported (as the frozen imports they are, rather than the stdlib modules they *may* have originated from), these changes maintain complete compatibility with past versions... including odd behavior for returning relative (below current directory, but not a C extension) vs. absolute (above current directory, or a C extension) paths.

Patch by Brandt Bucher.
2019-04-07 18:00:41 +10:00
Inada Naoki d6bf6f2d0c
bpo-36050: optimize HTTPResponse.read() (GH-12698)
* No need to chunking for now.
* No need to partial read caused by EINTR for now.
2019-04-06 18:06:19 +09:00
Zackery Spytz 50866e9ed3 bpo-25451: Add transparency methods to tkinter.PhotoImage. (GH-10406) 2019-04-05 13:17:13 +03:00
Matt Houglum 461c416dd7 bpo-36522: Print all values for headers with multiple values. (GH-12681) 2019-04-04 07:36:47 +03:00
tyomitch cb0748d393 bpo-36440: include node names in ParserError messages, instead of numeric IDs (GH-12565)
The error messages in the parser module are referring to numeric IDs for the nodes. To improve readability, use the node names when reporting errors.
2019-04-03 01:12:07 -04:00
Jeroen Demeyer fcef60f59d bpo-33261: guard access to __code__ attribute in inspect (GH-6448) 2019-04-02 16:03:42 +02:00
Zackery Spytz 487b73ab39 bpo-36504: Fix signed integer overflow in _ctypes.c's PyCArrayType_new(). (GH-12660) 2019-04-02 13:47:51 +03:00
Thomas Perl b8311cf5e5 bpo-36473: add maximum iteration check for dict .values() and .items() (GH-12619) 2019-04-02 18:30:10 +09:00
Joannah Nanjekye 8c61739def bpo-36157:Document PyInterpreterState_Main() (GH-12238)
I have added documentation for `PyInterpreterState_Main()`.
 I chose to place it under Advanced Debugger Support together with similar functions like `PyInterpreterState_Head()`, `PyInterpreterState_Next(`), and `PyInterpreterState_ThreadHead()` .


https://bugs.python.org/issue36157
2019-04-01 08:08:43 -07:00
Inada Naoki 10654c19b5
bpo-20844: open script file with "rb" mode (GH-12616) 2019-04-01 18:35:20 +09:00
Serhiy Storchaka 42a139ed88
bpo-36492: Deprecate passing some arguments as keyword arguments. (GH-12637)
Deprecated passing the following arguments as keyword arguments:

- "func" in functools.partialmethod(), weakref.finalize(),
  profile.Profile.runcall(), cProfile.Profile.runcall(),
  bdb.Bdb.runcall(), trace.Trace.runfunc() and
  curses.wrapper().
- "function" in unittest.addModuleCleanup() and
  unittest.TestCase.addCleanup().
- "fn" in the submit() method of concurrent.futures.ThreadPoolExecutor
  and concurrent.futures.ProcessPoolExecutor.
- "callback" in contextlib.ExitStack.callback(),
  contextlib.AsyncExitStack.callback() and
  contextlib.AsyncExitStack.push_async_callback().
- "c" and "typeid" in the create() method of multiprocessing.managers.Server
  and multiprocessing.managers.SharedMemoryServer.
- "obj" in weakref.finalize().

Also allowed to pass arbitrary keyword arguments (even "self" and "func")
if the above arguments are passed as positional argument.
2019-04-01 09:16:35 +03:00
Paul Moore e724152796 bpo-36010: Add venv to the nuget distribution (GH-12367) 2019-03-30 09:32:05 -07:00
Joel Croteau e653d4d8e8 bpo-36384: Remove check for leading zeroes in IPv4 addresses (GH-12577)
Stop rejecting IPv4 octets with leading zeroes as ambiguously octal.

Plenty of other tools generate decimal IPv4 octets with leading zeroes,
so keeping this check hurts interoperability.

Patch by Joel Croteau.
2019-03-31 00:53:48 +10:00
Serhiy Storchaka 2524fdefc9
bpo-36434: Properly handle writing errors in ZIP files. (GH-12559)
Errors during writing no longer prevent to properly close
the ZIP file.
2019-03-30 08:25:19 +02:00
Serhiy Storchaka 7a465cb5ee
bpo-24214: Fixed the UTF-8 incremental decoder. (GH-12603)
The bug occurred when the encoded surrogate character is passed
to the incremental decoder in two chunks.
2019-03-30 08:23:38 +02:00
Steve Dower 2438cdf0e9
bpo-36085: Enable better DLL resolution on Windows (GH-12302) 2019-03-29 16:37:16 -07:00
Paul Monson 32119e10b7 bpo-35947: Update Windows to the current version of libffi (GH-11797)
We now use a pre-built libffi binary from our binaries repository, and no longer vendor the full implementation.
2019-03-29 16:30:10 -07:00
Susan Su 081158e3ba bpo-33043: Add a Contributing to Docs link and Update the Found a Bug Page (#12006)
* changes to html file -> added contributing to docs link at the end of the page

* revisions to the dealing with bugs page. added more links in the documentation bugs section

* 📜🤖 Added by blurb_it.

* Update Doc/bugs.rst

Updated Doc/bugs.rst in accordance with willingc and JulienPalard suggestions.

Co-Authored-By: suhearsawho <susansu.software@gmail.com>
2019-03-28 18:55:24 -07:00
Wolfgang Maier 74510e2a57 bpo-30427: eliminate redundant type checks in os.path.normcase() (GH-1712)
https://bugs.python.org/issue30427
2019-03-28 14:47:18 -07:00
Xtreak 02b84cb1b4 bpo-36366: Return None on stopping unstarted patch object (GH-12472)
Return None after calling unittest.mock.patch.object.stop() regardless of whether the object was started. This makes the method idempotent.


https://bugs.python.org/issue36366
2019-03-28 14:08:43 -07:00
kctherookie d93fbbf88e bpo-35941: Fix ssl certificate enumeration for windows (GH-12486)
Add a function to collect certificates from several certificate stores into one certificate collection store that is then enumerated. This ensures we load as many certificates as we can access.
2019-03-28 10:59:06 -07:00
zhsj 45a5fdb91c bpo-36425: Add Simplified Chinese to the language switcher (GH-12537) 2019-03-28 16:32:25 +01:00
Giampaolo Rodola 3eca28c613
bpo-29515: add missing socket.IPPROTO_* constants on Windows (GH-12183) 2019-03-28 15:20:30 +01:00
Zackery Spytz cda139d1de bpo-36459: Fix a possible double PyMem_FREE() due to tokenizer.c's tok_nextc() (12601)
Remove the PyMem_FREE() call added in cb90c89.  The buffer will be
freed when PyTokenizer_Free() is called on the tokenizer state.
2019-03-28 15:53:00 +02:00
Thomas Perl 796cc6e3ad bpo-36452: dictiter: track maximum iteration count (GH-12596) 2019-03-28 15:03:25 +09:00
Ned Deily 738cb42a14
Fix NEWS entries with incorrect bpo numbers (GH-12599) 2019-03-28 00:19:30 -04:00
Philipp A d5a5a33f12 bpo-31292: Fixed distutils check --restructuredtext for include directives (GH-10605) 2019-03-27 17:34:19 -04:00
Victor Stinner d929f1838a
bpo-36443: Disable C locale coercion and UTF-8 Mode by default (GH-12589)
bpo-36443, bpo-36202: Since Python 3.7.0, calling Py_DecodeLocale()
before Py_Initialize() produces mojibake if the LC_CTYPE locale is
coerced and/or if the UTF-8 Mode is enabled by the user
configuration. This change fix the issue by disabling LC_CTYPE
coercion and UTF-8 Mode by default. They must now be enabled
explicitly (opt-in) using the new _Py_PreInitialize() API with
_PyPreConfig.

When embedding Python, set coerce_c_locale and utf8_mode attributes
of _PyPreConfig to -1 to enable automatically these parameters
depending on the LC_CTYPE locale, environment variables and command
line arguments

Alternative: Setting Py_UTF8Mode to 1 always explicitly enables the
UTF-8 Mode.

Changes:

* _PyPreConfig_INIT now sets coerce_c_locale and utf8_mode to 0 by
  default.
* _Py_InitializeFromArgs() and _Py_InitializeFromWideArgs() can now
  be called with config=NULL.
2019-03-27 18:28:46 +01:00
Steve Dower 4a9a505d6f
bpo-36441: Fixes creating a venv when debug binaries are installed. (#12566) 2019-03-27 08:14:53 -07:00
hliu0 f4333d0479 bpo-31904: Fix test_utf8_mode on VxWorks (GH-12428)
Python always use UTF-8 on VxWorks.
2019-03-27 16:11:12 +01:00
Eddie Elizondo 364f0b0f19 bpo-35810: Incref heap-allocated types in PyObject_Init (GH-11661)
* Incref heap-allocated types in PyObject_Init
* Add documentation and porting notes to What's New
2019-03-27 12:52:18 +01:00
Vladimir Surjaninov 384b81d923 bpo-36407: Fix writing indentations of CDATA section (xml.dom.minidom). (GH-12514) 2019-03-27 07:58:49 +02:00
Andre Delfino f760610bdd bpo-33832: Add "magic method" glossary entry (GH-7630) 2019-03-26 21:21:27 -04:00
Terry Jan Reedy 6a258c8890
bpo-36429: Fix starting IDLE with pyshell (#12548)
Add idlelib.pyshell alias at top; remove pyshell alias at bottom.
Remove obsolete __name__=='__main__' command.
2019-03-26 19:58:19 -04:00
Victor Stinner 414b1cde93
bpo-36436: Fix _testcapi.pymem_buffer_overflow() (GH-12560)
Handle memory allocation failure.
2019-03-26 14:35:30 +01:00
Inada Naoki 871309c775 bpo-36433: fix confusing error messages in classmethoddescr_call (GH-12556)
https://bugs.python.org/issue36433
2019-03-26 02:26:33 -07:00
Łukasz Langa 9f966dfd90 Python 3.8.0a3
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE4/8oOcBIslwITevpsmmV4xAlBWgFAlyZLdQACgkQsmmV4xAl
 BWi6Gg/+LvFG+HLD5t80y4mIJlPAxxhyFyPQNw4AlQ9Tt6perkkLVMa0ugvBPUE4
 fzwaofesW8CZEGR/wV/DGgZFACLbSNxBrrwMYD4QJ0+T/LoNeY9zGlnKOrlGbzvJ
 4tQtQF5hjaIxfTMq2pH+NP4ODuUm3zuVgtVdDmK02kRqvhiCiw1RFyDGaVgrm/uM
 i78NUsih5oq7uv+pfWIfrGx7Wn4hsqW/Xq+N5S9YsNAgCzqGhfVPdU2gtw8nhq/F
 SBBYkSsg4aOzlrZTot4SBffMBQ66nvvoAQfiAh2Etf7/PpzpdjG5BLTVx0UHIzCS
 PIhkeOf27HAi6ul9xzgnJMKQXjdpU2JinzURWqAIz/gd3Kyd7iM5TPRMoWNx0iaN
 MYdJaPjmBHTiMDYS9MoiSc8XWVqzmratMHMKsIq2oQbCHzU4OV5cLFE46f69BrVD
 Ml3JrxpvzT8GZhI9zTK1Su14gGdtsndv1P+nW5/eusCno4IWlDLpc9GAFBGklsl9
 5T5nvckfZc/RsyeQFuoayXYJi9WfX+ib2PUVk9aFx3j0zFN3yVXevKqOBWtn6dl1
 flUvu7bOCCHseFCGyraC8+Kr15i/Bf7LTZZZ9eaQVBpTSd5GwlRw68dAmcXZ8PIz
 8tYaZtF6WwAilYGBhxs7spN6ridGjJ3vMi7jGVZzwPVAy3Ky05k=
 =JzCc
 -----END PGP SIGNATURE-----
gpgsig -----BEGIN PGP SIGNATURE-----
 
 iQJEBAABCAAuFiEE4/8oOcBIslwITevpsmmV4xAlBWgFAlyZ7CQQHGx1a2FzekBs
 YW5nYS5wbAAKCRCyaZXjECUFaIFCD/4xM7QqftFYOhNdIibuoUeZZMpSNx85/fZL
 r8RLPSFQoUMS3fR21QmlXy2VHJzQfeyNBN8w2firnWBWL2vGmXL+Abfg2jG3MvD/
 cxQskRuarp+VAPiLYvN2dPavnq0p1ZNdqU74np+ZgH942+1TbBhQH+A8CpBKnNPq
 5K8hMiAx3alRq+Pkj0ro325xZeAvjc/mo866KB9kxNEl5S+uXJEEw3Z3FcrHaj98
 nxXdfTWFVhur2KeG44HxR+GTw35Sbbi6kbzmnUkiN+itSCZVPqyMQD514z6QYsnD
 JLjnaDt+kKKBTArMqLffmS057UcHJnhmfa9VkwkTIhtqwA388iL4A1lWOoGYe6Xq
 8S/BJZ7QCH3bawoIq1t8w51kWnWnsCITMN6oYhNQpGUKDtDAcgqyGqHaMLgIBn2L
 51ZVCAIc8QawWx0Jf1eEivTc66SDTR6LQhIuDbS5ElKAkwPCxW+iSWqyv3qsQq2q
 mwSGOMpB/xDvhUEDkgUrFflEKz92mESAcwO+Sz3nViKM46wj84PLyCAHDdoC3oJn
 5m+JwaKAhGob0aQNQd/FU61gP5qChKXVVB6vBo8b6iWZ8eOMs92HQk5zbKeC5L2A
 eLx5EioVp2ZFYCKusFaad2mu/wFYkRabtc5ZpA0UgNGHrHnwAmQUknmRajbn7h2U
 +58Rf0ozrw==
 =nJsD
 -----END PGP SIGNATURE-----

Merge tag 'v3.8.0a3'

Python 3.8.0a3
2019-03-26 10:08:49 +01:00
Zackery Spytz 0523c39e77 bpo-36430: Fix a possible reference leak in itertools.count(). (GH-12551) 2019-03-26 08:05:29 +02:00
Stéphane Wirtel 360e1e4c51 bpo-36345: Add a new example in the documentation of wsgiref (#12511) 2019-03-25 15:52:56 -07:00
Pablo Galindo 91759d9801
bpo-36143: Regenerate Lib/keyword.py from the Grammar and Tokens file using pgen (GH-12456)
Now that the parser generator is written in Python (Parser/pgen) we can make use of it to regenerate the Lib/keyword file that contains the language keywords instead of parsing the autogenerated grammar files. This also allows checking in the CI that the autogenerated files are up to date.
2019-03-25 22:01:12 +00:00
Raymond Hettinger d1e768a677
bpo-36326: Let inspect.getdoc() find docstrings for __slots__ (GH-12498) 2019-03-25 13:01:13 -07:00
Łukasz Langa 9a448855b5
v3.8.0a3 2019-03-25 20:36:40 +01:00
Zackery Spytz 5e333784f0 bpo-36421: Fix a possible double decref in _ctypes.c's PyCArrayType_new(). (GH-12530)
Set type_attr to NULL after the assignment to stgdict->proto (like
what is done with stgdict after the Py_SETREF() call) so that it is
not decrefed twice on error.
2019-03-25 10:07:47 +02:00
Rémi Lapeyre dd5417afcf bpo-36218: Fix handling of heterogeneous values in list.sort (GH-12209) 2019-03-25 00:25:37 -07:00
Raymond Hettinger 62be33870e
bpo-36401: Have help() show readonly properties separately (GH-12517) 2019-03-24 17:07:47 -07:00
Louie Lu 113d735e20 bpo-30348: IDLE: Add test_autocomplete unittest (GH-2209) 2019-03-24 19:33:12 -04:00
Terry Jan Reedy 0fe4513d9a
bpo-36405: IDLE - Restore __main__ and add tests (#12518)
Fix error in commit 2b75155 noticed by Serhiy Storchaka.
2019-03-24 17:12:28 -04:00
Zackery Spytz 3d07c1ee1d bpo-36412: fix a possible crash in dictobject.c's new_dict() (GH-12519) 2019-03-24 11:23:29 +09:00
AraHaan a7987e7193 bpo-32217: Correct usage of ABI tags in freeze. (GH-4719)
Check for sys.abiflags before using since not all platforms have it defined.
2019-03-23 12:29:49 -04:00
Inada Naoki d3c72a223a
bpo-36381: warn when no PY_SSIZE_T_CLEAN defined (GH-12473)
We will remove int support from 3.10 or 4.0.
2019-03-23 21:04:40 +09:00
Cheryl Sabella d60f658fc0
bpo-23205: IDLE: Add tests and refactor grep's findfiles (GH-12203)
* Add tests for grep findfiles.
* Move findfiles to module function.
* Change findfiles to use os.walk.

Based on a patch by Al Sweigart.
2019-03-23 07:33:42 -04:00
Terry Jan Reedy 2b75155590
bpo-36405: Use dict unpacking in idlelib (#12507)
Remove now unneeded imports.
2019-03-23 03:50:15 -04:00
Terry Jan Reedy c1419578a1
bpo-36396: Remove fgBg param of idlelib.config.GetHighlight() (GH-12491)
This param was only used once and changed the return type.
2019-03-22 18:23:41 -04:00
Brett Cannon 5086589305 bpo-36298: Raise ModuleNotFoundError in pyclbr when a module can't be found (GH-12358)
Before, an `AttributeError` was raised due to trying to access an attribute that exists on specs but having received `None` instead for a non-existent module.


https://bugs.python.org/issue36298
2019-03-22 15:16:50 -07:00
Rémi Lapeyre 96831c7fcf bpo-30670: Add pp function to the pprint module (GH-11769) 2019-03-22 10:22:20 -07:00
Kumar Akshay b0df45e55d bpo-21269: Provide args and kwargs attributes on mock call objects GH11807 2019-03-22 08:10:40 +00:00
Zackery Spytz 93e8012f2c bpo-36398: Fix a possible crash in structseq_repr(). (GH-12492)
If the first PyUnicode_DecodeUTF8() call fails in structseq_repr(),
_PyUnicodeWriter_Dealloc() will be called on an uninitialized
_PyUnicodeWriter.
2019-03-22 09:24:34 +02:00
Pablo Galindo 9a0000d15d
bpo-36256: Fix bug in parsermodule when parsing if statements (GH-12477)
bpo-36256: Fix bug in parsermodule when parsing if statements

In the parser module, when validating nodes before starting the parsing with to create a ST in "parser_newstobject" there is a problem that appears when two arcs in the same DFA state has transitions with labels with the same type. For example, the DFA for if_stmt has a state with
two labels with the same type: "elif" and "else" (type NAME). The algorithm tries one by one the arcs until the label that starts the arc transition has a label with the same type of the current child label we are trying to accept. In this case, the arc for "elif" comes before the arc for "else"and passes this test (because the current child label is "else" and has the same type as "elif"). This lead to expecting a namedexpr_test (305) instead of a colon (11). The solution is to compare also the string representation (in case there is one) of the labels to see if the transition that we have is the correct one.
2019-03-21 23:33:02 +00:00
CAM Gerlach e680c3db80 bpo-36268: Change default tar format to pax from GNU. (GH-12355) 2019-03-21 16:44:51 +02:00
sth aa3ecb8041 bpo-36285: Fix integer overflow in the array module. (GH-12317) 2019-03-20 21:49:39 +02:00
Serhiy Storchaka c1e2c288f4
bpo-36312: Fix decoders for some code pages. (GH-12369) 2019-03-20 21:45:18 +02:00
Zackery Spytz 9b4a1b1e23 bpo-36374: Fix a possible null pointer dereference (GH-12449)
https://bugs.python.org/issue36374
2019-03-20 02:16:25 -07:00
Victor Stinner c70ab02df2
bpo-36365: Rewrite structseq_repr() using _PyUnicodeWriter (GH-12440)
No longer limit repr(structseq) to 512 bytes. Use _PyUnicodeWriter
for better performance and to write directly Unicode rather than
encoding repr() value to UTF-8 and then decoding from UTF-8.
2019-03-20 00:05:51 +01:00
Victor Stinner dcf617152e
bpo-36236: Handle removed cwd at Python init (GH-12424)
At Python initialization, the current directory is no longer
prepended to sys.path if it has been removed.

Rename _PyPathConfig_ComputeArgv0() to
_PyPathConfig_ComputeSysPath0() to avoid confusion between argv[0]
and sys.path[0].
2019-03-19 16:09:27 +01:00
Stéphane Wirtel 943395fab9 bpo-36333: Fix leak _PyRuntimeState_Fini (GH-12400) 2019-03-19 11:51:32 +01:00
btharper e130a07eb2 bpo-36356: Fix memory leak in _PyPreConfig_Read() (GH-12425)
_PyPreConfig_Read() now free 'old_old' at exit.
2019-03-19 11:50:25 +01:00
Raymond Hettinger 714c60d7ac
bpo-36324: Add inv_cdf() to statistics.NormalDist() (GH-12377) 2019-03-18 20:17:14 -07:00
Victor Stinner faddaedd05
bpo-36352: Avoid hardcoded MAXPATHLEN size in getpath.c (GH-12423)
* Use Py_ARRAY_LENGTH() rather than hardcoded MAXPATHLEN in getpath.c.
* Pass string length to functions modifying strings.
2019-03-19 02:58:14 +01:00
Victor Stinner 5f9cf23502
bpo-36301: Error if decoding pybuilddir.txt fails (GH-12422)
Python initialization now fails if decoding pybuilddir.txt
configuration file fails at startup.

_PyPathConfig_Calculate() now reports memory allocation failure and
decoding error on decoding pybuilddir.txt content from
UTF-8/surrogateescape.
2019-03-19 01:46:25 +01:00
Raymond Hettinger f7b57df0c0 bpo-36320: Switch typing.NamedTuple from OrderedDict to regular dict (GH-12396)
Also,  deprecate the *_field_types* attributes which duplicated the information in *\__annotations__*.


https://bugs.python.org/issue36320
2019-03-18 09:53:56 -07:00
Pablo Galindo 0c9258a6d2
bpo-36332: Allow compile() to handle AST objects with assignment expressions (GH-12398) 2019-03-18 13:51:53 +00:00
Stéphane Wirtel 09a9f1799c bpo-36329: Declare the version of Python to use for Tools/scripts/serve.py (#12385)
* bpo-36329: Declare the version of Python to use for Tools/scripts/serve.py

* Add the blurb entry
2019-03-18 11:47:55 +01:00
Raymond Hettinger 23581c018f
bpo-36321: Fix misspelled attribute in namedtuple() (GH-12375) 2019-03-18 00:27:39 -07:00
Inada Naoki 6a16b18224
bpo-36297: remove "unicode_internal" codec (GH-12342) 2019-03-18 15:44:11 +09:00
Harmon 6fb544d8bc Fix "catchs" typos in NEWS entries (GH-12364) 2019-03-17 16:48:21 -07:00
Fantix King f683f46425 bpo-34745: Fix asyncio sslproto memory issues (GH-12386)
* Fix handshake timeout leak in asyncio/sslproto

Refs MagicStack/uvloop#222

* Break circular ref _SSLPipe <-> SSLProtocol

* bpo-34745: Fix asyncio ssl memory leak

* Break circular ref SSLProtocol <-> UserProtocol

* Add NEWS entry
2019-03-17 18:51:10 -04:00
Diego Rojas 06e1e68822 bpo-34160: Update news entry for XML order attributes (#12335) 2019-03-16 16:44:56 -07:00
Cheryl Sabella 0bb5e75cf8
bpo-23216: IDLE: Add docstrings to search modules (GH-12141) 2019-03-16 19:29:33 -04:00
Pablo Galindo 7c994549dc
bpo-35493: Use Process.sentinel instead of sleeping for polling worker status in multiprocessing.Pool (#11488)
* bpo-35493: Use Process.sentinel instead of sleeping for polling worker status in multiprocessing.Pool

* Use self-pipe pattern to avoid polling for changes

* Refactor some variable names and add comments

* Restore timeout and poll

* Use reader object only on wait()

* Recompute worker sentinels every time

* Remove timeout and use change notifier

* Refactor some methods to be overloaded by the ThreadPool, document the cache class and fix typos
2019-03-16 22:34:24 +00:00
Dave Chevell 962bdeab19 bpo-35715: Liberate return value of _process_worker (GH-11514)
ProcessPoolExecutor workers will hold the return value of their last task in memory until the next task is received. Since the return value has already been propagated to the parent process's Future (or has been discarded by this point), the object can be safely released.
2019-03-16 22:28:51 +00:00
Yasser A f40b4a0b62 bpo-36138: Clarify docs about converting datetime.timedelta to scalars. (GH-12137)
Be explicit that timedelta division converts an overall duration to the interval
units given by the denominator.
2019-03-16 13:56:58 +10:00
Eric Snow d2fdd1fedf
bpo-36124: Add PyInterpreterState.dict. (gh-12132) 2019-03-15 17:47:43 -06:00
Victor Stinner 86082c22d2
bpo-36235: Fix CFLAGS in distutils customize_compiler() (GH-12236)
Fix CFLAGS in customize_compiler() of distutils.sysconfig: when the
CFLAGS environment variable is defined, don't override CFLAGS variable with
the OPT variable anymore.

Initial patch written by David Malcolm.

Co-Authored-By: David Malcolm <dmalcolm@redhat.com>
2019-03-15 14:57:52 +01:00
Rémi Lapeyre 65f64b1903 bpo-36272: Logging now propagates RecursionError (GH-12312) 2019-03-15 06:53:34 +00:00
Inada Naoki 3fe7fa316f
bpo-30040: update news entry (GH-12324)
This optimization is not only for space, but also for speed.
2019-03-14 18:54:09 +09:00
Serhiy Storchaka f2f55e7f03
bpo-36282: Improved error message for too much positional arguments. (GH-12310) 2019-03-13 23:03:22 +02:00
Guido van Rossum 10f8ce6688 bpo-36280: Add Constant.kind field (GH-12295)
The value is a string for string and byte literals, None otherwise.
It is 'u' for u"..." literals, 'b' for b"..." literals, '' for "..." literals.
The 'r' (raw) prefix is ignored.
Does not apply to f-strings.

This appears sufficient to make mypy capable of using the stdlib ast module instead of typed_ast (assuming a mypy patch I'm working on).

WIP: I need to make the tests pass. @ilevkivskyi @serhiy-storchaka 



https://bugs.python.org/issue36280
2019-03-13 13:00:46 -07:00
pxinwr 8b5bdda5b4 bpo-31904: Adapt the _signal module to VxWorks RTOS (GH-12304)
Limited signal fields in VxWorks.
2019-03-13 18:18:25 +01:00
Victor Stinner 9776b0636a
bpo-36262: Fix _Py_dg_strtod() memory leak (goto undfl) (GH-12276)
Fix an unlikely memory leak on conversion from string to float in the
function _Py_dg_strtod() used by float(str), complex(str),
pickle.load(), marshal.load(), etc.

Fix an unlikely memory leak in _Py_dg_strtod() on "undfl:" label:
rewrite memory management in this function to always release all
memory before exiting the function. Initialize variables to NULL, and
set them to NULL after calling Bfree() at the "cont:" label.

Note: Bfree(NULL) is well defined: it does nothing.
2019-03-13 17:55:01 +01:00
Anthony Sottile 86900a4900 Fix stepping into a frame without a __name__ (GH-12064) 2019-03-12 20:57:09 -07:00
Ned Deily 20843a94a8
Correct minor edit to news entry. (GH-12298) 2019-03-12 19:44:20 -04:00
Ned Deily f45813df52
Minor edits to news entries (ported from 3.7) (GH-12293) 2019-03-12 12:21:22 -04:00
Anthony Sottile 25ec4a45dc bpo-36264: Don't honor POSIX HOME in os.path.expanduser on Windows (GH-12282) 2019-03-12 08:39:57 -07:00
Inada Naoki 410aea1ebf
canonicalize "Inada Naoki" in ACKS and 3.8 News (GH-12286) 2019-03-12 17:27:43 +09:00
Inada Naoki f2a186712b
bpo-30040: new empty dict uses key-sharing dict (GH-1080)
Sizeof new empty dict becomes 72 bytes from 240 bytes (amd64).
It is same size to empty dict created by dict.clear().
2019-03-12 17:25:44 +09:00
Daniel Hahler 3e936431e2 bpo-35931: Gracefully handle any exception in pdb debug command (GH-12103)
This is relevant for `debug doesnotexist()`, which would crash with a
NameError otherwise.
2019-03-11 20:29:04 -07:00
Lisa Roach 1ceb3a3d17
bpo-35132: Fixes missing target in gdb pep0393 check. (GH-11848) 2019-03-11 20:21:25 -07:00
Victor Stinner 876e82b4f3
bpo-36234: Add more tests to PosixUidGidTests (GH-12234)
test_posix.PosixUidGidTests:

* Add tests for invalid uid/gid type (str)
* Add UID_OVERFLOW and GID_OVERFLOW constants to replace (1 << 32)

Initial patch written by David Malcolm.

Co-Authored-By: David Malcolm <dmalcolm@redhat.com>
2019-03-11 13:57:53 +01:00
Terry Jan Reedy 491ef53c15
bpo-36176: Fix IDLE autocomplete & calltip popup colors. (#12262)
Prevent conflicts with Linux dark themes
(and slightly darken calltip background).
2019-03-10 20:18:40 -04:00
Xtreak 0e1f1f0105 bpo-35647: Fix path check in cookiejar (#11436)
* Refactor cookie path check as per RFC 6265

* Add tests for prefix match of path

* Add news entry

* Fix set_ok_path and refactor tests

* Use slice for last letter
2019-03-10 10:12:28 -07:00
Lysandros Nikolaou 1aeeaeb79e bpo-21314: Add a FAQ entry about positional only parameters (GH-10641) 2019-03-10 21:30:11 +10:00
sth 8b91edadc0 bpo-36251: Fix format strings used in match_repr() and stdprinter_repr(). (GH-12252) 2019-03-10 12:29:14 +02:00
Xtreak ca7fe50635 bpo-35121: prefix dot in domain for proper subdomain validation (GH-10258)
Don't send cookies of domain A without Domain attribute to domain B when domain A is a suffix match of domain B while using a cookiejar with `http.cookiejar.DefaultCookiePolicy` policy.  Patch by Karthikeyan Singaravelan.
2019-03-09 21:09:48 -05:00
Benjamin Peterson 738c19f4c5
closes bpo-33376: Update to Unicode 12.0.0. (GH-12256) 2019-03-09 16:25:55 -08:00
Cheryl Sabella d5a70c6b03
bpo-35661: Store the venv prompt in pyvenv.cfg (GH-11440) 2019-03-08 17:01:27 -05:00
Anthony Sottile ab9b31f947 bpo-35843: Implement __getitem__ for _NamespacePath (GH-11690) 2019-03-08 10:58:00 -08:00
Guido van Rossum 495da29225 bpo-35975: Support parsing earlier minor versions of Python 3 (GH-12086)
This adds a `feature_version` flag to `ast.parse()` (documented) and `compile()` (hidden) that allow tweaking the parser to support older versions of the grammar. In particular if `feature_version` is 5 or 6, the hacks for the `async` and `await` keyword from PEP 492 are reinstated. (For 7 or higher, these are unconditionally treated as keywords, but they are still special tokens rather than `NAME` tokens that the parser driver recognizes.)



https://bugs.python.org/issue35975
2019-03-07 12:38:08 -08:00
Steve Dower 16e6f7dee7
bpo-36216: Add check for characters in netloc that normalize to separators (GH-12201) 2019-03-07 08:02:26 -08:00
Raymond Hettinger 318d537daa
bpo-36169 : Add overlap() method to statistics.NormalDist (GH-12149) 2019-03-06 22:59:40 -08:00
Davide Rizzo bb9593af0a closes bpo-36139: release GIL around munmap(). (GH-12073) 2019-03-06 07:52:34 -08:00
Pradyun Gedam 01e0f439f5 bpo-35807: Upgrade ensurepip bundled pip and setuptools (GH-12189)
* Update pip to 19.0.3
* Update setuptools to 40.8.0
2019-03-06 06:42:21 -05:00
Martijn Pieters b727239575 closes bpo-36188: Clean up 'unbound' method left-overs. (GH-12169)
Methods are always bound, and `__self__` can no longer be `NULL`
(`method_new()` and `PyMethod_New()` both explicitly check for this).

Moreover, once a bound method is bound, it *stays* bound and won't be re-bound
to something else, so the section in the datamodel that talks about accessing
an methods in a different descriptor-binding context doesn't apply any more in
Python 3.
2019-03-04 21:19:34 -08:00
Christian Heimes b7bc283ab6 bpo-36179: Fix ref leaks in _hashopenssl (GH-12158)
Fix two unlikely reference leaks in _hashopenssl. The leaks only occur in
out-of-memory cases. Thanks to Charalampos Stratakis.

Signed-off-by: Christian Heimes <christian@python.org>



https://bugs.python.org/issue36179
2019-03-04 07:45:41 -08:00
Kevin Adler 800d5cd750 bpo-35198 Fix C++ extension compilation on AIX (GH-10437)
For C++ extensions, distutils tries to replace the C compiler with the
C++ compiler, but it assumes that C compiler is the first element after
any environment variables set. On AIX, linking goes through ld_so_aix,
so it is the first element and the compiler is the next element. Thus
the replacement is faulty:

ld_so_aix gcc ... -> g++ gcc ...

Also, it assumed that self.compiler_cxx had only 1 element or that
there were the same number of elements as the linker has and in the
same order. This might not be the case, so instead concatenate
everything together.
2019-03-04 15:48:40 +01:00
pxinwr f4b0a1c0da bpo-31904: Add encoding support for VxWorks RTOS (GH-12051)
Use UTF-8 as the system encoding on VxWorks.

The main reason are:

1. The locale is frequently misconfigured.
2. Missing some functions to deal with locale in VxWorks C library.
2019-03-04 10:02:06 +01:00
Brennan D Baraban 8b914d2767 bpo-35899: Fix Enum handling of empty and weird strings (GH-11891)
Co-authored-by: Maxwell <maxwellpxt@gmail.com>
Co-authored-by: Stéphane Wirtel <stephane@wirtel.be>





https://bugs.python.org/issue35899
2019-03-03 14:09:11 -08:00
Henry Chen 0a6a412fb2 bpo-36091: Remove reference to async generator in Lib/types.py. (GH-11996) 2019-03-03 16:35:24 +02:00
Inada Naoki 4f19030618
bpo-36103: change default buffer size of shutil.copyfileobj() (GH-12115)
It is changed from 16KiB to 64KiB.  The previous default value
is used since 1990.

coreutils chose 128 KiB as minimum buffer size for block device I/O.

But shutil.copyfileobj() can be used for non block devices.
So I choose more conservative value.

As my quick benchmark, performance difference between 64KiB and
128 KiB is up to ~5%.  On the other hand, performance difference
between 32 KiB and 64 KiB can be more than 10% when file is fully
buffered.

This is why 64 KiB is rational value.
2019-03-02 13:31:01 +09:00
Pablo Galindo 1f24a719e7
bpo-35808: Retire pgen and use pgen2 to generate the parser (GH-11814)
Pgen is the oldest piece of technology in the CPython repository, building it requires various #if[n]def PGEN hacks in other parts of the code and it also depends more and more on CPython internals. This commit removes the old pgen C code and replaces it for a new version implemented in pure Python. This is a modified and adapted version of lib2to3/pgen2 that can generate grammar files compatibles with the current parser.

This commit also eliminates all the #ifdef and code branches related to pgen, simplifying the code and making it more maintainable. The regen-grammar step now uses $(PYTHON_FOR_REGEN) that can be any version of the interpreter, so the new pgen code maintains compatibility with older versions of the interpreter (this also allows regenerating the grammar with the current CI solution that uses Python3.5). The new pgen Python module also makes use of the Grammar/Tokens file that holds the token specification, so is always kept in sync and avoids having to maintain duplicate token definitions.
2019-03-01 15:34:44 -08:00
Ned Deily 7eebbbd5b3 bpo-32129: Avoid blurry IDLE application icon on macOS with Tk 8.6. (GH-12031)
Patch by Kevin Walzer.
2019-03-01 17:53:50 -05:00
Stéphane Wirtel 4b219ce81e bpo-36043: FileCookieJar supports os.PathLike (GH-11945)
https://bugs.python.org/issue36043
2019-03-01 12:40:54 -08:00
Victor Stinner cfe172dc6b
bpo-36146: Add TEST_EXTENSIONS to setup.py (GH-12129)
Add TEST_EXTENSIONS constant to setup.py to allow to not build test
extensions like _testcapi.

Changes:

* Add add_ldflags_cppflags() subfunction
* Rename add_compiler_directories() to configure_compiler().
* Remove unused COMPILED_WITH_PYDEBUG constant.
* Use self.add() rather than accessing directly self.extensions.
* Remove module_enabled() function: check differently if curses
  extension is built or not.
2019-03-01 18:21:49 +01:00
Xtreak be7c460fb5 bpo-35178: Fix warnings._formatwarnmsg() (GH-12033)
Ensure custom formatwarning function can receive line as positional argument.

Co-Authored-By: Tashrif Billah <tashrifbillah@gmail.com>
2019-03-01 18:17:55 +01:00
Victor Stinner 96d81583be
bpo-36146: Fix inc_dirs in setup.py on macOS (GH-12098)
Fix setup.py on macOS: only add /usr/include/ffi to include
directories of _ctypes, not for all extensions.
2019-03-01 13:53:46 +01:00
Cheryl Sabella b9f0354efc
bpo-36152: IDLE: Remove unused parameter from colorizer (GH-12109)
Remove colorizer.ColorDelegator.close_when_done and the corresponding argument of .close().  In IDLE, both have always been None or False since 2007.
2019-03-01 05:19:40 -05:00
Victor Stinner f684d83d86
bpo-36142: Exclude coreconfig.h from Py_LIMITED_API (GH-12111)
The whole coreconfig.h header is now excluded from Py_LIMITED_API.
Move functions definitions into a new internal pycore_coreconfig.h
header.

* Move Include/coreconfig.h to Include/cpython/coreconfig.h
* coreconfig.h header is now excluded from Py_LIMITED_API
* Move functions to pycore_coreconfig.h
2019-03-01 03:44:13 +01:00
Victor Stinner ab71f8b793
bpo-29571: Fix test_re.test_locale_flag() (GH-12099)
Use locale.getpreferredencoding() rather than locale.getlocale() to
get the locale encoding. With some locales, locale.getlocale()
returns the wrong encoding.

For example, on Fedora 29, locale.getlocale() returns ISO-8859-1
encoding for the "en_IN" locale, whereas
locale.getpreferredencoding() reports the correct encoding: UTF-8.
2019-03-01 00:08:03 +01:00
Cheryl Sabella ed1deb0719
bpo-36096: IDLE: Refactor class variables in colorizer (GH-12002) 2019-02-27 08:21:16 -05:00
pxinwr 32f5fdd7f4 bpo-31904: Add cross-build support for VxWorks RTOS (GH-11968) 2019-02-27 12:09:28 +01:00
Joannah Nanjekye 53b9e1a1c1 bpo-36123: Fix test_socket.testWithTimeoutTriggeredSend() race condition (GH-12053)
Use longer timeout for accept() in the server and block on accept in the client.
The client now only sets the timeout once the socket is connected.
2019-02-26 17:18:23 +01:00
Giampaolo Rodola c606a9cbd4
bpo-35652: shutil.copytree(copy_function=...) erroneously pass DirEntry instead of path str (GH-11997) 2019-02-26 12:04:41 +01:00
Dima Pasechnik f57cd8288d bpo-36106: Resolve sinpi name clash with libm (IEEE-754 violation). (GH-12027)
The standard math library (libm) may follow IEEE-754 recommendation to
include an implementation of sinPi(), i.e. sinPi(x):=sin(pi*x).
And this triggers a name clash, found by FreeBSD developer
Steve Kargl, who worken on putting sinpi into libm used on FreeBSD
(it has to be named "sinpi", not "sinPi", cf. e.g.
https://en.cppreference.com/w/c/experimental/fpext4).
2019-02-26 08:36:11 +02:00
Zackery Spytz 6673decfa0 bpo-24643: Fix "#define timezone _timezone" clashes on Windows (GH-12019) 2019-02-25 15:56:44 -08:00
Anthony Sottile 8377cd4fcd Clean up code which checked presence of os.{stat,lstat,chmod} (#11643) 2019-02-25 23:32:27 +01:00
Xtreak 9c3f284de5 Autospec functions should propagate mock calls to parent GH-11273 2019-02-25 21:46:34 +00:00
Łukasz Langa 3025a7eebc Python 3.8.0a2
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE4/8oOcBIslwITevpsmmV4xAlBWgFAlxz2v0ACgkQsmmV4xAl
 BWjC4g/+OlySJwyU24YgF8XHihQiDSBwFGXgjdQ+KewOLbgCOofftXvEjhlozHVJ
 YxCiAGBPeFOUpsBZL1OHtihHSEYMS/XsGGWe/MKU+LaVtNnrLZAIFdX9uMkpfebg
 5Ndjs9PY5afxV0+QRfm0gz65V2JdHk4XuTbt5TUcji2i1rB021LKRXwnqex2R7Ly
 Mnwhmo/BDyT2t1+LYNEjBQ3QIQWZ9g/WbXdhRvxfccydVzwk9YX40g3yx0LkokDY
 Ej8AMk5A/A7a7gHOJebVqyLPbsg1rIfFe8r0JvCftH3YEQIshpbUgZsZCExgOsLm
 mGehPnOfc1D1mAKiWoDgm52+u9ae+IcxfktYM+spiUY69mQhwOhigPpfdUfZngW9
 H5sRu9dq9+KmDGrOb3jDY4coUiXnaRuGMcfosGqm+Q7gJXoRxE4zljVJwRdQpPea
 xpVeb4W3q58bfv+R60C88wuwv6OjdE0rL9VKLXndCoHV3YZTexS162S6d00rOOhB
 wYBViTxSh42+lUqn8CWJFDUQFnbTj2zArCMpDjIIDTWvWJEEczhEwKgDW7l5/sZE
 UOEm1xwn+3258gsxzPDRn1mZM99X1OyLx/hnSfAU2mv/2NFKRXYWgAmIhXtQqLHi
 t3BxwVgruP/yhswrPTf0Wz3RfZh4KH/5SH75O0YTykdzKMyPFIQ=
 =21b3
 -----END PGP SIGNATURE-----
gpgsig -----BEGIN PGP SIGNATURE-----
 
 iQJEBAABCAAuFiEE4/8oOcBIslwITevpsmmV4xAlBWgFAlx0SwkQHGx1a2FzekBs
 YW5nYS5wbAAKCRCyaZXjECUFaDW3D/9OM6xt4MNsIuRLfqI7NEqt6HRFrk96CCyW
 ++QPk+hKXKmRDNFRiriJoAZjCJRjvY7wqrt3dbJfVkKYnBvyxPCO+9rLgcI/J71G
 Wyi+mcm4TeVl5YfTFN9+/Y7rA3upcrGQsr/3eYDtT2y1x3XmxmpniwKNWyTDSMDg
 7lEwANLVj5u2FTojQjZHNVfc8D+V0w676s2IfGcIm8COckY+w1ti9QLF8qgLshAg
 ocEW8CKYF4S69QyYwngNiheGdO423kmAxOyWMCeWBdS5jq6wMmMIG9H31P35RdEi
 QtSkhq6sSabt+IwTYTe/SjxFXmSh8NgbnKJkgq3w33iEuNr1SVc3vCak+sDYtV6N
 dtyYUtYU+fqyrbhMP2bC0TR9T0aV63EXXXRAnhgIR0ZF3AtsnuLoqsWuhtobwypb
 DWem1jhhDywqZib1SCdgDBbW943yqcd86slC2cVlZzZi8odUKP6Lz6kSIadtWItb
 IKrSc8XCJ9J/1Kl1LxBtrm2IWBlH1VB+VHyT8lxuYR6NRrcFVzTIZBAjWtdUsE16
 hZocozrbkQkDPUs2qMkt6Op17dAfhb5geb7QdEsmTiMaiMYtESx7tmA5kihs7gkC
 QMPtPMQMFmPCnd8iE4W8kEKvHr+g7xbGU9JWUpAZomWZrzj6OnsCfVQRi8kmMZ+V
 RsFLW7aLLA==
 =MEsg
 -----END PGP SIGNATURE-----

Merge tag 'v3.8.0a2'

Python 3.8.0a2
2019-02-25 21:07:31 +01:00
Serhiy Storchaka 6a44f6eef3
bpo-36048: Use __index__() instead of __int__() for implicit conversion if available. (GH-11952)
Deprecate using the __int__() method in implicit conversions of Python
numbers to C integers.
2019-02-25 17:57:58 +02:00
Łukasz Langa 23f4589b4b
v3.8.0a2 2019-02-25 13:08:32 +01:00
Miro Hrončok 16323cb2c3 closes bpo-36083: Fix formatting of the manpage Synopsis. (GH-12017)
More specifically, the options of --check-hash-based-pycs.
2019-02-24 16:50:29 -08:00
Eric Snow ef4ac967e2
bpo-33608: Factor out a private, per-interpreter _Py_AddPendingCall(). (GH-11617)
This involves moving the global "pending calls" state to PyInterpreterState.

https://bugs.python.org/issue33608
2019-02-24 15:40:47 -08:00
Xtreak a875ea58b2 bpo-35512: Resolve string target to patch.dict decorator during function call GH#12000
* Resolve string target to patch.dict during function call

* Add NEWS entry

* Remove unneeded call

* Restore original value for support.target and refactor assertions

* Add extra assertion to verify unpatched dict
2019-02-24 18:54:49 +00:00
Ned Deily aeca373b33
bpo-27313: Avoid test_ttk_guionly ComboboxTest fail with macOS Cocoa Tk (GH-12011) 2019-02-24 02:28:24 -05:00
Terry Jan Reedy d610116a2e
bpo-24310: Document IDLE settings dialog font tab sample (GH-12007) 2019-02-23 23:04:53 -05:00
Raymond Hettinger 11c7953165
bpo-36018: Add the NormalDist class to the statistics module (GH-11973) 2019-02-23 14:44:07 -08:00
Eric Snow 64d6cc826d
bpo-35724: Explicitly require the main interpreter for signal-handling. (GH-11530)
Ensure that the main interpreter is active (in the main thread) for signal-handling operations. This is increasingly relevant as people use subinterpreters more.

https://bugs.python.org/issue35724
2019-02-23 15:40:43 -07:00
Gregory P. Smith 06babb2422
bpo-1054041: Add What's New docs. (GH-11999)
Add What's New docs about the Ctrl-C improvement. Correct the issue number in the NEWS entry.
2019-02-23 10:43:49 -08:00
Eric Snow be3b295838
bpo-35886: Make PyInterpreterState an opaque type in the public API. (GH-11731)
Move PyInterpreterState into the "internal" header files.
2019-02-23 11:35:52 -07:00
Pablo Galindo 175421b58c bpo-36016: Add generation option to gc.getobjects() (GH-11909) 2019-02-23 12:02:06 +09:00
Stéphane Wirtel a40681dd5d bpo-36019: Use pythontest.net instead of example.com in network tests (GH-11941) 2019-02-22 14:45:36 +01:00
Raymond Hettinger 47d9987247
bpo-35904: Add statistics.fmean() (GH-11892) 2019-02-21 15:06:29 -08:00
Stéphane Wirtel 3ad9167305 bpo-36052: Raise a SyntaxError when assigning a value to __debug__ with := (GH-11958)
Trying to assign a value to __debug__ using the assignment operator is supposed to fail, but
a missing check for forbidden names when setting the context in the ast was preventing this behaviour.
2019-02-21 10:11:53 +00:00
Zackery Spytz ebc793d6ac bpo-33989: Ensure that ms.key_compare is always initialized in list_sort_impl(). (GH-8710) 2019-02-21 09:47:14 +02:00
Stefan Behnel d8b9e1fc2e bpo-36012: Avoid linear slot search for non-dunder methods (GH-11907) 2019-02-20 09:29:24 -08:00
Stéphane Wirtel b5409dacc4 bpo-35993: Fix _PyInterpreterState_DeleteExceptMain() (GH-11852)
Fix a crash on fork when using subinterpreters.
2019-02-20 15:27:22 +01:00
Inada Naoki 001fee14e0
bpo-12822: use monotonic clock for condvar if possible (GH-11723) 2019-02-20 10:00:09 +09:00
Victor Stinner 3ef6344ee5
bpo-36037: Fix test_ssl for strict OpenSSL policy (GH-11940)
Fix test_ssl for strict OpenSSL configuration like RHEL8 strict crypto policy.
Use older TLS version for minimum TLS version of the server SSL context if
needed, to test TLS version older than default minimum TLS version.
2019-02-19 18:06:03 +01:00
Cédric Krier beda52ed36 bpo-35153: Add headers parameter to xmlrpc.client.ServerProxy (GH-10308)
Allow to add HTTP headers to XML-RPC requests sent to the server.
2019-02-19 17:18:50 +01:00
Sanyam Khurana 5105483acb bpo-31506: Clarify error messages for object.__new__ and object.__init__ (GH-11641)
`object.__new__` and `object.__init__` do take one argument each,
they just don't take extra user supplied arguments.

Patch by Sanyam Khurana.
2019-02-19 23:23:48 +10:00
Serhiy Storchaka e7a4bb554e
bpo-35798: Add test.support.check_syntax_warning(). (#11895)
It checks that a SyntaxWarning is raised when compile specified
statement, that it is raised only once, that it is converted to
a SyntaxError when raised as exception, and that both warning and
exception objects have corresponding attributes.
2019-02-19 08:30:15 +02:00
Cheryl Sabella ee0f927bd8 bpo-35689: IDLE: Add docstrings and unittests for colorizer.py (GH-11472) 2019-02-19 00:11:18 -05:00
tjb900 4371c0a9c0 bpo-34572: change _pickle unpickling to use import rather than retrieving from sys.modules (GH-9047)
Fix C implementation of pickle.loads to use importlib's locking mechanisms, and thereby avoid using partially-loaded modules.
2019-02-18 15:30:51 +00:00
animalize 4a7f44a2ed bpo-34294: re module, fix wrong capturing groups in rare cases. (GH-11546)
Need to reset capturing groups between two SRE(match) callings in loops, this fixes wrong capturing groups in rare cases.

Also add a missing index in re.rst.
2019-02-18 15:26:37 +02:00
Nick Coghlan 02c04f26df bpo-35704: Include correct NEWS entry (GH-11914)
The wrong NEWS snippet was inadvertently included in
GH-11500, this switches to the correct one.


https://bugs.python.org/issue35704
2019-02-18 03:23:36 -08:00
Michael Felt ef110b1807 bpo-35704: Prevent test_shutil fail result when AIX is 32-bit and MAXDATA < 0x20000000 (GH-11500)
https://bugs.python.org/issue35704
2019-02-18 03:02:44 -08:00
Pablo Galindo 09fbcd6085
bpo-35942: Improve the error message if __fspath__ returns invalid types in path_converter (GH-11831)
The error message emitted when returning invalid types from __fspath__ in interfaces that allow passing PathLike objects has been improved and now it does explain the origin of the error.
2019-02-18 10:46:34 +00:00
Ivan Levkivskyi ac28147e78
bpo-35992: Use PySequence_GetItem only if sq_item is not NULL (GH-11857)
Not using `__class_getitem__()` fallback if there is a non-subcriptable metaclass was caused by a certain asymmetry between how `PySequenceMethods` and `PyMappingMethods` are used in `PyObject_GetItem`. This PR removes this asymmetry. No tests failed, so I assume it was not intentional.
2019-02-17 23:13:46 +00:00
Michael Felt 1bf8845f74 bpo-34720: Fix test_importlib.test_bad_traverse for AIX (GH-9391)
Fix  Modules/_testmultiphase.c so that it exits with non-zero status on AIX just as other systems do (non zero exit status, e.g. as result of a segmentation fault) when a NULL pointer is accessed for data.


https://bugs.python.org/issue34720
2019-02-17 04:02:56 -08:00
Gregory P. Smith 38f11cc3f6
bpo-1054041: Exit properly after an uncaught ^C. (#11862)
* bpo-1054041: Exit properly by a signal after a ^C.

An uncaught KeyboardInterrupt exception means the user pressed ^C and
our code did not handle it.  Programs that install SIGINT handlers are
supposed to reraise the SIGINT signal to the SIG_DFL handler in order
to exit in a manner that their calling process can detect that they
died due to a Ctrl-C.  https://www.cons.org/cracauer/sigint.html

After this change on POSIX systems

 while true; do python -c 'import time; time.sleep(23)'; done

can be stopped via a simple Ctrl-C instead of the shell infinitely
restarting a new python process.

What to do on Windows, or if anything needs to be done there has not
yet been determined.  That belongs in its own PR.

TODO(gpshead): A unittest for this behavior is still needed.

* Do the unhandled ^C check after pymain_free.

* Return STATUS_CONTROL_C_EXIT on Windows.

* Fix ifdef around unistd.h include.

* 📜🤖 Added by blurb_it.

* Add STATUS_CTRL_C_EXIT to the os module on Windows

* Add unittests.

* Don't send CTRL_C_EVENT in the Windows test.

It was causing CI systems to bail out of the entire test suite.

See https://dev.azure.com/Python/cpython/_build/results?buildId=37980
for example.

* Correct posix test (fail on macOS?) check.

* STATUS_CONTROL_C_EXIT must be unsigned.

* Improve the error message.

* test typo :)

* Skip if the bash version is too old.

...and rename the windows test to reflect what it does.

* min bash version is 4.4, detect no bash.

* restore a blank line i didn't mean to delete.

* PyErr_Occurred() before the Py_DECREF(co);

* Don't add os.STATUS_CONTROL_C_EXIT as a constant.

* Update the Windows test comment.

* Refactor common logic into a run_eval_code_obj fn.
2019-02-16 12:57:40 -08:00
Serhiy Storchaka 62e4481238
bpo-15248: Emit a compiler warning when missed a comma before tuple or list. (GH-11757) 2019-02-16 08:12:19 +02:00
Anthony Sottile a16ab00c0b bpo-36007: Bump minimum sphinx version to 1.8 (GH-11887)
https://bugs.python.org/issue36007
2019-02-15 15:41:15 -08:00
Daniel Hahler 4327705cfa bpo-35931: Gracefully handle SyntaxError in pdb debug command (GH-11782)
Previously, `debug print(` would cause the interpreter to exit on a SyntaxError whereas `print(` would properly display the error and return to the pdb prompt.

This patch fixes this by pre-compiling the code before passing it to `Pdb.run`.


https://bugs.python.org/issue35931
2019-02-15 12:52:53 -08:00
Victor Stinner 355f16fd4b
bpo-35746: Credit Colin Read and Nicolas Edet (GH-11863)
Add credit for the cert parser vulnerability. Mention also Cisco
TALOS-2018-0758 identifier.
2019-02-15 12:25:47 +01:00
Paul Monson 8a1657b934 bpo-35976: Enable Windows projects to build with platform ARM32 (GH-11825)
This change adds the necessary items to the build projects to avoid erroring out right at the start. It does not add _support_ for targeting Windows on ARM32, but is a necessary prerequisite for adding it.
2019-02-14 08:31:30 -08:00
Zackery Spytz bb3c05d7ef closes bpo-35991: Fix a potential double free in Modules/_randommodule.c. (GH-11849) 2019-02-13 23:43:17 -08:00
Susan Su 2bdd5858e3 bpo-35500: align expected and actual calls on mock.assert_called_with error message. (GH-11804) 2019-02-13 18:22:29 -08:00
Victor Stinner dcb68f47f7
bpo-35961: Fix a crash in slice_richcompare() (GH-11830)
Fix a crash in slice_richcompare(): use strong references rather than
stolen references for the two temporary internal tuples.

The crash (or assertion error) occurred if a garbage collection
occurred during slice_richcompare(), especially while calling
PyObject_RichCompare(t1, t2, op).
2019-02-13 12:31:56 +01:00
Cheryl Sabella 5680f6546d bpo-18283: Add support for bytes to shutil.which (GH-11818) 2019-02-13 12:25:10 +01:00
Matěj Cepl 3dc67d0316 bpo-35505: Skip test_imap4_host_default_value if localhost listens on IMAP port (GH-11823)
Make test_imap4_host_default_value independent on whether the 
local IMAP server is running.
2019-02-12 19:30:19 +01:00
Christopher Hunt b01786c881 bpo-35960: Fix dataclasses.field throwing away empty metadata. (GH-11815) 2019-02-12 06:50:49 -05:00
Rémi Lapeyre a31f4cc881 bpo-35918: Remove broken has_key method and add test (#11819) 2019-02-11 16:37:24 -08:00
Pablo Galindo 3766f18c52
bpo-35378: Fix multiprocessing.Pool references (GH-11627)
Changes in this commit:

1. Use a _strong_ reference between the Pool and associated iterators
2. Rework PR #8450 to eliminate a cycle in the Pool.

There is no test in this commit because any test that automatically tests this behaviour needs to eliminate the pool before joining the pool to check that the pool object is garbaged collected/does not hang. But doing this will potentially leak threads and processes (see https://bugs.python.org/issue35413).
2019-02-11 17:29:00 +00:00
Eivind Teig 537b6caa56 bpo-22062: Updated docstring and documentation for pathlib (GH-8519)
Original patch by Mike Short


https://bugs.python.org/issue22062
2019-02-11 02:47:09 -08:00
Terry Jan Reedy 8a03ff2ff4
bpo-35833: Revise IDLE doc for control codes sent to Shell. (GH-11799)
Add  a code example block.
2019-02-08 22:51:51 -05:00
Fish 96d37dbcd2 bpo-35615: Fix crashes when copying a Weak{Key,Value}Dictionary. (GH-11384)
Protect dict iterations by wrapping them with _IterationGuard in the
following methods:

- WeakValueDictionary.copy()
- WeakValueDictionary.__deepcopy__()
- WeakKeyDictionary.copy()
- WeakKeyDictionary.__deepcopy__()
2019-02-07 19:51:59 +00:00
Pierre Glaser df8d2cde63 bpo-35911: add cell constructor (GH-11771)
Add a cell constructor, expose the cell type in the types module.
2019-02-07 19:36:48 +00:00
Jason R. Coombs f289084c83
bpo-24209: In http.server script, rely on getaddrinfo to bind to preferred address based on the bind parameter. (#11767)
In http.server script, rely on getaddrinfo to bind to preferred address based on the bind parameter.

As a result, now IPv6 is used as the default (including IPv4 on dual-stack systems). Enhanced tests.
2019-02-07 08:22:45 -05:00
Giampaolo Rodola 2848d9d299 bpo-35917: Test multiprocessing manager classes and shareable types (GH-11772)
multiprocessing: provide unittests for manager classes and shareable types
2019-02-07 11:03:11 +00:00
Pablo Galindo bc09851586 bpo-35606: Implement math.prod (GH-11359) 2019-02-06 23:04:02 -08:00
Steve Dower 85e102a2b0
bpo-35299: Fixed sysconfig and distutils during PGO profiling (GH-11744) 2019-02-04 17:15:13 -08:00
Nina Zakharenko 69091cb497 bpo-35321: Set the spec origin to frozen in frozen modules (#11732)
* bpo-35321: Set the spec origin to frozen in frozen modules

This fix correctly sets the spec origin to
"frozen" for the _frozen_importlib module. Note that the
origin was already correctly set in _frozen_importlib_external.

* 📜🤖 Added by blurb_it.
2019-02-04 16:56:26 -08:00
Paul Ganssle 89427cd0fe bpo-32417: Make timedelta arithmetic respect subclasses (#10902)
* Make timedelta return subclass types

Previously timedelta would always return the `date` and `datetime`
types, regardless of what it is added to. This makes it return
an object of the type it was added to.

* Add tests for timedelta arithmetic on subclasses

* Make pure python timedelta return subclass types

* Add test for fromtimestamp with tz argument

* Add tests for subclass behavior in now

* Add news entry.

Fixes:
bpo-32417
bpo-35364

* More descriptive variable names in tests

Addresses Victor's comments
2019-02-04 14:42:04 -05:00
Łukasz Langa e7afe1ab25 Python 3.8.0a1
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE4/8oOcBIslwITevpsmmV4xAlBWgFAlxW66AACgkQsmmV4xAl
 BWh3QQ//ZMI2a0KicKZ/MTnaW0cpTSQ5MoikOVL98z1Mx2MGDm5k5pzxN7E4qT39
 WPcnUJyK6o6juigq49qwpufaHRDhn/FoscFpQgTwf4POw23f6OB6karzV6xijOW7
 AoxjqKs7QhjIifAsxe76c4KzpR85tUgxJGsN4KHC6hwAbkri5HQ0372bjg+dkis1
 nmjnDwf6ZkzwtNqOwP5aAnm5z6XWXngQk1BHL1xG+ECXeAYTWEHbgjMlJpsc/28E
 rPDeL08BYiJYLuOghV71Yq3T1FSHicLwwKt0+Q61zBGsjwrEmqfRZpB4oOrr65Ca
 0QdqYbNwm9hkfp3/XnNV+JwBnBLJsd+vzEz3Lcu/hPb7YMAD7DMIrIeG6zVHVVEo
 9edurimM8w5QsmHyGFgVVEh4qImiGgmYSDFrU3l/u3SXjnSLnNJM2dGLMSmJ6N14
 komPeq4vg0daPqRPTbWHTZvjt7aidAJlSzrmjg45KOgv6TdJq8X/Sm/v70zopaXL
 gUW9SNHQY84WpHOni3+SKweh4IsmXT+ZagKHUVuf9h9W9U3u43eGYIWXlLbDUOTj
 tSx3scYJK97YIJjpz7RacAMrEcrn5gWrLnJemnlF5Bsv9tekKBP4vaF7hbkZn+0U
 6aPJ56OOSjtCMPrAl6uw9EbEz1ysPVY0wz6Lpcrb6j7dYCn81Ds=
 =g9Mg
 -----END PGP SIGNATURE-----
gpgsig -----BEGIN PGP SIGNATURE-----
 
 iQJEBAABCAAuFiEE4/8oOcBIslwITevpsmmV4xAlBWgFAlxX+EUQHGx1a2FzekBs
 YW5nYS5wbAAKCRCyaZXjECUFaIGiEACDEsoAjSDnlydMqQ94L5lpy+Tk9hOYiXjV
 ZG4JOhTSuWJzbEdjXHuwWf7rnXgu3xjgsct3wfkFkGuH11XxyyQRJ9u32zsCBE6k
 MO2CsNVVCKyFWM0CJ1A0BrILsVMY2N9mr3nmbiaGZt5eaIY7JUhTtpzhjmyrNIVL
 YWL+EfGgU/1xinsYXuhuOJXACBlJnxR8MOtdJ5X2HZb+6RokYj7efKP7Ty84vZIV
 bW7cngQsoac6C7QNalFRP6Z1pO81HoDQHxlAwh2dUU/4vYQTIZ3zNA7K8JbaJnWL
 bcQkUVvPr2OTqDkro2Hj6qKSpdroAu/7LzNjB3ky+iNkEtVonIX6H/sY9EfbwuYx
 3KS88d9ETdDkxVfRThp8pbedPJL8sIFVsR6E/PzUjLKyfqKQGdifKh+vzrfs642u
 3Xz7xJFyvCPT3xSeemRQRPODhWfPf1sd/rfHbk0SLBF5Se3lqFnNZ4/DhKXMr8/D
 BVozJSRD4P1A7xgEy59/jD5oIkkd8M9g6YtDTD5RYvfWxb/geLN+BAIXsIouRtTe
 ANPX0plC56FfY3xZYqeQi05f+C7j/P2FUKKL9tjiuxKJ8UQ3ftaDw39CneyDLMyy
 u/ynrv3H4EAWsV1qFjOeUIwc+k4rjlprcB+LRQguLBbBlFg1vj8MTOO+Kfgui0Lv
 y9Wpvhkeqw==
 =WuSS
 -----END PGP SIGNATURE-----

Merge tag 'v3.8.0a1'

Python 3.8.0a1
2019-02-04 09:30:59 +01:00
Steve Dower a8474d025c
bpo-35872 and bpo-35873: Clears __PYVENV_LAUNCHER__ variable (GH-11745)
After reading __PYVENV_LAUNCHER__ we now set sys._base_executable value for later use.
Make the same changes for macOS to avoid extra platform checks.
2019-02-03 23:19:38 -08:00
Steve Dower 2f6fae6e51
bpo-35692: pathlib no longer raises when checking file and directory existence on drives that are not ready (GH-11746) 2019-02-03 23:08:18 -08:00
Łukasz Langa 8efa3b69f6
[blurb] v3.8.0a1 2019-02-03 14:02:52 +01:00
Raymond Hettinger f75d59e1a8
bpo-35884: Add variable access benchmarking script (GH-11725) 2019-02-02 22:54:56 -08:00
Minmin Gong 8ebc6451f3 bpo-35890 : Fix some API calling consistency (GH-11742)
Unicode version of Windows APIs are used in places, but not for GetVersionEx in Python/sysmodule.c
The wcstok_s is called on Windows in Modules/main.c and PC/launcher.c, but not in Python/pathconfig.c
2019-02-02 20:26:55 -08:00
Steve Dower 4c70d9f79c
bpo-34691: Compile _contextvars module into main Python library (GH-11741) 2019-02-02 14:36:23 -08:00
Mark Becwar b82bfac436 bpo-29734: nt._getfinalpathname handle leak (GH-740)
Make sure that failure paths call CloseHandle outside of the function that failed
2019-02-02 13:08:23 -08:00
Shiva Saxena cb09047626 bpo-32560: inherit the py launcher's STARTUPINFO (GH-9000)
https://bugs.python.org/issue32560
2019-02-02 11:21:04 -08:00
Stefan Krah d08ea70464
bpo-35845: Add order={'C', 'F', 'A'} parameter to memoryview.tobytes(). (#11730) 2019-02-02 18:57:41 +01:00
Tony Roberts 4860f01ac0 bpo-33895: Relase GIL while calling functions that acquire Windows loader lock (GH-7789)
LoadLibrary, GetProcAddress, FreeLibrary and GetModuleHandle acquire the system loader lock. Calling these while holding the GIL will cause a deadlock on the rare occasion that another thread is detaching and needs to destroy its thread state at the same time.
2019-02-02 09:16:42 -08:00
Tzu-ping Chung 2de576e16d bpo-1104: msilib.SummaryInfo.GetProperty() truncates the string by one character (GH-4517)
Add one char to MsiSummaryInfoGetProperty() output
Based on the patch in bpo-1104 by Anthony Tuininga (atuining) and Mark McMahon (markm).
2019-02-02 09:13:23 -08:00
native-api 05e922136a bpo-33316: PyThread_release_lock always fails (GH-6541)
Use correct interpretation of return value from APIs.
2019-02-02 08:22:55 -08:00
Davin Potts e5ef45b8f5
bpo-35813: Added shared_memory submodule of multiprocessing. (#11664)
Added shared_memory submodule to multiprocessing in time for first alpha with cross-platform tests soon to follow.
2019-02-01 22:52:23 -06:00
Xtreak d4fceaafb8 bpo-35877: Make parenthesis optional for named expression in while statement (GH-11724)
* Add parenthesis optional in named expressions for while statement

* Add NEWS entry
2019-02-01 14:40:16 -07:00
Joannah Nanjekye 80c5dfe74b bpo-35537: Add setsid parameter to os.posix_spawn() and os.posix_spawnp() (GH-11608) 2019-02-01 11:05:22 +01:00
Guido van Rossum dcfcd146f8 bpo-35766: Merge typed_ast back into CPython (GH-11645) 2019-01-31 12:40:27 +01:00
Raymond Hettinger 0bb4bdf0d9
bpo-35864: Replace OrderedDict with regular dict in namedtuple() (#11708)
* Change from OrderedDict to a regular dict

* Add blurb
2019-01-31 00:59:50 -08:00
Michael Selik 9f3f0931cf bpo-34003: Use dict instead of OrderedDict in csv.DictReader (GH-8014) 2019-01-31 00:47:53 -08:00
Steve Dower a1f9a3332b
bpo-35854: Fix EnvBuilder and --symlinks in venv on Windows (GH-11700) 2019-01-30 13:49:14 -08:00
Giampaolo Rodola bafa8487f7 subprocess: close pipes/fds by using ExitStack (GH-11686)
Close pipes/fds in subprocess by using ExitStack.

"In case of premature failure on X.Close() or os.close(X) the remaining pipes/fds will remain "open". Perhaps it makes sense to use contextlib.ExitStack."
- Rationale: https://github.com/python/cpython/pull/11575#discussion_r250288394
2019-01-29 13:14:24 -08:00
Andreas Schwab 742d768656 bpo-35847: RISC-V needs CTYPES_PASS_BY_REF_HACK (GH-11694)
This fixes the ctypes.test.test_structures.StructureTestCase test.


https://bugs.python.org/issue35847
2019-01-29 08:16:10 -08:00
Raymond Hettinger d8080c0119
bpo-35780: Fix errors in lru_cache() C code (GH-11623) 2019-01-26 03:02:00 -05:00
Steve Dower adad9e6801
bpo-35811: Avoid propagating venv settings when launching via py.exe (GH-11677) 2019-01-25 14:59:58 -08:00
Steve Dower 4e02f8f8b4
bpo-35797: Fix default executable used by the multiprocessing module (GH-11676) 2019-01-25 14:59:12 -08:00
Ivan Levkivskyi 62c35a8a8f
bpo-35814: Allow same r.h.s. in annotated assignments as in normal ones (GH-11667) 2019-01-25 01:39:19 +00:00
Emily Morehouse 8f59ee01be
bpo-35224: PEP 572 Implementation (#10497)
* Add tokenization of :=
- Add token to Include/token.h. Add token to documentation in Doc/library/token.rst.
- Run `./python Lib/token.py` to regenerate Lib/token.py.
- Update Parser/tokenizer.c: add case to handle `:=`.

* Add initial usage of := in grammar.

* Update Python.asdl to match the grammar updates. Regenerated Include/Python-ast.h and Python/Python-ast.c

* Update AST and compiler files in Python/ast.c and Python/compile.c. Basic functionality, this isn't scoped properly

* Regenerate Lib/symbol.py using `./python Lib/symbol.py`

* Tests - Fix failing tests in test_parser.py due to changes in token numbers for internal representation

* Tests - Add simple test for := token

* Tests - Add simple tests for named expressions using expr and suite

* Tests - Update number of levels for nested expressions to prevent stack overflow

* Update symbol table to handle NamedExpr

* Update Grammar to allow assignment expressions in if statements.
Regenerate Python/graminit.c accordingly using `make regen-grammar`

* Tests - Add additional tests for named expressions in RoundtripLegalSyntaxTestCase, based on examples and information directly from PEP 572

Note: failing tests are currently commented out (4 out of 24 tests currently fail)

* Tests - Add temporary syntax test failure tests in test_parser.py

Note: There is an outstanding TODO for this -- syntax tests need to be
moved to a different file (presumably test_syntax.py), but this is
covering what needs to be tested at the moment, and it's more convenient
to run a single test for the time being

* Add support for allowing assignment expressions as function argument annotations. Uncomment tests for these cases because they all pass now!

* Tests - Move existing syntax tests out of test_parser.py and into test_named_expressions.py. Refactor syntax tests to use unittest

* Add TargetScopeError exception to extend SyntaxError

Note: This simply creates the TargetScopeError exception, it is not yet
used anywhere

* Tests - Update tests per PEP 572

Continue refactoring test suite:
The named expression test suite now checks for any invalid cases that
throw exceptions (no longer limited to SyntaxErrors), assignment tests
to ensure that variables are properly assigned, and scope tests to
ensure that variable availability and values are correct

Note:
- There are still tests that are marked to skip, as they are not yet
implemented
- There are approximately 300 lines of the PEP that have not yet been
addressed, though these may be deferred

* Documentation - Small updates to XXX/todo comments

- Remove XXX from child description in ast.c
- Add comment with number of previously supported nested expressions for
3.7.X in test_parser.py

* Fix assert in seq_for_testlist()

* Cleanup - Denote "Not implemented -- No keyword args" on failing test case. Fix PEP8 error for blank lines at beginning of test classes in test_parser.py

* Tests - Wrap all file opens in `with...as` to ensure files are closed

* WIP: handle f(a := 1)

* Tests and Cleanup - No longer skips keyword arg test. Keyword arg test now uses a simpler test case and does not rely on an external file. Remove print statements from ast.c

* Tests - Refactor last remaining test case that relied on on external file to use a simpler test case without the dependency

* Tests - Add better description of remaning skipped tests. Add test checking scope when using assignment expression in a function argument

* Tests - Add test for nested comprehension, testing value and scope. Fix variable name in skipped comprehension scope test

* Handle restriction of LHS for named expressions - can only assign to LHS of type NAME. Specifically, restrict assignment to tuples

This adds an alternative set_context specifically for named expressions,
set_namedexpr_context. Thus, context is now set differently for standard
assignment versus assignment for named expressions in order to handle
restrictions.

* Tests - Update negative test case for assigning to lambda to match new error message. Add negative test case for assigning to tuple

* Tests - Reorder test cases to group invalid syntax cases and named assignment target errors

* Tests - Update test case for named expression in function argument - check that result and variable are set correctly

* Todo - Add todo for TargetScopeError based on Guido's comment (2b3acd37bd (r30472562))

* Tests - Add named expression tests for assignment operator in function arguments

Note: One of two tests are skipped, as function arguments are currently treating
an assignment expression inside of parenthesis as one child, which does
not properly catch the named expression, nor does it count arguments
properly

* Add NamedStore to expr_context. Regenerate related code with `make regen-ast`

* Add usage of NamedStore to ast_for_named_expr in ast.c. Update occurances of checking for Store to also handle NamedStore where appropriate

* Add ste_comprehension to _symtable_entry to track if the namespace is a comprehension. Initialize ste_comprehension to 0. Set set_comprehension to 1 in symtable_handle_comprehension

* s/symtable_add_def/symtable_add_def_helper. Add symtable_add_def to handle grabbing st->st_cur and passing it to symtable_add_def_helper. This now allows us to call the original code from symtable_add_def by instead calling symtable_add_def_helper with a different ste.

* Refactor symtable_record_directive to take lineno and col_offset as arguments instead of stmt_ty. This allows symtable_record_directive to be used for stmt_ty and expr_ty

* Handle elevating scope for named expressions in comprehensions.

* Handle error for usage of named expression inside a class block

* Tests - No longer skip scope tests. Add additional scope tests

* Cleanup - Update error message for named expression within a comprehension within a class. Update comments. Add assert for symtable_extend_namedexpr_scope to validate that we always find at least a ModuleScope if we don't find a Class or FunctionScope

* Cleanup - Add missing case for NamedStore in expr_context_name. Remove unused var in set_namedexpr_content

* Refactor - Consolidate set_context and set_namedexpr_context to reduce duplicated code. Special cases for named expressions are handled by checking if ctx is NamedStore

* Cleanup - Add additional use cases for ast_for_namedexpr in usage comment. Fix multiple blank lines in test_named_expressions

* Tests - Remove unnecessary test case. Renumber test case function names

* Remove TargetScopeError for now. Will add back if needed

* Cleanup - Small comment nit for consistency

* Handle positional argument check with named expression

* Add TargetScopeError exception definition. Add documentation for TargetScopeError in c-api docs. Throw TargetScopeError instead of SyntaxError when using a named expression in a comprehension within a class scope

* Increase stack size for parser by 200. This is a minimal change (approx. 5kb) and should not have an impact on any systems. Update parser test to allow 99 nested levels again

* Add TargetScopeError to exception_hierarchy.txt for test_baseexception.py_

* Tests - Major update for named expression tests, both in test_named_expressions and test_parser

- Add test for TargetScopeError
- Add tests for named expressions in comprehension scope and edge cases
- Add tests for named expressions in function arguments (declarations
and call sites)
- Reorganize tests to group them more logically

* Cleanup - Remove unnecessary comment

* Cleanup - Comment nitpicks

* Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0)

- Add check for LHS types to detect a parenthesis then a name (see note)
- Add test for this scenario
- Update tests for changed error message for named assignment to a tuple
(also, see note)

Note: This caused issues with the previous error handling for named assignment
to a LHS that contained an expression, such as a tuple. Thus, the check
for the LHS of a named expression must be changed to be more specific if
we wish to maintain the previous error messages

* Cleanup - Wrap lines more strictly in test file

* Revert "Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0)"

This reverts commit f1531400ca7d7a2d148830c8ac703f041740896d.

* Add NEWS.d entry

* Tests - Fix error in test_pickle.test_exceptions by adding TargetScopeError to list of exceptions

* Tests - Update error message tests to reflect improved messaging convention (s/can't/cannot)

* Remove cases that cannot be reached in compile.c. Small linting update.

* Update Grammar/Tokens to add COLONEQUAL. Regenerate all files

* Update TargetScopeError PRE_INIT and POST_INIT, as this was purposefully left out when fixing rebase conflicts

* Add NamedStore back and regenerate files

* Pass along line number and end col info for named expression

* Simplify News entry

* Fix compiler warning and explicity mark fallthrough
2019-01-24 16:49:56 -07:00
Rémi Lapeyre 1fd06f1eca bpo-35717: Fix KeyError exception raised when using enums and compile (GH-11523)
https://bugs.python.org/issue17467
2019-01-24 11:43:13 -08:00
Manjusaka da6424e96a bpo-35726: Prevented QueueHandler formatting from affecting other handlers (GH-11537)
QueueHandler.prepare() now makes a copy of the record before modifying and enqueueing it, to avoid affecting other handlers in the chain.
2019-01-23 07:08:38 +00:00
Victor Stinner 6d43f6f081
bpo-35713: Split _Py_InitializeCore into subfunctions (GH-11650)
* Split _Py_InitializeCore_impl() into subfunctions: add multiple pycore_init_xxx() functions
* Preliminary sys.stderr is now set earlier to get an usable
  sys.stderr ealier.
* Move code into _Py_Initialize_ReconfigureCore() to be able to call
  it from _Py_InitializeCore().
* Split _PyExc_Init(): create a new _PyBuiltins_AddExceptions()
  function.
* Call _PyExc_Init() earlier in _Py_InitializeCore_impl()
  and new_interpreter() to get working exceptions earlier.
* _Py_ReadyTypes() now returns _PyInitError rather than calling
  Py_FatalError().
* Misc code cleanup
2019-01-22 21:18:05 +01:00
Steve Dower 28f6cb34f6
bpo-35683: Improve Azure Pipelines steps (GH-11493) 2019-01-22 10:49:52 -08:00
Victor Stinner bf4ac2d2fd
bpo-35713: Rework Python initialization (GH-11647)
* The PyByteArray_Init() and PyByteArray_Fini() functions have been
  removed. They did nothing since Python 2.7.4 and Python 3.2.0, were
  excluded from the limited API (stable ABI), and were not
  documented.
* Move "_PyXXX_Init()" and "_PyXXX_Fini()" declarations from
  Include/cpython/pylifecycle.h to
  Include/internal/pycore_pylifecycle.h. Replace
  "PyAPI_FUNC(TYPE)" with "extern TYPE".
* _PyExc_Init() now returns an error on failure rather than calling
  Py_FatalError(). Move macros inside _PyExc_Init() and undefine them
  when done. Rewrite macros to make them look more like statement:
  add ";" when using them, add "do { ... } while (0)".
* _PyUnicode_Init() now returns a _PyInitError error rather than call
  Py_FatalError().
* Move stdin check from _PySys_BeginInit() to init_sys_streams().
* _Py_ReadyTypes() now returns a _PyInitError error rather than
  calling Py_FatalError().
2019-01-22 17:39:03 +01:00
Lucas Cimon 35ca1820e1 bpo-35720: Fixing a memory leak in pymain_parse_cmdline_impl() (GH-11528)
When the loop in the pymain_read_conf function in this same file
calls pymain_init_cmdline_argv() a 2nd time, the pymain->command
buffer of wchar_t is overriden and the previously allocated memory
is never freed.
2019-01-22 17:15:01 +01:00
Ivan Levkivskyi 9932a22897
bpo-33416: Add end positions to Python AST (GH-11605)
The majority of this PR is tediously passing `end_lineno` and `end_col_offset` everywhere. Here are non-trivial points:
* It is not possible to reconstruct end positions in AST "on the fly", some information is lost after an AST node is constructed, so we need two more attributes for every AST node `end_lineno` and `end_col_offset`.
* I add end position information to both CST and AST.  Although it may be technically possible to avoid adding end positions to CST, the code becomes more cumbersome and less efficient.
* Since the end position is not known for non-leaf CST nodes while the next token is added, this requires a bit of extra care (see `_PyNode_FinalizeEndPos`). Unless I made some mistake, the algorithm should be linear.
* For statements, I "trim" the end position of suites to not include the terminal newlines and dedent (this seems to be what people would expect), for example in
  ```python
  class C:
      pass

  pass
  ```
  the end line and end column for the class definition is (2, 8).
* For `end_col_offset` I use the common Python convention for indexing, for example for `pass` the `end_col_offset` is 4 (not 3), so that `[0:4]` gives one the source code that corresponds to the node.
* I added a helper function `ast.get_source_segment()`, to get source text segment corresponding to a given AST node. It is also useful for testing.

An (inevitable) downside of this PR is that AST now takes almost 25% more memory. I think however it is probably justified by the benefits.
2019-01-22 11:18:22 +00:00
Minmin Gong 7a2368063f bpo-35758: Fix building on ARM + MSVC (gh-11531)
* Disable x87 control word for non-x86 targets

On msvc, x87 control word is only available on x86 target. Need to disable it for other targets to prevent compiling problems.

* Include immintrin.h on x86 and x64 only

Immintrin.h is only available on x86 and x64. Need to disable it for other targets to prevent compiling problems.
2019-01-21 21:49:40 +01:00
Victor Stinner b2385458ce
bpo-35772: Fix test_tarfile on ppc64 (GH-11606)
Fix sparse file tests of test_tarfile on ppc64le with the tmpfs
filesystem.

Fix the function testing if the filesystem supports sparse files:
create a file which contains data and "holes", instead of creating a
file which contains no data.

tmpfs effective block size is a page size (tmpfs lives in the page
cache). RHEL uses 64 KiB pages on aarch64, ppc64 and ppc64le, only
s390x and x86_64 use 4 KiB pages, whereas the test punch holes of
4 KiB.

test.pythoninfo: Add resource.getpagesize().
2019-01-21 10:24:12 +01:00
Pablo Galindo 222d303ade bpo-20239: Allow repeated deletion of unittest.mock.Mock attributes (#11057)
* Allow repeated deletion of unittest.mock.Mock attributes

* fixup! Allow repeated deletion of unittest.mock.Mock attributes

* fixup! fixup! Allow repeated deletion of unittest.mock.Mock attributes
2019-01-21 08:57:46 +00:00
Marc Schlaich b2dc4a3313 bpo-35699: fix distuils cannot detect Build Tools 2017 anymore (GH-11495) 2019-01-20 13:47:42 -05:00
Anthony Sottile 7417622617 bpo-35733: Make isinstance(ast.Constant(boolean), ast.Num) be false. (GH-11547) 2019-01-18 21:30:28 +02:00
Terry Jan Reedy 39ed289a35
bpo-35770: IDLE macosx deletes Options => Configure IDLE. (GH-11614)
It previously deleted Window => Zoom Height by mistake.
(Zoom Height is now on the Options menu).  On Mac, the settings
dialog is accessed via Preferences on the IDLE menu.
2019-01-18 14:00:45 -05:00
Ashwin Ramaswami 478f829132 bpo-21257: document http.client.parse_headers (GH-11443)
Document http.client.parse_headers
2019-01-18 08:49:16 -07:00
Christian Heimes 34de2d312b bpo-35045: Accept TLSv1 default in min max test (GH-11510)
Make ssl tests less strict and also accept TLSv1 as system default. The
changes unbreaks test_min_max_version on Fedora 29.

Signed-off-by: Christian Heimes <christian@python.org>
2019-01-18 16:09:30 +01:00
Terry Jan Reedy a902239f22
bpo-35769: Change IDLE's name for new files from 'Untitled' to 'untitled' (GH-11602)
'Untitled' violates the PEP 8 standard for .py files
2019-01-18 02:09:53 -05:00
Serhiy Storchaka 3bcbedc9f1
bpo-34850: Emit a warning for "is" and "is not" with a literal. (GH-9642) 2019-01-18 07:47:48 +02:00
Timo Furrer 17f05bbc78 bpo-33687: Fix call to os.chmod() in uu.decode() (GH-7282) 2019-01-17 17:15:53 +03:00
Dong-hee Na 89669ffe10 bpo-35283: Add deprecation warning for Thread.isAlive (GH-11454)
Add a deprecated warning for the threading.Thread.isAlive() method.
2019-01-17 13:14:45 +01:00
Joannah Nanjekye 92b8322e7e bpo-35674: Add os.posix_spawnp() (GH-11554)
Add a new os.posix_spawnp() function.
2019-01-16 14:29:26 +01:00