Commit Graph

157 Commits

Author SHA1 Message Date
Terry Jan Reedy 667522efa8 bpo-31414: IDLE -- fix tk entry box tests by deleting first. (#3501)
Adding to an int entry is not the same as deleting and inserting
because int('') will fail.
2017-09-11 16:02:33 -04:00
Oren Milman 252033d50e bpo-31411: Prevent raising a SystemError in case warnings.onceregistry is not a dictionary. (#3485) 2017-09-11 09:28:39 +03:00
wohlganger 58fc71c447 bpo-27099: IDLE - Convert built-in extensions to regular features (#2494)
About 10 IDLE features were implemented as supposedly optional
extensions.  Their different behavior could be confusing or worse for
users and not good for maintenance.  Hence the conversion.

The main difference for users is that user configurable key bindings
for builtin features are now handled uniformly.  Now, editing a binding
in a keyset only affects its value in the keyset.  All bindings are
defined together in the system-specific default keysets in config-
extensions.def.  All custom keysets are saved as a whole in config-
extension.cfg.  All take effect as soon as one clicks Apply or Ok.

The affected events are '<<force-open-completions>>', '<<expand-word>>',
'<<force-open-calltip>>', '<<flash-paren>>', '<<format-paragraph>>',
'<<run-module>>', '<<check-module>>', and '<<zoom-height>>'.  Any
(global) customizations made before 3.6.3 will not affect their keyset-
specific customization after 3.6.3. and vice versa.

Inital patch by Charles Wohlganger, revised by Terry Jan Reedy.
2017-09-10 17:19:47 -04:00
Raymond Hettinger 8b57d73639 bpo-28638: Optimize namedtuple() creation time by minimizing use of exec() (#3454)
* Working draft without _source

* Re-use itemgetter() instances

* Speed-up calls to __new__() with a pre-bound tuple.__new__()

* Add note regarding string interning

* Remove unnecessary create function wrappers

* Minor sync-ups with PR-2736.  Mostly formatting and f-strings

* Bring-in qualname/__module fix-ups from PR-2736

* Formally remove the verbose flag and _source attribute

* Restore a test of potentially problematic field names

* Restore kwonly_args test but without the verbose option

* Adopt Inada's idea to reuse the docstrings for the itemgetters

* Neaten-up a bit

* Add news blurb

* Serhiy pointed-out the need for interning

* Jelle noticed as missing f on an f-string

* Add whatsnew entry for feature removal

* Accede to request for dict literals instead keyword arguments

* Leave the method.__module__ attribute pointing the actual location of the code

* Improve variable names and add a micro-optimization for an non-public helper function

* Simplify by in-lining reuse_itemgetter()

* Arrange steps in more logical order

* Save docstring in local cache instead of interning
2017-09-10 10:23:36 -07:00
Steve Dower b84bcc48ae bpo-31392: Update SSL build for 1.1.0 (#3448) 2017-09-09 06:13:06 -07:00
Gregory P. Smith efb1d0a3c0 bpo-29639: change test.support.HOST to "localhost"
test.support.HOST should be "localhost" as it was in the past. See the bpo-29639.

Tests that need the IP address should use HOSTv4 (added) or the existing HOSTv6 constant.

This changes the definition and fixes tests that needed updating to deal with HOST being
the hostname rather than the hardcoded IP address.

This is only the first step in addressing https://bugs.python.org/issue29639.
2017-09-09 00:30:15 -07:00
Steve Dower e6eb48c10d bpo-31400: Improve SSL error handling on Windows (#3463)
* bpo-31392: Improve SSL error handling on Windows

* Remove unnecessary Windows mention in NEWS
2017-09-08 15:16:15 -07:00
octaviansoldea 4c81401b3a bpo-31354: Let configure --with-lto work on all builds
Allow configure --with-lto to apply to all builds, not just profile-opt builds.

Whether this is actually useful or not must be determined by the person
building CPython using their own toolchain.

My own quick test on x86_64 Debian 9 (gcc 6.3, binutils 2.28) seemed
to suggest that it wasn't, but I expect better toolchains can or will exist
at some point.  The point is to allow it at all.
2017-09-08 12:14:33 -07:00
Christian Heimes b3ad0e5127 bpo-28182: Expose OpenSSL verification results (#3412)
The SSL module now raises SSLCertVerificationError when OpenSSL fails to
verify the peer's certificate. The exception contains more information about
the error.

Original patch by Chi Hsuan Yen

Signed-off-by: Christian Heimes <christian@python.org>
2017-09-08 12:00:19 -07:00
Eric Snow 2ebc5ce42a bpo-30860: Consolidate stateful runtime globals. (#3397)
* group the (stateful) runtime globals into various topical structs
* consolidate the topical structs under a single top-level _PyRuntimeState struct
* add a check-c-globals.py script that helps identify runtime globals

Other globals are excluded (see globals.txt and check-c-globals.py).
2017-09-07 23:51:28 -06:00
Christian Heimes cb5b68abde bpo-29136: Add TLS 1.3 cipher suites and OP_NO_TLSv1_3 (#1363)
* bpo-29136: Add TLS 1.3 support

TLS 1.3 introduces a new, distinct set of cipher suites. The TLS 1.3
cipher suites don't overlap with cipher suites from TLS 1.2 and earlier.
Since Python sets its own set of permitted ciphers, TLS 1.3 handshake
will fail as soon as OpenSSL 1.1.1 is released. Let's enable the common
AES-GCM and ChaCha20 suites.

Additionally the flag OP_NO_TLSv1_3 is added. It defaults to 0 (no op) with
OpenSSL prior to 1.1.1. This allows applications to opt-out from TLS 1.3
now.

Signed-off-by: Christian Heimes <christian@python.org>
2017-09-07 18:07:00 -07:00
Nick Coghlan 5a8516701f bpo-31344: Per-frame control of trace events (GH-3417)
f_trace_lines: enable/disable line trace events
f_trace_opcodes: enable/disable opcode trace events

These are intended primarily for testing of the interpreter
itself, as they make it much easier to emulate signals
arriving at unfortunate times.
2017-09-08 10:14:16 +10:00
Christian Heimes 888bbdc192 bpo-27340: Use memoryview in SSLSocket.sendall() (#3384)
* bpo-27340: Use memoryview in SSLSocket.sendall()

SSLSocket.sendall() now uses memoryview to create slices of data. This fix
support for all bytes-like object. It is also more efficient and avoids
costly copies.

Signed-off-by: Christian Heimes <christian@python.org>

* Cast view to bytes, fix typo

Signed-off-by: Christian Heimes <christian@python.org>
2017-09-07 14:18:21 -07:00
Christian Heimes 17c9ac927b bpo-28958: Improve SSLContext error reporting. (#3414)
Signed-off-by: Christian Heimes <christian@python.org>
2017-09-07 14:14:00 -07:00
Pablo Galindo 586c0502b5 bpo-31294: Fix ZeroMQSocketListener and ZeroMQSocketHandler examples (#3229)
* Fix ZeroMQSocketListener and ZeroMQSocketHandler examples

* Use send_json and recv_json to simplify pyzmq interfacing

* Add News entry
2017-09-07 13:53:13 -07:00
Steve Dower 05f01d8525 bpo-30389 Adds detection of VS 2017 to distutils._msvccompiler (#1632) 2017-09-07 11:49:23 -07:00
Benjamin Peterson a853a8ba78 bpo-31373: fix undefined floating-point demotions (#3396) 2017-09-07 11:13:59 -07:00
Antoine Pitrou a6a4dc816d bpo-31370: Remove support for threads-less builds (#3385)
* Remove Setup.config
* Always define WITH_THREAD for compatibility.
2017-09-07 18:56:24 +02:00
Zachary Ware d01db1c2a2 bpo-31358: Pull zlib out of the repository (GH-3375)
Also enable building externals by default on Windows, use PCbuild\build.bat's -E option to disable it.
2017-09-06 17:29:37 -07:00
Łukasz Langa e7c566caf1 bpo-30465: Fix lineno and col_offset in fstring AST nodes (#1800)
For f-string ast nodes, fix the line and columns so that tools such as flake8 can identify them correctly.
2017-09-06 17:27:58 -07:00
R. David Murray 0f6b9d2306 bpo-14191 Add parse_intermixed_args. (#3319)
This adds support for parsing a command line where options and positionals are intermixed as is common in many unix commands. This is paul.j3's patch with a few tweaks.
2017-09-06 20:25:40 -04:00
Christian Heimes ad0ffa033e bpo-21649: Add RFC 7525 and Mozilla server side TLS (#3387)
Signed-off-by: Christian Heimes <christian@python.org>
2017-09-06 16:19:56 -07:00
caavery effc12f8e9 bpo-27584: New addition of vSockets to the python socket module (#2489)
* bpo-27584: New addition of vSockets to the python socket module

Support for AF_VSOCK on Linux only

* bpo-27584: Fixes for V2

Fixed syntax and naming problems.
Fixed #ifdef AF_VSOCK checking
Restored original aclocal.m4

* bpo-27584: Fixes for V3

Added checking for fcntl and thread modules.

* bpo-27584: Fixes for V4

Fixed white space error

* bpo-27584: Fixes for V5

Added back comma in (CID, port).

* bpo-27584: Fixes for V6

Added news file.
socket.rst now reflects first Linux introduction of AF_VSOCK.
Fixed get_cid in test_socket.py.
Replaced PyLong_FromLong with PyLong_FromUnsignedLong in socketmodule.c
Got rid of extra AF_VSOCK #define.
Added sockaddr_vm to sock_addr.

* bpo-27584: Fixes for V7

Minor cleanup.

* bpo-27584: Fixes for V8

Put back #undef AF_VSOCK as it is  necessary when vm_sockets.h is not installed.
2017-09-06 15:18:10 -07:00
Devin Jeanpierre c5bace2bf7 bpo-29505: Add fuzz tests for float(str), int(str), unicode(str) (#2878)
Add basic fuzz tests for a few common builtin functions.

This is an easy place to start, and these functions are probably safe.
We'll want to add more fuzz tests later.  Lets bootstrap using these.

While the fuzz tests are included in CPython and compiled / tested on a
very basic level inside CPython itself, the actual fuzzing happens as
part of oss-fuzz (https://github.com/google/oss-fuzz). The reason to
include the tests in CPython is to make sure that they're maintained
as part of the CPython project, especially when (as some eventually
will) they use internal implementation details in the test.

(This will be necessary sometimes because e.g. the fuzz test should
never enter Python's interpreter loop, whereas some APIs only expose
themselves publicly as Python functions.)

This particular set of changes is part of testing Python's builtins,
tracked internally at Google by b/37562550.

The _xxtestfuzz module that this change adds need not be shipped with binary distributions of Python.
2017-09-06 11:15:35 -07:00
Steve Dower 5fcd5e64ee bpo-31340: Change to building with MSVC v141 (included with Visual Studio 2017) (#3311) 2017-09-06 10:01:38 -07:00
Ammar Askar 3fc499bca1 bpo-31178: Avoid concatenating bytes with str in subprocess error (#3066)
Avoid concatenating bytes with str in the typically rare subprocess error path (exec failed). Includes a mock based unittest to exercise the codepath.
2017-09-05 23:41:30 -07:00
Christian Heimes 6877111648 bpo-29781: Fix SSLObject.version before handshake (#3364)
SSLObject.version() now correctly returns None when handshake over BIO has
not been performed yet.

Signed-off-by: Christian Heimes <christian@python.org>
2017-09-05 21:55:40 -07:00
Eric Snow 05351c1bd8 Revert "bpo-30860: Consolidate stateful runtime globals." (#3379)
Windows buildbots started failing due to include-related errors.
2017-09-05 21:43:08 -07:00
Eric Snow 76d5abc868 bpo-30860: Consolidate stateful runtime globals. (#2594)
* group the (stateful) runtime globals into various topical structs
* consolidate the topical structs under a single top-level _PyRuntimeState struct
* add a check-c-globals.py script that helps identify runtime globals

Other globals are excluded (see globals.txt and check-c-globals.py).
2017-09-05 18:26:16 -07:00
jimmylai 80bbe6a7b6 bpo-31350: Optimize get_event_loop and _get_running_loop (#3347)
* call remove_done_callback in finally section

* Optimize get_event_loop and _get_running_loop

* rename _loop_pid as loop_pid and add blurb news

* rename _loop_pid as loop_pid and add blurb news

* add back _RunningLoop

* Update 2017-09-05-10-30-48.bpo-31350.dXJ-7N.rst

* Update 2017-09-05-10-30-48.bpo-31350.dXJ-7N.rst
2017-09-05 20:36:59 -04:00
Christian Heimes 305e56c27a bpo-31320: No traceback to sys.stderr in test_ssl (#3360)
In case PROTOCOL_TLS_SERVER is used for both client context and server
context, the test thread dies with OSError. Catch OSError to avoid
traceback on sys.stderr

Signed-off-by: Christian Heimes <christian@python.org>
2017-09-05 16:37:09 -07:00
Gregory P. Smith 564a2c68ad Link to blurb on PyPI in the NEWS.d READMEs. (#3323) 2017-09-05 10:38:05 -07:00
Christian Heimes 75b961869a bpo-31343: Include sys/sysmacros.h (#3318)
Include sys/sysmacros.h for major(), minor(), and makedev(). GNU C libray
plans to remove the functions from sys/types.h.

Signed-off-by: Christian Heimes <christian@python.org>
2017-09-05 15:53:09 +02:00
Christian Heimes c941e6238a bpo-30102: Call OPENSSL_add_all_algorithms_noconf (#3112)
The ssl and hashlib modules now call OPENSSL_add_all_algorithms_noconf() on
OpenSSL < 1.1.0. The function detects CPU features and enables optimizations
on some CPU architectures such as POWER8. Patch is based on research from
Gustavo Serra Scalet.

Signed-off-by: Christian Heimes <christian@python.org>
2017-09-05 15:47:11 +02:00
Ned Deily e295b82609 Simplify NEWS entry to prevent suspicious warnings. (#3339) 2017-09-05 00:45:36 -07:00
Benjamin Peterson a3070d530c bpo-31347: _PyObject_FastCall_Prepend: do not call memcpy if args might not be null (#3329)
Passing NULL as the second argument to to memcpy is undefined behavior even if the size is 0.
2017-09-04 22:23:42 -07:00
Neil Schemenauer db564238db Revert "bpo-17852: Maintain a list of BufferedWriter objects. Flush them on exit. (#1908)" (#3337)
This reverts commit e38d12ed34.
2017-09-04 22:13:17 -07:00
Neil Schemenauer e38d12ed34 bpo-17852: Maintain a list of BufferedWriter objects. Flush them on exit. (#1908)
* Maintain a list of BufferedWriter objects.  Flush them on exit.

In Python 3, the buffer and the underlying file object are separate
and so the order in which objects are finalized matters.  This is
unlike Python 2 where the file and buffer were a single object and
finalization was done for both at the same time.  In Python 3, if
the file is finalized and closed before the buffer then the data in
the buffer is lost.

This change adds a doubly linked list of open file buffers.  An atexit
hook ensures they are flushed before proceeding with interpreter
shutdown.  This is addition does not remove the need to properly close
files as there are other reasons why buffered data could get lost during
finalization.

Initial patch by Armin Rigo.

* Use weakref.WeakSet instead of WeakKeyDictionary.

* Simplify buffered double-linked list types.

* In _flush_all_writers(), suppress errors from flush().

* Remove NEWS entry, use blurb.
2017-09-04 20:18:38 -07:00
Victor Stinner 759e30ec47 bpo-31170: Update libexpat from 2.2.3 to 2.2.4 (#3315)
* bpo-31170: Update libexpat from 2.2.3 to 2.2.4

Fix copying of partial characters for UTF-8 input (libexpat bug 115):
https://github.com/libexpat/libexpat/issues/115

* Add NEWS entry.
2017-09-05 01:58:08 +02:00
Eric Snow 86b7afdfee bpo-28411: Remove "modules" field from Py_InterpreterState. (#1638)
sys.modules is the one true source.
2017-09-04 17:54:09 -06:00
Benjamin Peterson 069306312a remove IRIX support (closes bpo-31341) (#3310)
See PEP 11.
2017-09-04 16:36:05 -07:00
Serhiy Storchaka e503ca5288 bpo-30502: Fix handling of long oids in ssl. (#2909) 2017-09-05 00:28:53 +02:00
Barry Warsaw ba4279683f bpo-1198569: Allow string.Template braced pattern to be different (#3288)
* bpo-1198569: Allow the braced pattern to be different

``string.Template`` subclasses can optionally define ``braceidpattern`` if
they want to specify different placeholder patterns inside and outside the
braces.  If None (the default) it falls back to ``idpattern``.
2017-09-04 16:32:10 -04:00
larryhastings f9f17346d7 Blurbify master branch. (#3298)
Blurbify master branch.
2017-09-04 13:30:19 -07:00
Christian Heimes 002d64039b bpo-25674: remove sha256.tbs-internet.com ssl test (#3297)
Signed-off-by: Christian Heimes <christian@python.org>
2017-09-04 22:26:01 +02:00
Zhiming Wang 06de1aeff9 bpo-31281: Fix pathlib.Path incompatibility in fileinput (gh-3208)
Fix fileinput with inplace=True to accept pathlib.Path objects.
2017-09-04 13:37:24 -04:00
Gregory P. Smith 4f013881cb bpo-9146: add the missing NEWS entry. (#3275) 2017-09-03 14:08:48 -07:00
Serhiy Storchaka bca4939d80 bpo-31185: Fixed miscellaneous errors in asyncio speedup module. (#3076) 2017-09-03 08:10:14 +03:00
Victor Stinner b713adf27a bpo-31326: ProcessPoolExecutor waits for the call queue thread (#3265)
* bpo-31326: ProcessPoolExecutor waits for the call queue thread

concurrent.futures.ProcessPoolExecutor.shutdown() now explicitly
closes the call queue. Moreover, shutdown(wait=True) now also join
the call queue thread, to prevent leaking a dangling thread.

* Fix for shutdown() being called twice.
2017-09-02 00:25:11 +02:00
Grzegorz Grzywacz 97e1b1c814 bpo-27144: concurrent.futures as_complete and map iterators do not keep reference to returned object (#1560)
* bpo-27144: concurrent.futures as_complie and map iterators do not keep
reference to returned object

* Some nits.  Improve wordings in docstrings and comments, and avoid relying on
sys.getrefcount() in tests.
2017-09-01 18:54:00 +02:00