Commit Graph

24438 Commits

Author SHA1 Message Date
Miss Islington (bot) 4cffe2f66b bpo-32929: Dataclasses: Change the tri-state hash parameter to the boolean unsafe_hash. (GH-5891) (GH-5902)
unsafe_hash=False is now the default. It is the same behavior as the old hash=None parameter. unsafe_hash=True will try to add __hash__. If it already exists, TypeError is raised.
(cherry picked from commit dbf9cff48a)

Co-authored-by: Eric V. Smith <ericvsmith@users.noreply.github.com>
2018-02-26 04:43:35 -05:00
Miss Islington (bot) 1a5790689f
Fix 'deecorator' typo in test/test_dataclasses (GH-5899)
(cherry picked from commit 973cae07d6)

Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
2018-02-25 19:09:05 -08:00
Miss Islington (bot) 632c1cb571
bpo-32622: Native sendfile on windows (GH-5565)
* Support sendfile on Windows Proactor event loop naively.
(cherry picked from commit a19fb3c6aa)

Co-authored-by: Andrew Svetlov <andrew.svetlov@gmail.com>
2018-02-25 09:10:58 -08:00
Miss Islington (bot) b6b6669cfd Exhaustively test dataclass hashing when no hash= value is provided. This is in anticipation of changing how non-default hashing is handled. (GH-5834) (GH-5889)
(cherry picked from commit 718070db26)

Co-authored-by: Eric V. Smith <ericvsmith@users.noreply.github.com>
2018-02-25 11:56:30 -05:00
Miss Islington (bot) ff7528f089 [3.7] bpo-31809: test secp ECDH curves (GH-4036) (#5872)
Add tests to verify connection with secp384r1 ECDH curves.
(cherry picked from commit b7b9225831)

Co-authored-by: Christian Heimes <christian@python.org>
2018-02-25 10:56:11 +01:00
Miss Islington (bot) 8fa8478dde [3.7] bpo-24334: Cleanup SSLSocket (GH-5252) (#5857)
* The SSLSocket is no longer implemented on top of SSLObject to
  avoid an extra level of indirection.
* Owner and session are now handled in the internal constructor.
* _ssl._SSLSocket now uses the same method names as SSLSocket and
  SSLObject.
* Channel binding type check is now handled in C code. Channel binding
  is always available.

The patch also changes the signature of SSLObject.__init__(). In my
opinion it's fine. A SSLObject is not a user-constructable object.
SSLContext.wrap_bio() is the only valid factory.
(cherry picked from commit 141c5e8c24)

Co-authored-by: Christian Heimes <christian@python.org>
2018-02-24 21:51:56 +01:00
Miss Islington (bot) e49bf0f353
bpo-31966: Fixed WindowsConsoleIO.write() for writing empty data. (GH-5754)
(cherry picked from commit 42c35d9c0c)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2018-02-24 09:39:18 -08:00
Miss Islington (bot) 46632f4d3c [3.7] bpo-32819: Simplify and improve ssl.match_hostname (GH-5620) (#5847)
ssl.match_hostname() has been simplified and no longer depends on re and
ipaddress module for wildcard and IP addresses. Error reporting for invalid
wildcards has been improved.

Signed-off-by: Christian Heimes <christian@python.org>
(cherry picked from commit aef1283ba4)

Co-authored-by: Christian Heimes <christian@python.org>
2018-02-24 15:06:46 +01:00
Miss Islington (bot) 1c37e27719 [bpo-28414] Make all hostnames in SSL module IDN A-labels (GH-5128) (GH-5843)
Previously, the ssl module stored international domain names (IDNs)
as U-labels. This is problematic for a number of reasons -- for
example, it made it impossible for users to use a different version
of IDNA than the one built into Python.

After this change, we always convert to A-labels as soon as possible,
and use them for all internal processing. In particular, server_hostname
attribute is now an A-label, and on the server side there's a new
sni_callback that receives the SNI servername as an A-label rather than
a U-label.
(cherry picked from commit 11a1493bc4)

Co-authored-by: Christian Heimes <christian@python.org>
2018-02-23 19:18:28 -08:00
Miss Islington (bot) 694c5e0e1f
bpo-30028: make test.support.temp_cwd() fork-safe (GH-1066)
Make test.support.temp_cwd() fork-safe. The context manager test.support.temp_cwd() no longer removes the temporary directory when executing in a process other than the parent it entered from.
If a forked child exits the context manager it won't do the cleanup.
(cherry picked from commit 33dddac00b)

Co-authored-by: Anselm Kruis <a.kruis@science-computing.de>
2018-02-22 21:39:02 -08:00
Miss Islington (bot) 24d6e3c56b Test that new_timezone can return the UTC singleton (gh-5318) (#5819)
(cherry picked from commit a049f5790e)

Co-authored-by: Paul Ganssle <pganssle@users.noreply.github.com>
2018-02-22 16:18:54 -05:00
Miss Islington (bot) e5a9b3574c
bpo-32457: Improves handling of denormalized executable path when launching Python (GH-5756)
(cherry picked from commit 48e8c82fc6)

Co-authored-by: Steve Dower <steve.dower@microsoft.com>
2018-02-22 11:15:16 -08:00
Miss Islington (bot) 01dd52fb29
bpo-32556: nt._getfinalpathname, nt._getvolumepathname and nt._getdiskusage now correctly convert from bytes. (GH-5761)
(cherry picked from commit 23ad6d0d1a)

Co-authored-by: Steve Dower <steve.dower@microsoft.com>
2018-02-22 11:02:12 -08:00
Miss Islington (bot) eed3c7af4e bpo-32604: Swap threads only if the interpreter is different. (gh-5783)
The CPython runtime assumes that there is a one-to-one relationship (for a given interpreter) between PyThreadState and OS threads. Sending and receiving on a channel in the same interpreter was causing crashes because of this (specifically due to a check in PyThreadState_Swap()). The solution is to not switch threads if the interpreter is the same.
(cherry picked from commit f53d9f2778)

Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
2018-02-20 17:09:41 -07:00
Miss Islington (bot) 1d927d4feb
bpo-31848: Fix broken error handling in Aifc_read.initfp() when the SSND chunk is not found (GH-5240)
Initialize self._ssnd_chunk so that aifc.Error is raised as intended,
not AttributeError.
(cherry picked from commit 80d20b918b)

Co-authored-by: Zackery Spytz <zspytz@gmail.com>
2018-02-20 14:06:58 -08:00
Miss Islington (bot) a3d6c1b23b bpo-32409: Ensures activate.bat can handle Unicode contents (GH-5765)
(cherry picked from commit 6240917b77)

Co-authored-by: Steve Dower <steve.dower@microsoft.com>
2018-02-19 17:45:02 -08:00
Miss Islington (bot) 622a824802
bpo-30121: Fix test_subprocess for Windows Debug builds (GH-5758)
(cherry picked from commit 5537646bfa)

Co-authored-by: Zachary Ware <zachary.ware@gmail.com>
2018-02-19 13:00:22 -08:00
Miss Islington (bot) 0cd35817e8
bpo-32682: Improve libz version parsing in test_zilb (GH-5347)
(cherry picked from commit 4c7108a771)

Co-authored-by: pmp-p <pmp-p@users.noreply.github.com>
2018-02-18 20:09:59 -08:00
Ivan Levkivskyi 3892899288
bpo-31333: Re-implement ABCMeta in C (GH-5733)
This adds C versions of methods used by ABCMeta that
improve performance of various ABC operations.
2018-02-18 17:39:43 +00:00
Miss Islington (bot) fa9a502a38 [3.7] Improves the ability to build in CI (GH-5729) 2018-02-17 19:31:37 -08:00
Miss Islington (bot) afb5e55836
bpo-32852: Fix trace changing sys.argv to tuple. (GH-5692)
(cherry picked from commit 9f4223261f)

Co-authored-by: Kyle Altendorf <sda@fstab.net>
2018-02-16 22:53:24 -08:00
Miss Islington (bot) 3db05a3a9c bpo-32604: Clean up created subinterpreters before runtime finalization. (gh-5710)
(cherry picked from commit 4c6955e2b0)

Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
2018-02-16 19:15:24 -07:00
Miss Islington (bot) 8caee0fa57
bpo-32841: Fix cancellation in awaiting asyncio.Condition (GH-5665)
(cherry picked from commit 5746510b7a)

Co-authored-by: Bar Harel <bzvi7919@gmail.com>
2018-02-14 01:47:30 -08:00
Miss Islington (bot) e81a6c86c8
bpo-31787: Skip refleak check when _hashlib is not available (GH-5660)
(cherry picked from commit f0bc645dfe)

Co-authored-by: INADA Naoki <methane@users.noreply.github.com>
2018-02-13 07:07:34 -08:00
Miss Islington (bot) ef20abed7f
bpo-31787: Prevent refleaks when calling __init__() more than once (GH-3995)
(cherry picked from commit d019bc8319)

Co-authored-by: Oren Milman <orenmn@gmail.com>
2018-02-13 03:32:18 -08:00
Miss Islington (bot) 74ebbaeb56
bpo-29248: Fix os.readlink() on Windows (GH-5577)
The PrintNameOffset field of the reparse data buffer
was treated as a number of characters instead of bytes.
(cherry picked from commit 3c34aad4e7)

Co-authored-by: SSE4 <tomskside@gmail.com>
2018-02-12 13:39:42 -08:00
Miss Islington (bot) 0442599961 bpo-32221: makeipaddr(): remove interface part + speedup (GH-5449) (GH-5449) (#5641)
(cherry picked from commit 7766b96ab8)

Co-authored-by: Коренберг Марк <socketpair@gmail.com>
2018-02-12 15:12:24 -05:00
Miss Islington (bot) 170b3f7950 bpo-32792: Preserve mapping order in ChainMap() (GH-5586) (#GH-5617)
(cherry picked from commit 3793f95f98)

Co-authored-by: Raymond Hettinger <rhettinger@users.noreply.github.com>
2018-02-11 01:09:52 -08:00
Miss Islington (bot) 2ef69a1d45 bpo-30157: Fix csv.Sniffer.sniff() regex pattern. (GH-5601) (GH-5602)
Co-authored-by: Jake Davis <jcdavis@awedge.net>
(cherry picked from commit 2411292ba8)
2018-02-10 00:00:48 +02:00
Miss Islington (bot) 0e361730b0 bpo-32775: Fix regular expression warnings in fnmatch. (GH-5583) (GH-5596)
fnmatch.translate() no longer produces patterns which contain set
operations.

Sets starting with '[' or containing '--', '&&', '~~' or '||' will
be interpreted differently in regular expressions in future versions.
Currently they emit warnings. fnmatch.translate() now avoids producing
patterns containing such sets by accident.
(cherry picked from commit 23cdbfa744)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2018-02-09 13:56:50 +02:00
Miss Islington (bot) 2c6f668276 bpo-30693: Fix tarfile test cleanup on MSWindows (GH-5557) (GH-5567)
it was using our mocked listdir to check when the files were gone.
(cherry picked from commit 4ad703b7ca)

Co-authored-by: Bernhard M. Wiedemann <githubbmw@lsmod.de>
2018-02-06 20:33:27 +02:00
Miss Islington (bot) e6499a07d4 Use assertTrue() instead of deprecated assert_(). (GH-5526) (GH-5527)
(cherry picked from commit 95c3262762)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2018-02-04 18:38:56 +02:00
Miss Islington (bot) 1a0239e12e [3.7] bpo-32691: Use mod_spec.parent when running modules with pdb (GH-5510)
Previously the module name was used, which broke relative imports when pdb was run against a plain module or submodule.
(cherry picked from commit 38bfa8418f)

Co-authored-by: Mario Corchero <mariocj89@gmail.com>
2018-02-04 18:07:16 +10:00
Miss Islington (bot) e86db34dd3 [3.7] bpo-32746: Fix multiple typos (GH-5144) (GH-5520)
Fix typos found by codespell in docs, docstrings, and comments.
(cherry picked from commit c3d9508ff2)

Co-authored-by: Leo Arias <leo.arias@canonical.com>
2018-02-03 20:41:43 -05:00
Miss Islington (bot) f33ecedcad bpo-32604: Fix memory leaks in the new _xxsubinterpreters module. (GH-5507)
(cherry picked from commit 4e9da0d163)

Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
2018-02-02 22:38:57 -07:00
Barry Warsaw a23d30f64b
bpo-32303 - Consistency fixes for namespace loaders (GH-5481) (#5503)
* Make sure ``__spec__.loader`` matches ``__loader__`` for namespace packages.
* Make sure ``__spec__.origin` matches ``__file__`` for namespace packages.

https://bugs.python.org/issue32303
https://bugs.python.org/issue32305
(cherry picked from commit bbbcf8693b)

Co-authored-by: Barry Warsaw <barry@python.org>
2018-02-02 19:49:25 -05:00
Miss Islington (bot) 2b5937ec0a bpo-32734: Fix asyncio.Lock multiple acquire safety issue (GH-5466) (#5501)
(cherry picked from commit 2f79c01493)

Co-authored-by: Bar Harel <bzvi7919@gmail.com>
2018-02-02 18:14:38 -05:00
Miss Islington (bot) 29fd9eae43 Revert "bpo-31356: Add context manager to temporarily disable GC GH-5495 (#5496)
This reverts commit 72a0d218dc.

The reverted commit had a few issues so it was unanimously decided
to undo it. See the bpo issue for details.
(cherry picked from commit 383b32fe10)

Co-authored-by: Yury Selivanov <yury@magic.io>
2018-02-02 10:04:32 -05:00
Miss Islington (bot) 709f6ad7f8 bpo-32733: Make test_coroutines robust against -Werror (GH-5487) (GH-5490)
(cherry picked from commit 2efb973589)

Co-authored-by: Nathaniel J. Smith <njs@pobox.com>
2018-02-01 21:57:28 -08:00
Miss Islington (bot) 71a0b0e6e8 Implement TimerHandle.when() (GH-5473) (GH-5480)
(cherry picked from commit 3d4dbd8f01)

Co-authored-by: Andrew Svetlov <andrew.svetlov@gmail.com>
2018-02-01 21:56:03 +02:00
Xiang Zhang 86fdad093b bpo-32583: Fix possible crashing in builtin Unicode decoders (#5325)
When using customized decode error handlers, it is possible for builtin decoders
to write out-of-bounds and then crash.
2018-01-31 17:02:12 -05:00
Bernhard M. Wiedemann 57750be4ad bpo-30693: zip+tarfile: sort directory listing (#2263)
tarfile and zipfile now sort directory listing to generate tar and zip archives
in a more reproducible way.

See also https://reproducible-builds.org/docs/stable-inputs/ on that topic.
2018-01-31 17:01:55 -05:00
Stéphane Wirtel 209108bd69 bpo-32727: smtplib's SMTP.send_message behaves differently with from_addr and to_addrs (#5451)
Do not pass the name field in the 'from' address in the SMTP envelope.
2018-01-31 17:01:31 -05:00
Ned Deily 763f094b1f
bpo-28440: Don't add /Library/Python/3.x/site-packages to sys.path (#5445)
No longer add /Library/Python/3.x/site-packages, the Apple-supplied
system Python site-packages directory, to sys.path for macOS framework
builds in case Apple ships a version of Python 3. A similar change
was made earlier to Python 2.7 where it was found that the coupling
between the system Python and a user-installed framework Python often
caused confusion or pip install failures.
2018-01-30 05:14:09 -05:00
John Jolly 066df4fd45 bpo-22908: Add seek and tell functionality to ZipExtFile (GH-4966)
This allows for nested zip files, tar files within zip files, zip files within tar files, etc.

Contributed by: John Jolly
2018-01-30 00:51:35 -08:00
Christian Heimes 2e0ecde8d7
bpo-28134: Ignore proto in unknown socket test (GH-5435)
Band-aid for macOS: Some platforms seem to ignore unknown protocols.

Signed-off-by: Christian Heimes <christian@python.org>
2018-01-30 08:55:46 +01:00
Anders Lorentsen dd42cb71f2 bpo-31961: subprocess now accepts path-like args (GH-4329)
Allow os.PathLike args in subprocess APIs.
2018-01-29 23:27:28 -08:00
Bo Bayles ce0f33d045 bpo-32102 Add "capture_output=True" to subprocess.run (GH-5149)
Add "capture_output=True" option to subprocess.run, this is equivalent to
setting stdout=PIPE, stderr=PIPE but is much more readable.
2018-01-29 22:40:39 -08:00
Benjamin Peterson 95441809ef
closes bpo-32721: do not fail test_hashlib if _md5 isn't available (GH-5441) 2018-01-29 22:14:17 -08:00
Bo Bayles ce237c7d58 bpo-21417: Add compresslevel= to the zipfile module (GH-5385)
This allows the compression level to be specified when writing zipfiles
(for the entire file *and* overridden on a per-file basis).

Contributed by Bo Bayles
2018-01-29 21:54:07 -08:00
Gregory P. Smith f4d644f36f
bpo-25942: make subprocess more graceful on ^C (GH-5026)
Do not allow receiving a SIGINT to cause the subprocess module to trigger an
immediate SIGKILL of the child process.  SIGINT is normally sent to all child
processes by the OS at the same time already as was the established normal
behavior in 2.7 and 3.2.  This behavior change was introduced during the fix to https://bugs.python.org/issue12494 and is generally surprising to command line
tool users who expect other tools launched in child processes to get their own
SIGINT and do their own cleanup.

In Python 3.3-3.6 subprocess.call and subprocess.run would immediately
SIGKILL the child process upon receiving a SIGINT (which raises a
KeyboardInterrupt).  We now give the child a small amount of time to
exit gracefully before resorting to a SIGKILL.

This is also the case for subprocess.Popen.__exit__ which would
previously block indefinitely waiting for the child to die.  This was
hidden from many users by virtue of subprocess.call and subprocess.run
sending the signal immediately.

Behavior change: subprocess.Popen.__exit__ will not block indefinitely
when the exiting exception is a KeyboardInterrupt.  This is done for
user friendliness as people expect their ^C to actually happen.  This
could cause occasional orphaned Popen objects when not using `call` or
`run` with a child process that hasn't exited.

Refactoring involved: The Popen.wait method deals with the
KeyboardInterrupt second chance, existing platform specific internals
have been renamed to _wait().
Also fixes comment typos.
2018-01-29 21:27:39 -08:00
Benjamin Peterson 77526f05fa replace dynamic import with 'exec' with importlib.import_module (#5433) 2018-01-29 18:03:01 -08:00
Eric Snow 7f8bfc9b9a
bpo-32604: Expose the subinterpreters C-API in a "private" stdlib module. (gh-1748)
The module is primarily intended for internal use in the test suite.  Building the module under Windows will come in a follow-up PR.
2018-01-29 18:23:44 -07:00
Mark Shannon 332cd5ee4f bpo-32550. Remove the STORE_ANNOTATION bytecode. (GH-5181) 2018-01-29 16:41:04 -08:00
Christian Heimes b6e43af669
bpo-28134: Auto-detect socket values from file descriptor (#1349)
Fix socket(fileno=fd) by auto-detecting the socket's family, type,
and proto from the file descriptor. The auto-detection can be overruled
by passing in family, type, and proto explicitly.

Without the fix, all socket except for TCP/IP over IPv4 are basically broken:

>>> s = socket.create_connection(('www.python.org', 443))
>>> s
<socket.socket fd=3, family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=6, laddr=('2003:58:bc4a:3b00:56ee:75ff:fe47:ca7b', 59730, 0, 0), raddr=('2a04:4e42:1b::223', 443, 0, 0)>
>>> socket.socket(fileno=s.fileno())
<socket.socket fd=3, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('2003:58:bc4a:3b00::%2550471192', 59730, 0, 2550471192), raddr=('2a04:4e42:1b:0:700c:e70b:ff7f:0%2550471192', 443, 0, 2550471192)>

Signed-off-by: Christian Heimes <christian@python.org>
2018-01-29 22:37:58 +01:00
Pablo Galindo 72a0d218dc bpo-31356: Add context manager to temporarily disable GC (GH-4224) 2018-01-29 12:37:09 -08:00
Yury Selivanov 2a2270db9b
bpo-32703: Fix coroutine resource warning in case where there's an error (GH-5410)
The commit removes one unnecessary "if" clause in genobject.c.  That "if" clause was masking un-awaited coroutines warnings just to make writing unittests more convenient.
2018-01-29 14:31:47 -05:00
jayyyin aa218d1649 bpo-27931: Fix email address header parsing error (#5329)
Correctly handle addresses whose username is an empty quoted string.
2018-01-29 13:07:44 -05:00
Andrew Svetlov 4f4ef0acba
Add a test for pdb until command in coroutine (#5427) 2018-01-29 16:17:45 +02:00
Christian Heimes 66e5742bec
bpo-28414: ssl module idna test (#5395)
Add test cases for IDNA 2003 and 2008 host names. IDNA 2003
internationalized host names are working since bpo-31399 has landed. IDNA
2008 deviations are still broken and will be fixed in another patch.

Signed-off-by: Christian Heimes <christian@python.org>
2018-01-29 14:25:13 +01:00
Christian Heimes 892d66e422
bpo-31429: Define TLS cipher suite on build time (#3532)
Until now Python used a hard coded white list of default TLS cipher
suites. The old approach has multiple downsides. OpenSSL's default
selection was completely overruled. Python did neither benefit from new
cipher suites (ChaCha20, TLS 1.3 suites) nor blacklisted cipher suites.
For example we used to re-enable 3DES.

Python now defaults to OpenSSL DEFAULT cipher suite selection and black
lists all unwanted ciphers. Downstream vendors can override the default
cipher list with --with-ssl-default-suites.

Signed-off-by: Christian Heimes <christian@python.org>
2018-01-29 14:10:18 +01:00
Victor Stinner d951157268
bpo-20891: Reenable test_embed.test_bpo20891() (GH-5420) 2018-01-29 14:04:44 +01:00
Victor Stinner 51500f3745
bpo-32706: Skip test_ftplib.test_check_hostname() (#5422)
This test is unstable and currently prevents to make any new change
since the test always fails on Travis CI.

Skip the test to get more time to fix it.
2018-01-29 13:21:34 +01:00
Nathaniel J. Smith 95e214713a
bpo-32591: silence deprecation warnings in test_coroutine (GH-5412) 2018-01-28 23:34:26 -08:00
Yury Selivanov 1e5b25b8c0
bpo-23749: Make start-tls tests more stable on win7 buildbot (GH-5409)
To mitigate the situation when the buildbot is under load
and is unable to send/receive data fast enough:

* reduce the size of the payload
* set a generous timeout for socket ops
2018-01-29 00:25:05 -05:00
Yury Selivanov 9ee1bf9ab5
bpo-32650: Add an asyncgen pdb test (#5406) 2018-01-28 22:43:46 -05:00
embg 1e34da49ef bpo-28685: Optimize sorted() list.sort() with type-specialized comparisons (#582) 2018-01-28 19:03:23 -08:00
Pablo Galindo 6c6ddf97c4 bpo-20104: Expose `posix_spawn` in the os module (GH-5109)
Add os.posix_spawn to wrap the low level POSIX API of the same name.

Contributed by Pablo Galindo.
2018-01-28 17:56:10 -08:00
Pablo Galindo c7ab581db2 bpo-32650 Add support for async generators and more test for coroutines in pdb (#5403) 2018-01-28 20:31:00 -05:00
Pablo Galindo 4687702442 bpo-32650: Add native coroutine support to bdb when stepping over line (GH-5400) 2018-01-29 02:25:05 +02:00
Eric V. Smith 7c99e931a9
Fix trivial typo in test_dataclasses.py. (GH-5398) 2018-01-28 19:18:55 -05:00
Yury Selivanov f11b460d87
bpo-32373: Add socket.getblocking() method. (#4926) 2018-01-28 17:27:38 -05:00
Yury Selivanov 631fd38dbf
bpo-32251: Implement asyncio.BufferedProtocol. (#4755) 2018-01-28 16:30:26 -05:00
Yury Selivanov 0ceb717689
Revert "bpo-32221: makeipaddr(): remove interface part + speedup (GH-4724)" (#5394)
This reverts commit 47c0b1f7d4.
2018-01-28 16:08:32 -05:00
larryhastings f36ba12809
bpo-32697: Definition order of kwonly params is now guaranteed preserved. (#5391)
Definition order of kwonly params is now guaranteed preserved.
2018-01-28 11:13:09 -08:00
Yury Selivanov bec2372b7e
bpo-32327: Revert loop.run_in_executor behaviour: return a Future. (#5392)
I've run some tests on 3.7 asyncio and it appears that too many
things assume that run_in_executor returns a Future.
2018-01-28 14:09:40 -05:00
Nitish Chandra 059f58ce93 bpo-32228: Reset raw_pos after unwinding the raw stream (#4858)
Ensure that ``truncate()`` preserves the file position (as reported by ``tell()``) after writes longer than the buffer size.
2018-01-28 17:00:09 +01:00
Nitish Chandra 43c0f1ac5e bpo-32685: Improve suggestion for print statement (GH-5375)
Better account for single-line compound statements and
semi-colon separated statements when suggesting
Py3 replacements for Py2 print statements.

Initial patch by Nitish Chandra.
2018-01-28 20:56:02 +10:00
Mario Corchero fcf8b4c102 bpo-32206: Update pdb usage to include new module option (GH-5111) 2018-01-28 14:58:47 +10:00
Martin Teichmann dd0e087edc bpo-30306: release arguments of contextmanager (GH-1500)
The arguments to a generator function which is declared as a
contextmanager are stored inside the context manager, and
thus are kept alive, even when it is used as a regular context
manager, and not as a function decorator (where it needs
the original arguments to recreate the generator on each
call).

This is a possible unnecessary memory leak, so this changes
contextmanager.__enter__ to release the saved arguments,
as that method being called means that particular CM instance
isn't going to need to recreate the underlying generator.

Patch by Martin Teichmann.
2018-01-28 14:17:46 +10:00
INADA Naoki bea57060c8
bpo-32677: Optimize str.isascii() (GH-5356) 2018-01-28 09:59:12 +09:00
Eric V. Smith ea8fc52e75
bpo-32513: Make it easier to override dunders in dataclasses. (GH-5366)
Class authors no longer need to specify repr=False if they want to provide a custom __repr__ for dataclasses. The same thing applies for the other dunder methods that the dataclass decorator adds. If dataclass finds that a dunder methods is defined in the class, it will not overwrite it.
2018-01-27 19:07:40 -05:00
Коренберг Марк 47c0b1f7d4 bpo-32221: makeipaddr(): remove interface part + speedup (GH-4724) 2018-01-27 17:20:50 -05:00
Andrew Svetlov 0f54e00e96
Fix tests if Python compiled without SSL (#5367) 2018-01-28 00:00:37 +02:00
Yury Selivanov b1a6ac4c40
bpo-32622: Enforce sendfile fallback policy for FALLBACK transports (#5364) 2018-01-27 15:52:52 -05:00
Andrew Svetlov 7c684073f9
bpo-32622: Implement loop.sendfile() (#5271) 2018-01-27 21:22:47 +02:00
Yury Selivanov f13f12d8da
bpo-32630: Use contextvars in decimal (GH-5278) 2018-01-27 13:46:46 -05:00
Pablo Galindo 4defba3b95 bpo-31368: Expose preadv and pwritev in the os module (#5239) 2018-01-27 17:16:37 +01:00
Christian Heimes 61d478c71c
bpo-31399: Let OpenSSL verify hostname and IP address (#3462)
bpo-31399: Let OpenSSL verify hostname and IP

The ssl module now uses OpenSSL's X509_VERIFY_PARAM_set1_host() and
X509_VERIFY_PARAM_set1_ip() API to verify hostname and IP addresses.

* Remove match_hostname calls
* Check for libssl with set1_host, libssl must provide X509_VERIFY_PARAM_set1_host()
* Add documentation for OpenSSL 1.0.2 requirement
* Don't support OpenSSL special mode with a leading dot, e.g. ".example.org" matches "www.example.org". It's not standard conform.
* Add hostname_checks_common_name

Signed-off-by: Christian Heimes <christian@python.org>
2018-01-27 15:51:38 +01:00
Christian Heimes d0e31b980f
bpo-32454: socket closefd (#5048)
Add close(fd) function to the socket module

Signed-off-by: Christian Heimes <christian@python.org>
2018-01-27 09:54:13 +01:00
Christian Heimes 2f050c7e1b
bpo-32433: Optimized HMAC digest (#5023)
The hmac module now has hmac.digest(), which provides an optimized HMAC
digest for short messages. hmac.digest() is up to three times faster
than hmac.HMAC().digest().

Signed-off-by: Christian Heimes <christian@python.org>
2018-01-27 09:53:43 +01:00
INADA Naoki a49ac99029
bpo-32677: Add .isascii() to str, bytes and bytearray (GH-5342) 2018-01-27 14:06:21 +09:00
Yury Selivanov 43c47fe096
bpo-32670: Enforce PEP 479. (#5327) 2018-01-26 15:24:24 -05:00
Nathaniel J. Smith dba976b8a2 bpo-32591: fix abort in _PyErr_WarnUnawaitedCoroutine during shutdown (#5337)
When an unawaited coroutine is collected very late in shutdown --
like, during the final GC at the end of PyImport_Cleanup -- then it
was triggering an interpreter abort, because we'd try to look up the
"warnings" module and not only was it missing (we were prepared for
that), but the entire module system was missing (which we were not
prepared for).

I've tried to fix this at the source, by making the utility function
get_warnings_attr robust against this in general. Note that it already
has the convention that it can return NULL without setting an error,
which is how it signals that the attribute it was asked to fetch is
missing, and that all callers already check for NULL returns.

There's a similar check for being late in shutdown at the top of
warn_explicit, which might be unnecessary after this fix, but I'm not
sure so I'm going to leave it.
2018-01-26 14:28:31 -05:00
Guido van Rossum 95e4d58913 String annotations [PEP 563] (#4390)
* Document `from __future__ import annotations`
* Provide plumbing and tests for `from __future__ import annotations`
* Implement unparsing the AST back to string form

This is required for PEP 563 and as such only implements a part of the
unparsing process that covers expressions.
2018-01-26 08:20:18 -08:00
bennorth d7773d92bd bpo-18533: Avoid RecursionError from repr() of recursive dictview (#4823)
dictview_repr(): Use a Py_ReprEnter() / Py_ReprLeave() pair to check
for recursion, and produce "..." if so.

test_recursive_repr(): Check for the string rather than a
RecursionError.  (Test cannot be any tighter as contents are
implementation-dependent.)

test_deeply_nested_repr(): Add new test, replacing the original
test_recursive_repr().  It checks that a RecursionError is raised in
the case of a non-recursive but deeply nested structure.  (Very
similar to what test_repr_deep() in test/test_dict.py does for a
normal dict.)

OrderedDictTests: Add new test case, to test behavior on OrderedDict
instances containing their own values() or items().
2018-01-26 07:46:01 -08:00
Yury Selivanov 4112c5b97d
bpo-32662: Try making test_asyncio.test_server more reliable (#5338) 2018-01-26 01:30:57 -05:00
mwidjaja 863b1e4d0e bpo-29237: Create enum for pstats sorting options (GH-5103) 2018-01-25 20:49:56 -08:00
José Melero Fernández c47dacb690 bpo-32574: Fix leaks in asyncio.Queue.put() and .get() (#5208) 2018-01-25 18:45:43 -05:00
Yury Selivanov c9070d03f5
bpo-32662: Implement Server.start_serving() and Server.serve_forever() (#5312)
* bpo-32662: Implement Server.start_serving() and Server.serve_forever()

New methods:

* Server.start_serving(),
* Server.serve_forever(), and
* Server.is_serving().

Add 'start_serving' keyword parameter to loop.create_server() and
loop.create_unix_server().
2018-01-25 18:08:09 -05:00
Ilya Kulakov 1aa094f740 bpo-29302: Implement contextlib.AsyncExitStack. (#4790) 2018-01-25 15:51:18 -05:00
Victor Stinner b31206a223
bpo-32667: Fix tests when $PATH contains a file (#5322)
Some tests failed when the PATH environment variable contained a path
to an existing file. Fix tests to ignore also NotADirectoryError, not
only FileNotFoundError and PermissionError.
2018-01-25 19:06:05 +01:00
Victor Stinner 2b822a0bb1
bpo-32652: Defer pymain_set_global_config() call (#5303)
In Py_Main(), don't call pymain_set_global_config() early: only call
it when the whole configuration has been read.

Add an unit test to prevent future regression.
2018-01-25 09:18:36 +01:00
Andrew Svetlov fe133aad52
bpo-32391: Implement StreamWriter.wait_closed() (#5281) 2018-01-25 00:30:30 +02:00
Paul Ganssle 04af5b1ba9 bpo-10381: Add timezone to datetime C API (#5032)
* Add timezone to datetime C API

* Add documentation for timezone C API macros

* Add dedicated tests for datetime type check macros

* Remove superfluous C API test

* Drop support for TimeZoneType in datetime C API

* Expose UTC singleton to the datetime C API

* Update datetime C-API documentation to include links

* Add reference count information for timezone constructors
2018-01-24 17:29:30 -05:00
Bernhard M. Wiedemann ccbe5818af bpo-29708: Setting SOURCE_DATE_EPOCH forces hash-based .pyc files (GH-5200)
To support reproducible builds, the setting of of SOURCE_DATE_EPOCH triggers the py_compile module -- and by extension, compileall -- to forcibly compile with hash-based .pyc files. This eliminates the possibility of timestamp-based .pyc files which vary between builds.
2018-01-24 13:26:18 -08:00
Victor Stinner 789e359f51 bpo-32636: Fix two bugs in test_asyncio (#5302) 2018-01-24 15:15:14 -05:00
Nathaniel J. Smith fb5a7ad421 bpo-32636: Fix @asyncio.coroutine debug mode bug exposed by gh-5250 (#5291) 2018-01-24 15:14:33 -05:00
Andrew Svetlov 0a5e71b4c7
bpo-32410: Improve sock_sendfile tests (#5294)
* Rename sock_sendfile test methods
* Make sock_sendfile tests more stable
2018-01-24 21:40:35 +02:00
Yury Selivanov 22feeb88b4
bpo-32643: Drop support for a few private Task and Future APIs. (#5293)
Specifically, it's not possible to subclass Task/Future classes
and override the following methods:

* Future._schedule_callbacks
* Task._step
* Task._wakeup
2018-01-24 11:31:01 -05:00
Bo Bayles 6b273f7f40 bpo-32502: Discard 64-bit (and other invalid) hardware addresses (#5254) 2018-01-23 20:11:44 -05:00
Xiang Zhang 370d04d1dc
bpo-32618: Fix test_mutatingdecodehandler not testing test.mutating (#5269)
* bpo-32618: Fix test_mutatingdecodehandler not testing test.mutating

It should test both test.replacing and test.mutating instead of test test.replacing twice.
2018-01-23 22:50:50 +08:00
Nathaniel J. Smith 6934831e43 bpo-32633: Fix some warnings in test_asyncio.test_tasks (#5280) 2018-01-23 11:09:31 +02:00
Yury Selivanov f23746a934
bpo-32436: Implement PEP 567 (#5027) 2018-01-22 19:11:18 -05:00
Victor Stinner 13ff24582c
bpo-32593: Drop FreeBSD 9 and older support (#5232)
Drop support of FreeBSD 9 and older.
2018-01-22 18:32:50 +01:00
Yury Selivanov b0a7a037b8
bpo-31179: Make dict.copy() up to 5.5 times faster. (#3067) 2018-01-22 11:54:41 -05:00
Ethan Furman a4b1bb4801
bpo-31801: Enum: add _ignore_ as class option (#5237)
* bpo-31801:  Enum:  add _ignore_ as class option

_ignore_ is a list, or white-space seperated str, of names that will not
be candidates for members; these names, and _ignore_ itself, are removed
from the final class.

* bpo-31801:  Enum:  add documentation for _ignore_

* bpo-31801: Enum: remove trailing whitespace

* bpo-31801: Enum: fix bulleted list format

* bpo-31801: add version added for _ignore_
2018-01-22 07:56:37 -08:00
Yury Selivanov a4afcdfa55
bpo-32314: Fix asyncio.run() to cancel runinng tasks on shutdown (#5262) 2018-01-21 14:56:59 -05:00
Nathaniel J. Smith fc2f407829 bpo-32591: Add native coroutine origin tracking (#5250)
* Add coro.cr_origin and sys.set_coroutine_origin_tracking_depth
* Use coroutine origin information in the unawaited coroutine warning
* Stop using set_coroutine_wrapper in asyncio debug mode
* In BaseEventLoop.set_debug, enable debugging in the correct thread
2018-01-21 09:44:07 -05:00
Serhiy Storchaka 1211c9a989
bpo-32503: Avoid creating too small frames in pickles. (#5127) 2018-01-20 16:42:44 +02:00
Christian Heimes bd5c7d238c
bpo-32602: Test ECDSA certs (#5247)
Add test certs and test for ECDSA cert and EC/RSA dual mode.

I'm also adding certs for IDNA 2003/2008 tests and simplify some test
data handling.

Signed-off-by: Christian Heimes <christian@python.org>
2018-01-20 15:16:30 +01:00
Ivan Levkivskyi d911e40e78
bpo-32226: PEP 560: improve typing module (#4906)
This PR re-designs the internal typing API using the new PEP 560 features.
However, there are only few minor changes in the public API.
2018-01-20 11:23:59 +00:00
Sanyam Khurana d57f26c753 bpo-32028: Fix suggestions for indented print statements (GH-4688)
The suggested replacement for print statements previously failed to account
for leading whitespace and hence could end up including unwanted text in
the proposed call to the print builtin.

Patch by Sanyam Khurana.
2018-01-20 13:12:22 +10:00
Andrew Svetlov 7464e87a65
bpo-32410: Make SendfileNotAvailableError exception public (#5243) 2018-01-19 20:04:29 +02:00
Gregory P. Smith 3941499d6c
Use assertCountEqual instead of assertEqual. (#5223)
This test doesn't care about order, the underlying filesystem APIs do not
guarantee directory listings on subsequent calls will be in the same order.
2018-01-17 14:57:57 -08:00
Victor Stinner 7d91c02504
pythoninfo: add time.time and datetime.datetime.now (#5214) 2018-01-17 16:35:45 +01:00
Victor Stinner 3a0cf93179
bpo-32248: Fix test_importlib.test_open() (#5213)
Use the binary.file instead of utf-8.file to avoid issues with
Unix newlines vs Windows newlines.
2018-01-17 15:21:50 +01:00
Paul Ganssle 9f1b7b93f5 bpo-32403: Faster date and datetime constructors (#4993)
* Add tests for date subclass alternate constructors

* Switch over alternate date constructors to fast path

* Switch datetime constructors to fastpath, fix bpo-32404

* Add fast path for datetime in date subclass constructor

* Set fold in constructor in datetime.combine

* Add news entries.
2018-01-16 13:06:31 -05:00
Andrew Svetlov 6b5a27975a
bpo-32410: Implement loop.sock_sendfile() (#4976) 2018-01-16 19:59:34 +02:00
Victor Stinner c495e799ed
Skip test_readline.test_nonascii() on C locale (#5203)
bpo-29240: On FreeBSD, if the LC_CTYPE locale is "C" or "POSIX",
writing and reading non-ASCII bytes into/from a TTY works,
but readline or ncurses ignores non-ASCII bytes on read.
2018-01-16 17:34:34 +01:00
Antoine Pitrou 94e1696d04
bpo-14976: Reentrant simple queue (#3346)
Add a queue.SimpleQueue class, an unbounded FIFO queue with a reentrant C implementation of put().
2018-01-16 00:27:16 +01:00
Barry Warsaw 5ec0feeeec
Implement the get_resource_reader() API for file system imports (#5168) 2018-01-15 15:07:11 -08:00
Yury Selivanov 21102f0dc2
Make asyncio tests run when there's no SSL module (#5196) 2018-01-15 17:56:32 -05:00
Raymond Hettinger d37258dd2e
Add itertools recipe for directly finding the n-th combination (#5161) 2018-01-13 10:35:40 -08:00
Serhiy Storchaka 5b76bdba07
bpo-31993: Do not use memoryview when pickle large strings. (#5154)
PyMemoryView_FromMemory() created a memoryview referring to
the internal data of the string.  When the string is destroyed
the memoryview become referring to a freed memory.
2018-01-13 00:28:31 +02:00
Raymond Hettinger 02556fbade
bpo-32467: Let collections.abc.ValuesView inherit from Collection (#5152) 2018-01-11 21:53:49 -08:00
Serhiy Storchaka 782d6fe443
bpo-31113: Get rid of recursion in the compiler for normal control flow. (#3015) 2018-01-11 20:20:13 +02:00
Serhiy Storchaka 0a2da50e18
bpo-31993: Do not create frames for large bytes and str objects (#5114)
when serialize into memory buffer with C pickle implementations.

This optimization already is performed when serialize into memory
with Python pickle implementations or into a file with both
implementations.
2018-01-11 13:03:20 +02:00
Raymond Hettinger 3948207c61
bpo-32320: Add default value support to collections.namedtuple() (#4859) 2018-01-10 21:45:19 -08:00
Victor Stinner 2cba6b8579
bpo-29240: readline now ignores the UTF-8 Mode (#5145)
Add new fuctions ignoring the UTF-8 mode:

* _Py_DecodeCurrentLocale()
* _Py_EncodeCurrentLocale()
* _PyUnicode_DecodeCurrentLocaleAndSize()
* _PyUnicode_EncodeCurrentLocale()

Modify the readline module to use these functions.

Re-enable test_readline.test_nonascii().
2018-01-10 22:46:15 +01:00
Mario Corchero f80c0ca133 Fix when parsing tz offsets microseconds shorter than 6 (#4781)
As the remainder was directly parsed as an int, strings like
.600 were parsed as 600 microseconds rather than milliseconds.
2018-01-09 16:37:26 -05:00
Serhiy Storchaka d4864c61e3
bpo-24340: Fix estimation of the code stack size. (#5076) 2018-01-09 21:54:52 +02:00
Nick Coghlan 9b99747386
bpo-31975 (PEP 565): Show DeprecationWarning in __main__ (GH-4458)
- primary change is to add a new default filter entry for
  'default::DeprecationWarning:__main__'
- secondary change is an internal one to cope with plain
  strings in the warning module's internal filter list
  (this avoids the need to create a compiled regex object
  early on during interpreter startup)
- assorted documentation updates, including many more
  examples of configuring the warnings settings
- additional tests to ensure that both the pure Python and
  the C accelerated warnings modules have the expected
  default configuration
2018-01-08 12:45:02 +10:00
Serhiy Storchaka 3460198f6b
bpo-31802: Fix importing native path module before importing os. (#4017) 2018-01-07 17:54:31 +02:00
Nathaniel J. Smith e46a8af450 bpo-30579: Allow TracebackType creation and tb_next mutation from Python (GH-4793)
Third party projects may wish to hide their own internal machinery in
order to present more comprehensible tracebacks to end users
(e.g. Jinja2 and Trio both do this).

Previously such projects have had to rely on ctypes to do so:

  fe3dadacdf/jinja2/debug.py (L345)
  1e86b1aee8/trio/_core/_multierror.py (L296)

This provides a Python level API for creating and modifying real
Traceback objects, allowing tracebacks to be edited at runtime.

Patch by Nathaniel Smith.
2018-01-07 23:30:18 +10:00
Eric V. Smith d80b443f02
bpo-32279: Add additional params to make_dataclass(), pass through to dataclass(). (gh-5117) 2018-01-06 17:09:58 -05:00
Eric V. Smith ed7d429ebb
bpo-32278: Allow dataclasses.make_dataclass() to omit type information. (gh-5115) 2018-01-06 16:14:03 -05:00
Eric V. Smith e7ba013d87
bpo-32499: Add dataclasses.is_dataclass(obj), which returns True if obj is a dataclass or an instance of one. (#5113) 2018-01-06 12:41:53 -05:00
Olivier Grisel 3cd7c6e6eb bpo-31993: Do not allocate large temporary buffers in pickle dump. (#4353)
The picklers do no longer allocate temporary memory when dumping large
bytes and str objects into a file object. Instead the data is
directly streamed into the underlying file object.

Previously the C implementation would buffer all content and issue a
single call to file.write() at the end of the dump. With protocol 4
this behavior has changed to issue one call to file.write() per frame.

The Python pickler with protocol 4 now dumps each frame content as a
memoryview to an IOBytes instance that is never reused and the
memoryview is no longer released after the call to write. This makes it
possible for the file object to delay access to the memoryview of
previous frames without forcing any additional memory copy as was
already possible with the C pickler.
2018-01-06 17:18:54 +02:00
Mario Corchero 9f1e5f1b7f bpo-32206: Pdb can now run modules (GH-4752)
Add a new argument "-m" to the pdb module to allow
users to run `python -m pdb -m my_module_name`.

This relies on private APIs in the runpy module to work,
but we can get away with that since they're both part of
the standard library and can be updated together if
the runpy internals get refactored.
2018-01-06 17:53:05 +10:00
Thomas Moreau 94459fd7dc bpo-31699 Deadlocks in `concurrent.futures.ProcessPoolExecutor` with pickling error (#3895)
Fix deadlocks in :class:`concurrent.futures.ProcessPoolExecutor` when task arguments or results cause pickling or unpickling errors.
This should make sure that calls to the :class:`ProcessPoolExecutor` API always eventually return.
2018-01-05 11:15:54 +01:00
Serhiy Storchaka ce5b0e9db1
bpo-32226: Make __class_getitem__ an automatic class method. (#5098) 2018-01-05 00:21:41 +02:00
Serhiy Storchaka 87be28f4a1
bpo-31672: Restore the former behavior when override flags in Template. (#5099)
Overriding flags to 0 will make the default pattern matching only
lower case letters.
2018-01-04 19:20:11 +02:00
Serhiy Storchaka d8ac4d1d5a
bpo-31778: Make ast.literal_eval() more strict. (#4035)
Addition and subtraction of arbitrary numbers no longer allowed.
2018-01-04 11:15:39 +02:00
Serhiy Storchaka fbb490fd2f
bpo-32308: Replace empty matches adjacent to a previous non-empty match in re.sub(). (#4846) 2018-01-04 11:06:13 +02:00
Serhiy Storchaka 0cc99c8cd7
bpo-32482: Fix suspicious code in tests for syntax and grammar. (#5086) 2018-01-04 10:36:35 +02:00
Serhiy Storchaka 7cc42c356b
bpo-32478: Add tests for 'break' and 'return' inside 'finally' clause. (#5078) 2018-01-02 02:38:35 +02:00
Serhiy Storchaka e8ed96550c
bpo-32416: Add two new tests in test_sys_settrace. (#5072)
Move other test to more proper place.
2018-01-01 19:39:38 +02:00
Antoine Pitrou 14709144b5
bpo-32468: Better frame repr() (#5067)
bpo-32468: Better frame repr()
2017-12-31 22:35:22 +01:00
Yury Selivanov 0c36bed1c4
bpo-32458: Temporarily mask start-tls proactor test on Windows (#5054) 2017-12-30 15:40:20 -05:00
Barry Warsaw deae6b451f
bpo-32248 - Implement importlib.resources (#4911)
Port importlib_resources to importlib.resources
2017-12-30 15:18:06 -05:00
Andrew Svetlov ffcb4c0165
bpo-32418: Postfix, raise NotImplementdError and close resources in tests (#5052) 2017-12-30 18:52:56 +02:00
Srinivas Reddy Thatiparthy (శ్రీనివాస్ రెడ్డి తాటిపర్తి) 1634fc289a bpo-32418: Add get_loop() method on Server, AbstractServer classes (#4997)
* Add abstract get_loop() method to Server, AbstractServer classes.
* Add test cases for get_loop() method in Server, AbstractServer classes
* Add documentation for get_loop() method
2017-12-30 17:09:32 +02:00
Yury Selivanov f111b3dcb4
bpo-23749: Implement loop.start_tls() (#5039) 2017-12-30 00:35:36 -05:00
Benjamin Peterson bbdb17d19b
return the new file descriptor from os.dup2 (closes bpo-32441) (#5041) 2017-12-29 13:13:06 -08:00
Eric V. Smith 03220fdb26
bpo-32427: Expose dataclasses.MISSING object. (#5045) 2017-12-29 13:59:58 -05:00
gfyoung 531dd7e27e Fix minor spelling error in test_sys_settrace.py. (#5025) 2017-12-28 10:28:25 +02:00
Serhiy Storchaka 53f9135667
bpo-32416: Refactor tests for the f_lineno setter and add new tests. (#4991) 2017-12-27 00:38:56 +02:00
Serhiy Storchaka 13a6c098c2
bpo-32259: Make a TypeError message when unpack non-iterable more specific. (#4903) 2017-12-26 12:30:41 +02:00
Andrew Svetlov a8f4e15f3d
bpo-26133: Fix typos (#5010)
* Fix typos
* Change warning text
* Add test
2017-12-26 11:53:38 +02:00
Yury Selivanov e0aef4f3cd
bpo-31721: Allow Future._log_traceback to only be set to False (#5009) 2017-12-25 16:16:10 -05:00
Yury Selivanov 0cf16f9ea0
bpo-32363: Disable Task.set_exception() and Task.set_result() (#4923) 2017-12-25 10:48:15 -05:00
Yury Selivanov 719ccbca69
bpo-32415: Fix "error is already set" (#4999) 2017-12-23 16:29:26 -05:00
Andrew Svetlov a330f483e2
Fix check for run_in_executor on closed loop. (#4996) 2017-12-23 23:03:27 +02:00
Yury Selivanov a8fb079789
bpo-32415: Add more tests (#4995) 2017-12-23 15:42:27 -05:00
Yury Selivanov ca9b36cd1a
bpo-32415: Add asyncio.Task.get_loop() and Future.get_loop() (#4992) 2017-12-23 15:04:15 -05:00
Yury Selivanov 558aa30f79
bpo-32357: Fix tests in refleak mode (#4989) 2017-12-23 12:44:29 -05:00
Victor Stinner 424315fa86
bpo-29240: Skip test_readline.test_nonascii() (#4968)
Skip the test which fails on FreeBSD with POSIX locale.

Skip the test to fix FreeBSD buildbots, until a fix can be found, so
the buildbots can catch other regressions.
2017-12-22 00:09:26 +01:00
Victor Stinner 550ee051d6
bpo-20891: Skip test_embed.test_bpo20891() (#4967)
Skip the test failing randomly because of known race condition.

Skip the test to fix macOS buildbots until a decision is made on the
proper fix for the race condition.
2017-12-22 00:05:05 +01:00
Коренберг Марк fbd605151f bpo-32323: urllib.parse.urlsplit() must not lowercase() IPv6 scope value (#4867) 2017-12-21 14:16:17 +02:00
Paul Ganssle 09dc2f508c bpo-15873: Implement [date][time].fromisoformat (#4699)
Closes bpo-15873.
2017-12-21 00:33:49 -05:00
INADA Naoki 507434fd50
bpo-15216: io: TextIOWrapper.reconfigure() accepts encoding, errors and newline (GH-2343) 2017-12-21 09:59:53 +09:00
Andrew Svetlov 51eb1c6b9c
bpo-29970: Make ssh_handshake_timeout None by default (#4939)
* Make ssh_handshake_timeout None by default.
* Raise ValueError if ssl_handshake_timeout is used without ssl.
* Raise ValueError if ssl_handshake_timeout is not positive.
2017-12-20 20:24:43 +02:00
Antoine Pitrou 1f1a34c314
bpo-32379: Faster MRO computation for single inheritance (#4932)
* bpo-32379: Faster MRO computation for single inheritance
2017-12-20 15:58:21 +01:00
Marcel Plch 776407fe89 bpo-31901: atexit callbacks should be run at subinterpreter shutdown (#4611)
Change atexit behavior and PEP-489 multiphase init support.
2017-12-20 11:17:58 +01:00
Neil Aspinall f7686c1f55 bpo-29970: Add timeout for SSL handshake in asyncio
10 seconds by default.
2017-12-19 21:45:42 +02:00
Julien Duponchelle 319c0345cd bpo-29711: Fix stop_serving in proactor loop kill all listening servers (#431) 2017-12-19 07:23:17 -05:00
Yury Selivanov a9d7e552c7
bpo-32357: Optimize asyncio.iscoroutine() for non-native coroutines (#4915) 2017-12-19 07:18:45 -05:00
Andrew Svetlov 21be85f520
Improve test coverage (#4924) 2017-12-19 08:51:16 +02:00
Yury Selivanov 9818142b1b
bpo-32331: Fix socket.type when SOCK_NONBLOCK is available (#4877) 2017-12-18 20:02:54 -05:00
Yury Selivanov d757aaf9dd
bpo-32356: idempotent pause_/resume_reading; new is_reading method. (#4914) 2017-12-18 17:03:23 -05:00
izbyshev 2d8f06382e bpo-32369: test_subprocess: Fix pass_fds check in test_close_fds() (#4920)
The last part of test_close_fds() doesn't match its own comment.
The following assertion always holds because fds_to_keep and open_fds
are disjoint by construction.

self.assertFalse(remaining_fds & fds_to_keep & open_fds,
                 "Some fds not in pass_fds were left open")

Fix the code to match the message in the assertion.
2017-12-18 12:26:49 -08:00
Segev Finer b2a6083eb0 bpo-19764: Implemented support for subprocess.Popen(close_fds=True) on Windows (#1218)
Even though Python marks any handles it opens as non-inheritable there
is still a race when using `subprocess.Popen` since creating a process
with redirected stdio requires temporarily creating inheritable handles.
By implementing support for `subprocess.Popen(close_fds=True)` we fix
this race.

In order to implement this we use PROC_THREAD_ATTRIBUTE_HANDLE_LIST
which is available since Windows Vista. Which allows to pass an explicit
list of handles to inherit when creating a process.

This commit also adds `STARTUPINFO.lpAttributeList["handle_list"]`
which can be used to control PROC_THREAD_ATTRIBUTE_HANDLE_LIST
directly.
2017-12-18 10:28:19 +01:00
Nathaniel J. Smith 902ab80b59 bpo-30050: Allow disabling full buffer warnings in signal.set_wakeup_fd (#4792) 2017-12-17 23:10:18 -05:00
Yury Selivanov 1b7c11ff0e
bpo-32348: Optimize asyncio.Future schedule/add/remove callback. (#4907) 2017-12-17 20:19:47 -05:00
Andrew Svetlov 44d1a5912e
bpo-32250: Implement asyncio.current_task() and asyncio.all_tasks() (#4799) 2017-12-16 21:58:38 +02:00
Serhiy Storchaka 45700fb757 Add tests for using PEP560 with classes implemented in C. (#4883)
Based on tests from #4878
2017-12-16 10:25:56 +01:00
Nick Coghlan 9c19b02024
bpo-32002: Refactor C locale coercion tests (GH-4369)
Exactly which locale requests will end up giving
you the "C" locale is actually platform dependent.

A blank locale and "POSIX" will translate to "C"
on most Linux distros, but may not do so on other platforms, so this adjusts the way the tests are structured to better account for that.

This is an initial step towards fixing the current
test failure on Cygwin (hence the issue reference)
2017-12-16 21:51:19 +13:00
Victor Stinner 9454060e84
bpo-29240, bpo-32030: Py_Main() re-reads config if encoding changes (#4899)
bpo-29240, bpo-32030: If the encoding change (C locale coerced or
UTF-8 Mode changed), Py_Main() now reads again the configuration with
the new encoding.

Changes:

* Add _Py_UnixMain() called by main().
* Rename pymain_free_pymain() to pymain_clear_pymain(), it can now be
  called multipled times.
* Rename pymain_parse_cmdline_envvars() to pymain_read_conf().
* Py_Main() now clears orig_argc and orig_argv at exit.
* Remove argv_copy2, Py_Main() doesn't modify argv anymore. There is
  no need anymore to get two copies of the wchar_t** argv.
* _PyCoreConfig: add coerce_c_locale and coerce_c_locale_warn.
* Py_UTF8Mode is now initialized to -1.
* Locale coercion (PEP 538) now respects -I and -E options.
2017-12-16 04:54:22 +01:00
Yury Selivanov e796b2fe26
bpo-27456: Ensure TCP_NODELAY is set on linux (#4231) 2017-12-15 19:32:25 -05:00
Brett Cannon 4ac5150e06
bpo-32248: Implement importlib.abc.ResourceReader (GH-4892) 2017-12-15 16:29:35 -08:00
Victor Stinner 9402c8367b
bpo-32294: Fix multiprocessing test_semaphore_tracker() (#4885)
Run the child process with -E option to ignore the PYTHONWARNINGS
environment variable.
2017-12-15 16:29:24 +01:00
Serhiy Storchaka 3327a2ddf1
bpo-32265: Classify class and static methods of builtin types. (#4776)
Add types.ClassMethodDescriptorType for unbound class methods.
2017-12-15 14:13:41 +02:00
Serhiy Storchaka 2e3f570185
bpo-30416: Protect the optimizer during constant folding. (#4860)
It no longer spends much time doing complex calculations and no
longer consumes much memory for creating large constants that will
be dropped later.

This fixes also bpo-21074.
2017-12-15 14:11:43 +02:00
Serhiy Storchaka 3325a6780c
bpo-27169: The __debug__ constant is now optimized out at compile time. (#4880)
This fixes also bpo-22091.
2017-12-15 12:35:48 +02:00
Andrew Svetlov f74ef458ab
bpo-32311: Implement asyncio.create_task() shortcut (#4848)
* Implement functionality
* Add documentation
2017-12-15 07:04:38 +02:00
Yury Selivanov 19a44f63c7
bpo-32327: Convert asyncio functions documented as coroutines to coroutines. (#4872) 2017-12-14 20:53:26 -05:00
Victor Stinner 358e5e17a5
bpo-32329: Fix -R option for hash randomization (#4873)
bpo-32329, bpo-32030:

* The -R option now turns on hash randomization when the
  PYTHONHASHSEED environment variable is set to 0 Previously, the
  option was ignored.
* sys.flags.hash_randomization is now properly set to 0 when hash
  randomization is turned off by PYTHONHASHSEED=0.
* _PyCoreConfig_ReadEnv() now reads the PYTHONHASHSEED environment
  variable. _Py_HashRandomization_Init() now only apply the
  configuration, it doesn't read PYTHONHASHSEED anymore.
2017-12-15 00:51:22 +01:00
Giuseppe Scrivano 96a5e50a5d bpo-32143: add f_fsid to os.statvfs() (#4571)
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2017-12-14 17:46:46 -05:00
Ivan Levkivskyi 2b5fd1e9ca
bpo-32226: Implementation of PEP 560 (core components) (#4732)
This part of the PEP implementation adds support for
__mro_entries__ and __class_getitem__ by updating
__build_class__ and PyObject_GetItem.
2017-12-14 23:32:56 +01:00
Yury Selivanov 02a0a19206
bpo-32314: Implement asyncio.run() (#4852) 2017-12-14 09:42:21 -05:00
Mike 53f7a7c281 bpo-32297: Few misspellings found in Python source code comments. (#4803)
* Fix multiple typos in code comments

* Add spacing in comments (test_logging.py, test_math.py)

* Fix spaces at the beginning of comments in test_logging.py
2017-12-14 13:04:53 +02:00
Ivan Levkivskyi 5364b5cd75
bpo-32225: Implementation of PEP 562 (#4731)
Implement PEP 562: module __getattr__ and __dir__.
The implementation simply updates module_getattro and
module_dir.
2017-12-14 11:59:44 +01:00
Jelle Zijlstra 176baa326b bpo-30241: implement contextlib.AbstractAsyncContextManager (#1412) 2017-12-13 20:19:17 -05:00
Yury Selivanov bfbf04ef18
bpo-32296: Unbreak tests on Windows (#4850) 2017-12-13 17:28:41 -05:00
Yury Selivanov a70232f288
bpo-32296: Implement asyncio.get_event_loop and _get_running_loop in C. (#4827)
asyncio.get_event_loop(), and, subsequently asyncio._get_running_loop()
are one of the most frequently executed functions in asyncio.  They also
can't be sped up by third-party event loops like uvloop.

When implemented in C they become 4x faster.
2017-12-13 14:49:42 -05:00
Victor Stinner 981469794a
pythoninfo: Add builtins, test.support, ... (#4840)
Collect more info from builtins, resource, test.test_socket and
test.support modules.

Co-Authored-By: Christian Heimes <christian@python.org>
2017-12-13 17:27:40 +01:00
Andrew Svetlov 7a6706bf48
Fix couple typos (#4839) 2017-12-13 17:50:16 +02:00
Victor Stinner 91106cd9ff
bpo-29240: PEP 540: Add a new UTF-8 Mode (#855)
* Add -X utf8 command line option, PYTHONUTF8 environment variable
  and a new sys.flags.utf8_mode flag.
* If the LC_CTYPE locale is "C" at startup: enable automatically the
  UTF-8 mode.
* Add _winapi.GetACP(). encodings._alias_mbcs() now calls
  _winapi.GetACP() to get the ANSI code page
* locale.getpreferredencoding() now returns 'UTF-8' in the UTF-8
  mode. As a side effect, open() now uses the UTF-8 encoding by
  default in this mode.
* Py_DecodeLocale() and Py_EncodeLocale() now use the UTF-8 encoding
  in the UTF-8 Mode.
* Update subprocess._args_from_interpreter_flags() to handle -X utf8
* Skip some tests relying on the current locale if the UTF-8 mode is
  enabled.
* Add test_utf8mode.py.
* _Py_DecodeUTF8_surrogateescape() gets a new optional parameter to
  return also the length (number of wide characters).
* pymain_get_global_config() and pymain_set_global_config() now
  always copy flag values, rather than only copying if the new value
  is greater than the old value.
2017-12-13 12:29:09 +01:00
Antoine Pitrou fc5db95e00
Test atexit shutdown mechanism in a subprocess (#4828)
* Test atexit shutdown mechanism in a subprocess
2017-12-13 02:29:07 +01:00
Victor Stinner 721e25c653
bpo-32101: Fix tests for PYTHONDEVMODE=1 (#4821)
test_asycio: remove also aio_path which was used when asyncio was
developed outside the stdlib.
2017-12-12 23:15:00 +01:00
Victor Stinner 747f48e2e9
bpo-32230: Set sys.warnoptions with -X dev (#4820)
Rather than supporting dev mode directly in the warnings module, this
instead adjusts the initialisation code to add an extra 'default'
entry to sys.warnoptions when dev mode is enabled.

This ensures that dev mode behaves *exactly* as if `-Wdefault` had
been passed on the command line, including in the way it interacts
with `sys.warnoptions`, and with other command line flags like `-bb`.

Fix also bpo-20361: have -b & -bb options take precedence over any
other warnings options.

Patch written by Nick Coghlan, with minor modifications of Victor Stinner.
2017-12-12 22:59:48 +01:00
Serhiy Storchaka b748e3b258
Fix improper use of re.escape() in tests. (#4814) 2017-12-12 19:21:50 +02:00
Serhiy Storchaka 0e069a1597
Fix implementation dependent assertion in test_plistlib. (#4813)
It is failed with an advanced optimizer.
2017-12-12 19:03:08 +02:00
Licht Takeuchi 2001900b0c bpo-32255: Always quote a single empty field when write into a CSV file. (#4769)
This allows to distinguish an empty row from a row consisting of a single empty field.
2017-12-12 11:57:06 +02:00
Łukasz Langa e56975351b
bpo-32227: functools.singledispatch supports registering via type annotations (#4733) 2017-12-11 13:56:31 -08:00
Yury Selivanov abae67ebc2
Add asyncio.get_running_loop() function. (#4782) 2017-12-11 10:07:44 -05:00
Yury Selivanov 3e9751819a
bpo-32273: Move asyncio.test_utils to test.test_asyncio (#4785) 2017-12-11 10:04:40 -05:00
Sanyam Khurana 1b74f9b77a bpo-22671: Clarify and test default read method implementations (#4568)
Original patch written by Martin Panter, enhanced by Sanyam Khurana.
2017-12-11 14:42:09 +01:00
Yury Selivanov 6370f345e1
bpo-32262: Fix codestyle; use f-strings formatting where necessary. (#4775) 2017-12-10 18:36:12 -05:00
Yury Selivanov c4d9df5fd7
Skip test_read_pty_output test on macOS for poll and select. (#4774)
Starting with 10.13.2 the test hangs forever.
2017-12-10 12:40:19 -05:00
Sanyam Khurana 780acc89bc bpo-31506: Improve the error message logic for class instantiation (GH-4740)
The error messages in `object.__new__` and `object.__init__` now aim
to point the user more directly at the name of the class being instantiated
in cases where they *haven't* been overridden (on the assumption that
the actual problem is a missing `__new__` or `__init__` definition in the
class body).

When they *have* been overridden, the errors still report themselves as
coming from object, on the assumption that the problem is with the call
up to the base class in the method implementation, rather than with the
way the constructor is being called.
2017-12-10 10:14:22 +10:00
Benjamin Peterson 42aa93b8ff
closes bpo-31650: PEP 552 (Deterministic pycs) implementation (#4575)
Python now supports checking bytecode cache up-to-dateness with a hash of the
source contents rather than volatile source metadata. See the PEP for details.

While a fairly straightforward idea, quite a lot of code had to be modified due
to the pervasiveness of pyc implementation details in the codebase. Changes in
this commit include:

- The core changes to importlib to understand how to read, validate, and
  regenerate hash-based pycs.

- Support for generating hash-based pycs in py_compile and compileall.

- Modifications to our siphash implementation to support passing a custom
  key. We then expose it to importlib through _imp.

- Updates to all places in the interpreter, standard library, and tests that
  manually generate or parse pyc files to grok the new format.

- Support in the interpreter command line code for long options like
  --check-hash-based-pycs.

- Tests and documentation for all of the above.
2017-12-09 10:26:52 -08:00
Andrew Svetlov 28d8d14013
bpo-32253: Deprecate with statement and bare await for asyncio locks (GH-4764)
* Add test for 'with (yield from lock)'
* Deprecate with statement for asyncio locks
* Document the deprecation
2017-12-09 20:00:05 +02:00
Andrew Svetlov 5f841b5538
bpo-32193: Convert asyncio to async/await usage (#4753)
* Convert asyncio/tasks.py to async/await

* Convert asyncio/queues.py to async/await

* Convert asyncio/test_utils.py to async/await

* Convert asyncio/base_subprocess.py to async/await

* Convert asyncio/subprocess.py to async/await

* Convert asyncio/streams.py to async/await

* Fix comments

* Convert asyncio/locks.py to async/await

* Convert asyncio.sleep to async def

* Add a comment

* Add missing news

* Convert stubs from AbstrctEventLoop to async functions

* Convert subprocess_shell/subprocess_exec

* Convert connect_read_pipe/connect_write_pip to async/await syntax

* Convert create_datagram_endpoint

* Convert create_unix_server/create_unix_connection

* Get rid of old style coroutines in unix_events.py

* Convert selector_events.py to async/await

* Convert wait_closed and create_connection

* Drop redundant line

* Convert base_events.py

* Code cleanup

* Drop redundant comments

* Fix indentation

* Add explicit tests for compatibility between old and new coroutines

* Convert windows event loop to use async/await

* Fix double awaiting of async function

* Convert asyncio/locks.py

* Improve docstring

* Convert tests to async/await

* Convert more tests

* Convert more tests

* Convert more tests

* Convert tests

* Improve test
2017-12-09 00:23:48 +02:00
Garrett Berg a0374dd34a bpo-32208: update threading.Semaphore docs and add unit test (#4709)
* fix issue32208: update threading.Semaphore docs and add unit test to validate correct behavior

* add test for blocking

* Update threading.rst

* semaphore: remove documentation validation tests and move 'return value' test to BaseSemaphore
2017-12-07 20:04:26 +02:00
Pablo Galindo fb77e0d855 Fix hardcoded value in test_os.py (#4744) 2017-12-07 07:55:44 +01:00
Eric V. Smith 24e77f9265
Use raw strings for regex to avoid invalid escape sequences. (GH-4741) 2017-12-06 14:00:34 -05:00
Ned Deily b3edde8dd4
bpo-31380: Skip test_httpservers test_undecodable_file on macOS. (#4720)
The undecodable file name cannot be created on macOS APFS file systems.
2017-12-04 23:42:02 -05:00
Ivan Levkivskyi 29bc193210 [bpo-28556] Minor fixes for typing module (#4710) 2017-12-04 21:43:58 -05:00
Eric V. Smith f0db54a0a1
bpo-32214: Implement PEP 557: Data Classes (#4704) 2017-12-04 16:58:55 -05:00
Serhiy Storchaka 70d56fb525
bpo-25054, bpo-1647489: Added support of splitting on zerowidth patterns. (#4471)
Also fixed searching patterns that could match an empty string.
2017-12-04 14:29:05 +02:00
Serhiy Storchaka e69fbb6a56
Fix a regression in uuid added in bpo-32107. (#4677)
uuid.get_node() always must return a stable result.
Also added a test for non-reproducibility of _random_getnode().
Original patch by Xavier de Gaye.
2017-12-04 11:51:55 +02:00
R. David Murray 85d5c18c9d
bpo-27240 Rewrite the email header folding algorithm. (#3488)
The original algorithm tried to delegate the folding to the tokens so
that those tokens whose folding rules differed could specify the
differences.  However, this resulted in a lot of duplicated code because
most of the rules were the same.

The new algorithm moves all folding logic into a set of functions
external to the token classes, but puts the information about which
tokens can be folded in which ways on the tokens...with the exception of
mime-parameters, which are a special case (which was not even
implemented in the old folder).

This algorithm can still probably be improved and hopefully simplified
somewhat.

Note that some of the test expectations are changed.  I believe the
changes are toward more desirable and consistent behavior: in general
when (re) folding a line the canonical version of the tokens is
generated, rather than preserving errors or extra whitespace.
2017-12-03 18:51:41 -05:00
Serhiy Storchaka 1fb72d2ad2
bpo-32137: The repr of deeply nested dict now raises a RecursionError (#4570)
instead of crashing due to a stack overflow.

This perhaps will fix similar problems in other extension types.
2017-12-03 22:12:11 +02:00
Nick Coghlan 078f1814f1
bpo-32176: Set CO_NOFREE in the code object constructor (GH-4675)
Previously, CO_NOFREE was set in the compiler, which meant
it could end up being set incorrectly when code objects
were created directly. Setting it in the constructor based
on freevars and cellvars ensures it is always accurate,
regardless of how the code object is defined.
2017-12-03 11:12:20 +10:00
Neil Schemenauer 1eaa1169c5
Add a minimal unit test for Python/frozen.c. (#2995)
If the marshal or bytecode formats get changed, frozen.c needs to
be updated as well.  It can be easy to miss this step and not doing
so can cause test_importlib to crash in mysterious ways.  Add an
explict unit test to make it easier to track down the problem.
2017-12-02 13:54:00 -08:00
xdegaye bf2b65e413
bpo-28668: test.support.requires_multiprocessing_queue is removed (GH-4560)
Skip tests with test.support.import_module('multiprocessing.synchronize')
instead when the semaphore implementation is broken or missing.
2017-12-01 08:08:49 +01:00
Serhiy Storchaka 73a7e9b10b bpo-10544: Deprecate "yield" in comprehensions and generator expressions. (GH-4579)
The current behaviour of yield expressions inside comprehensions  and
generator expressions is essentially an accident of implementation - it
arises implicitly from the way the compiler handles yield expressions inside
nested functions and generators.

Since the current behaviour wasn't deliberately designed, and is inherently
confusing, we're deprecating it, with no current plans to reintroduce it.
Instead, our advice will be to use a named nested generator definition
for cases where this behaviour is desired.
2017-12-01 14:54:17 +10:00
Serhiy Storchaka a897aeeef6
bpo-32072: Fix issues with binary plists. (#4455)
* Fixed saving bytearrays.
* Identical objects will be saved only once.
* Equal references will be load as identical objects.
* Added support for saving and loading recursive data structures.
2017-11-30 23:26:11 +02:00