Commit Graph

1255 Commits

Author SHA1 Message Date
Vladimir Matveev 91cb298f81 bpo-6700: Fix inspect.getsourcelines for module level frames/tracebacks (GH-8864) 2018-08-24 17:18:00 +03:00
Paul Ganssle 096329f0b2 bpo-34454: fix .fromisoformat() methods crashing on inputs with surrogate code points (GH-8862)
The current C implementations **crash** if the input includes a surrogate
Unicode code point, which is not possible to encode in UTF-8.

Important notes:

1.  It is possible to pass a non-UTF-8 string as a separator to the
    `.isoformat()` methods.
2.  The pure-Python `datetime.fromisoformat()` implementation accepts
    strings with a surrogate as the separator.

In `datetime.fromisoformat()`, in the special case of non-UTF-8 separators,
this implementation will take a performance hit by making a copy of the
input string and replacing the separator with 'T'.

Co-authored-by: Alexey Izbyshev <izbyshev@ispras.ru>
Co-authored-by: Paul Ganssle <paul@ganssle.io>
2018-08-23 18:06:20 +03:00
Michael Osipov 48ce4897f8 bpo-34412: Make signal.strsignal() work on HP-UX (GH-8786)
Introduce a configure check for strsignal(3) which defines HAVE_STRSIGNAL for
signalmodule.c. Add some common signals on HP-UX. This change applies for
Windows and HP-UX.
2018-08-23 16:27:19 +03:00
Berker Peksag e7d4b2f205
bpo-2122: Make mmap.flush() behave same on all platforms (GH-8692)
Previously, its behavior was platform-dependent and there was no error checking
under Windows.
2018-08-22 21:21:05 +03:00
Alexey Izbyshev cdbf50cba1 bpo-34441: Fix ABC.__subclasscheck__ crash on classes with invalid __subclasses__ (GH-8835)
The missing NULL check was reported by Svace static analyzer.
2018-08-20 23:04:19 +03:00
Serhiy Storchaka 77d5781835
bpo-34318: Convert deprecation warnings to errors in assertRaises() etc. (GH-8623) 2018-08-19 10:00:11 +03:00
Zackery Spytz e349bf2358 bpo-22602: Raise an exception in the UTF-7 decoder for ill-formed sequences starting with "+". (GH-8741)
The UTF-7 decoder now raises UnicodeDecodeError for ill-formed
sequences starting with "+" (as specified in RFC 2152).
2018-08-19 07:43:38 +03:00
davidair 2b32da2fea Improve error message when mock.assert_has_calls fails (GH-8205)
This makes the assertion error message more useful, aiding debugging.

Thanks @davidair!
2018-08-17 12:09:58 -07:00
Christian Heimes 1590c39336
bpo-34391: Fix ftplib test for TLS 1.3 (GH-8787)
Read from data socket to avoid "[SSL] shutdown while in init" exception
during shutdown of the dummy server.

Signed-off-by: Christian Heimes <christian@python.org>


<!-- issue-number: [bpo-34391](https://www.bugs.python.org/issue34391) -->
https://bugs.python.org/issue34391
<!-- /issue-number -->
2018-08-16 19:43:44 +02:00
Steve Dower 864a892af3
bpo-34405: Updated to OpenSSL 1.1.0i for Windows builds. (GH-8775) 2018-08-15 13:29:24 -07:00
Berker Peksag e0b5b2096e
bpo-34384: Fix os.readlink() on Windows (GH-8740)
os.readlink() now accepts path-like and bytes objects on Windows.
Previously, support for path-like and bytes objects was only
implemented on Unix.

This commit also merges Unix and Windows implementations of
os.readlink() in one function and adds basic unit tests to increase
test coverage of the function.
2018-08-15 13:03:41 +03:00
Zackery Spytz 7c4ab2afb1 closes bpo-34400: Fix undefined behavior in parsetok(). (GH-4439)
Avoid undefined pointer arithmetic with NULL.
2018-08-14 23:27:26 -07:00
Christian Heimes 88bfd0bce0
bpo-34399: 2048 bits RSA keys and DH params (#8762)
Downstream vendors have started to deprecate weak keys. Update all RSA keys
and DH params to use at least 2048 bits.

Finite field DH param file use RFC 7919 values, generated with

    certtool --get-dh-params --sec-param=high

Signed-off-by: Christian Heimes <christian@python.org>
2018-08-14 12:54:19 +02:00
Vadim Pushtaev 4d12e4dc28 bpo-34213: Allow dataclasses to work with a field named 'object'. (GH-8452) 2018-08-12 07:46:05 -04:00
Sergey Fedoseev 2fc46979b8 bpo-34151: Improve performance of some list operations (GH-8332) 2018-08-11 21:12:07 +08:00
Paul Price db6075ab3a closes bpo-34377: Update Valgrind suppressions. (GH-8729)
Valgrind isn't seeing PyObject_Free/PyObject_Realloc, but using _PyObject_Free/_PyObject_Realloc works.
2018-08-10 23:41:34 -07:00
Berker Peksag 84a13fbda0
bpo-9372: Deprecate several __getitem__ methods (GH-8609)
The __getitem__ methods of DOMEventStream, FileInput,
and FileWrapper classes ignore their 'index' parameters
and return the next item instead.
2018-08-11 09:05:04 +03:00
Berker Peksag 423d05f6f5
bpo-34333: Fix %-formatting in Path.with_suffix() (GH-8663) 2018-08-11 08:45:06 +03:00
Tal Einat 077059e0f0
bpo-34047: IDLE: fix mousewheel scrolling direction on macOS (GH-8678) 2018-08-10 09:02:08 +03:00
GPery b92c526ed5 closes bpo-34353: Add sockets to stat.filemode fallback python implementation. (GH-8703) 2018-08-09 22:12:08 -07:00
Alex Grönholm cca4eec3c0 bpo-34270: Make it possible to name asyncio tasks (GH-8547)
Co-authored-by: Antti Haapala <antti.haapala@anttipatterns.com>
2018-08-08 17:06:47 -04:00
Berker Peksag e4dcbbd7f4
bpo-18540: Fix EAI_NONAME in imaplib.IMAP4*() (GH-8634) 2018-08-07 05:12:18 +03:00
Anthony Sottile b2499669ef bpo-19891: Ignore error while writing history file (GH-8483) 2018-08-06 17:28:19 +09:00
Tal Einat 87e59ac11e
bpo-33839: refactor IDLE's tooltips & calltips, add docstrings and tests (GH-7683)
* make CallTip and ToolTip sub-classes of a common abstract base class
* remove ListboxToolTip (unused and ugly)
* greatly increase test coverage
* tested on Windows, Linux and macOS
2018-08-05 09:21:08 +03:00
Terry Jan Reedy 9beaef6225
bpo-34275: Make IDLE calltips always visible on Mac. (GH-8639)
Some MacOS-tk combinations need .update_idletasks().
The call is both unneeded and innocuous on Linux and Windows.
Patch by Kevin Waltzer.
2018-08-02 22:50:01 -04:00
Marcel Plch a2fe1e52eb bpo-34097: Add support for zipping files older than 1980-01-01 (GH-8270)
ZipFile can zip files older than 1980-01-01 and newer than 2107-12-31 using
a new strict_timestamps parameter at the cost of setting the timestamp
to the limit.
2018-08-02 15:04:52 +02:00
Tal Einat 10ea9409ce
bpo-34120: fix IDLE freezing after closing dialogs (GH-8603)
Added missing .grab_release() calls to all places where we call .grab_set().
2018-08-02 09:18:29 +03:00
costypetrisor 8ed317f1ca bpo-34113: Fix a crash when using LLTRACE is on (GH-8517)
Fix a crash on negative STACKADJ() when Low-Level trace (LLTRACE) is enabled.
2018-07-31 22:55:14 +02:00
MartinAltmayer 944451cd8d bpo-34263 Cap timeout submitted to epoll/select etc. to one day. (GH-8532) 2018-07-31 10:06:12 -04:00
Raymond Hettinger 9c18b1ae52
bpo-33089: Add math.dist() for computing the Euclidean distance between two points (GH-8561) 2018-07-31 00:45:49 -07:00
Serhiy Storchaka 9d5727326a
bpo-33871: Fix os.sendfile(), os.writev(), os.readv(), etc. (GH-7931)
* Fix integer overflow in os.readv(), os.writev(), os.preadv()
  and os.pwritev() and in os.sendfile() with headers or trailers
  arguments (on BSD-based OSes and MacOS).

* Fix sending the part of the file in os.sendfile() on MacOS.
  Using the trailers argument could cause sending more bytes from
  the input file than was specified.

Thanks Ned Deily for testing on 32-bit MacOS.
2018-07-31 10:24:54 +03:00
Serhiy Storchaka f1d36d8efa
bpo-33729: Fix issues with arguments parsing in hashlib. (GH-8346)
* help(hashlib) didn't work because of incorrect module name in blake2b and
  blake2s classes.
* Constructors blake2*(), sha3_*(), shake_*() and keccak_*() incorrectly
  accepted keyword argument "string" for binary data, but documented as
  accepting the "data" keyword argument. Now this parameter is positional-only.
* Keyword-only parameters in blake2b() and blake2s() were not documented as
  keyword-only.
* Default value for some parameters of blake2b() and blake2s() was None,
  which is not acceptable value.
* The length argument for shake_*.digest() was wrapped out to 32 bits.
* The argument for shake_128.digest() and shake_128.hexdigest() was not
  positional-only as intended.
* TypeError messages for incorrect arguments in all constructors sha3_*(),
  shake_*() and keccak_*() incorrectly referred to sha3_224.

Also made the following enhancements:

* More accurately specified input and result types for strings, bytes and
  bytes-like objects.
* Unified positional parameter names for update() and constructors.
* Improved formatting.
2018-07-31 09:50:16 +03:00
Serhiy Storchaka 4b8a7f51da
Revert "closes bpo-27494: Fix 2to3 handling of trailing comma after a generator expression (GH-3771)" (#8241)
This reverts commit af810b35b4.

This is not valid syntax (see bpo-32012).
2018-07-31 09:34:30 +03:00
Serhiy Storchaka ac20e0f98d
bpo-1617161: Make the hash and equality of methods not depending on the value of self. (GH-7848)
* The hash of BuiltinMethodType instances no longer depends on the hash
  of __self__. It depends now on the hash of id(__self__).
* The hash and equality of ModuleType and MethodWrapperType instances no
  longer depend on the hash and equality of __self__. They depend now on
  the hash and equality of id(__self__).
* MethodWrapperType instances no longer support ordering.
2018-07-31 09:18:24 +03:00
twisteroid ambassador 9045199c5a bpo-33833: Fix ProactorSocketTransport AssertionError (#7893) 2018-07-30 21:58:50 +03:00
Elvis Pranskevichus 22d25085db bpo-34075: Deprecate non-ThreadPoolExecutor in loop.set_default_executor() (GH-8533)
Various asyncio internals expect that the default executor is a
`ThreadPoolExecutor`, so deprecate passing anything else to
`loop.set_default_executor()`.
2018-07-30 12:42:43 +02:00
Mickaël Schoentgen 3f8c6913b8 bpo-34035: Fix several AttributeError in zipfile seek() methods. (GH-8527) 2018-07-29 21:26:52 +03:00
Franz Wöllert d2e902e4fb bpo-31047: Fix ntpath.abspath for invalid paths (GH-8544) 2018-07-29 13:47:09 +01:00
Berker Peksag 11eb1a9470 bpo-34251: Restore msilib.Win64 to preserve compatibility (GH-8510) 2018-07-28 16:02:56 -05:00
Raymond Hettinger c6dabe37e3
bpo-33089: Multidimensional math.hypot() (GH-8474) 2018-07-28 07:48:04 -07:00
Dong-hee Na 8fe9eed937 bpo-33476: Fix _header_value_parser when address group is missing final ';' (GH-7484) 2018-07-28 13:55:11 +01:00
Berker Peksag 612dbefe9d
bpo-30722: Fix NEWS entries (GH-8501) 2018-07-27 18:49:58 +03:00
Steve Dower 5473f061f5
bpo-34225: Ensure INCLUDE and LIB directories do not end with a backslash. (GH-8464) 2018-07-26 04:23:10 -07:00
Serhiy Storchaka 7cb7bcff20
bpo-20260: Implement non-bitwise unsigned int converters for Argument Clinic. (GH-8434) 2018-07-26 13:22:16 +03:00
Serhiy Storchaka 323748ad74
bpo-34197: Make _csv.Dialect attributes booleans. (GH-8440)
Attributes skipinitialspace, doublequote and strict are now
booleans instead of integers 0 or 1.
2018-07-26 13:21:09 +03:00
Victor Stinner ecf411c59e
bpo-34170: Enhance _PyCoreConfig_Read() (GH-8468)
* Inline cmdline_get_env_flags() into config_read_env_vars():
  _PyCoreConfig_Read() now reads much more environment variables like
  PYTHONVERBOSE.
* Allow to override faulthandler and allocator even if dev_mode=1.
  PYTHONMALLOC is now the priority over PYTHONDEVMODE.
* Fix _PyCoreConfig_Copy(): copy also install_signal_handlers,
  coerce_c_locale and coerce_c_locale_warn
* _PyCoreConfig.install_signal_handlers default is now 1: install
  signals by default
* Fix also a compiler warning: don't define _PyPathConfig type twice.
2018-07-26 02:37:22 +02:00
chason 48ed88a93b bpo-32663 Make SMTPUTF8SimTests run (#5314)
Enable and fix SMTPUTF8SimTests in test_smtplib.

The tests for SMTPUTF8SimTests in test_smtplib.py were not actually
being run because test_smtplib was still using the 'test_main' pattern,
and the class was never added to test_main.

Additionally, one of the tests needed to be moved to the non-UTF8 server
class because it relies on the server not being UTF-8 compatible (and it
had a bug in in).
2018-07-25 15:01:28 -04:00
Victor Stinner 60b04c9f6f
bpo-34228: Allow PYTHONTRACEMALLOC=0 (GH-8467)
PYTHONTRACEMALLOC=0 environment variable and -X tracemalloc=0 command
line option are now allowed to disable explicitly tracemalloc at
startup.
2018-07-25 19:23:53 +02:00
Ammar Askar 96d1e69a12 bpo-29097: Forego fold detection on windows for low timestamp values (GH-2385)
On Windows, passing a negative value to local results in an OSError because localtime_s on Windows does not support negative timestamps. Unfortunately this means that fold detection for timestamps between 0 and max_fold_seconds will result in this OSError since we subtract max_fold_seconds from the timestamp to detect a fold. However, since we know there haven't been any folds in the interval [0, max_fold_seconds) in any timezone, we can hackily just forego fold detection for this time range on Windows.
2018-07-25 12:54:58 -04:00
Berker Peksag 74102c9a5f
bpo-13041: Use shutil.get_terminal_size() in argparse.HelpFormatter (GH-8459) 2018-07-25 18:23:44 +03:00
Serhiy Storchaka ac0b3c2f4d
bpo-34164: Fix handling of incorrect padding in base64.b32decode(). (GH-8351)
Now base64.Error is always raised instead of UnboundLocalError or
OverflowError.
2018-07-24 12:52:51 +03:00
Serhiy Storchaka 02ec92fa7b
bpo-29209: Remove old-deprecated features in ElementTree. (GH-6769)
Also make getchildren() and getiterator() emitting
a DeprecationWarning instead of PendingDeprecationWarning.
2018-07-24 12:03:34 +03:00
Alexander Marshalov e22072fb11 bpo-34149: Behavior of the min/max with key=None (GH-8328)
Improve consistency with the signature for sorted(), heapq.nsmallest(), heapq.nlargest(), and itertools.groupby().
2018-07-23 20:58:21 -07:00
Serhiy Storchaka a6fdddb7df
bpo-32500: Fix error messages for sequence and mapping C API. (GH-7846)
Fix error messages for PySequence_Size(), PySequence_GetItem(),
PySequence_SetItem() and PySequence_DelItem() called with a mapping
and PyMapping_Size() called with a sequence.
2018-07-23 23:43:42 +03:00
Matěj Cepl caa331d492 bpo-33336, imaplib: Legalize MOVE command (GH-6569)
imaplib now allows MOVE command in IMAP4.uid() (RFC 6851:
IMAP MOVE Extension) and potentially as a name of supported
method of IMAP4 object.
2018-07-23 13:28:54 +02:00
Berker Peksag 7a3056fa7d
bpo-21446: Update reload fixer to use importlib (GH-8391) 2018-07-23 09:49:08 +03:00
Berker Peksag d04f46c59f
bpo-940286: Fix pydoc to show cross refs correctly (GH-8390) 2018-07-23 08:37:47 +03:00
Xtreak 1426daa4fe bpo-34127: Fix grammar in error message with respect to argument count (GH-8395) 2018-07-22 13:13:26 -07:00
jdemeyer 56868f940e bpo-34126: Fix crashes while profiling invalid calls. (GH-8300) 2018-07-21 11:30:59 +03:00
Vinay Sajip 94487d4570
bpo-34011: Update code copying DLLs and init.tcl into venvs. (GH-8253) 2018-07-20 17:07:38 +01:00
Victor Stinner fb47bca9ee
bpo-34008: Allow to call Py_Main() after Py_Initialize() (GH-8043)
Py_Main() can again be called after Py_Initialize(), as in Python
3.6. The new configuration is ignored, except of
_PyMainInterpreterConfig.argv which is used to update sys.argv.
2018-07-20 17:34:23 +02:00
ValeriyaSinevich ce75df3031 bpo-30237: Output error when ReadConsole is canceled by CancelSynchronousIo. (GH-7911) 2018-07-19 15:34:03 -07:00
Serhiy Storchaka 5d4cb54800
bpo-34141: Optimized pickling simple non-recursive values. (GH-8318) 2018-07-18 10:10:49 +03:00
Zackery Spytz 28f07364f0 bpo-34068: _io__IOBase_close_impl could call _PyObject_SetAttrId with an exception set (GH-8282) 2018-07-17 09:31:44 +03:00
Antoine Pitrou 961d54c5c1
bpo-32430: Rename Modules/Setup.dist to Modules/Setup (GH-8229)
bpo-32430: Rename Modules/Setup.dist to Modules/Setup

Remove the necessity to copy the former manually to the latter when updating the local source tree.
2018-07-16 19:03:03 +02:00
Serhiy Storchaka bd47384e07
bpo-24618: Add a check in the code constructor. (GH-8283)
Check that the size of the varnames tuple is enough at least for all arguments.
2018-07-16 09:10:19 +03:00
Benjamin Peterson 15c7b2abdf
bpo-34121: Fix detection of C11 atomic support on clang. (GH-8288) 2018-07-15 17:01:42 -07:00
INADA Naoki 16dfca4d82
bpo-34087: Fix buffer overflow in int(s) and similar functions (GH-8274)
`_PyUnicode_TransformDecimalAndSpaceToASCII()` missed trailing NUL char.
It caused buffer overflow in `_Py_string_to_number_with_underscores()`.

This bug is introduced in 9b6c60cb.
2018-07-14 12:06:43 +09:00
Jason R. Coombs cafaf0447b
bpo-34108: Fix double carriage return in 2to3 on Windows (#8271)
* Add test capturing failure.
* Honor newlines as present in the original file.
2018-07-13 11:26:03 -04:00
Serhiy Storchaka 993030aac5 bpo-34080: Fix a memory leak in the compiler. (GH-8222) 2018-07-11 23:17:53 +02:00
Serhiy Storchaka 504373c59b
bpo-23927: Make getargs.c skipitem() skipping 'w*'. (GH-8192) 2018-07-11 17:41:43 +03:00
Eitan Adler b91a3a0d61 bpo-33648: Remove PY_WARN_ON_C_LOCALE (GH-7114)
This code does not appear to be used anywhere in the python code base.
The use was removed in eb81795d7d.
2018-07-11 20:01:27 +09:00
Chih-Hsuan Yen 09b2bece78 bpo-29442: Replace optparse with argparse in setup.py (GH-139) 2018-07-11 17:48:43 +09:00
INADA Naoki 5ac9e6eee5
bpo-33597: Reduce PyGC_Head size (GH-7043) 2018-07-10 17:19:53 +09:00
Dong-hee Na 445f1b35ce bpo-33967: Fix singledispatch raised IndexError when no args (GH-8184) 2018-07-10 16:26:36 +09:00
Serhiy Storchaka 3f4d90d4d7 bpo-34066: Disabled interruption before SETUP_WITH and BEFORE_ASYNC_WITH. (GH-8159)
This will prevent emitting a resource warning when the execution was
interrupted by Ctrl-C between calling open() and entering a 'with' block
in "with open()".
2018-07-09 22:40:14 +10:00
Serhiy Storchaka cf7303ed2a
bpo-33305: Improve SyntaxError for invalid numerical literals. (GH-6517) 2018-07-09 15:09:35 +03:00
Serhiy Storchaka 2a9b8babf0
bpo-26544: Fixed implementation of platform.libc_ver(). (GH-7684) 2018-07-09 11:47:45 +03:00
Serhiy Storchaka 25b804a9c2
bpo-31014: Fix the webbrowser module. (GH-7267)
webbrowser._synthesize() called webbrowser.register() with
outdated signature.

Co-Authored-By: John Still <john@jmsdvl.com>
2018-07-08 10:22:32 +03:00
Sergey Fedoseev 0830858aee bpo-34041: Allow creating deterministic functions in Connection.create_function() (GH-8086) 2018-07-08 10:09:20 +03:00
Dong-hee Na 2800dcf656 bpo-34065: Improve the markup of logging.basicConfig() arguments (GH-8153) 2018-07-07 15:36:40 +03:00
Benjamin Peterson b0274f2cdd
closes bpo-34056: Always return bytes from _HackedGetData.get_data(). (GH-8130)
* Always return bytes from _HackedGetData.get_data().

Ensure the imp.load_source shim always returns bytes by reopening the file in
binary mode if needed. Hash-based pycs have to receive the source code in bytes.

It's tempting to change imp.get_suffixes() to always return 'rb' as a mode, but
that breaks some stdlib tests and likely 3rdparty code, too.
2018-07-06 20:41:06 -07:00
Yury Selivanov 0b75228700
bpo-34042: Fix dict.copy() to maintain correct total refcount (GH-8119) 2018-07-06 12:20:07 -04:00
Victor Stinner c2368cbc83
bpo-34054: multiprocessing uses time.monotonic() (GH-8118)
The multiprocessing module now uses the monotonic clock
time.monotonic() instead of the system clock time.time() to implement
timeouts.
2018-07-06 13:51:52 +02:00
Ammar Askar c4ef4896ea bpo-33899: Make tokenize module mirror end-of-file is end-of-line behavior (GH-7891)
Most of the change involves fixing up the test suite, which previously made
the assumption that there wouldn't be a new line if the input didn't end in
one.

Contributed by Ammar Askar.
2018-07-06 10:19:08 +03:00
INADA Naoki 8d130913cb
bpo-34043: Optimize tarfile uncompress performance (GH-8089)
tarfile._Stream has two buffer for compressed and uncompressed data.
Those buffers are not aligned so unnecessary bytes slicing happens
for every reading chunks.

This commit bypass compressed buffering.

In this benchmark [1], user time become 250ms from 300ms.

[1]: https://bugs.python.org/msg320763
2018-07-06 14:06:00 +09:00
Victor Stinner 483422f57e
bpo-34044: subprocess.Popen copies startupinfo (GH-8090)
subprocess.Popen now copies the startupinfo argument to leave it
unchanged: it will modify the copy, so that the same STARTUPINFO
object can be used multiple times.

Add subprocess.STARTUPINFO.copy() method.
2018-07-05 22:54:17 +02:00
hajoscher 12a08c4760 bpo-34010: Fix tarfile read performance regression (GH-8020)
During buffered read, use a list followed by join instead of extending a bytes object.
This is how it was done before but changed in commit b506dc32c1.
2018-07-04 17:13:18 +09:00
INADA Naoki 3c452404ae
bpo-33418: Add tp_clear for function object (GH-8058)
Without tp_clear, GC can't break cyclic reference.
It will cause memory leak when cyclic reference is
created intentionally.
2018-07-04 11:15:50 +09:00
Zackery Spytz d8cba5d16f bpo-24596: Decref module in PyRun_SimpleFileExFlags() on SystemExit (GH-7918)
PyErr_Print() will not return when the exception is a SystemExit, so
decref the __main__ module object in that case.
2018-07-03 21:47:22 +02:00
Bumsik Kim 3cf1f154ed bpo-34019: Fix wrong arguments for Opera Browser (#8047)
The Opera Browser was using a outdated command line invocation that resulted in an incorrect URL being opened in the browser when requested using the webbrowser module.

* Correct the arguments passed to the Opera Browser when opening a new URL.
2018-07-03 12:30:06 +01:00
Zachary Ware d824ca7f4d bpo-34006: Revert line length limit for Windows help docs (GH-8051)
The line-length limit is not needed because the pages appear in a separate app rather
 than on a browser tab.  It can also interact badly with the DPI setting.
2018-07-02 16:31:42 -04:00
Xtreak 087570af6d bpo-33978: Close existing handlers before logging (re-)configuration. (GH-8008) 2018-07-02 09:57:46 +01:00
Tal Einat 0cdf5f4289
bpo-32568: make select.epoll() and its docs consistent (#7840)
* `flags` is indeed deprecated, but there is a validation on its value for
  backwards compatibility reasons.  This adds mention of this in the docs.
* The docs say that `sizehint` is deprecated and ignored, but it is still
  used when `epoll_create1()` is unavailable. This adds mention of this in
  the docs.
* `sizehint=-1` is acceptable again, and is replaced with `FD_SETSIZE-1`.
  This is needed to have a default value available at the Python level,
  since `FD_SETSIZE` is not exposed to Python. (see: bpo-31938)
* Reject `sizehint=0` since it is invalid to pass on to `epoll_create()`.

The relevant tests have also been updated.
2018-06-30 15:43:23 +03:00
Serhiy Storchaka 5bb5bbfca8
bpo-33974: Fix passing special characters to ttk widgets. (GH-7986)
Fix passing lists and tuples of strings containing special characters
'"', '\\', '{', '}' and '\n' as options to tkinter.ttk widgets.
2018-06-30 09:20:28 +03:00
Zackery Spytz 23db935bcf bpo-25862: Fix assertion failures in io.TextIOWrapper.tell(). (GH-3918) 2018-06-29 13:14:58 +03:00
Terry Jan Reedy 891a1f86d4
bpo-14117: Make minor tweaks to turtledemo (GH-8002)
The 'wikipedia' example is now 'rosette', describing what it draws.
The 'penrose' print output is reduced.  The 'tree' '1024'
output is eliminated.
2018-06-29 01:10:05 -04:00
Yury Selivanov d904c238ca
bpo-27500: Fix static version of getaddrinfo to resolve IPv6 (GH-7993) 2018-06-28 21:59:32 -04:00
Yury Selivanov 41cb0baea9
bpo-33985: Implement ContextVar.name attribute. (GH-7980) 2018-06-28 13:20:29 -04:00
Thomas A Caswell 9b9d58f0d8 bpo-31546: Fix input hook integration (GH-7978) 2018-06-28 09:29:44 -07:00
Ned Deily e76ac9d4ef Forward port rest of NEWS changes as of 3.7.0 2018-06-28 04:18:35 -04:00
INADA Naoki 461a1c4b49
bpo-33842: Remove tarfile.filemode (GH-7661) 2018-06-28 17:10:36 +09:00
Ned Deily aee5df5e16 Forward port 3.7.0 final changes 2018-06-27 18:45:50 -04:00
Zackery Spytz d2cbfffc84 bpo-25007: Add copy protocol support to zlib compressors and decompressors (GH-7940) 2018-06-27 21:04:51 +03:00
twisteroid ambassador 4a8b037d2b bpo-31647: Fix bpo typo in NEWS entry. (GH-7964) 2018-06-27 10:58:08 -04:00
Victor Stinner 2cc9d21fff
bpo-33929: multiprocessing: fix handle leak on race condition (GH-7921)
Fix a race condition in Popen of
multiprocessing.popen_spawn_win32. The child process now duplicates
the read end of pipe instead of "stealing" it.

Previously, the read end of pipe was "stolen" by the child process,
but it leaked a handle if the child process had been terminated
before it could steal the handle from the parent process.
2018-06-27 11:40:24 +02:00
Raymond Hettinger ddf7171911
bpo-24567: Random subnormal.diff (#7954)
Handle subnormal weights for choices()
2018-06-27 01:08:31 -07:00
Terry Jan Reedy 3c8043d8fa
bpo-33975: Avoid small type when running IDLE's htests. (GH-7944)
Import pyshell first in htest to call SetProcessDpiAwareness on Windows
before tkinter.Tk() is called for the htest. Apparently, 'root.destroy()'
undoes a previous 'root = Tk()'. Since IDLE unittests always destroy roots,
a unittest before an htest does not require anything more to work right.
Since part of the purpose of human-viewed tests is to determine that
widgets look right, it is important that they look the same for testing
as when running IDLE.
2018-06-27 00:07:30 -04:00
Benjamin Peterson 4e21100fa7
bpo-33956: update vendored expat to 2.2.5 (GH-7925) 2018-06-26 19:25:45 -07:00
Pablo Galindo 58ed7307ea
bpo-33873: Fix bug in `runtest.py` and add checks for invalid `-R` parameters (GH-7735)
Fix bug in `Lib/test/libregrtest/runtest.py` that makes running tests an extra time than the specified number of runs.

Add check for invalid --huntrleaks/-R parameters.
2018-06-26 15:17:26 +01:00
Dong-hee Na cf67d6a934 bpo-33897: Add a 'force' keyword argument to logging.basicConfig(). (GH-7873) 2018-06-25 05:11:09 +01:00
Zackery Spytz ea737751b1 bpo-33451: Close pyc files before calling PyEval_EvalCode() (GH-7884)
Directly executed pyc files were being kept open longer than necessary.
2018-06-24 13:15:24 +10:00
Dong-hee Na 3d70f7aef6 bpo-33805: Improve error message of dataclasses.replace() (GH-7580) 2018-06-23 10:46:32 -04:00
Victor Stinner 9b7cf75721
bpo-33916: Fix bz2 and lzma init when called twice (GH-7843)
bz2, lzma: When Decompressor.__init__() is called twice, free the old
lock to not leak memory.
2018-06-23 10:35:23 +02:00
Victor Stinner 209abf7469
bpo-33932: Calling Py_Initialize() twice does nothing (GH-7845)
Calling Py_Initialize() twice does nothing, instead of failing with a
fatal error: restore the Python 3.6 behaviour.
2018-06-22 19:14:51 +02:00
Terry Jan Reedy bcd3a1a18d
bpo-33905: Add test for idlelib.stackview.StackBrowser. (GH-7852)
Increases coverage by 44%.
2018-06-21 22:19:56 -04:00
Terry Jan Reedy 33c7420e7d
bpo-33924: Change IDLE mainmenu.menudefs key 'windows' to 'window' (GH-7836)
Every other menudef key is the lowercase version of the
corresponding main menu entry (in this case, 'Window').
2018-06-20 22:49:55 -04:00
Terry Jan Reedy a361e89d5a
bpo-33906: Rename idlelib.windows as window (#7833)
Match Window on the main menu and remove last plural module name.
Change imports, test, and attribute references to match new name.
2018-06-20 21:25:59 -04:00
Terry Jan Reedy 87a927325e
bpo-33917: Fix and document idlelib/idle_test/template.py (GH-7830)
The revised file compiles, runs, and tests OK. idle_test/README.txt
explains how to use it to create new IDLE test files.
2018-06-20 17:08:31 -04:00
Victor Stinner fd8fbce495
bpo-33746: Fix test_unittest.testRegisterResult() in verbose mode (GH-7799)
Only make sure that the result is in unittest.signals._results, don't
check the full content of unittest.signals._results.

support._run_suite() uses TextTestRunner in verbose mode, but
TextTestRunner.run() calls registerResult(result) which made the test
fail with "odd object in result set".

Call also removeResult() to restore unittest.signals._results to
avoid test side effect.
2018-06-20 11:29:33 +02:00
Srinivas Reddy Thatiparthy (శ్రీనివాస్ రెడ్డి తాటిపర్తి) 9bb92235f6 bpo-33904: In IDLE's rstrip, rename class RstripExtension as Rstrip (GH-7811) 2018-06-20 03:42:13 -04:00
Terry Jan Reedy 9af1836664
bpo-33907: Rename an IDLE module and classes. (GH-7810)
Fix-up class name duplication in PR #7807. Combined effect is that
module calltips and its class CallTips are now calltip and Calltip.
In module calltip_w class CallTip is now CalltipWindow.
2018-06-20 02:18:49 -04:00
Terry Jan Reedy 06e2029dfa
bpo-33907: Rename an IDLE module and class. (GH-7807)
Improve consistency and appearance. Module idlelib.calltips is now calltip.
Class idlelib.calltip_w.CallTip is now Calltip.
2018-06-19 23:00:35 -04:00
Victor Stinner 06fe77a84b
bpo-30345: Add -g to LDFLAGS for LTO (GH-7709)
Add -g to LDFLAGS when compiling with LTO to get debug symbols.
2018-06-19 18:24:58 +02:00
Victor Stinner c44d8e5db6
bpo-33901: Better test_dbm_gnu.test_reorganize() fix (GH-7795)
Fix test_dbm_gnu.test_reorganize() on macOS with gdbm 1.15: add a
larger value to make sure that the file size changes.
2018-06-19 17:37:07 +02:00
Giampaolo Rodola c7f02a9659
bpo-33671 / shutil.copyfile: use memoryview() with dynamic size on Windows (#7681)
bpo-33671
* use memoryview() with size == file size on Windows, see https://github.com/python/cpython/pull/7160#discussion_r195405230
* release intermediate (sliced) memoryview immediately
* replace "OSX" occurrences with "macOS"
* add some unittests for copyfileobj()
2018-06-19 08:27:29 -07:00
Marco Strigl 936f03e7fa bpo-33365: print the header values beside the keys (GH-6611)
with debuglevel=1 only the header keys got printed. With
this change the header values get printed as well and the single
header entries get '\n' as a separator.
2018-06-19 16:20:58 +03:00
Victor Stinner 1261bfa83d
bpo-33901: Fix test_dbm_gnu for gdbm 1.15 (GH-7791)
Using gdbm 1.15, creating a database creates a file of 16 MiB. Adding
a small entry and then modifying the small entry doesn't change the
file size. Modify test_dbm_gnu to be less strict: allow that the file
size doesn't change.
2018-06-19 14:19:54 +02:00
INADA Naoki 698865dcbb
bpo-33843: Remove deprecated stuff in cgi module (GH-7662) 2018-06-19 17:28:50 +09:00
ValeriyaSinevich b36b0a3765 bpo-33663: Convert content length to string before putting to header (GH-7754) 2018-06-18 14:17:53 -07:00
Stéphane Wirtel 9d49f85064 bpo-33856: Add "help" to the welcome message of IDLE (GH-7755)
Make it the same as when one runs 'python'.
2018-06-16 17:20:56 -04:00
Carl Meyer b193fa996a bpo-33499: Add PYTHONPYCACHEPREFIX env var for alt bytecode cache location. (GH-6834)
In some development setups it is inconvenient or impossible to write bytecode
caches to the code tree, but the bytecode caches are still useful. The
PYTHONPYCACHEPREFIX environment variable allows specifying an alternate
location for cached bytecode files, within which a directory tree mirroring the code
tree will be created. This cache tree is then used (for both reading and writing)
instead of the local `__pycache__` subdirectory within each source directory.

Exposed at runtime as sys.pycache_prefix (defaulting to None), and can
be set from the CLI as "-X pycache_prefix=path".

Patch by Carl Meyer.
2018-06-16 14:40:56 +10:00
Terry Jan Reedy ee5ef309c7
bpo-33855: Minimally test all IDLE modules. (GH-7689)
Create a template for minimally testing a tkinter-using module by importing it and instantiating its class(es).  Add a test file for all non-startup IDLE modules.  Edit existing files and update coverage.  This is part 1 of 3, covering the 21 autocomplete to help modules and touching 33 idlelib files.
2018-06-15 18:20:55 -04:00
Victor Stinner 6c5a4b3156
bpo-33824, bpo-32030: Fix pymain_read_conf() (GH-7712)
Fix "LC_ALL=C python3.7 -V": reset properly the command line parser
when the encoding changes after reading the Python configuration.

Fix pymain_read_conf(): use memset(0) to reset properly cmdline.
2018-06-16 00:06:28 +02:00
Victor Stinner 2f9cbaa8b2
Revert "bpo-32962: Fix test_gdb failure in debug build with -mcet -fcf-protection -O0 (GH-6754)" (#7723)
This reverts commit 9b7c74ca32.
2018-06-15 22:54:35 +02:00
Andrés Delfino 695118600f bpo-33847: Add '@' operator entry to index (GH-7669) 2018-06-15 15:23:00 -04:00
Marcel Plch 9b7c74ca32 bpo-32962: Fix test_gdb failure in debug build with -mcet -fcf-protection -O0 (#6754)
When Python is built with the intel control-flow protection flags,
-mcet -fcf-protection, gdb is not able to read the stack without
actually jumping inside the function. This means an extra
'next' command is required to make the $pc (program counter)
enter the function and make the stack of the function exposed to gdb.
2018-06-15 17:56:24 +02:00
Wonsup Yoon d134809cd3 bpo-29456: Fix bugs in unicodedata.normalize: u1176, u11a7 and u11c3 (GH-1958)
Hangul composition check boundaries are wrong for the second character
([0x1161, 0x1176) instead of [0x1161, 0x1176]) and third character ((0x11A7, 0x11C3)
instead of [0x11A7, 0x11C3]).
2018-06-15 20:03:14 +08:00
Serhiy Storchaka ceeef10cdb
bpo-33818: PyExceptionClass_Name() will now return "const char *". (GH-7581) 2018-06-15 11:09:43 +03:00
Serhiy Storchaka 08f127a3ca
bpo-33851: Fix ast.get_docstring() for a node that lacks a docstring. (GH-7682) 2018-06-15 11:05:15 +03:00
Eric Snow 9e7c92193c
bpo-17045: Improve C-API doc for PyTypeObject. (gh-7413)
The existing doc had a number of info gaps and was a little hard to use.  This patch provides several quick-reference tables as well as examples.
2018-06-14 15:46:35 -06:00
Victor Stinner d22fc0bc7d
bpo-32962: python-gdb catchs UnicodeDecodeError (GH-7693)
python-gdb now catchs UnicodeDecodeError exceptions when calling
string().
2018-06-14 22:34:52 +02:00
Victor Stinner 019d33b7a4
bpo-32962: python-gdb catchs ValueError on read_var() (GH-7692)
python-gdb now catchs ValueError on read_var(): when Python has no
debug symbols for example.
2018-06-14 16:28:07 +02:00
Giampaolo Rodola 4a172ccc73
bpo-33671: efficient zero-copy for shutil.copy* functions (Linux, OSX and Win) (#7160)
* have shutil.copyfileobj use sendfile() if possible

* refactoring: use ctx manager

* add test with non-regular file obj

* emulate case where file size can't be determined

* reference _copyfileobj_sendfile directly

* add test for offset() at certain position

* add test for empty file

* add test for non regular file dst

* small refactoring

* leave copyfileobj() alone in order to not introduce any incompatibility

* minor refactoring

* remove old test

* update docstring

* update docstring; rename exception class

* detect platforms which only support file to socket zero copy

* don't run test on platforms where file-to-file zero copy is not supported

* use tempfiles

* reset verbosity

* add test for smaller chunks

* add big file size test

* add comment

* update doc

* update whatsnew doc

* update doc

* catch Exception

* remove unused import

* add test case for error on second sendfile() call

* turn docstring into comment

* add one more test

* update comment

* add Misc/NEWS entry

* get rid of COPY_BUFSIZE; it belongs to another PR

* update doc

* expose posix._fcopyfile() for OSX

* merge from linux branch

* merge from linux branch

* expose fcopyfile

* arg clinic for the win implementation

* convert path type to path_t

* expose CopyFileW

* fix windows tests

* release GIL

* minor refactoring

* update doc

* update comment

* update docstrings

* rename functions

* rename test classes

* update doc

* update doc

* update docstrings and comments

* avoid do import nt|posix modules if unnecessary

* set nt|posix modules to None if not available

* micro speedup

* update description

* add doc note

* use better wording in doc

* rename function using 'fastcopy' prefix instead of 'zerocopy'

* use :ref: in rst doc

* change wording in doc

* add test to make sure sendfile() doesn't get called aymore in case it doesn't support file to file copies

* move CopyFileW in _winapi and actually expose CopyFileExW instead

* fix line endings

* add tests for mode bits

* add docstring

* remove test file mode class; let's keep it for later when Istart addressing OSX fcopyfile() specific copies

* update doc to reflect new changes

* update doc

* adjust tests on win

* fix argument clinic error

* update doc

* OSX: expose copyfile(3) instead of fcopyfile(3); also expose flags arg to python

* osx / copyfile: use path_t instead of char

* do not set dst name in the OSError exception in order to remain consistent with platforms which cannot do that (e.g. linux)

* add same file test

* add test for same file

* have osx copyfile() pre-emptively check if src and dst are the same, otherwise it will return immedialtey and src file content gets deleted

* turn PermissionError into appropriate SameFileError

* expose ERROR_SHARING_VIOLATION in order to raise more appropriate SameFileError

* honour follow_symlinks arg when using CopyFileEx

* update Misc/NEWS

* expose CreateDirectoryEx mock

* change C type

* CreateDirectoryExW actual implementation

* provide specific makedirs() implementation for win

* fix typo

* skeleton for SetNamedSecurityInfo

* get security info for src path

* finally set security attrs

* add unit tests

* mimick os.makedirs() behavior and raise if dst dir exists

* set 2 paths for OSError object

* set 2 paths for OSError object

* expand windows test

* in case of exception on os.sendfile() set filename and filename2 exception attributes

* set 2 filenames (src, dst) for OSError in case copyfile() fails on OSX

* update doc

* do not use CreateDirectoryEx() in copytree() if source dir is a symlink (breaks test_copytree_symlink_dir); instead just create a plain dir and remain consistent with POSIX implementation

* use bytearray() and readinto()

* use memoryview() with bytearray()

* refactoring + introduce a new _fastcopy_binfileobj() fun

* remove CopyFileEx and other C wrappers

* remove code related to CopyFileEx

* Recognize binary files in copyfileobj()
...and use fastest _fastcopy_binfileobj() when possible

* set 1MB copy bufsize on win; also add a global _COPY_BUFSIZE variable

* use ctx manager for memoryview()

* update doc

* remove outdated doc

* remove last CopyFileEx remnants

* OSX - use fcopyfile(3) instead of copyfile(3)

...as an extra safety measure: in case src/dst are "exotic" files (non
regular or living on a network fs etc.) we better fail on open() instead
of copyfile(3) as we're not quite sure what's gonna happen in that
case.

* update doc
2018-06-12 23:04:50 +02:00
Cheryl Sabella 33cd058f21 bpo-32108: Don't clear configparser values if key is assigned to itself (GH-7588) 2018-06-12 13:37:51 -07:00
Tal Einat c3f55be7dd
bpo-27397: Make email module properly handle invalid-length base64 strings (#7583)
When attempting to base64-decode a payload of invalid length (1 mod 4),
properly recognize and handle it.  The given data will be returned as-is,
i.e. not decoded, along with a new defect, InvalidBase64LengthDefect.
2018-06-12 15:46:22 +03:00
Christian Heimes ef24b6c54d bpo-31432: Clarify ssl CERT_NONE/OPTIONAL/REQUIRED docs. (GH-3530)
The documentation for CERT_NONE, CERT_OPTIONAL, and CERT_REQUIRED were
misleading and partly wrong. It fails to explain that OpenSSL behaves
differently in client and server mode. Also OpenSSL does validate the
cert chain everytime. With SSL_VERIFY_NONE a validation error is not
fatal in client mode and does not request a client cert in server mode.
Also discourage people from using CERT_OPTIONAL in client mode.
2018-06-11 18:59:45 -04:00
Matthias Bussonnier 46c5cd0f6e bpo-33582: Emit deprecation warning for `formatargspec` (GH-6994) 2018-06-11 16:08:16 -04:00
Terry Jan Reedy 800415e3df
bpo-33656: On Windows, add API call saying that tk scales for DPI (GH-7137)
On Windows 8.1+ or 10, with DPI compatibility properties of the Python binary
unchanged, and a monitor resolution greater than 96 DPI, this should
make text and lines sharper. It should otherwise have no effect.

Using a magnifier, I determined that the improvement comes from horizontal and
lines being better lined up with the monitor pixels. I checked that this call causes
no problem on any Windows buildbot, including the Win7 buildbots. Unlike most
IDLE patches, this one can be easily reverted by users by removing a few lines,
at the top of idlelib/pyshell.py.
2018-06-11 14:14:32 -04:00
Serhiy Storchaka 5cbefa9919
Clean up after bpo-33738. (GH-7627)
* Add declarations even if they are overridden by macros.
* Make the declaration and the definition of PyExceptionClass_Name
  consistent.
2018-06-11 15:01:47 +03:00
Steve Weber 2487f30d55 bpo-30167: Prevent site.main() exception if PYTHONSTARTUP is set. (GH-6731)
Before Python 3.6, os.path.abspath(None) used to report an AttributeError which was properly caught inside site.abs_paths, making it ignore __main__, one of sys.modules, which has __file__ and __cached__ set to None. With 3.6, os.path.abspath(None) raises TypeError instead which site.abs_path was not expecting.  This resulted in an uncaught exception if a user had PYTHONSTARTUP set and the application called site.main() which a number of third-party programs do.
2018-06-10 20:49:34 -04:00
Christian Tismer 8398713cea bpo-33738: Address review comments in GH #7477 (GH-7585) 2018-06-10 18:48:28 -04:00
Alexander Belopolsky 877b23202b
bpo-33812: Corrected astimezone for naive datetimes. (GH-7578)
A datetime object d is aware if d.tzinfo is not None and
d.tzinfo.utcoffset(d) does  not return None. If d.tzinfo is None,
or if d.tzinfo is not None but d.tzinfo.utcoffset(d) returns None,
 d is naive.

This commit ensures that instances with non-None d.tzinfo, but
d.tzinfo.utcoffset(d) returning None are treated as naive.

In addition, C acceleration code will raise TypeError if
d.tzinfo.utcoffset(d) returns an object with the type other than
timedelta.

* Updated the documentation.

Assume that the term "naive" is defined elsewhere and remove the
not entirely correct clarification.  Thanks, Tim.
2018-06-10 17:02:58 -04:00
Tal Einat 1b85c71a21
bpo-33770: improve base64 exception message for encoded inputs of invalid length (#7416) 2018-06-10 10:01:50 +03:00
Serhiy Storchaka 98a0e466cd
Fix spaces added after hyphens in news entries. (GH-7579)
Seems they were added by double applying blurb.
2018-06-10 09:46:50 +03:00
Ned Deily ced0adb263
bpo-32493: Correct test for uuid_enc_be availability in configure.ac. (GH-7511) (GH-7567) 2018-06-09 18:19:57 -04:00
Christian Tismer ea62ce7f4f bpo-33738: Fix macros which contradict PEP 384 (GH-7477)
During development of the limited API support for PySide,
we saw an error in a macro that accessed a type field.

This patch fixes the 7 errors in the Python headers.
Macros which were not written as capitals were implemented
as function.

To do the necessary analysis again, a script was included that
parses all headers and looks for "->tp_" in serctions which can
be reached with active limited API.

It is easily possible to call this script as a test.

Error listing:

../../Include/objimpl.h:243
#define PyObject_IS_GC(o) (PyType_IS_GC(Py_TYPE(o)) && \
    (Py_TYPE(o)->tp_is_gc == NULL || Py_TYPE(o)->tp_is_gc(o)))
Action: commented only

../../Include/objimpl.h:362
#define PyType_SUPPORTS_WEAKREFS(t) ((t)->tp_weaklistoffset > 0)
Action: commented only

../../Include/objimpl.h:364
#define PyObject_GET_WEAKREFS_LISTPTR(o) \
    ((PyObject **) (((char *) (o)) + Py_TYPE(o)->tp_weaklistoffset))
Action: commented only

../../Include/pyerrors.h:143
#define PyExceptionClass_Name(x) \
     ((char *)(((PyTypeObject*)(x))->tp_name))
Action: implemented function

../../Include/abstract.h:593
#define PyIter_Check(obj) \
    ((obj)->ob_type->tp_iternext != NULL && \
     (obj)->ob_type->tp_iternext != &_PyObject_NextNotImplemented)
Action: implemented function

../../Include/abstract.h:713
#define PyIndex_Check(obj)                              \
    ((obj)->ob_type->tp_as_number != NULL &&            \
     (obj)->ob_type->tp_as_number->nb_index != NULL)
Action: implemented function

../../Include/abstract.h:924
#define PySequence_ITEM(o, i)\
    ( Py_TYPE(o)->tp_as_sequence->sq_item(o, i) )
Action: commented only
2018-06-09 14:32:25 -04:00
Nick Coghlan 1bcb8a6368
bpo-33409: Clarify PEP 538/540 relationship (GH-7534)
While locale coercion and UTF-8 mode turned out to
be complementary ideas rather than competing ones,
it isn't immediately obvious why it's useful to
have both, or how they interact at runtime.

This updates both the Python 3.7 What's New doc
and the PYTHONCOERCECLOCALE and PYTHONUTF8
documentation in an attempt to clarify that
relationship:

- in the respective What's New sections, add a closing paragraph
  explaining which problem each one solves, and pointing to the
  other PEP's section for the specific aspects it relies on the other
  PEP to solve
- use "locale-aware mode" as a more descriptive term for the
  default non-UTF-8 mode
- improve wording conistenccy between the PYTHONCOERCECLOCALE
  and PYTHONUTF8 docs when they cover the same thing (mostly
  related to legacy locale detection and setting the standard
  stream error handler)
- improve the description of the locale coercion trigger conditions
  (including pointing out that setting LC_ALL turns off locale coercion)
- port the full description of the UTF-8 mode behaviour changes
  from PEP 540 into the PYTHONUTF8 documentation
- be explicit that PYTHONIOENCODING still overrides the settings
  for the standard streams
- mention concrete examples of things that do and don't get their
  text encoding assumptions adjusted by the two text encoding
  assumption override techniques
2018-06-09 16:54:08 +10:00
Yury Selivanov 12f482e0ae
bpo-30805: Avoid race condition with debug logging (GH-7545)
Supersedes https://github.com/python/cpython/pull/2490
2018-06-08 18:24:37 -04:00
Thomas Kluyver 11a896652e bpo-33375: Get filename for warnings from frame.f_code.co_filename (GH-6622)
More consistent with how other parts of Python find the filename (e.g. tracebacks and pdb).
2018-06-08 12:28:37 -07:00
Dong-hee Na 4f54867e29 bpo-33197: Add versionadded tag to the documentation of ParameterKind (GH-7536) 2018-06-08 12:07:52 -04:00
Elvis Pranskevichus c0d062f523 bpo-33736: Improve the documentation of asyncio stream APIs (GH-7326) 2018-06-08 11:36:00 -04:00
wim glenn 66f02aa32f bpo-11874: fix assertion failure in argparse metavar handling (GH-1826)
- bugfix and test for fragile metavar handling in argparse (see
  bpo-24089, bpo-14046, bpo-25058, bpo-11874)
- also fixes some incorrect tests that did not make 1-element tuples correctly
2018-06-08 20:12:49 +10:00
Cheryl Sabella 041272b657 bpo-33768: IDLE: Clicking on code context line moves it to top of editor (GH-7411) 2018-06-08 01:21:15 -04:00
Dong-hee Na 4aa3006619 bpo-33197: Add description property for _ParameterKind. (GH-7206) 2018-06-07 23:46:31 -04:00
Yury Selivanov 8f4042964d
bpo-33792: Add selector and proactor windows policies (GH-7487) 2018-06-07 20:44:57 -04:00
Yury Selivanov 52698c7ad9
bpo-33786: Fix asynchronous generators to handle GeneratorExit in athrow() (GH-7467) 2018-06-07 20:31:26 -04:00
Yury Selivanov 378c53cc31
bpo-33803: Fix a crash in hamt.c (#7504) 2018-06-07 20:29:55 -04:00
Terry Jan Reedy bed523ba03
bpo-33642 and bpo-33679: Revise and condense blurbs (GH-7500) 2018-06-07 19:11:41 -04:00
Victor Stinner 79790bc35f
bpo-33694: Fix race condition in asyncio proactor (GH-7498)
The cancellation of an overlapped WSARecv() has a race condition
which causes data loss because of the current implementation of
proactor in asyncio.

No longer cancel overlapped WSARecv() in _ProactorReadPipeTransport
to work around the race condition.

Remove the optimized recv_into() implementation to get simple
implementation of pause_reading() using the single _pending_data
attribute.

Move _feed_data_to_bufferred_proto() to protocols.py.

Remove set_protocol() method which became useless.
2018-06-08 00:25:52 +02:00
Scott Sanderson cebe80b59b bpo-29235: Update document for Profiler's context manager (GH-7331) 2018-06-07 18:46:42 +09:00
arikrupnik 5bfa058e65 bpo-33274: Compliance with DOM L1: return removed attribute (#7465)
* bpo-33274: Compliance with DOM L1: return removed attribute

* Update 2018-06-06-22-01-33.bpo-33274.teYqv8.rst
2018-06-07 00:42:38 -04:00
Benjamin Peterson 7c69c1c0fb
update to Unicode 11.0.0 (closes bpo-33778) (GH-7439)
Also, standardize indentation of generated tables.
2018-06-06 20:14:28 -07:00
John Reese 3a5b0d8988 bpo-33504: Migrate configparser from OrderedDict to dict. (#6819)
With 3.7+, dictionary are ordered by design.  Configparser still uses
collections.OrderedDict, which is unnecessary.  This updates the module
to use the standard dict implementation by default, and changes the
docs and tests to match.
2018-06-05 16:31:33 -07:00
Thomas Kluyver c56b17bd8c bpo-12486: Document tokenize.generate_tokens() as public API (#6957)
* Document tokenize.generate_tokens()

* Add news file

* Add test for generate_tokens

* Document behaviour around ENCODING token

* Add generate_tokens to __all__
2018-06-05 10:26:39 -07:00
Vinay Sajip dde9fdbe45
bpo-33165: Added stacklevel parameter to logging APIs. (GH-7424) 2018-06-05 17:24:18 +01:00
Yury Selivanov 415bc46a78
bpo-33769: start_tls: Fix error message; cancel callbacks on error (GH-7403)
In addition to that, mark SSLTransport as "closed" in its "abort()" method to prevent bogus warnings.
2018-06-05 08:59:58 -04:00
Zackery Spytz e9e3976057 bpo-33767: Fix improper use of SystemError by mmap.mmap objects (GH-7381)
Raise TypeError instead of SystemError for unsupported operations.
2018-06-05 15:59:41 +03:00
INADA Naoki e336484847
bpo-5755: Move -Wstrict-prototypes to CFLAGS_NODIST (GH-7395) 2018-06-05 20:40:53 +09:00
Steve Dower 2a4a62ba4a
bpo-33720: Reduces maximum marshal recursion depth on release builds. (GH-7401) 2018-06-04 13:25:00 -07:00
Cheryl Sabella b609e687a0 bpo-33763: IDLE: Replace label widget with text widget in code context (GH-7367) 2018-06-04 11:58:44 -04:00
Cheryl Sabella d49dbd9acc bpo-33664: Scroll IDLE editor text by lines (GH-7351)
Previously, the mouse wheel and scrollbar slider moved text by a fixed
number of pixels, resulting in partial lines at the top of the editor
box. The change also applies to the shell and grep output windows,
but not to read-only text views.
2018-06-04 11:48:21 -04:00
Yury Selivanov 9602643120
bpo-33734: asyncio/ssl: a bunch of bugfixes (#7321)
* Fix AttributeError (not all SSL exceptions have 'errno' attribute)

* Increase default handshake timeout from 10 to 60 seconds
* Make sure start_tls can be cancelled correctly
* Make sure any error in SSLProtocol gets propagated (instead of just being logged)
2018-06-04 11:32:35 -04:00
A. Jesse Jiryu Davis a8eb58546b bpo-31849: Fix warning in pyhash.c (GH-6799) 2018-06-04 19:57:08 +09:00
Brett Cannon 8425de4147
bpo-33562: Check the global asyncio event loop policy isn't set after any tests (GH-7328) 2018-06-01 20:34:09 -07:00
Cheryl Sabella de6516264e bpo-33679: IDLE: Re-enable color configuration for code context (GH-7199)
The difference from before is that the settings are now on the
Highlights tab instead of the Extensions tab and only change one theme
at a time instead of all themes. The default for light themes is black
on light gray, as before. The default for the IDLE Dark theme is white
on dark gray, which better fits the dark theme.

When one starts IDLE from a console and loads a custom theme without
definitions for 'context', one will see a warning message on the console.
To stop the warning, go to Options => Configure IDLE => Highlights,
select the custom theme if not selected already, select 'Code Context',
and select foreground and background colors.
2018-06-01 21:45:54 -04:00
Cheryl Sabella 29996a1c4e bpo-33642: IDLE: Use variable number of lines in CodeContext. (GH-7106)
Instead of displaying a fixed number of lines, some blank, Code Context
now displays the variable number of actual context lines.  When there
are no context lines, it shows a single blank line to indicate that the
feature is turned on.

The Code Context configuration option is changed from 'numlines'
(default 3) to 'maxlines' (default 15) to avoid possible interference
between user settings for the old and new versions of Code Context.
2018-06-01 19:23:00 -04:00
Scott Sanderson 2e01b75884 bpo-29235: Make cProfile.Profile a context manager (GH-6808) 2018-06-01 13:36:23 -07:00
pkerling e905c84494 bpo-30654: Do not reset SIGINT handler to SIG_DFL in finisignal (GH-7146) 2018-06-01 11:47:18 +02:00
Victor Stinner 58d1683255
bpo-33706: Fix pymain_parse_cmdline_impl() (GH-7283)
Fix a crash in Python initialization when parsing the command line
options.

Fix memcpy() size parameter: previously, we read one wchar_t after
the end of _PyOS_optarg. Moreover, don't copy the trailingg NUL
character: we write it manually anyway.

Thanks Christoph Gohlke for the bug report and the fix!
2018-05-31 15:09:28 +02:00
Ned Deily 126869f308
Fix "ditribution" typo in NEWS entry. (GH-7274) 2018-05-31 01:18:49 -04:00
jhaydaman 0a28c0d12e bpo-33238: Add InvalidStateError to concurrent.futures. (GH-7056)
Future.set_result and Future.set_exception now raise InvalidStateError
if the futures are not pending or running. This mirrors the behavior
of asyncio.Future, and prevents AssertionErrors in asyncio.wrap_future
when set_result is called multiple times.
2018-05-30 10:15:06 +03:00
Yury Selivanov 99279ad823
Revert "bpo-22087: Fix Policy.get_event_loop() to detect fork (GH-7208)" (#7232)
This reverts commit 5d97b7bcc1.
2018-05-29 20:47:56 -04:00
Elvis Pranskevichus f9aeca2085 bpo-23859: Document that asyncio.wait() does not cancel its futures (#7217)
Unlike `asyncio.wait_for()`, `asyncio.wait()` does not cancel the passed
futures when a timeout accurs.
2018-05-29 18:21:44 -04:00
Elvis Pranskevichus e2b340ab41 bpo-32751: Wait for task cancellation in asyncio.wait_for() (GH-7216)
Currently, asyncio.wait_for(fut), upon reaching the timeout deadline,
cancels the future and returns immediately.  This is problematic for
when *fut* is a Task, because it will be left running for an arbitrary
amount of time.  This behavior is iself surprising and may lead to
related bugs such as the one described in bpo-33638:

    condition = asyncio.Condition()
    async with condition:
        await asyncio.wait_for(condition.wait(), timeout=0.5)

Currently, instead of raising a TimeoutError, the above code will fail
with `RuntimeError: cannot wait on un-acquired lock`, because
`__aexit__` is reached _before_ `condition.wait()` finishes its
cancellation and re-acquires the condition lock.

To resolve this, make `wait_for` await for the task cancellation.
The tradeoff here is that the `timeout` promise may be broken if the
task decides to handle its cancellation in a slow way.  This represents
a behavior change and should probably not be back-patched to 3.6 and
earlier.
2018-05-29 17:31:01 -04:00
Yury Selivanov 863b674909
bpo-32684: Fix gather to propagate cancel of itself with return_exceptions (GH-7209) 2018-05-29 17:20:02 -04:00
Géry Ogam 1cee216cf3 bpo-31639: Change ThreadedHTTPServer to ThreadingHTTPServer class name (GH-7195) 2018-05-29 22:10:30 +02:00
Yury Selivanov 5d97b7bcc1
bpo-22087: Fix Policy.get_event_loop() to detect fork (GH-7208)
Original patch by Dan O'Reilly.
2018-05-29 15:38:07 -04:00
Dong-hee Na a9cab433bb bpo-33197: Update a error message of invalid inspect.Parameters. (GH-6636) 2018-05-29 11:04:08 -04:00
Serhiy Storchaka 73cbe7a01a
bpo-32911: Revert bpo-29463. (GH-7121) (GH-7197)
Remove the docstring attribute of AST types and restore docstring
expression as a first stmt in their body.

Co-authored-by: INADA Naoki <methane@users.noreply.github.com>
2018-05-29 12:04:55 +03:00
Yury Selivanov 2179022d94 bpo-33654: Support protocol type switching in SSLTransport.set_protocol() (#7194) 2018-05-29 12:02:40 +03:00
Yury Selivanov f295587c45
bpo-33674: Pause the transport as early as possible (#7192) 2018-05-29 01:00:12 -04:00
Victor Stinner be00a5583a
bpo-33674: asyncio: Fix SSLProtocol race (GH-7175)
Fix a race condition in SSLProtocol.connection_made() of
asyncio.sslproto: start immediately the handshake instead of using
call_soon(). Previously, data_received() could be called before the
handshake started, causing the handshake to hang or fail.
2018-05-29 01:33:35 +02:00
Yury Selivanov 7165754b6b
bpo-32410: Avoid blocking on file IO in sendfile fallback code (GH-7172) 2018-05-28 18:31:55 -04:00
Yury Selivanov 416c1ebd98
bpo-32610: Fix asyncio.all_tasks() to return only pending tasks. (GH-7174) 2018-05-28 17:54:02 -04:00
Yury Selivanov fdccfe09f0
bpo-33469: RuntimeError after closing loop that used run_in_executor (GH-7171) 2018-05-28 17:10:20 -04:00
Yury Selivanov 989b9e0e6d
bpo-33672: Fix Task.__repr__ crash with Cython's bogus coroutines (GH-7161) 2018-05-28 16:27:34 -04:00
Steve Dower e97ba4c690
bpo-33614: Ensures module definition files for the stable ABI on Windows are correctly regenerated. (GH-7165) 2018-05-28 12:32:05 -07:00
Yury Selivanov dbf102271f
bpo-33654: Support BufferedProtocol in set_protocol() and start_tls() (GH-7130)
In this commit:

* Support BufferedProtocol in set_protocol() and start_tls()
* Fix proactor to cancel readers reliably
* Update tests to be compatible with OpenSSL 1.1.1
* Clarify BufferedProtocol docs
* Bump TLS tests timeouts to 60 seconds; eliminate possible race from start_serving
* Rewrite test_start_tls_server_1
2018-05-28 14:31:28 -04:00
jimmylai e549c4be5f bpo-33505: Optimize asyncio.ensure_future by reordering if conditions (GH-6836) 2018-05-28 12:42:05 -04:00
twisteroid ambassador 23f587e395 bpo-31647: Fix write_eof() after close() for SelectorSocketTransport (GH-7149)
Fixed bug where calling write_eof() on a _SelectorSocketTransport after
it's already closed raises AttributeError.
2018-05-28 11:16:45 -04:00
Yury Selivanov 35230d08e0
bpo-33623: Fix possible SIGSGV when asyncio.Future is created in __del__ (#7080) 2018-05-28 11:11:31 -04:00
Ethan Smith c651275afe bpo-32380: Create functools.singledispatchmethod (#6306) 2018-05-26 13:38:33 -07:00
Ned Deily 09c4a7dee2
bpo-33655: Also ignore test_posix_fallocate failures on BSD platforms (GH-7134)
The failure may be due to the use oF ZFS, a case we already ignore
for Solaris-based systems where ZFS is frequently used.
2018-05-26 16:30:46 -04:00
Serhiy Storchaka 09f3221fbb bpo-33652: Improve pickle support in the typing module. (GH-7123)
Pickles of type variables and subscripted generics are now future-proof
and compatible with older Python versions.
2018-05-26 11:19:26 -07:00
Serhiy Storchaka 17d8830312 bpo-32493: Fix uuid.uuid1() on FreeBSD. (GH-7099)
Use uuid_enc_be() if available to encode UUID to bytes as big endian.
2018-05-25 00:45:09 +02:00
Serhiy Storchaka 301e3cc8a5
bpo-33622: Fix issues with handling errors in the GC. (GH-7078)
* Fixed a leak when the GC fails to add an object with __del__ into
  the gc.garbage list.
* PyGC_Collect() can now be called when an exception is set and
  preserves it.
* Fixed an undefined behavior with comparing a dead pointer with NULL.
2018-05-24 15:19:29 +03:00
Cheryl Sabella 8506016f90 bpo-33628: IDLE: Minor code cleanup of codecontext.py and its tests (GH-7085) 2018-05-23 22:18:15 -04:00
Ned Deily 8ebf5ceb0f
bpo-33109: argparse subparsers are once again not required by default (GH-6919)
bpo-26510 in 3.7.0a2 changed the behavior of argparse to make
subparsers required by default, returning to the behavior of 2.7
and 3.2. The behavior was changed in 3.3 to be no longer required.
While it might make more sense to have the default to required,
compatibility with 3.3 through 3.6 is probably less disruptive
than trying to reintroduce compatibility with 2.7 at this point.
This change restores the 3.6 behavior.
2018-05-23 21:55:15 -04:00
Victor Stinner 453bd0bc65
bpo-33540: Add block_on_close attr to socketserver (GH-6911)
Add a new block_on_close class attribute to ForkingMixIn and
ThreadingMixIn classes of socketserver to opt-in for pre-3.7 behaviour.
2018-05-24 03:14:44 +02:00
Christian Heimes 529525fb5a
bpo-33618: Enable TLS 1.3 in tests (GH-7079)
TLS 1.3 behaves slightly different than TLS 1.2. Session tickets and TLS
client cert auth are now handled after the initialy handshake. Tests now
either send/recv data to trigger session and client certs. Or tests
ignore ConnectionResetError / BrokenPipeError on the server side to
handle clients that force-close the socket fd.

To test TLS 1.3, OpenSSL 1.1.1-pre7-dev (git master + OpenSSL PR
https://github.com/openssl/openssl/pull/6340) is required.

Signed-off-by: Christian Heimes <christian@python.org>
2018-05-23 22:24:45 +02:00
Yury Selivanov 28b9178023
bpo-32436: Document PEP 567 changes to asyncio. (GH-7073) 2018-05-23 13:35:04 -04:00
sth 825aab95fd bpo-27300: Add the errors parameter to tempfile classes. (GH-6696) 2018-05-23 08:07:01 +03:00
Matthias Bussonnier 8bb0b5b03c bpo-33604: Remove Pending from hmac Deprecation warning. (GH-7062)
bpo-33604: Bump removal notice from 3.6 to 3.8 and change PendingDeprecationWarning to DeprecationWarning as we had intended to do earlier...
2018-05-22 15:55:31 -07:00
Christian Heimes e8eb6cb792
bpo-33570: TLS 1.3 ciphers for OpenSSL 1.1.1 (GH-6976)
Change TLS 1.3 cipher suite settings for compatibility with OpenSSL
1.1.1-pre6 and newer. OpenSSL 1.1.1 will have TLS 1.3 cipers enabled by
default.

Also update multissltests and Travis config to test with latest OpenSSL.

Signed-off-by: Christian Heimes <christian@python.org>
2018-05-22 22:50:12 +02:00
John Reese 6c4fab0f4b bpo-33516: Add support for __round__ in MagicMock (GH-6880)
unittest.mock.MagicMock now supports the __round__() magic method.
2018-05-22 22:01:10 +02:00
Serhiy Storchaka ae00fb1d4f
bpo-30877: Fix clearing a cache in the the JSON decoder. (GH-7048) 2018-05-22 14:55:07 +03:00
Vlad Starostin a84d0b361a bpo-33263: Fix FD leak in _SelectorSocketTransport (GH-6450)
* bpo-33263 Fix FD leak in _SelectorSocketTransport. (GH-6450)

Under particular circumstances _SelectorSocketTransport can try to add a reader
even the transport is already being closed. This can lead to FD leak and
invalid stated of the following connections. Fixed the SelectorSocketTransport
to add the reader only if the trasport is still active.
2018-05-21 11:13:45 +03:00
Aaron Hall, MBA 4054b172ab bpo-26103: Fix inspect.isdatadescriptor() and a data descriptor definition. (GH-1959)
Look for '__set__' or '__delete__'.
2018-05-21 02:46:42 +03:00
Serhiy Storchaka aef639f626
Fix line breaks added after hyphens by blurb. (GH-7002)
Also remove bullet asterisks from IDLE entries.
2018-05-21 02:36:05 +03:00
Mario Corchero 89a25ce4fd bpo-33541: Remove unused __pad function (GH-4377)
Function was added with the initial implementation 00efe7e798.
2018-05-20 13:42:30 -04:00
CtrlZvi c66c342cb4 bpo-33542: Ignore DUID in uuid.get_node on Windows. (GH-6922)
uuid._ipconfig_getnode did not validate the maximum length of the value,
so long as the value had the same type of formatting as a MAC address.
This let it select DUIDs as MAC addresses. It now requires an exact
length match.
2018-05-20 18:03:25 +03:00
CtrlZvi 4151061855 bpo-26819: Prevent proactor double read on resume (#6921)
The proactor event loop has a race condition when reading with
pausing/resuming. `resume_reading()` unconditionally schedules the read
function to read from the current future. If `resume_reading()` was
called before the previously scheduled done callback fires, this results
in two attempts to get the data from the most recent read and an
assertion failure. This commit tracks whether or not `resume_reading`
needs to reschedule the callback to restart the loop, preventing a
second attempt to read the data.
2018-05-20 13:21:10 +03:00
Serhiy Storchaka f5e7b1999f
bpo-23722: Raise a RuntimeError for absent __classcell__. (GH-6931)
A DeprecationWarning was emitted in Python 3.6-3.7.
2018-05-20 08:48:12 +03:00
Cheryl Sabella 654038d896 bpo-32831: IDLE: Add docstrings and tests for codecontext (GH-5638) 2018-05-19 15:34:03 -04:00
Ivan Levkivskyi f65e31fee3
bpo-28556: Don't simplify unions at runtime (GH-6841) 2018-05-18 16:00:38 -07:00
Terry Jan Reedy d89ca94847
bpo-33564: Add async to IDLE's code context block openers. (GH-6960) 2018-05-17 20:38:41 -04:00
Zvezdan Petkovic c2f082e9d1 bpo-13631: Fix the order of initialization for readline libedit on macOS. (GH-6915)
The editline emulation needs to be initialized *after* the name is
defined. This fixes the long open issue.
2018-05-17 02:45:10 -04:00
Serhiy Storchaka 64fddc423f bpo-33475: Fix and improve converting annotations to strings. (GH-6774) 2018-05-16 23:17:48 -04:00
Steve Dower e5f41d2f1e
bpo-33522: Enable CI builds on Visual Studio Team Services (#6865) 2018-05-16 17:50:29 -04:00
Nina Zakharenko 2d2d3b170b Fix ClassVar as string fails when getting type hints (GH-6824) 2018-05-16 12:27:03 -04:00
Petr Viktorin 8b94b41ab7
bpo-28167: Remove platform.linux_distribution (GH-6871)
* test_ssl: Remove skip_if_broken_ubuntu_ssl

We no longer support OpenSSL 0.9.8.15.15.

* bpo-28167: Remove platform.linux_distribution
2018-05-16 11:51:18 -04:00
Eric V. Smith 4e81296b18
bpo-33536: Validate make_dataclass() field names. (GH-6906) 2018-05-16 11:31:29 -04:00
Christopher Beacham 5db5c0669e bpo-21475: Support the Sitemap extension in robotparser (GH-6883) 2018-05-16 10:52:07 -04:00
Eitan Adler 3055c947f9 closes bpo-33512: use standard for detecting long double (GH-6847) 2018-05-15 22:58:09 -07:00
Eric V. Smith 2a7bacbd91
bpo-33453: Handle string type annotations in dataclasses. (GH-6768) 2018-05-15 22:44:27 -04:00
Christian Heimes 67c4801663 bpo-32257: Add ssl.OP_NO_RENEGOTIATION (GH-5904)
The ssl module now contains OP_NO_RENEGOTIATION constant, available with
OpenSSL 1.1.0h or 1.1.1.

Note, OpenSSL 1.1.0h hasn't been released yet.

Signed-off-by: Christian Heimes <christian@python.org>
2018-05-15 16:25:40 -04:00
Stéphane Wirtel 19177fbd5d bpo-33503: Fix the broken pypi link in the source and the documentation (GH-6814) 2018-05-15 14:58:35 -04:00
Victor Stinner b056562860
bpo-33509: Fix _warnings for module_globals=None (#6833)
Don't crash on warnings.warn_explicit() if module_globals is not a dict.
2018-05-15 20:42:12 +02:00
Matthias Bussonnier 9eb40bc38d bpo-28167: bump platform.linux_distribution removal to 3.8 (GH-6669)
Also bump PendingDeprecationWarning to DeprecationWarning.
2018-05-15 11:17:00 -04:00