Commit Graph

830 Commits

Author SHA1 Message Date
Srinivas Thatiparthy (శ్రీనివాస్ తాటిపర్తి) dd13c88b53 bpo-33947: dataclasses no longer can raise RecursionError in repr (GF9916)
The reprlib code was copied here instead of importing reprlib. I'm not sure if we really need to avoid the import, but since I expect dataclasses to be more common that reprlib, it seems wise. Plus, the code is small.
2018-10-19 12:54:50 -04:00
matthewbelisle-wf 209144831b bpo-34866: Adding max_num_fields to cgi.FieldStorage (GH-9660)
Adding `max_num_fields` to `cgi.FieldStorage` to make DOS attacks harder by
limiting the number of `MiniFieldStorage` objects created by `FieldStorage`.
2018-10-19 03:52:59 -07:00
Cheryl Sabella d16f012f84 bpo-31522: mailbox.get_string: pass `from_` parameter to `get_bytes` (#9857)
This allows *from_* to be successfully set to a non-default value when calling mbox.get_string.
2018-10-18 20:21:47 -04:00
Juliette Monsel 1deea5e539 bpo-34936: Fix TclError in tkinter.Spinbox.selection_element(). (GH-9760) 2018-10-18 22:28:31 +03:00
Serhiy Storchaka 6f906b3d72
bpo-35008: Fix possible leaks in Element.__setstate__(). (GH-9924)
C implementation of xml.etree.ElementTree.Element.__setstate__()
leaked references to children when called for already initialized
element.
2018-10-18 09:49:54 +03:00
Stéphane Wirtel fcd5e84a51 bpo-23420: Verify the value of '-s' when execute the CLI of cProfile (GH-9925)
Verify the value for the parameter '-s' of the cProfile CLI. Patch by Robert
Kuska.

Co-authored-by: Robert Kuska <rkuska@gmail.com>
2018-10-17 12:03:40 +02:00
BNMetrics 18fb1fb943 bpo-34844: logging.Formatter enhancement - Ensure style and format string matches in logging.Formatter (GH-9703) 2018-10-15 19:41:36 +01:00
Pablo Galindo bd036d3d15
bpo-34521: Add NEWS entry for changes in GH-9613 (GH-9850)
* Add News entry for the change in multiprocessing.reduction.recvfds
made in GH-9613.
2018-10-14 17:41:11 +01:00
Serhiy Storchaka b11c5667f9
bpo-34941: Fix searching Element subclasses. (GH-9766)
Methods find(), findtext() and findall() of xml.etree.ElementTree.Element
were not able to find chldren which are instances of Element subclasses.
2018-10-14 10:32:19 +03:00
Andrew Svetlov 97cf082872 bpo-34970: Protect tasks weak set manipulation in asyncio.all_tasks() (GH-9837)
https://bugs.python.org/issue34970
2018-10-13 11:12:40 -07:00
Zackery Spytz d4d60134b2 bpo-16965: 2to3 now rewrites execfile() to open with 'rb'. (GH-8569) 2018-10-13 12:27:30 +03:00
Zackery Spytz 0461704060 bpo-22872: multiprocessing.Queue's put() and get() now raise ValueError if the queue is closed. (GH-9010)
Previously, put() and get() would raise AssertionError and OSError,
respectively.
2018-10-13 12:26:09 +03:00
Juliette Monsel bf034715db bpo-23831: Add moveto method to the tkinter.Canvas widget. (GH-9768) 2018-10-12 19:44:10 +03:00
Bruno Oliveira da2bf9f66d bpo-34900: Make TestCase.debug() work with subtests (GH-9707) 2018-10-12 13:35:55 +03:00
Serhiy Storchaka 9b8c2e7676
bpo-34922: Fix integer overflow in the digest() and hexdigest() methods (GH-9751)
for the SHAKE algorithm in the hashlib module.
2018-10-11 07:41:00 +03:00
Elvis Pranskevichus a6b3ec5b6d bpo-34022: Stop forcing of hash-based invalidation with SOURCE_DATE_EPOCH (GH-9607)
Unconditional forcing of ``CHECKED_HASH`` invalidation was introduced in
3.7.0 in bpo-29708.  The change is bad, as it unconditionally overrides
*invalidation_mode*, even if it was passed as an explicit argument to
``py_compile.compile()`` or ``compileall``.  An environment variable
should *never* override an explicit argument to a library function.
That change leads to multiple test failures if the ``SOURCE_DATE_EPOCH``
environment variable is set.

This changes ``py_compile.compile()`` to only look at
``SOURCE_DATE_EPOCH`` if no explicit *invalidation_mode* was specified.
I also made various relevant tests run with explicit control over the
value of ``SOURCE_DATE_EPOCH``.

While looking at this, I noticed that ``zipimport`` does not work
with hash-based .pycs _at all_, though I left the fixes for
subsequent commits.
2018-10-10 18:43:14 +02:00
Mayank Asthana 7e18deef65 bpo-34926: Make mimetypes.guess_type accept os.PathLike objects (GH-9777)
:meth:`mimetypes.MimeTypes.guess_type` now accepts :term:`path-like object` in addition to url strings.
2018-10-10 16:46:44 +02:00
twisteroid ambassador c880ffe7d2 bpo-34769: Thread safety for _asyncgen_finalizer_hook(). (GH-9716) 2018-10-09 11:30:21 -04:00
Romuald Brunet 7b31397180 bpo-32680 add default "sock" on SMTP objects (#5345)
By default the smtplib.SMTP objects did not have a sock attribute, it
was only created during connect()
2018-10-09 16:31:55 +02:00
travisoneill 199a280af5 bpo-34758: add .wasm to recognized file extensions in mimetypes module (GH-9464) 2018-10-09 07:43:58 +03:00
Bradley Meck 0854b92cd2 bpo-31715 Add mimetype for extension .mjs (#3908) 2018-10-08 23:04:55 +03:00
Paul Bailey 4c33997057 bpo-34911: Added support for secure websocket cookies (GH-9734) 2018-10-08 21:49:29 +03:00
Juliette Monsel af5658ae93 bpo-34829: Add missing selection_ methods to the Tkinter Spinbox. (GH-9617)
Implement the methods selection_from(), selection_range(), selection_present()
and selection_to() for Tkinter Spinbox.
2018-10-08 19:29:24 +03:00
Raymond Hettinger de2e448414
bpo-34925: Optimize common case for bisect() argument parsing (#9753) 2018-10-08 08:02:41 -07:00
Stéphane Wirtel 683281f536 bpo-34906: Doc: Fix typos (2) (GH-9735)
Fix typos
2018-10-06 16:35:53 +02:00
E. M. Bray 7a7693e9cb bpo-28441: Ensure `.exe` suffix in `sys.executable` on MinGW and Cygwin (GH-4348)
This is needed to even the run the test suite on buildbots for affected platforms; e.g.:

```
./python.exe  ./Tools/scripts/run_tests.py -j 1 -u all -W --slowest --fail-env-changed --timeout=11700 -j2
/home/embray/src/python/test-worker/3.x.test-worker/build/python -u -W default -bb -E -W error::BytesWarning -m test -r -w -j 1 -u all -W --slowest --fail-env-changed --timeout=11700 -j2
Traceback (most recent call last):
  File "./Tools/scripts/run_tests.py", line 56, in <module>
    main(sys.argv[1:])
  File "./Tools/scripts/run_tests.py", line 52, in main
    os.execv(sys.executable, args)
PermissionError: [Errno 13] Permission denied
make: *** [Makefile:1073: buildbottest] Error 1
```
2018-10-05 20:38:50 +09:00
INADA Naoki 6f85b826b5 bpo-34871: inspect: Don't pollute sys.modules (GH-9696)
https://bugs.python.org/issue34871
2018-10-04 09:47:09 -07:00
Elvis Pranskevichus 0c797a6aca bpo-34872: Fix self-cancellation in C implementation of asyncio.Task (GH-9679)
The C implementation of asyncio.Task currently fails to perform the
cancellation cleanup correctly in the following scenario.

    async def task1():
        async def task2():
            await task3     # task3 is never cancelled

        asyncio.current_task().cancel()
        await asyncio.create_task(task2())

The actuall error is a hardcoded call to `future_cancel()` instead of
calling the `cancel()` method of a future-like object.

Thanks to Vladimir Matveev for noticing the code discrepancy and to
Yury Selivanov for coming up with a pathological scenario.
2018-10-03 10:30:31 -04:00
tzickel 97bfe8d3eb bpo-34172: multiprocessing.Pool leaks resources after being deleted (GH-8450)
Fix a reference issue inside multiprocessing.Pool that caused the pool to remain alive if it was deleted without being closed or terminated explicitly.
2018-10-02 23:01:23 +02:00
Andrew Svetlov d5bd036138 bpo-34849: Don't log wating for selector.select in asyncio loop iteration (GH-9641)
The waiting is pretty normal for any asyncio program, logging its time just adds
a noise to logs without any useful information provided.



https://bugs.python.org/issue34849
2018-09-29 22:28:40 -07:00
Zsolt Cserna 9df346bf98 bpo-34248: Add filename to error raised in {gnu,ndbm}.open() (GH-8590)
Report the filename to the exception when raising {gdbm,dbm.ndbm}.error in
dbm.gnu.open() and dbm.ndbm.open() functions, so it gets printed when the
exception is raised, and can also be obtained by the filename attribute of the
exception object.
2018-09-27 22:54:34 +03:00
Serhiy Storchaka 3f22811fef
bpo-32892: Use ast.Constant instead of specific constant AST types. (GH-9445) 2018-09-27 17:42:37 +03:00
orlnub123 a94ee12c26 bpo-34819: Use a monotonic clock to compute timeouts in concurrent.futures (GH-9599)
Use a monotonic clock to compute timeouts in :meth:`Executor.map` and :func:`as_completed`, in order to prevent timeouts from deviating when the system clock is adjusted.

This may not be sufficient on all systems. On POSIX for example, the actual waiting (e.g. in ``sem_timedwait``) is specified to rely on the CLOCK_REALTIME clock.
2018-09-27 13:16:26 +02:00
Tal Einat f55c64c632
bpo-31425: fix versionadded in docs and add attribution in NEWS (GH-9595) 2018-09-27 00:20:38 +03:00
Bjorn Andersson bb8165172a bpo-31425: Expose AF_QIPCRTR in socket module (GH-3706)
The AF_QIPCRTR address family was introduced in Linux v4.7.

Co-authored-by: Bjorn Andersson <bjorn.andersson@linaro.org>
2018-09-26 16:47:52 +03:00
Cheryl Sabella d345bb4d9b bpo-34334: Don't log traceback twice in QueueHandler (GH-9537) 2018-09-26 00:00:08 +01:00
Zackery Spytz 5a5ce064b3 bpo-5950: Support reading zips with comments in zipimport (#9548)
* bpo-5950: Support reading zips with comments in zipimport
2018-09-25 15:15:47 -04:00
Victor Stinner 6ea29c5e90 bpo-34687: Make asynico use ProactorEventLoop by default (GH-9538) 2018-09-25 11:27:08 -04:00
João Júnior 558c49bcf3 bpo-34728: Remove deprecate *loop* argument in asyncio.sleep (GH-9415)
* Insert the warn in the asyncio.sleep when the loop argument is used

* Insert the warn in the asyncio.wait and asyncio.wait_for when the loop argument is used

* Better format of the code

* Add news file

* change calls for get_event_loop() to calls for get_running_loop()

* Change message to be more clear in News

* Improve the comments in test_tasks
2018-09-24 05:51:22 -04:00
Lisa Roach 9718b59ee5
bpo-34659: Adds initial kwarg to itertools.accumulate() (GH-9345) 2018-09-23 17:34:59 -07:00
Christian Heimes 9fb051f032 bpo-34670: Add TLS 1.3 post handshake auth (GH-9460)
Add SSLContext.post_handshake_auth and
SSLSocket.verify_client_post_handshake for TLS 1.3 post-handshake
authentication.

Signed-off-by: Christian Heimes <christian@python.org>q


https://bugs.python.org/issue34670
2018-09-22 23:32:31 -07:00
Ethan Furman 5bdab641da
bpo-29577: Enum: mixin classes don't mix well with already mixed Enums (GH-9328)
* bpo-29577: allow multiple mixin classes
2018-09-21 19:03:09 -07:00
Brett Cannon d64ee1a5ba bpo-32718: Make Activate.ps1 for venv cross-platform and available on all platforms (GH-9321)
PowerShell Core 6.1 is the cross-platform port of Windows PowerShell. This change updates Activate.ps1 to not make Windows assumptions as well as installing it into the bin/Scripts directory on all operating systems.

Requires PowerShell Core 6.1 for proper readline support once the shell has been activated for the virtual environment.
2018-09-21 15:27:26 -07:00
Berker Peksag 8d1e190fc5
bpo-32215: Fix performance regression in sqlite3 (GH-8511) 2018-09-20 14:10:49 +03:00
Silas Sewell 4ba3b50bfe bpo-34472: Add data descriptor signature to zipfile (GH-8871)
This makes streamed zips compatible with MacOS Archive Utility and
other applications.
2018-09-18 20:00:05 +03:00
Serhiy Storchaka 0185f34ddc
bpo-33721: Make some os.path functions and pathlib.Path methods be tolerant to invalid paths. (#7695)
Such functions as os.path.exists(), os.path.lexists(), os.path.isdir(),
os.path.isfile(), os.path.islink(), and os.path.ismount() now return False
instead of raising ValueError or its subclasses UnicodeEncodeError
and UnicodeDecodeError for paths that contain characters or bytes
unrepresentative at the OS level.
2018-09-18 11:28:51 +03:00
Serhiy Storchaka 7bdf28265a
bpo-32455: Add jump parameter to dis.stack_effect(). (GH-6610)
Add C API function PyCompile_OpcodeStackEffectWithJump().
2018-09-18 09:54:26 +03:00
Serhiy Storchaka 9bdb7be482
bpo-34341: Fix appending to ZIP archives with the ZIP64 extension. (GH-8683) 2018-09-17 15:36:40 +03:00
Serhiy Storchaka e0e5065dae
bpo-34610: Fixed iterator of multiprocessing.managers.DictProxy. (GH-9113) 2018-09-17 14:24:01 +03:00
Monson Shao 10a428b64b closes bpo-34515: Support non-ASCII identifiers in lib2to3. (GH-8950) 2018-09-15 10:32:29 -07:00
Benjamin Peterson 5633c4f342
bpo-34672: Try to pass the C library's own timezone strings back to it. (GH-9288) 2018-09-14 09:09:04 -07:00
Eric V. Smith 9b9d97dd13
bpo-34363: dataclasses.asdict() and .astuple() now handle fields which are namedtuples. (GH-9151) 2018-09-14 11:32:16 -04:00
Gregory P. Smith 19003841e9
bpo-6721: Hold logging locks across fork() (GH-4071)
bpo-6721: When os.fork() was called while another thread holds a logging lock, the child process may deadlock when it tries to log.  This fixes that by acquiring all logging locks before fork and releasing them afterwards.

A regression test that fails before this change is included.

Within the new unittest itself: There is a small _potential_ due to mixing of fork and a thread in the child process if the parent's thread happened to hold a non-reentrant library call lock (malloc?) when the os.fork() happens.  buildbots and time will tell if this actually manifests itself in this test or not.  :/  A functionality test that avoids that would be a challenge.

An alternate test that isn't trying to produce the deadlock itself but just checking that the release and acquire calls are made would be the next best alternative if so.
2018-09-13 22:08:31 -07:00
Andrew Svetlov 11194c877c
bpo-34666: Implement stream.awrite() and stream.aclose() (GH-9274) 2018-09-13 16:53:49 -07:00
Gregory P. Smith a20b6adb5a bpo-34658: Fix rare subprocess prexec_fn fork error. (GH-9255)
[bpo-34658](https://www.bugs.python.org/issue34658): Fix a rare interpreter unhandled exception state SystemError only
seen when using subprocess with a preexec_fn while an after_parent handler has
been registered with os.register_at_fork and the fork system call fails.



https://bugs.python.org/issue34658
2018-09-13 04:30:10 -07:00
Benjamin Peterson 40caa05fa4
closes bpo-34652: Always disable lchmod on Linux. (GH-9234) 2018-09-12 15:52:40 -07:00
Tony Flury 2087023fde bpo-32933: Implement __iter__ method on mock_open() (GH-5974) 2018-09-13 01:21:16 +03:00
Andrew Svetlov 0dd71807a9
bpo-34630: Skip logging SSL certificate errors by asyncio code (GH-9169) 2018-09-12 14:03:54 -07:00
Ethan Furman 019f0a0cb8
bpo-34536: raise error for invalid _missing_ results (GH-9147)
* raise exception if _missing_ returns None or invalid type
2018-09-12 11:43:34 -07:00
Andrew Svetlov a5d1eb8d8b
bpo-34638: Store a weak reference to stream reader to break strong references loop (GH-9201)
Store a weak reference to stream readerfor breaking strong references

It breaks the strong reference loop between reader and protocol and allows to detect and close the socket if the stream is deleted (garbage collected)
2018-09-12 11:43:04 -07:00
orlnub123 0fb9fadd3b bpo-34282: Fix Enum._convert shadowing members named _convert (GH-8568)
* Fix enum members getting shadowed by parent attributes
* Move Enum._convert to EnumMeta._convert_
* Deprecate _convert
2018-09-12 10:28:53 -07:00
Sergey Fedoseev ec014a101a bpo-34636: Use fast path for more chars in SRE category macros. (GH-9170)
When handling \s, \d, or \w (and their inverse) escapes in bytes regexes this a small but measurable performance improvement.

<!-- issue-number: [bpo-34636](https://www.bugs.python.org/issue34636) -->
https://bugs.python.org/issue34636
<!-- /issue-number -->
2018-09-11 15:47:59 -07:00
Oren Milman 24bd50bdcc closes bpo-31608: Fix a crash in methods of a subclass of _collections.deque with a bad __new__(). (GH-3788) 2018-09-11 11:46:55 -07:00
Andrew Svetlov 0baa72f4b2
bpo-34622: Extract asyncio exceptions into a separate module (GH-9141) 2018-09-11 10:13:04 -07:00
Zackery Spytz 73870bfeb9 closes bpo-32490: Fix filename duplication in subprocess exception message. (GH-9163)
8621bb5d93 sets the filename in directly in the FileNotFoundError, so we may revert the earlier fix 5f78040057.
2018-09-11 08:54:07 -07:00
Benjamin Peterson 5033aa77aa
bpo-34625: Update vendorized expat version to 2.2.6. (GH-9150) 2018-09-10 21:04:00 -07:00
Gregory P. Smith ce34410b8b
bpo-32270: Don't close stdin/out/err in pass_fds (GH-6242)
When subprocess.Popen() stdin= stdout= or stderr= handles are specified
and appear in pass_fds=, don't close the original fds after dup'ing them.

This implementation and unittest primarily came from @izbyshev (see the PR)

See also b89b52f284

This also removes the old manual p2cread, c2pwrite, and errwrite closing logic
as inheritable flags and _close_open_fds takes care of that properly today without special treatment.

This code is within child_exec() where it is the only thread so there is no
race condition between the dup and _Py_set_inheritable_async_safe call.
2018-09-10 17:46:22 -07:00
Zachary Ware 880d42a3b2
bpo-8110: Refactor platform detection in subprocess (GH-9053)
Check for functionality via imports rather than checking sys.platform
specifically for Windows
2018-09-10 16:16:08 -07:00
Rahul Jha 9430652535 bpo-33217: Raise TypeError for non-Enum lookups in Enums (GH-6651)
* bpo-33217: Raise TypeError for non-Enum lookups in Enums
2018-09-10 11:21:04 -07:00
Matthias Bussonnier 51a4743d19 bpo-33604: Remove deprecated HMAC default value marked for removal in 3.8 (GH-7063)
HMAC's digestmod was deprecated marked for removal, this removes it as planned.
2018-09-10 11:10:01 -07:00
Berker Peksag 9797b7ae44 bpo-26502: Implement FrameSummary.__len__() (GH-8632) 2018-09-10 10:02:33 -07:00
Julien Malard 0afada163c bpo-34421 avoid unicode error in distutils logging (GH-8799)
This caused installation errors in some cases on Windows.
Patch by Julien Malard.
2018-09-08 16:31:26 -04:00
Pablo Aguiar d5fbe9b1a3 bpo-34246: Use no mutable default args in smtplib (GH-8554)
Some methods of the SMTP class use mutable default arguments. Specially
`send_message` is affected as it mutates one of the args by appending items
to it, which has side effects on further calls.
2018-09-07 23:04:48 +01:00
William Grzybowski 28658485a5 bpo-34604: Fix possible mojibake in pwd.getpwnam() and grp.getgrnam() (GH-9098)
Pass the user/group name as Unicode to the formatting function,
instead of always decoding a bytes string from UTF-8.
2018-09-07 19:10:39 +02:00
William Grzybowski 23e65b2555 bpo-33625: Release GIL for grp.getgr{nam,gid} and pwd.getpw{nam,uid} (GH-7081)
Release GIL on grp.getgrnam(), grp.getgrgid(), pwd.getpwnam() and
pwd.getpwuid() if reentrant variants of these functions are available.

Patch by William Grzybowski.
2018-09-07 14:06:15 +02:00
Tal Einat 3e2b29dccc
bpo-30977: make uuid.UUID use __slots__ (GH-9078)
Co-Authored-By: Wouter Bolsterlee.
2018-09-06 14:34:25 +03:00
Alexander Buchkovsky 266f4904a2 bpo-34563: Fix for invalid assert on big output of multiprocessing.Process (GH-9027)
Fix for invalid assert on big output of multiprocessing.Process.
2018-09-04 18:10:28 +02:00
Victor Stinner 39487196c8
bpo-34530: Fix distutils find_executable() (GH-9049)
distutils.spawn.find_executable() now falls back on os.defpath if the
PATH environment variable is not set.
2018-09-04 11:01:09 +02:00
Pablo Galindo ec74d187f5 bpo-33613, test_semaphore_tracker_sigint: fix race condition (#7850)
Fail `test_semaphore_tracker_sigint` if no warnings are expected and one is received.

Fix race condition when the child receives SIGINT before it can register signal handlers for it.

The race condition occurs when the parent calls
`_semaphore_tracker.ensure_running()` (which in turn spawns the
semaphore_tracker using `_posixsubprocess.fork_exec`), the child
registers the signal handlers and the parent tries to kill the child.
What seem to happen is that in some slow systems, the parent sends the
signal to kill the child before the child protects against the signal.
2018-09-04 10:53:54 +02:00
Michael Felt 172a71f19b bpo-34558: Add missing parentheses in _aix.py (GH-9017)
p.wait()
2018-08-31 19:31:07 -07:00
Naris R 1b5f9c9653 bpo-34427: Fix infinite loop when calling MutableSequence.extend() on self (GH-8813) 2018-08-30 09:56:14 -07:00
Carl Meyer d658deac60 bpo-21145: Add cached_property decorator in functools (#6982)
Robust caching of calculated properties is
harder than it looks at first glance, so add
a solid, well-tested implementation to the
standard library.
2018-08-28 17:11:56 +10:00
Elias Zamaria 393f1ff62e bpo-32968: Make modulo and floor division involving Fraction and float consistent with other operations (#5956)
Make mixed-type `%` and `//` operations involving `Fraction` and `float` objects behave like all other mixed-type arithmetic operations: first the `Fraction` object is converted to a `float`, then the `float` operation is performed as normal. This fixes some surprising corner cases, like `Fraction('1/3') % inf` giving a NaN.

Thanks Elias Zamaria for the patch.
2018-08-27 07:59:28 +01:00
Serhiy Storchaka c406d5cd74
bpo-34171: Prevent creating Lib/trace.cover when run the trace module. (GH-8841) 2018-08-25 10:27:55 +03:00
Gregory P. Smith 76be0fffff
bpo-13312: Avoid int underflow in time year. (GH-8912)
Avoids an integer underflow in the time module's year handling code.
2018-08-24 18:08:50 -07:00
Vladimir Matveev 91cb298f81 bpo-6700: Fix inspect.getsourcelines for module level frames/tracebacks (GH-8864) 2018-08-24 17:18:00 +03:00
Paul Ganssle 096329f0b2 bpo-34454: fix .fromisoformat() methods crashing on inputs with surrogate code points (GH-8862)
The current C implementations **crash** if the input includes a surrogate
Unicode code point, which is not possible to encode in UTF-8.

Important notes:

1.  It is possible to pass a non-UTF-8 string as a separator to the
    `.isoformat()` methods.
2.  The pure-Python `datetime.fromisoformat()` implementation accepts
    strings with a surrogate as the separator.

In `datetime.fromisoformat()`, in the special case of non-UTF-8 separators,
this implementation will take a performance hit by making a copy of the
input string and replacing the separator with 'T'.

Co-authored-by: Alexey Izbyshev <izbyshev@ispras.ru>
Co-authored-by: Paul Ganssle <paul@ganssle.io>
2018-08-23 18:06:20 +03:00
Michael Osipov 48ce4897f8 bpo-34412: Make signal.strsignal() work on HP-UX (GH-8786)
Introduce a configure check for strsignal(3) which defines HAVE_STRSIGNAL for
signalmodule.c. Add some common signals on HP-UX. This change applies for
Windows and HP-UX.
2018-08-23 16:27:19 +03:00
Berker Peksag e7d4b2f205
bpo-2122: Make mmap.flush() behave same on all platforms (GH-8692)
Previously, its behavior was platform-dependent and there was no error checking
under Windows.
2018-08-22 21:21:05 +03:00
Alexey Izbyshev cdbf50cba1 bpo-34441: Fix ABC.__subclasscheck__ crash on classes with invalid __subclasses__ (GH-8835)
The missing NULL check was reported by Svace static analyzer.
2018-08-20 23:04:19 +03:00
Serhiy Storchaka 77d5781835
bpo-34318: Convert deprecation warnings to errors in assertRaises() etc. (GH-8623) 2018-08-19 10:00:11 +03:00
Zackery Spytz e349bf2358 bpo-22602: Raise an exception in the UTF-7 decoder for ill-formed sequences starting with "+". (GH-8741)
The UTF-7 decoder now raises UnicodeDecodeError for ill-formed
sequences starting with "+" (as specified in RFC 2152).
2018-08-19 07:43:38 +03:00
Berker Peksag e0b5b2096e
bpo-34384: Fix os.readlink() on Windows (GH-8740)
os.readlink() now accepts path-like and bytes objects on Windows.
Previously, support for path-like and bytes objects was only
implemented on Unix.

This commit also merges Unix and Windows implementations of
os.readlink() in one function and adds basic unit tests to increase
test coverage of the function.
2018-08-15 13:03:41 +03:00
Vadim Pushtaev 4d12e4dc28 bpo-34213: Allow dataclasses to work with a field named 'object'. (GH-8452) 2018-08-12 07:46:05 -04:00
Berker Peksag 84a13fbda0
bpo-9372: Deprecate several __getitem__ methods (GH-8609)
The __getitem__ methods of DOMEventStream, FileInput,
and FileWrapper classes ignore their 'index' parameters
and return the next item instead.
2018-08-11 09:05:04 +03:00
Berker Peksag 423d05f6f5
bpo-34333: Fix %-formatting in Path.with_suffix() (GH-8663) 2018-08-11 08:45:06 +03:00
Alex Grönholm cca4eec3c0 bpo-34270: Make it possible to name asyncio tasks (GH-8547)
Co-authored-by: Antti Haapala <antti.haapala@anttipatterns.com>
2018-08-08 17:06:47 -04:00
Berker Peksag e4dcbbd7f4
bpo-18540: Fix EAI_NONAME in imaplib.IMAP4*() (GH-8634) 2018-08-07 05:12:18 +03:00
Anthony Sottile b2499669ef bpo-19891: Ignore error while writing history file (GH-8483) 2018-08-06 17:28:19 +09:00
Marcel Plch a2fe1e52eb bpo-34097: Add support for zipping files older than 1980-01-01 (GH-8270)
ZipFile can zip files older than 1980-01-01 and newer than 2107-12-31 using
a new strict_timestamps parameter at the cost of setting the timestamp
to the limit.
2018-08-02 15:04:52 +02:00
MartinAltmayer 944451cd8d bpo-34263 Cap timeout submitted to epoll/select etc. to one day. (GH-8532) 2018-07-31 10:06:12 -04:00
Raymond Hettinger 9c18b1ae52
bpo-33089: Add math.dist() for computing the Euclidean distance between two points (GH-8561) 2018-07-31 00:45:49 -07:00
Serhiy Storchaka 9d5727326a
bpo-33871: Fix os.sendfile(), os.writev(), os.readv(), etc. (GH-7931)
* Fix integer overflow in os.readv(), os.writev(), os.preadv()
  and os.pwritev() and in os.sendfile() with headers or trailers
  arguments (on BSD-based OSes and MacOS).

* Fix sending the part of the file in os.sendfile() on MacOS.
  Using the trailers argument could cause sending more bytes from
  the input file than was specified.

Thanks Ned Deily for testing on 32-bit MacOS.
2018-07-31 10:24:54 +03:00
Serhiy Storchaka f1d36d8efa
bpo-33729: Fix issues with arguments parsing in hashlib. (GH-8346)
* help(hashlib) didn't work because of incorrect module name in blake2b and
  blake2s classes.
* Constructors blake2*(), sha3_*(), shake_*() and keccak_*() incorrectly
  accepted keyword argument "string" for binary data, but documented as
  accepting the "data" keyword argument. Now this parameter is positional-only.
* Keyword-only parameters in blake2b() and blake2s() were not documented as
  keyword-only.
* Default value for some parameters of blake2b() and blake2s() was None,
  which is not acceptable value.
* The length argument for shake_*.digest() was wrapped out to 32 bits.
* The argument for shake_128.digest() and shake_128.hexdigest() was not
  positional-only as intended.
* TypeError messages for incorrect arguments in all constructors sha3_*(),
  shake_*() and keccak_*() incorrectly referred to sha3_224.

Also made the following enhancements:

* More accurately specified input and result types for strings, bytes and
  bytes-like objects.
* Unified positional parameter names for update() and constructors.
* Improved formatting.
2018-07-31 09:50:16 +03:00
Serhiy Storchaka 4b8a7f51da
Revert "closes bpo-27494: Fix 2to3 handling of trailing comma after a generator expression (GH-3771)" (#8241)
This reverts commit af810b35b4.

This is not valid syntax (see bpo-32012).
2018-07-31 09:34:30 +03:00
twisteroid ambassador 9045199c5a bpo-33833: Fix ProactorSocketTransport AssertionError (#7893) 2018-07-30 21:58:50 +03:00
Elvis Pranskevichus 22d25085db bpo-34075: Deprecate non-ThreadPoolExecutor in loop.set_default_executor() (GH-8533)
Various asyncio internals expect that the default executor is a
`ThreadPoolExecutor`, so deprecate passing anything else to
`loop.set_default_executor()`.
2018-07-30 12:42:43 +02:00
Mickaël Schoentgen 3f8c6913b8 bpo-34035: Fix several AttributeError in zipfile seek() methods. (GH-8527) 2018-07-29 21:26:52 +03:00
Franz Wöllert d2e902e4fb bpo-31047: Fix ntpath.abspath for invalid paths (GH-8544) 2018-07-29 13:47:09 +01:00
Berker Peksag 11eb1a9470 bpo-34251: Restore msilib.Win64 to preserve compatibility (GH-8510) 2018-07-28 16:02:56 -05:00
Raymond Hettinger c6dabe37e3
bpo-33089: Multidimensional math.hypot() (GH-8474) 2018-07-28 07:48:04 -07:00
Dong-hee Na 8fe9eed937 bpo-33476: Fix _header_value_parser when address group is missing final ';' (GH-7484) 2018-07-28 13:55:11 +01:00
Serhiy Storchaka 323748ad74
bpo-34197: Make _csv.Dialect attributes booleans. (GH-8440)
Attributes skipinitialspace, doublequote and strict are now
booleans instead of integers 0 or 1.
2018-07-26 13:21:09 +03:00
Victor Stinner 60b04c9f6f
bpo-34228: Allow PYTHONTRACEMALLOC=0 (GH-8467)
PYTHONTRACEMALLOC=0 environment variable and -X tracemalloc=0 command
line option are now allowed to disable explicitly tracemalloc at
startup.
2018-07-25 19:23:53 +02:00
Berker Peksag 74102c9a5f
bpo-13041: Use shutil.get_terminal_size() in argparse.HelpFormatter (GH-8459) 2018-07-25 18:23:44 +03:00
Serhiy Storchaka ac0b3c2f4d
bpo-34164: Fix handling of incorrect padding in base64.b32decode(). (GH-8351)
Now base64.Error is always raised instead of UnboundLocalError or
OverflowError.
2018-07-24 12:52:51 +03:00
Serhiy Storchaka 02ec92fa7b
bpo-29209: Remove old-deprecated features in ElementTree. (GH-6769)
Also make getchildren() and getiterator() emitting
a DeprecationWarning instead of PendingDeprecationWarning.
2018-07-24 12:03:34 +03:00
Matěj Cepl caa331d492 bpo-33336, imaplib: Legalize MOVE command (GH-6569)
imaplib now allows MOVE command in IMAP4.uid() (RFC 6851:
IMAP MOVE Extension) and potentially as a name of supported
method of IMAP4 object.
2018-07-23 13:28:54 +02:00
Berker Peksag 7a3056fa7d
bpo-21446: Update reload fixer to use importlib (GH-8391) 2018-07-23 09:49:08 +03:00
Berker Peksag d04f46c59f
bpo-940286: Fix pydoc to show cross refs correctly (GH-8390) 2018-07-23 08:37:47 +03:00
Jason R. Coombs cafaf0447b
bpo-34108: Fix double carriage return in 2to3 on Windows (#8271)
* Add test capturing failure.
* Honor newlines as present in the original file.
2018-07-13 11:26:03 -04:00
Dong-hee Na 445f1b35ce bpo-33967: Fix singledispatch raised IndexError when no args (GH-8184) 2018-07-10 16:26:36 +09:00
Serhiy Storchaka 2a9b8babf0
bpo-26544: Fixed implementation of platform.libc_ver(). (GH-7684) 2018-07-09 11:47:45 +03:00
Serhiy Storchaka 25b804a9c2
bpo-31014: Fix the webbrowser module. (GH-7267)
webbrowser._synthesize() called webbrowser.register() with
outdated signature.

Co-Authored-By: John Still <john@jmsdvl.com>
2018-07-08 10:22:32 +03:00
Sergey Fedoseev 0830858aee bpo-34041: Allow creating deterministic functions in Connection.create_function() (GH-8086) 2018-07-08 10:09:20 +03:00
Benjamin Peterson b0274f2cdd
closes bpo-34056: Always return bytes from _HackedGetData.get_data(). (GH-8130)
* Always return bytes from _HackedGetData.get_data().

Ensure the imp.load_source shim always returns bytes by reopening the file in
binary mode if needed. Hash-based pycs have to receive the source code in bytes.

It's tempting to change imp.get_suffixes() to always return 'rb' as a mode, but
that breaks some stdlib tests and likely 3rdparty code, too.
2018-07-06 20:41:06 -07:00
Victor Stinner c2368cbc83
bpo-34054: multiprocessing uses time.monotonic() (GH-8118)
The multiprocessing module now uses the monotonic clock
time.monotonic() instead of the system clock time.time() to implement
timeouts.
2018-07-06 13:51:52 +02:00
Ammar Askar c4ef4896ea bpo-33899: Make tokenize module mirror end-of-file is end-of-line behavior (GH-7891)
Most of the change involves fixing up the test suite, which previously made
the assumption that there wouldn't be a new line if the input didn't end in
one.

Contributed by Ammar Askar.
2018-07-06 10:19:08 +03:00
INADA Naoki 8d130913cb
bpo-34043: Optimize tarfile uncompress performance (GH-8089)
tarfile._Stream has two buffer for compressed and uncompressed data.
Those buffers are not aligned so unnecessary bytes slicing happens
for every reading chunks.

This commit bypass compressed buffering.

In this benchmark [1], user time become 250ms from 300ms.

[1]: https://bugs.python.org/msg320763
2018-07-06 14:06:00 +09:00
Victor Stinner 483422f57e
bpo-34044: subprocess.Popen copies startupinfo (GH-8090)
subprocess.Popen now copies the startupinfo argument to leave it
unchanged: it will modify the copy, so that the same STARTUPINFO
object can be used multiple times.

Add subprocess.STARTUPINFO.copy() method.
2018-07-05 22:54:17 +02:00
hajoscher 12a08c4760 bpo-34010: Fix tarfile read performance regression (GH-8020)
During buffered read, use a list followed by join instead of extending a bytes object.
This is how it was done before but changed in commit b506dc32c1.
2018-07-04 17:13:18 +09:00
Bumsik Kim 3cf1f154ed bpo-34019: Fix wrong arguments for Opera Browser (#8047)
The Opera Browser was using a outdated command line invocation that resulted in an incorrect URL being opened in the browser when requested using the webbrowser module.

* Correct the arguments passed to the Opera Browser when opening a new URL.
2018-07-03 12:30:06 +01:00
Xtreak 087570af6d bpo-33978: Close existing handlers before logging (re-)configuration. (GH-8008) 2018-07-02 09:57:46 +01:00
Tal Einat 0cdf5f4289
bpo-32568: make select.epoll() and its docs consistent (#7840)
* `flags` is indeed deprecated, but there is a validation on its value for
  backwards compatibility reasons.  This adds mention of this in the docs.
* The docs say that `sizehint` is deprecated and ignored, but it is still
  used when `epoll_create1()` is unavailable. This adds mention of this in
  the docs.
* `sizehint=-1` is acceptable again, and is replaced with `FD_SETSIZE-1`.
  This is needed to have a default value available at the Python level,
  since `FD_SETSIZE` is not exposed to Python. (see: bpo-31938)
* Reject `sizehint=0` since it is invalid to pass on to `epoll_create()`.

The relevant tests have also been updated.
2018-06-30 15:43:23 +03:00
Serhiy Storchaka 5bb5bbfca8
bpo-33974: Fix passing special characters to ttk widgets. (GH-7986)
Fix passing lists and tuples of strings containing special characters
'"', '\\', '{', '}' and '\n' as options to tkinter.ttk widgets.
2018-06-30 09:20:28 +03:00
Terry Jan Reedy 891a1f86d4
bpo-14117: Make minor tweaks to turtledemo (GH-8002)
The 'wikipedia' example is now 'rosette', describing what it draws.
The 'penrose' print output is reduced.  The 'tree' '1024'
output is eliminated.
2018-06-29 01:10:05 -04:00
Yury Selivanov d904c238ca
bpo-27500: Fix static version of getaddrinfo to resolve IPv6 (GH-7993) 2018-06-28 21:59:32 -04:00
INADA Naoki 461a1c4b49
bpo-33842: Remove tarfile.filemode (GH-7661) 2018-06-28 17:10:36 +09:00
Ned Deily aee5df5e16 Forward port 3.7.0 final changes 2018-06-27 18:45:50 -04:00
Zackery Spytz d2cbfffc84 bpo-25007: Add copy protocol support to zlib compressors and decompressors (GH-7940) 2018-06-27 21:04:51 +03:00
twisteroid ambassador 4a8b037d2b bpo-31647: Fix bpo typo in NEWS entry. (GH-7964) 2018-06-27 10:58:08 -04:00
Victor Stinner 2cc9d21fff
bpo-33929: multiprocessing: fix handle leak on race condition (GH-7921)
Fix a race condition in Popen of
multiprocessing.popen_spawn_win32. The child process now duplicates
the read end of pipe instead of "stealing" it.

Previously, the read end of pipe was "stolen" by the child process,
but it leaked a handle if the child process had been terminated
before it could steal the handle from the parent process.
2018-06-27 11:40:24 +02:00
Raymond Hettinger ddf7171911
bpo-24567: Random subnormal.diff (#7954)
Handle subnormal weights for choices()
2018-06-27 01:08:31 -07:00
Dong-hee Na cf67d6a934 bpo-33897: Add a 'force' keyword argument to logging.basicConfig(). (GH-7873) 2018-06-25 05:11:09 +01:00
Dong-hee Na 3d70f7aef6 bpo-33805: Improve error message of dataclasses.replace() (GH-7580) 2018-06-23 10:46:32 -04:00
Victor Stinner 9b7cf75721
bpo-33916: Fix bz2 and lzma init when called twice (GH-7843)
bz2, lzma: When Decompressor.__init__() is called twice, free the old
lock to not leak memory.
2018-06-23 10:35:23 +02:00
Giampaolo Rodola c7f02a9659
bpo-33671 / shutil.copyfile: use memoryview() with dynamic size on Windows (#7681)
bpo-33671
* use memoryview() with size == file size on Windows, see https://github.com/python/cpython/pull/7160#discussion_r195405230
* release intermediate (sliced) memoryview immediately
* replace "OSX" occurrences with "macOS"
* add some unittests for copyfileobj()
2018-06-19 08:27:29 -07:00
Marco Strigl 936f03e7fa bpo-33365: print the header values beside the keys (GH-6611)
with debuglevel=1 only the header keys got printed. With
this change the header values get printed as well and the single
header entries get '\n' as a separator.
2018-06-19 16:20:58 +03:00
INADA Naoki 698865dcbb
bpo-33843: Remove deprecated stuff in cgi module (GH-7662) 2018-06-19 17:28:50 +09:00
ValeriyaSinevich b36b0a3765 bpo-33663: Convert content length to string before putting to header (GH-7754) 2018-06-18 14:17:53 -07:00
Wonsup Yoon d134809cd3 bpo-29456: Fix bugs in unicodedata.normalize: u1176, u11a7 and u11c3 (GH-1958)
Hangul composition check boundaries are wrong for the second character
([0x1161, 0x1176) instead of [0x1161, 0x1176]) and third character ((0x11A7, 0x11C3)
instead of [0x11A7, 0x11C3]).
2018-06-15 20:03:14 +08:00
Serhiy Storchaka 08f127a3ca
bpo-33851: Fix ast.get_docstring() for a node that lacks a docstring. (GH-7682) 2018-06-15 11:05:15 +03:00
Giampaolo Rodola 4a172ccc73
bpo-33671: efficient zero-copy for shutil.copy* functions (Linux, OSX and Win) (#7160)
* have shutil.copyfileobj use sendfile() if possible

* refactoring: use ctx manager

* add test with non-regular file obj

* emulate case where file size can't be determined

* reference _copyfileobj_sendfile directly

* add test for offset() at certain position

* add test for empty file

* add test for non regular file dst

* small refactoring

* leave copyfileobj() alone in order to not introduce any incompatibility

* minor refactoring

* remove old test

* update docstring

* update docstring; rename exception class

* detect platforms which only support file to socket zero copy

* don't run test on platforms where file-to-file zero copy is not supported

* use tempfiles

* reset verbosity

* add test for smaller chunks

* add big file size test

* add comment

* update doc

* update whatsnew doc

* update doc

* catch Exception

* remove unused import

* add test case for error on second sendfile() call

* turn docstring into comment

* add one more test

* update comment

* add Misc/NEWS entry

* get rid of COPY_BUFSIZE; it belongs to another PR

* update doc

* expose posix._fcopyfile() for OSX

* merge from linux branch

* merge from linux branch

* expose fcopyfile

* arg clinic for the win implementation

* convert path type to path_t

* expose CopyFileW

* fix windows tests

* release GIL

* minor refactoring

* update doc

* update comment

* update docstrings

* rename functions

* rename test classes

* update doc

* update doc

* update docstrings and comments

* avoid do import nt|posix modules if unnecessary

* set nt|posix modules to None if not available

* micro speedup

* update description

* add doc note

* use better wording in doc

* rename function using 'fastcopy' prefix instead of 'zerocopy'

* use :ref: in rst doc

* change wording in doc

* add test to make sure sendfile() doesn't get called aymore in case it doesn't support file to file copies

* move CopyFileW in _winapi and actually expose CopyFileExW instead

* fix line endings

* add tests for mode bits

* add docstring

* remove test file mode class; let's keep it for later when Istart addressing OSX fcopyfile() specific copies

* update doc to reflect new changes

* update doc

* adjust tests on win

* fix argument clinic error

* update doc

* OSX: expose copyfile(3) instead of fcopyfile(3); also expose flags arg to python

* osx / copyfile: use path_t instead of char

* do not set dst name in the OSError exception in order to remain consistent with platforms which cannot do that (e.g. linux)

* add same file test

* add test for same file

* have osx copyfile() pre-emptively check if src and dst are the same, otherwise it will return immedialtey and src file content gets deleted

* turn PermissionError into appropriate SameFileError

* expose ERROR_SHARING_VIOLATION in order to raise more appropriate SameFileError

* honour follow_symlinks arg when using CopyFileEx

* update Misc/NEWS

* expose CreateDirectoryEx mock

* change C type

* CreateDirectoryExW actual implementation

* provide specific makedirs() implementation for win

* fix typo

* skeleton for SetNamedSecurityInfo

* get security info for src path

* finally set security attrs

* add unit tests

* mimick os.makedirs() behavior and raise if dst dir exists

* set 2 paths for OSError object

* set 2 paths for OSError object

* expand windows test

* in case of exception on os.sendfile() set filename and filename2 exception attributes

* set 2 filenames (src, dst) for OSError in case copyfile() fails on OSX

* update doc

* do not use CreateDirectoryEx() in copytree() if source dir is a symlink (breaks test_copytree_symlink_dir); instead just create a plain dir and remain consistent with POSIX implementation

* use bytearray() and readinto()

* use memoryview() with bytearray()

* refactoring + introduce a new _fastcopy_binfileobj() fun

* remove CopyFileEx and other C wrappers

* remove code related to CopyFileEx

* Recognize binary files in copyfileobj()
...and use fastest _fastcopy_binfileobj() when possible

* set 1MB copy bufsize on win; also add a global _COPY_BUFSIZE variable

* use ctx manager for memoryview()

* update doc

* remove outdated doc

* remove last CopyFileEx remnants

* OSX - use fcopyfile(3) instead of copyfile(3)

...as an extra safety measure: in case src/dst are "exotic" files (non
regular or living on a network fs etc.) we better fail on open() instead
of copyfile(3) as we're not quite sure what's gonna happen in that
case.

* update doc
2018-06-12 23:04:50 +02:00
Cheryl Sabella 33cd058f21 bpo-32108: Don't clear configparser values if key is assigned to itself (GH-7588) 2018-06-12 13:37:51 -07:00
Tal Einat c3f55be7dd
bpo-27397: Make email module properly handle invalid-length base64 strings (#7583)
When attempting to base64-decode a payload of invalid length (1 mod 4),
properly recognize and handle it.  The given data will be returned as-is,
i.e. not decoded, along with a new defect, InvalidBase64LengthDefect.
2018-06-12 15:46:22 +03:00
Matthias Bussonnier 46c5cd0f6e bpo-33582: Emit deprecation warning for `formatargspec` (GH-6994) 2018-06-11 16:08:16 -04:00
Steve Weber 2487f30d55 bpo-30167: Prevent site.main() exception if PYTHONSTARTUP is set. (GH-6731)
Before Python 3.6, os.path.abspath(None) used to report an AttributeError which was properly caught inside site.abs_paths, making it ignore __main__, one of sys.modules, which has __file__ and __cached__ set to None. With 3.6, os.path.abspath(None) raises TypeError instead which site.abs_path was not expecting.  This resulted in an uncaught exception if a user had PYTHONSTARTUP set and the application called site.main() which a number of third-party programs do.
2018-06-10 20:49:34 -04:00
Alexander Belopolsky 877b23202b
bpo-33812: Corrected astimezone for naive datetimes. (GH-7578)
A datetime object d is aware if d.tzinfo is not None and
d.tzinfo.utcoffset(d) does  not return None. If d.tzinfo is None,
or if d.tzinfo is not None but d.tzinfo.utcoffset(d) returns None,
 d is naive.

This commit ensures that instances with non-None d.tzinfo, but
d.tzinfo.utcoffset(d) returning None are treated as naive.

In addition, C acceleration code will raise TypeError if
d.tzinfo.utcoffset(d) returns an object with the type other than
timedelta.

* Updated the documentation.

Assume that the term "naive" is defined elsewhere and remove the
not entirely correct clarification.  Thanks, Tim.
2018-06-10 17:02:58 -04:00
Tal Einat 1b85c71a21
bpo-33770: improve base64 exception message for encoded inputs of invalid length (#7416) 2018-06-10 10:01:50 +03:00
Ned Deily ced0adb263
bpo-32493: Correct test for uuid_enc_be availability in configure.ac. (GH-7511) (GH-7567) 2018-06-09 18:19:57 -04:00
Yury Selivanov 12f482e0ae
bpo-30805: Avoid race condition with debug logging (GH-7545)
Supersedes https://github.com/python/cpython/pull/2490
2018-06-08 18:24:37 -04:00
Thomas Kluyver 11a896652e bpo-33375: Get filename for warnings from frame.f_code.co_filename (GH-6622)
More consistent with how other parts of Python find the filename (e.g. tracebacks and pdb).
2018-06-08 12:28:37 -07:00
wim glenn 66f02aa32f bpo-11874: fix assertion failure in argparse metavar handling (GH-1826)
- bugfix and test for fragile metavar handling in argparse (see
  bpo-24089, bpo-14046, bpo-25058, bpo-11874)
- also fixes some incorrect tests that did not make 1-element tuples correctly
2018-06-08 20:12:49 +10:00
Dong-hee Na 4aa3006619 bpo-33197: Add description property for _ParameterKind. (GH-7206) 2018-06-07 23:46:31 -04:00
Yury Selivanov 8f4042964d
bpo-33792: Add selector and proactor windows policies (GH-7487) 2018-06-07 20:44:57 -04:00
Victor Stinner 79790bc35f
bpo-33694: Fix race condition in asyncio proactor (GH-7498)
The cancellation of an overlapped WSARecv() has a race condition
which causes data loss because of the current implementation of
proactor in asyncio.

No longer cancel overlapped WSARecv() in _ProactorReadPipeTransport
to work around the race condition.

Remove the optimized recv_into() implementation to get simple
implementation of pause_reading() using the single _pending_data
attribute.

Move _feed_data_to_bufferred_proto() to protocols.py.

Remove set_protocol() method which became useless.
2018-06-08 00:25:52 +02:00
Scott Sanderson cebe80b59b bpo-29235: Update document for Profiler's context manager (GH-7331) 2018-06-07 18:46:42 +09:00
arikrupnik 5bfa058e65 bpo-33274: Compliance with DOM L1: return removed attribute (#7465)
* bpo-33274: Compliance with DOM L1: return removed attribute

* Update 2018-06-06-22-01-33.bpo-33274.teYqv8.rst
2018-06-07 00:42:38 -04:00
Benjamin Peterson 7c69c1c0fb
update to Unicode 11.0.0 (closes bpo-33778) (GH-7439)
Also, standardize indentation of generated tables.
2018-06-06 20:14:28 -07:00
John Reese 3a5b0d8988 bpo-33504: Migrate configparser from OrderedDict to dict. (#6819)
With 3.7+, dictionary are ordered by design.  Configparser still uses
collections.OrderedDict, which is unnecessary.  This updates the module
to use the standard dict implementation by default, and changes the
docs and tests to match.
2018-06-05 16:31:33 -07:00
Thomas Kluyver c56b17bd8c bpo-12486: Document tokenize.generate_tokens() as public API (#6957)
* Document tokenize.generate_tokens()

* Add news file

* Add test for generate_tokens

* Document behaviour around ENCODING token

* Add generate_tokens to __all__
2018-06-05 10:26:39 -07:00
Vinay Sajip dde9fdbe45
bpo-33165: Added stacklevel parameter to logging APIs. (GH-7424) 2018-06-05 17:24:18 +01:00
Yury Selivanov 415bc46a78
bpo-33769: start_tls: Fix error message; cancel callbacks on error (GH-7403)
In addition to that, mark SSLTransport as "closed" in its "abort()" method to prevent bogus warnings.
2018-06-05 08:59:58 -04:00
Zackery Spytz e9e3976057 bpo-33767: Fix improper use of SystemError by mmap.mmap objects (GH-7381)
Raise TypeError instead of SystemError for unsupported operations.
2018-06-05 15:59:41 +03:00
Yury Selivanov 9602643120
bpo-33734: asyncio/ssl: a bunch of bugfixes (#7321)
* Fix AttributeError (not all SSL exceptions have 'errno' attribute)

* Increase default handshake timeout from 10 to 60 seconds
* Make sure start_tls can be cancelled correctly
* Make sure any error in SSLProtocol gets propagated (instead of just being logged)
2018-06-04 11:32:35 -04:00
Scott Sanderson 2e01b75884 bpo-29235: Make cProfile.Profile a context manager (GH-6808) 2018-06-01 13:36:23 -07:00
Ned Deily 126869f308
Fix "ditribution" typo in NEWS entry. (GH-7274) 2018-05-31 01:18:49 -04:00
jhaydaman 0a28c0d12e bpo-33238: Add InvalidStateError to concurrent.futures. (GH-7056)
Future.set_result and Future.set_exception now raise InvalidStateError
if the futures are not pending or running. This mirrors the behavior
of asyncio.Future, and prevents AssertionErrors in asyncio.wrap_future
when set_result is called multiple times.
2018-05-30 10:15:06 +03:00
Yury Selivanov 99279ad823
Revert "bpo-22087: Fix Policy.get_event_loop() to detect fork (GH-7208)" (#7232)
This reverts commit 5d97b7bcc1.
2018-05-29 20:47:56 -04:00
Elvis Pranskevichus e2b340ab41 bpo-32751: Wait for task cancellation in asyncio.wait_for() (GH-7216)
Currently, asyncio.wait_for(fut), upon reaching the timeout deadline,
cancels the future and returns immediately.  This is problematic for
when *fut* is a Task, because it will be left running for an arbitrary
amount of time.  This behavior is iself surprising and may lead to
related bugs such as the one described in bpo-33638:

    condition = asyncio.Condition()
    async with condition:
        await asyncio.wait_for(condition.wait(), timeout=0.5)

Currently, instead of raising a TimeoutError, the above code will fail
with `RuntimeError: cannot wait on un-acquired lock`, because
`__aexit__` is reached _before_ `condition.wait()` finishes its
cancellation and re-acquires the condition lock.

To resolve this, make `wait_for` await for the task cancellation.
The tradeoff here is that the `timeout` promise may be broken if the
task decides to handle its cancellation in a slow way.  This represents
a behavior change and should probably not be back-patched to 3.6 and
earlier.
2018-05-29 17:31:01 -04:00
Yury Selivanov 863b674909
bpo-32684: Fix gather to propagate cancel of itself with return_exceptions (GH-7209) 2018-05-29 17:20:02 -04:00
Géry Ogam 1cee216cf3 bpo-31639: Change ThreadedHTTPServer to ThreadingHTTPServer class name (GH-7195) 2018-05-29 22:10:30 +02:00
Yury Selivanov 5d97b7bcc1
bpo-22087: Fix Policy.get_event_loop() to detect fork (GH-7208)
Original patch by Dan O'Reilly.
2018-05-29 15:38:07 -04:00
Dong-hee Na a9cab433bb bpo-33197: Update a error message of invalid inspect.Parameters. (GH-6636) 2018-05-29 11:04:08 -04:00
Yury Selivanov 2179022d94 bpo-33654: Support protocol type switching in SSLTransport.set_protocol() (#7194) 2018-05-29 12:02:40 +03:00
Yury Selivanov f295587c45
bpo-33674: Pause the transport as early as possible (#7192) 2018-05-29 01:00:12 -04:00
Victor Stinner be00a5583a
bpo-33674: asyncio: Fix SSLProtocol race (GH-7175)
Fix a race condition in SSLProtocol.connection_made() of
asyncio.sslproto: start immediately the handshake instead of using
call_soon(). Previously, data_received() could be called before the
handshake started, causing the handshake to hang or fail.
2018-05-29 01:33:35 +02:00
Yury Selivanov 7165754b6b
bpo-32410: Avoid blocking on file IO in sendfile fallback code (GH-7172) 2018-05-28 18:31:55 -04:00
Yury Selivanov 416c1ebd98
bpo-32610: Fix asyncio.all_tasks() to return only pending tasks. (GH-7174) 2018-05-28 17:54:02 -04:00
Yury Selivanov fdccfe09f0
bpo-33469: RuntimeError after closing loop that used run_in_executor (GH-7171) 2018-05-28 17:10:20 -04:00
Yury Selivanov 989b9e0e6d
bpo-33672: Fix Task.__repr__ crash with Cython's bogus coroutines (GH-7161) 2018-05-28 16:27:34 -04:00
Yury Selivanov dbf102271f
bpo-33654: Support BufferedProtocol in set_protocol() and start_tls() (GH-7130)
In this commit:

* Support BufferedProtocol in set_protocol() and start_tls()
* Fix proactor to cancel readers reliably
* Update tests to be compatible with OpenSSL 1.1.1
* Clarify BufferedProtocol docs
* Bump TLS tests timeouts to 60 seconds; eliminate possible race from start_serving
* Rewrite test_start_tls_server_1
2018-05-28 14:31:28 -04:00
jimmylai e549c4be5f bpo-33505: Optimize asyncio.ensure_future by reordering if conditions (GH-6836) 2018-05-28 12:42:05 -04:00
twisteroid ambassador 23f587e395 bpo-31647: Fix write_eof() after close() for SelectorSocketTransport (GH-7149)
Fixed bug where calling write_eof() on a _SelectorSocketTransport after
it's already closed raises AttributeError.
2018-05-28 11:16:45 -04:00
Yury Selivanov 35230d08e0
bpo-33623: Fix possible SIGSGV when asyncio.Future is created in __del__ (#7080) 2018-05-28 11:11:31 -04:00
Ethan Smith c651275afe bpo-32380: Create functools.singledispatchmethod (#6306) 2018-05-26 13:38:33 -07:00
Serhiy Storchaka 09f3221fbb bpo-33652: Improve pickle support in the typing module. (GH-7123)
Pickles of type variables and subscripted generics are now future-proof
and compatible with older Python versions.
2018-05-26 11:19:26 -07:00
Serhiy Storchaka 17d8830312 bpo-32493: Fix uuid.uuid1() on FreeBSD. (GH-7099)
Use uuid_enc_be() if available to encode UUID to bytes as big endian.
2018-05-25 00:45:09 +02:00
Ned Deily 8ebf5ceb0f
bpo-33109: argparse subparsers are once again not required by default (GH-6919)
bpo-26510 in 3.7.0a2 changed the behavior of argparse to make
subparsers required by default, returning to the behavior of 2.7
and 3.2. The behavior was changed in 3.3 to be no longer required.
While it might make more sense to have the default to required,
compatibility with 3.3 through 3.6 is probably less disruptive
than trying to reintroduce compatibility with 2.7 at this point.
This change restores the 3.6 behavior.
2018-05-23 21:55:15 -04:00
Victor Stinner 453bd0bc65
bpo-33540: Add block_on_close attr to socketserver (GH-6911)
Add a new block_on_close class attribute to ForkingMixIn and
ThreadingMixIn classes of socketserver to opt-in for pre-3.7 behaviour.
2018-05-24 03:14:44 +02:00
Christian Heimes 529525fb5a
bpo-33618: Enable TLS 1.3 in tests (GH-7079)
TLS 1.3 behaves slightly different than TLS 1.2. Session tickets and TLS
client cert auth are now handled after the initialy handshake. Tests now
either send/recv data to trigger session and client certs. Or tests
ignore ConnectionResetError / BrokenPipeError on the server side to
handle clients that force-close the socket fd.

To test TLS 1.3, OpenSSL 1.1.1-pre7-dev (git master + OpenSSL PR
https://github.com/openssl/openssl/pull/6340) is required.

Signed-off-by: Christian Heimes <christian@python.org>
2018-05-23 22:24:45 +02:00
sth 825aab95fd bpo-27300: Add the errors parameter to tempfile classes. (GH-6696) 2018-05-23 08:07:01 +03:00
Christian Heimes e8eb6cb792
bpo-33570: TLS 1.3 ciphers for OpenSSL 1.1.1 (GH-6976)
Change TLS 1.3 cipher suite settings for compatibility with OpenSSL
1.1.1-pre6 and newer. OpenSSL 1.1.1 will have TLS 1.3 cipers enabled by
default.

Also update multissltests and Travis config to test with latest OpenSSL.

Signed-off-by: Christian Heimes <christian@python.org>
2018-05-22 22:50:12 +02:00
John Reese 6c4fab0f4b bpo-33516: Add support for __round__ in MagicMock (GH-6880)
unittest.mock.MagicMock now supports the __round__() magic method.
2018-05-22 22:01:10 +02:00
Serhiy Storchaka ae00fb1d4f
bpo-30877: Fix clearing a cache in the the JSON decoder. (GH-7048) 2018-05-22 14:55:07 +03:00
Vlad Starostin a84d0b361a bpo-33263: Fix FD leak in _SelectorSocketTransport (GH-6450)
* bpo-33263 Fix FD leak in _SelectorSocketTransport. (GH-6450)

Under particular circumstances _SelectorSocketTransport can try to add a reader
even the transport is already being closed. This can lead to FD leak and
invalid stated of the following connections. Fixed the SelectorSocketTransport
to add the reader only if the trasport is still active.
2018-05-21 11:13:45 +03:00
Aaron Hall, MBA 4054b172ab bpo-26103: Fix inspect.isdatadescriptor() and a data descriptor definition. (GH-1959)
Look for '__set__' or '__delete__'.
2018-05-21 02:46:42 +03:00
Mario Corchero 89a25ce4fd bpo-33541: Remove unused __pad function (GH-4377)
Function was added with the initial implementation 00efe7e798.
2018-05-20 13:42:30 -04:00
CtrlZvi c66c342cb4 bpo-33542: Ignore DUID in uuid.get_node on Windows. (GH-6922)
uuid._ipconfig_getnode did not validate the maximum length of the value,
so long as the value had the same type of formatting as a MAC address.
This let it select DUIDs as MAC addresses. It now requires an exact
length match.
2018-05-20 18:03:25 +03:00
CtrlZvi 4151061855 bpo-26819: Prevent proactor double read on resume (#6921)
The proactor event loop has a race condition when reading with
pausing/resuming. `resume_reading()` unconditionally schedules the read
function to read from the current future. If `resume_reading()` was
called before the previously scheduled done callback fires, this results
in two attempts to get the data from the most recent read and an
assertion failure. This commit tracks whether or not `resume_reading`
needs to reschedule the callback to restart the loop, preventing a
second attempt to read the data.
2018-05-20 13:21:10 +03:00
Ivan Levkivskyi f65e31fee3
bpo-28556: Don't simplify unions at runtime (GH-6841) 2018-05-18 16:00:38 -07:00
Steve Dower e5f41d2f1e
bpo-33522: Enable CI builds on Visual Studio Team Services (#6865) 2018-05-16 17:50:29 -04:00
Nina Zakharenko 2d2d3b170b Fix ClassVar as string fails when getting type hints (GH-6824) 2018-05-16 12:27:03 -04:00
Petr Viktorin 8b94b41ab7
bpo-28167: Remove platform.linux_distribution (GH-6871)
* test_ssl: Remove skip_if_broken_ubuntu_ssl

We no longer support OpenSSL 0.9.8.15.15.

* bpo-28167: Remove platform.linux_distribution
2018-05-16 11:51:18 -04:00
Eric V. Smith 4e81296b18
bpo-33536: Validate make_dataclass() field names. (GH-6906) 2018-05-16 11:31:29 -04:00
Christopher Beacham 5db5c0669e bpo-21475: Support the Sitemap extension in robotparser (GH-6883) 2018-05-16 10:52:07 -04:00
Eric V. Smith 2a7bacbd91
bpo-33453: Handle string type annotations in dataclasses. (GH-6768) 2018-05-15 22:44:27 -04:00
Christian Heimes 67c4801663 bpo-32257: Add ssl.OP_NO_RENEGOTIATION (GH-5904)
The ssl module now contains OP_NO_RENEGOTIATION constant, available with
OpenSSL 1.1.0h or 1.1.1.

Note, OpenSSL 1.1.0h hasn't been released yet.

Signed-off-by: Christian Heimes <christian@python.org>
2018-05-15 16:25:40 -04:00
Matthias Bussonnier 9eb40bc38d bpo-28167: bump platform.linux_distribution removal to 3.8 (GH-6669)
Also bump PendingDeprecationWarning to DeprecationWarning.
2018-05-15 11:17:00 -04:00
Segev Finer 735abadd5b bpo-16865: Support arrays >=2GB in ctypes. (GH-3006) 2018-05-15 02:54:29 +03:00
Ned Deily d063b84d9e
bpo-33497: Add NEWS and ACKS entries. (GH-6838) 2018-05-14 18:15:15 -04:00
Eric V. Smith 2473eea65d
bpo-33494: Change dataclasses.Fields repr to use the repr of each of its members (GH-6798) 2018-05-14 11:37:28 -04:00
Michael Lazar bd08a0af2d bpo-32861: urllib.robotparser fix incomplete __str__ methods. (GH-5711)
The urllib.robotparser's __str__ representation now includes wildcard
entries and the "Crawl-delay" and "Request-rate" fields. Also removes extra
newlines that were being appended to the end of the string.
2018-05-14 17:10:41 +03:00
sblondon 8cf4b34b36 bpo-33311: Do not display parameters displayed in parentheses for module call. (GH-6677) 2018-05-09 12:39:32 +03:00
Ivan Levkivskyi 43d12a6bd8
bpo-28556: Minor fixes for typing module (GH-6732)
This also fixes https://bugs.python.org/issue33420
2018-05-09 02:23:46 +01:00
Serhiy Storchaka cedc9b7420
bpo-20087: Update locale alias mapping with glibc 2.27 supported locales. (ПР-6708) 2018-05-06 08:46:15 +03:00
Andrés Delfino b2043bbe60 bpo-33422: Fix quotation marks getting deleted when looking up byte/string literals on pydoc. (GH-6701)
Also update the list of string prefixes.
2018-05-05 19:07:32 +03:00
Antoine Pitrou 9d3627e311
bpo-33332: Add signal.valid_signals() (GH-6581) 2018-05-04 13:00:50 +02:00
Ned Deily 69a013ec18
bpo-33281: NEWS and ACK (GH-6681) 2018-05-01 22:41:43 -04:00
Serhiy Storchaka ef347535f2
bpo-20104: Improve error handling and fix a reference leak in os.posix_spawn(). (#6332) 2018-05-01 16:45:04 +03:00
Michael Selik 47ab15470d bpo-31908: Fix output of cover files for trace module command-line tool. (GH-4205)
Previously emitted cover files only when --missing option was used.
2018-05-01 06:46:52 +03:00
Brett Cannon 3ab9365dca
bpo-33254: do not return an empty list when asking for the contents of a namespace package (GH-6467) 2018-04-30 11:31:45 -07:00
sblondon 7d68bfa826 bpo-33256: Replace angle brackets around python object repr to display it in html (GH-6442) 2018-04-29 20:48:33 +03:00
Serhiy Storchaka 2e38cc3933
bpo-33383: Fix crash in get() of the dbm.ndbm database object. (#6630) 2018-04-29 12:38:06 +03:00
Cheryl Sabella 0250de4819 bpo-27485: Rename and deprecate undocumented functions in urllib.parse (GH-2205) 2018-04-25 16:51:54 -07:00
Serhiy Storchaka 57faf34887
bpo-33334: Support NOP and EXTENDED_ARG in dis.stack_effect(). (#6566)
Added tests to ensure that all defined opcodes are supported.
2018-04-25 22:04:06 +03:00
Chris Bradbury 1d4a733cce bpo-33251: Prevent ConfigParser.items returning items present in vars. (#6446)
* bpo-33251: ConfigParser.items no longer returns items present in vars.

Documentation for `ConfigParser.items()` states:
'Items present in vars no longer appear in the result.'

This fix aligns behaviour to that specified in the documentation.
2018-04-23 12:16:17 -07:00
Antoine Pitrou 25038ecfb6
bpo-33329: Fix multiprocessing regression on newer glibcs (GH-6575)
Starting with glibc 2.27.9000-xxx, sigaddset() can return EINVAL for some
reserved signal numbers between 1 and NSIG.  The `range(1, NSIG)` idiom
is commonly used to select all signals for blocking with `pthread_sigmask`.
So we ignore the sigaddset() return value until we expose sigfillset()
to provide a better idiom.
2018-04-23 20:53:33 +02:00
Berker Peksag d5a2377c3d
bpo-991266: Fix quoting of Comment attribute of SimpleCookie (GH-6555) 2018-04-23 02:48:11 +03:00
Paul Moore 0399cf9b5e
bpo-33131: Upgrade ensurepip to bundle pip 10.0.1 (GH-6546)
Upgrade ensurepip to bundle pip 10.0.1
2018-04-20 21:06:51 +01:00
Serhiy Storchaka e5362eaa75
bpo-33308: Fix a crash in the parser module when convert an ST object. (#6519)
Converting with line_info=False and col_info=True crashed before.
2018-04-19 01:55:37 +03:00
Aaron Ang c127a86e18 bpo-11594: Ensure line-endings are respected when using 2to3 (GH-6483) 2018-04-17 14:34:14 -07:00
Wolfgang Maier ba3a87aca3 bpo-33144: random.Random and subclasses: split _randbelow implementation (GH-6291) 2018-04-17 08:16:17 -07:00
Łukasz Langa 76618061b9
[lib2to3] Make grammar pickling faster (#6491)
* Now uses pickle protocol 4

* Doesn't wrap the grammar's `__dict__` in ordered dictionaries anymore as
dictionaries in Python 3.6+ are ordered by default

This still produces deterministic pickles (that hash the same with MD5).
Tested with different PYTHONHASHSEED values.
2018-04-16 17:33:59 -07:00
Łukasz Langa 2bea947628
acks += Zsolt Dollenstein (#6490) 2018-04-16 16:24:05 -07:00
Nick Coghlan 1a5c4bdb6e
bpo-33185: Improve wording and markup (GH-6477)
Adds some working and markup fixes that I missed
in the initial commit for this issue.

(Follow-up to GH-6419)
2018-04-15 23:32:05 +10:00
Nick Coghlan 82a9481059
bpo-33185: Fix regression in pydoc CLI sys.path handling (GH-6419)
The pydoc CLI assumed -m pydoc would add the empty string
to sys.path, and hence got confused when it switched to
adding the full initial working directory instead.

This refactors the pydoc CLI path manipulation to be
more testable, and ensures it won't accidentally
remove the standard library directory containing
pydoc itself from sys.path.
2018-04-15 21:52:57 +10:00
jdemeyer 23ab5ee667 bpo-33265: use an actual method instead of a method-like function in ExitStack (GH-6456)
`MethodType` has the exact semantics that `ExitStack` needs,
so we can avoid creating a Python level closure.
2018-04-13 22:22:46 +10:00
Mark Nemec c4b695f85e bpo-33097: Fix submit accepting callable after executor shutdown by interpreter exit (GH-6144)
Executors in concurrent.futures accepted tasks after executor was shutdown by interpreter exit. Tasks were left in PENDING state forever. This fix changes submit to instead raise a RuntimeError.
2018-04-10 19:23:14 +02:00
Antoine Pitrou e4679cd644
bpo-32759: Free unused arenas in multiprocessing.heap (GH-5827)
Large shared arrays allocated using multiprocessing would remain allocated
until the process ends.
2018-04-09 17:37:55 +02:00