Commit Graph

21589 Commits

Author SHA1 Message Date
Serhiy Storchaka 6543912c90
bpo-32912: Replace a DeprecationWarning with a SyntaxWarning (GH-9652)
for invalid escape sequences in string and bytes literals.
2018-10-19 17:42:06 +03:00
matthewbelisle-wf 209144831b bpo-34866: Adding max_num_fields to cgi.FieldStorage (GH-9660)
Adding `max_num_fields` to `cgi.FieldStorage` to make DOS attacks harder by
limiting the number of `MiniFieldStorage` objects created by `FieldStorage`.
2018-10-19 03:52:59 -07:00
Cheryl Sabella d16f012f84 bpo-31522: mailbox.get_string: pass `from_` parameter to `get_bytes` (#9857)
This allows *from_* to be successfully set to a non-default value when calling mbox.get_string.
2018-10-18 20:21:47 -04:00
Juliette Monsel 1deea5e539 bpo-34936: Fix TclError in tkinter.Spinbox.selection_element(). (GH-9760) 2018-10-18 22:28:31 +03:00
stratakis b7ad31c892 bpo-34765: Update the install-sh file (GH-9592)
Update the outdated install-sh file to the latest
revision from automake v1.16.1
2018-10-18 10:51:53 +02:00
Serhiy Storchaka 6f906b3d72
bpo-35008: Fix possible leaks in Element.__setstate__(). (GH-9924)
C implementation of xml.etree.ElementTree.Element.__setstate__()
leaked references to children when called for already initialized
element.
2018-10-18 09:49:54 +03:00
Gregory P. Smith 9d4712bc8f
bpo-35011: Restore use of pyexpatns.h in libexpat (GH-9939)
Restores the use of pyexpatns.h to isolate our embedded copy of the expat C
library so that its symbols do not conflict at link or dynamic loading time
with an embedding application or other extension modules with their own
version of libexpat.

5dc3f23b5f (diff-3afaf7274c90ce1b7405f75ad825f545) inadvertently removed it when upgrading expat.
2018-10-17 18:10:46 -07:00
Stéphane Wirtel 74a8b6ea7e bpo-24658: Fix read/write greater than 2 GiB on macOS (GH-1705)
On macOS, fix reading from and writing into a file with a size larger than 2 GiB.
2018-10-18 01:05:04 +02:00
Stéphane Wirtel fcd5e84a51 bpo-23420: Verify the value of '-s' when execute the CLI of cProfile (GH-9925)
Verify the value for the parameter '-s' of the cProfile CLI. Patch by Robert
Kuska.

Co-authored-by: Robert Kuska <rkuska@gmail.com>
2018-10-17 12:03:40 +02:00
Victor Stinner 2e438cc255
bpo-34989: python-gdb.py: fix current_line_num() (GH-9889)
python-gdb.py now handles errors on computing the line number
of a Python frame.

Changes:

* PyFrameObjectPtr.current_line_num() now catchs any Exception on
  calling addr2line(), instead of failing with a surprising "<class
  'TypeError'> 'FakeRepr' object is not subscriptable" error.
* All callers of current_line_num() now handle current_line_num()
  returning None.
* PyFrameObjectPtr.current_line() now also catchs IndexError on
  getting a line from the Python source file.
2018-10-15 23:19:57 +02:00
BNMetrics 18fb1fb943 bpo-34844: logging.Formatter enhancement - Ensure style and format string matches in logging.Formatter (GH-9703) 2018-10-15 19:41:36 +01:00
Serhiy Storchaka e890421e33
bpo-34974: Do not replace unexpected errors in bytes() and bytearray(). (GH-9852)
bytes and bytearray constructors converted unexpected exceptions
(e.g. MemoryError and KeyboardInterrupt) to TypeError.
2018-10-15 00:02:57 +03:00
Pablo Galindo de2aea0ff0
bpo-34939: Allow annotated global names in module namespace (GH-9844)
Allow annotated global names in the module namespace after the symbol is
declared as global. Previously, only symbols annotated before they are declared
as global (i.e. inside a function) were allowed. This change allows symbols to be
declared as global before the annotation happens in the global scope.
2018-10-14 18:01:03 +01:00
Pablo Galindo bd036d3d15
bpo-34521: Add NEWS entry for changes in GH-9613 (GH-9850)
* Add News entry for the change in multiprocessing.reduction.recvfds
made in GH-9613.
2018-10-14 17:41:11 +01:00
Serhiy Storchaka b11c5667f9
bpo-34941: Fix searching Element subclasses. (GH-9766)
Methods find(), findtext() and findall() of xml.etree.ElementTree.Element
were not able to find chldren which are instances of Element subclasses.
2018-10-14 10:32:19 +03:00
Andrew Svetlov 97cf082872 bpo-34970: Protect tasks weak set manipulation in asyncio.all_tasks() (GH-9837)
https://bugs.python.org/issue34970
2018-10-13 11:12:40 -07:00
Zackery Spytz d4d60134b2 bpo-16965: 2to3 now rewrites execfile() to open with 'rb'. (GH-8569) 2018-10-13 12:27:30 +03:00
Zackery Spytz 0461704060 bpo-22872: multiprocessing.Queue's put() and get() now raise ValueError if the queue is closed. (GH-9010)
Previously, put() and get() would raise AssertionError and OSError,
respectively.
2018-10-13 12:26:09 +03:00
Stéphane Wirtel e385d0661e bpo-34967: Sphinx is deprecating add_description_unit, use add_object_type (GH-9827) 2018-10-13 08:14:08 +02:00
Juliette Monsel bf034715db bpo-23831: Add moveto method to the tkinter.Canvas widget. (GH-9768) 2018-10-12 19:44:10 +03:00
Cheryl Sabella 2d6097d027 bpo-11233: Create availability directive for documentation (GH-9692)
Replace "Availability: xxx" with ".. availability:: xxx" in the doc.
Original patch by Georg Brandl.

Co-Authored-By: Georg Brandl <georg@python.org>
2018-10-12 16:55:20 +02:00
Bruno Oliveira da2bf9f66d bpo-34900: Make TestCase.debug() work with subtests (GH-9707) 2018-10-12 13:35:55 +03:00
Stéphane Wirtel 859c068e52 bpo-34962: make doctest in Doc/ now passes, and is enforced in CI (GH-9806) 2018-10-12 09:51:05 +02:00
Serhiy Storchaka 9b8c2e7676
bpo-34922: Fix integer overflow in the digest() and hexdigest() methods (GH-9751)
for the SHAKE algorithm in the hashlib module.
2018-10-11 07:41:00 +03:00
Elvis Pranskevichus a6b3ec5b6d bpo-34022: Stop forcing of hash-based invalidation with SOURCE_DATE_EPOCH (GH-9607)
Unconditional forcing of ``CHECKED_HASH`` invalidation was introduced in
3.7.0 in bpo-29708.  The change is bad, as it unconditionally overrides
*invalidation_mode*, even if it was passed as an explicit argument to
``py_compile.compile()`` or ``compileall``.  An environment variable
should *never* override an explicit argument to a library function.
That change leads to multiple test failures if the ``SOURCE_DATE_EPOCH``
environment variable is set.

This changes ``py_compile.compile()`` to only look at
``SOURCE_DATE_EPOCH`` if no explicit *invalidation_mode* was specified.
I also made various relevant tests run with explicit control over the
value of ``SOURCE_DATE_EPOCH``.

While looking at this, I noticed that ``zipimport`` does not work
with hash-based .pycs _at all_, though I left the fixes for
subsequent commits.
2018-10-10 18:43:14 +02:00
Mayank Asthana 7e18deef65 bpo-34926: Make mimetypes.guess_type accept os.PathLike objects (GH-9777)
:meth:`mimetypes.MimeTypes.guess_type` now accepts :term:`path-like object` in addition to url strings.
2018-10-10 16:46:44 +02:00
Stéphane Wirtel 7c817e620b bpo-34913: Document gzip command line interface (GH-9782) 2018-10-10 08:28:26 +02:00
Stéphane Wirtel e8bbc52deb bpo-23596: Use argparse for the command line of gzip (GH-9781)
Co-authored-by: Antony Lee <anntzer.lee@gmail.com>
2018-10-10 00:41:33 +02:00
twisteroid ambassador c880ffe7d2 bpo-34769: Thread safety for _asyncgen_finalizer_hook(). (GH-9716) 2018-10-09 11:30:21 -04:00
Victor Stinner 79d21331e6
bpo-32962: Fix test_gdb failure in debug build with -mcet -fcf-protection -O0 (GH-9656)
When Python is built with the intel control-flow protection flags,
-mcet -fcf-protection, gdb is not able to read the stack without
actually jumping inside the function. This means an extra
'next' command is required to make the $pc (program counter)
enter the function and make the stack of the function exposed to gdb.

Co-Authored-By: Marcel Plch <gmarcel.plch@gmail.com>

(cherry picked from commit 9b7c74ca32)
2018-10-09 16:54:04 +02:00
Romuald Brunet 7b31397180 bpo-32680 add default "sock" on SMTP objects (#5345)
By default the smtplib.SMTP objects did not have a sock attribute, it
was only created during connect()
2018-10-09 16:31:55 +02:00
travisoneill 199a280af5 bpo-34758: add .wasm to recognized file extensions in mimetypes module (GH-9464) 2018-10-09 07:43:58 +03:00
animalize 6261ae9b01 bpo-32174: Let .chm document display non-ASCII characters properly (GH-9758)
Let .chm document display non-ASCII characters properly

Escape the `body` part of .chm source file to 7-bit ASCII, to fix visual effect on some MBCS Windows systems.
2018-10-08 14:20:54 -07:00
Bradley Meck 0854b92cd2 bpo-31715 Add mimetype for extension .mjs (#3908) 2018-10-08 23:04:55 +03:00
Paul Bailey 4c33997057 bpo-34911: Added support for secure websocket cookies (GH-9734) 2018-10-08 21:49:29 +03:00
Juliette Monsel af5658ae93 bpo-34829: Add missing selection_ methods to the Tkinter Spinbox. (GH-9617)
Implement the methods selection_from(), selection_range(), selection_present()
and selection_to() for Tkinter Spinbox.
2018-10-08 19:29:24 +03:00
Raymond Hettinger de2e448414
bpo-34925: Optimize common case for bisect() argument parsing (#9753) 2018-10-08 08:02:41 -07:00
Zackery Spytz 365ad2ead5 bpo-34824: Fix a possible NULL pointer dereference in _ssl.c (GH-9606)
On failure, _PyBytes_Resize() will deallocate the bytes object and set
"result" to NULL.


https://bugs.python.org/issue34824
2018-10-06 10:41:45 -07:00
Stéphane Wirtel 683281f536 bpo-34906: Doc: Fix typos (2) (GH-9735)
Fix typos
2018-10-06 16:35:53 +02:00
Zackery Spytz ae62f01524 bpo-34910: Ensure that PyObject_Print() always returns -1 on error. (GH-9733) 2018-10-06 09:44:25 +03:00
E. M. Bray 7a7693e9cb bpo-28441: Ensure `.exe` suffix in `sys.executable` on MinGW and Cygwin (GH-4348)
This is needed to even the run the test suite on buildbots for affected platforms; e.g.:

```
./python.exe  ./Tools/scripts/run_tests.py -j 1 -u all -W --slowest --fail-env-changed --timeout=11700 -j2
/home/embray/src/python/test-worker/3.x.test-worker/build/python -u -W default -bb -E -W error::BytesWarning -m test -r -w -j 1 -u all -W --slowest --fail-env-changed --timeout=11700 -j2
Traceback (most recent call last):
  File "./Tools/scripts/run_tests.py", line 56, in <module>
    main(sys.argv[1:])
  File "./Tools/scripts/run_tests.py", line 52, in main
    os.execv(sys.executable, args)
PermissionError: [Errno 13] Permission denied
make: *** [Makefile:1073: buildbottest] Error 1
```
2018-10-05 20:38:50 +09:00
INADA Naoki 6f85b826b5 bpo-34871: inspect: Don't pollute sys.modules (GH-9696)
https://bugs.python.org/issue34871
2018-10-04 09:47:09 -07:00
Elvis Pranskevichus 0c797a6aca bpo-34872: Fix self-cancellation in C implementation of asyncio.Task (GH-9679)
The C implementation of asyncio.Task currently fails to perform the
cancellation cleanup correctly in the following scenario.

    async def task1():
        async def task2():
            await task3     # task3 is never cancelled

        asyncio.current_task().cancel()
        await asyncio.create_task(task2())

The actuall error is a hardcoded call to `future_cancel()` instead of
calling the `cancel()` method of a future-like object.

Thanks to Vladimir Matveev for noticing the code discrepancy and to
Yury Selivanov for coming up with a pathological scenario.
2018-10-03 10:30:31 -04:00
Zackery Spytz 96c5932794 bpo-34879: Fix a possible null pointer dereference in bytesobject.c (GH-9683)
formatfloat() was not checking if PyBytes_FromStringAndSize()
failed, which could lead to a null pointer dereference in
_PyBytes_FormatEx().
2018-10-03 09:01:30 +03:00
tzickel 97bfe8d3eb bpo-34172: multiprocessing.Pool leaks resources after being deleted (GH-8450)
Fix a reference issue inside multiprocessing.Pool that caused the pool to remain alive if it was deleted without being closed or terminated explicitly.
2018-10-02 23:01:23 +02:00
Victor Stinner e972c13624
bpo-30156: Remove property_descr_get() optimization (GH-9541)
property_descr_get() uses a "cached" tuple to optimize function
calls. But this tuple can be discovered in debug mode with
sys.getobjects(). Remove the optimization, it's not really worth it
and it causes 3 different crashes last years.

Microbenchmark:

./python -m perf timeit -v \
    -s "from collections import namedtuple; P = namedtuple('P', 'x y'); p = P(1, 2)" \
    --duplicate 1024 "p.x"

Result:

Mean +- std dev: [ref] 32.8 ns +- 0.8 ns -> [patch] 40.4 ns +- 1.3 ns: 1.23x slower (+23%)
2018-10-01 03:03:22 -07:00
Serhiy Storchaka 2a2940e5c3
bpo-34854: Fix compiling string annotations containing lambdas. (GH-9645)
* Compiling a string annotation containing a lambda with keyword-only
argument without default value caused a crash.

* Remove the final "*" (it is incorrect syntax) in the representation of
lambda without *args and keyword-only arguments when compile from AST.

* Improve the representation of lambda without arguments.
2018-09-30 21:07:05 +03:00
Andrew Svetlov d5bd036138 bpo-34849: Don't log wating for selector.select in asyncio loop iteration (GH-9641)
The waiting is pretty normal for any asyncio program, logging its time just adds
a noise to logs without any useful information provided.



https://bugs.python.org/issue34849
2018-09-29 22:28:40 -07:00
Zsolt Cserna 9df346bf98 bpo-34248: Add filename to error raised in {gnu,ndbm}.open() (GH-8590)
Report the filename to the exception when raising {gdbm,dbm.ndbm}.error in
dbm.gnu.open() and dbm.ndbm.open() functions, so it gets printed when the
exception is raised, and can also be obtained by the filename attribute of the
exception object.
2018-09-27 22:54:34 +03:00
Serhiy Storchaka 3f22811fef
bpo-32892: Use ast.Constant instead of specific constant AST types. (GH-9445) 2018-09-27 17:42:37 +03:00
orlnub123 a94ee12c26 bpo-34819: Use a monotonic clock to compute timeouts in concurrent.futures (GH-9599)
Use a monotonic clock to compute timeouts in :meth:`Executor.map` and :func:`as_completed`, in order to prevent timeouts from deviating when the system clock is adjusted.

This may not be sufficient on all systems. On POSIX for example, the actual waiting (e.g. in ``sem_timedwait``) is specified to rely on the CLOCK_REALTIME clock.
2018-09-27 13:16:26 +02:00
Tal Einat f55c64c632
bpo-31425: fix versionadded in docs and add attribution in NEWS (GH-9595) 2018-09-27 00:20:38 +03:00
Bjorn Andersson bb8165172a bpo-31425: Expose AF_QIPCRTR in socket module (GH-3706)
The AF_QIPCRTR address family was introduced in Linux v4.7.

Co-authored-by: Bjorn Andersson <bjorn.andersson@linaro.org>
2018-09-26 16:47:52 +03:00
INADA Naoki 2aaf98c16a bpo-34320: Fix dict(o) didn't copy order of dict subclass (GH-8624)
When dict subclass overrides order (`__iter__()`, `keys()`, and `items()`), `dict(o)`
should use it instead of dict ordering.


https://bugs.python.org/issue34320
2018-09-25 20:59:00 -07:00
Cheryl Sabella d345bb4d9b bpo-34334: Don't log traceback twice in QueueHandler (GH-9537) 2018-09-26 00:00:08 +01:00
Zackery Spytz 5a5ce064b3 bpo-5950: Support reading zips with comments in zipimport (#9548)
* bpo-5950: Support reading zips with comments in zipimport
2018-09-25 15:15:47 -04:00
Yury Selivanov 996859a90d
bpo-34790: [docs] Passing coroutines to asyncio.wait() can be confusing. (GH-9543) 2018-09-25 14:51:21 -04:00
Victor Stinner 6ea29c5e90 bpo-34687: Make asynico use ProactorEventLoop by default (GH-9538) 2018-09-25 11:27:08 -04:00
Joe Pamer c8c0249c9e bpo-32557: allow shutil.disk_usage to take a file path on Windows also (GH-9372)
https://bugs.python.org/issue32557
2018-09-25 07:57:36 -07:00
Tal Einat 604e7b9931
bpo-1529353: IDLE: squeeze large output in the shell (GH-7626) 2018-09-25 15:10:14 +03:00
Zackery Spytz f6c8007a29 bpo-34770: Fix a possible null pointer dereference in pyshellext.cpp (GH-9497)
The GlobalLock() call in UpdateDropDescription() was not checked for
failure.



https://bugs.python.org/issue34770
2018-09-24 21:25:23 -07:00
Ammar Askar 025eb98dc0 bpo-34683: Make SyntaxError column offsets consistently 1-indexed (gh-9338)
Also point to start of tokens in parsing errors.

Fixes bpo-34683
2018-09-24 14:12:49 -07:00
Christian Heimes 223e501fb9 bpo-34791: xml package obeys ignore env flags (GH-9544)
The xml.sax and xml.dom.domreg modules now obey
sys.flags.ignore_environment.

Signed-off-by: Christian Heimes <christian@python.org>



https://bugs.python.org/issue34791
2018-09-24 10:21:12 -07:00
João Júnior 558c49bcf3 bpo-34728: Remove deprecate *loop* argument in asyncio.sleep (GH-9415)
* Insert the warn in the asyncio.sleep when the loop argument is used

* Insert the warn in the asyncio.wait and asyncio.wait_for when the loop argument is used

* Better format of the code

* Add news file

* change calls for get_event_loop() to calls for get_running_loop()

* Change message to be more clear in News

* Improve the comments in test_tasks
2018-09-24 05:51:22 -04:00
Tim Hoffmann a0fd7f1b55 Migrate datetime.date.fromtimestamp to Argument Clinic (GH-8535) 2018-09-24 10:39:02 +02:00
Lisa Roach 9718b59ee5
bpo-34659: Adds initial kwarg to itertools.accumulate() (GH-9345) 2018-09-23 17:34:59 -07:00
Tal Einat c87d9f406b bpo-34548: IDLE: use configured theme colors in TextView (GH-9008)
https://bugs.python.org/issue34548
2018-09-23 05:23:15 -07:00
Christian Heimes 17b1d5d4e3 bpo-17239: Disable external entities in SAX parser (GH-9217)
The SAX parser no longer processes general external entities by default
to increase security. Before, the parser created network connections
to fetch remote files or loaded local files from the file system for DTD
and entities.

Signed-off-by: Christian Heimes <christian@python.org>



https://bugs.python.org/issue17239
2018-09-23 00:50:25 -07:00
Christian Heimes 9fb051f032 bpo-34670: Add TLS 1.3 post handshake auth (GH-9460)
Add SSLContext.post_handshake_auth and
SSLSocket.verify_client_post_handshake for TLS 1.3 post-handshake
authentication.

Signed-off-by: Christian Heimes <christian@python.org>q


https://bugs.python.org/issue34670
2018-09-22 23:32:31 -07:00
jChapman 8fabae3b00 bpo-32117: Iterable unpacking in return and yield documentation (GH-9487)
News entry clean up, added to what's new

Requested by @gvanrossum in https://github.com/python/cpython/pull/4509 


https://bugs.python.org/issue32117
2018-09-22 18:13:10 -07:00
Christian Heimes 026337a710 bpo-34623: Mention CVE-2018-14647 in news entry (GH-9482)
https://bugs.python.org/issue34623
2018-09-21 21:42:29 -07:00
Ethan Furman 5bdab641da
bpo-29577: Enum: mixin classes don't mix well with already mixed Enums (GH-9328)
* bpo-29577: allow multiple mixin classes
2018-09-21 19:03:09 -07:00
David Cuthbert fd97d1f1af bpo-32117: Allow tuple unpacking in return and yield statements (gh-4509)
Iterable unpacking is now allowed without parentheses in yield and return
statements, e.g. ``yield 1, 2, 3, *rest``. Thanks to David Cuthbert for the
change and jChapman for added tests.
2018-09-21 18:31:15 -07:00
Elvis Pranskevichus 7279b5125e bpo-34537: Fix test_gdb:test_strings with LC_ALL=C (GH-9483)
We cannot simply call locale.getpreferredencoding() here,
as GDB might have been linked against a different version
of Python with a different encoding and coercion policy
with respect to PEP 538 and PEP 540.

Thanks to Victor Stinner for a hint on how to fix this.
2018-09-21 18:13:16 -07:00
Brett Cannon d64ee1a5ba bpo-32718: Make Activate.ps1 for venv cross-platform and available on all platforms (GH-9321)
PowerShell Core 6.1 is the cross-platform port of Windows PowerShell. This change updates Activate.ps1 to not make Windows assumptions as well as installing it into the bin/Scripts directory on all operating systems.

Requires PowerShell Core 6.1 for proper readline support once the shell has been activated for the virtual environment.
2018-09-21 15:27:26 -07:00
Yury Selivanov 2ec872b31e
bpo-34762: Fix contextvars C API to use PyObject* pointer types. (GH-9473) 2018-09-21 15:33:56 -04:00
Zackery Spytz 91e6c8717b bpo-34735: Fix a memory leak in Modules/timemodule.c (GH-9418)
There was a missing PyMem_Free(format) in time_strftime().
2018-09-21 09:09:48 +03:00
Berker Peksag 8d1e190fc5
bpo-32215: Fix performance regression in sqlite3 (GH-8511) 2018-09-20 14:10:49 +03:00
Victor Stinner 06e7608207
Revert "bpo-34589: Add -X coerce_c_locale command line option (GH-9378)" (GH-9430)
* Revert "bpo-34589: Add -X coerce_c_locale command line option (GH-9378)"

This reverts commit dbdee0073c.

* Revert "bpo-34589: C locale coercion off by default (GH-9073)"

This reverts commit 7a0791b699.

* Revert "bpo-34589: Make _PyCoreConfig.coerce_c_locale private (GH-9371)"

This reverts commit 188ebfa475.
2018-09-19 14:56:36 -07:00
jdemeyer e89de73987 bpo-34125: Enable profiling of method_descriptor in all cases (GH-8416)
`list.append([], None)` was profiled but `list.append([], None, **{})` was not profiled.
Enable profiling for later case.

https://bugs.python.org/issue34125
2018-09-19 03:06:20 -07:00
Ross Burton 2a9c3805dd closes bpo-34585: Don't do runtime test to get float byte order. (GH-9085)
Currently configure.ac uses AC_RUN_IFELSE to determine the byte order of doubles, but this silently fails under cross compilation and Python doesn't do floats properly.

Instead, steal a macro from autoconf-archive which compiles code using magic doubles (which encode to ASCII) and grep for the representation in the binary.

RFC because this doesn't yet handle the weird ancient ARMv4 OABI 'mixed-endian' encoding properly. This encoding is ancient and I don't believe the union of "Python 3.8 users" and "OABI users" has anything in. Should the support for this just be dropped too? Alternatively, someone will need to find an OABI toolchain to verify the encoding of the magic double.
2018-09-18 23:25:48 -07:00
Serhiy Storchaka 79d1c2e6c9
bpo-25711: Rewrite zipimport in pure Python. (GH-6809) 2018-09-18 22:22:29 +03:00
Silas Sewell 4ba3b50bfe bpo-34472: Add data descriptor signature to zipfile (GH-8871)
This makes streamed zips compatible with MacOS Archive Utility and
other applications.
2018-09-18 20:00:05 +03:00
Steve Dower d0f49d2f50
bpo-34582: Adds JUnit XML output for regression tests (GH-9210) 2018-09-18 09:10:26 -07:00
Christian Heimes cb5778f00c bpo-34623: Use XML_SetHashSalt in _elementtree (GH-9146)
The C accelerated _elementtree module now initializes hash randomization
salt from _Py_HashSecret instead of libexpat's default CPRNG.

Signed-off-by: Christian Heimes <christian@python.org>



https://bugs.python.org/issue34623
2018-09-18 05:38:58 -07:00
Serhiy Storchaka 0185f34ddc
bpo-33721: Make some os.path functions and pathlib.Path methods be tolerant to invalid paths. (#7695)
Such functions as os.path.exists(), os.path.lexists(), os.path.isdir(),
os.path.isfile(), os.path.islink(), and os.path.ismount() now return False
instead of raising ValueError or its subclasses UnicodeEncodeError
and UnicodeDecodeError for paths that contain characters or bytes
unrepresentative at the OS level.
2018-09-18 11:28:51 +03:00
Serhiy Storchaka 7bdf28265a
bpo-32455: Add jump parameter to dis.stack_effect(). (GH-6610)
Add C API function PyCompile_OpcodeStackEffectWithJump().
2018-09-18 09:54:26 +03:00
Victor Stinner dbdee0073c
bpo-34589: Add -X coerce_c_locale command line option (GH-9378)
Add a new -X coerce_c_locale command line option to control C locale
coercion (PEP 538).
2018-09-17 17:19:26 -07:00
Victor Stinner 7a0791b699
bpo-34589: C locale coercion off by default (GH-9073)
Py_Initialize() and Py_Main() cannot enable the C locale coercion
(PEP 538) anymore: it is always disabled. It can now only be enabled
by the Python program ("python3).

test_embed: get_filesystem_encoding() doesn't have to set PYTHONUTF8
nor PYTHONCOERCECLOCALE, these variables are already set in the
parent.
2018-09-17 16:22:29 -07:00
Victor Stinner 7484bdfd1e
bpo-34587, test_socket: remove RDSTest.testCongestion() (GH-9277)
The test tries to fill the receiver's socket buffer and expects an
error. But the RDS protocol doesn't require that. Moreover, the Linux
implementation of RDS expects that the producer of the messages
reduces its rate, it's not the role of the receiver to trigger an
error.

The test fails on Fedora 28 by design, so remove it.
2018-09-17 14:01:20 -07:00
Steve Dower c6fd1c1c3a
bpo-32533: Fixed thread-safety of error handling in _ssl. (GH-7158) 2018-09-17 11:34:47 -07:00
Serhiy Storchaka 9bdb7be482
bpo-34341: Fix appending to ZIP archives with the ZIP64 extension. (GH-8683) 2018-09-17 15:36:40 +03:00
Serhiy Storchaka da8d72c953
bpo-12458: Fix line numbers for multiline expressions. (GH-8774) 2018-09-17 15:17:29 +03:00
Alexandru Ardelean b3a271fc0c bpo-34710: fix SSL module build (GH-9347)
Include ``openssl/dh.h`` header file to fix implicit function declaration of ``DH_free()``.

Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
2018-09-17 04:53:31 -07:00
Serhiy Storchaka e0e5065dae
bpo-34610: Fixed iterator of multiprocessing.managers.DictProxy. (GH-9113) 2018-09-17 14:24:01 +03:00
Vladimir Matveev 7843caeb90 bpo-34603, ctypes/libffi_msvc: Fix returning structs from functions (GH-9258) 2018-09-15 22:36:29 -07:00
Monson Shao 10a428b64b closes bpo-34515: Support non-ASCII identifiers in lib2to3. (GH-8950) 2018-09-15 10:32:29 -07:00
Michael Felt d2067318c7 bpo-34579: Fix test_embed DEFAULT_CON AIX (GH-9063)
* Modify DEFAULT_CONFIG for AIX

* bedevere/news did not like old name

* Modify NEWS entry

* Modified per peer review

* Define and use NULL_STR constant to account for AIX libc behavior

* Modify per peer review

* Modify NEWS
2018-09-15 02:28:31 -07:00
Eric Snow 5903296045
bpo-34651: Only allow the main interpreter to fork. (gh-9279)
When os.fork() is called (on platforms that support it) all threads but the current one are destroyed in the child process. Consequently we must ensure that all but the associated interpreter are likewise destroyed. The main interpreter is critical for runtime operation, so we must ensure that fork only happens in the main interpreter.

https://bugs.python.org/issue34651
2018-09-14 14:17:20 -07:00
Benjamin Peterson 5633c4f342
bpo-34672: Try to pass the C library's own timezone strings back to it. (GH-9288) 2018-09-14 09:09:04 -07:00
Eric V. Smith 9b9d97dd13
bpo-34363: dataclasses.asdict() and .astuple() now handle fields which are namedtuples. (GH-9151) 2018-09-14 11:32:16 -04:00
Lisa Roach 5ac704306f bpo-33073: Adding as_integer_ratio to ints. (GH-8750) 2018-09-13 23:56:23 -07:00
Gregory P. Smith 19003841e9
bpo-6721: Hold logging locks across fork() (GH-4071)
bpo-6721: When os.fork() was called while another thread holds a logging lock, the child process may deadlock when it tries to log.  This fixes that by acquiring all logging locks before fork and releasing them afterwards.

A regression test that fails before this change is included.

Within the new unittest itself: There is a small _potential_ due to mixing of fork and a thread in the child process if the parent's thread happened to hold a non-reentrant library call lock (malloc?) when the os.fork() happens.  buildbots and time will tell if this actually manifests itself in this test or not.  :/  A functionality test that avoids that would be a challenge.

An alternate test that isn't trying to produce the deadlock itself but just checking that the release and acquire calls are made would be the next best alternative if so.
2018-09-13 22:08:31 -07:00
Windson yang 1aeba7458d bpo-34552: Clarify built-in types comparisons (GH-9035)
Some updates to ancient text about comparisons; fixes bp-34552.
2018-09-13 21:50:18 -07:00
Andrew Svetlov 11194c877c
bpo-34666: Implement stream.awrite() and stream.aclose() (GH-9274) 2018-09-13 16:53:49 -07:00
Michael Felt 413118ebf3 Fix test_asyncio for AIX - do not call transport.get_extra_info('sockname') (#8907) 2018-09-13 16:35:56 -07:00
Benjamin Peterson a710ebd21b
closes bpo-34661: Fix test_shutil if unzip doesn't support -t. (GH-9262) 2018-09-13 10:08:46 -07:00
Eric V. Smith 53c427e839 bpo-34653: Removed unused function PyParser_SimpleParseStringFilename. (GH-9260)
This function was not in any .h file and was not used by Python, so removing it is safe.


https://bugs.python.org/issue34653
2018-09-13 09:34:55 -07:00
Gregory P. Smith a20b6adb5a bpo-34658: Fix rare subprocess prexec_fn fork error. (GH-9255)
[bpo-34658](https://www.bugs.python.org/issue34658): Fix a rare interpreter unhandled exception state SystemError only
seen when using subprocess with a preexec_fn while an after_parent handler has
been registered with os.register_at_fork and the fork system call fails.



https://bugs.python.org/issue34658
2018-09-13 04:30:10 -07:00
Gregory P. Smith 4ae8ece5cd bpo-34200: Fix non-determinism of test_pkg (GH-9248)
This causes the tearDown code to only unimport the test modules specifically created as part of each test via the self.mkhier method rather than abusing test.support.modules_setup() and the scary test.support.modules_cleanup() code.


https://bugs.python.org/issue34200
2018-09-12 17:58:40 -07:00
Benjamin Peterson c9a71dd223
closes bpo-34641: Further restrict the LHS of keyword argument function call syntax. (GH-9212) 2018-09-12 17:14:39 -07:00
Benjamin Peterson 40caa05fa4
closes bpo-34652: Always disable lchmod on Linux. (GH-9234) 2018-09-12 15:52:40 -07:00
Tony Flury 2087023fde bpo-32933: Implement __iter__ method on mock_open() (GH-5974) 2018-09-13 01:21:16 +03:00
Andrew Svetlov 0dd71807a9
bpo-34630: Skip logging SSL certificate errors by asyncio code (GH-9169) 2018-09-12 14:03:54 -07:00
Oren Milman 0bd1a2dcfd bpo-31577: Fix a crash in os.utime() in case of a bad ns argument. (GH-3752) 2018-09-12 22:14:35 +03:00
Ethan Furman 019f0a0cb8
bpo-34536: raise error for invalid _missing_ results (GH-9147)
* raise exception if _missing_ returns None or invalid type
2018-09-12 11:43:34 -07:00
Andrew Svetlov a5d1eb8d8b
bpo-34638: Store a weak reference to stream reader to break strong references loop (GH-9201)
Store a weak reference to stream readerfor breaking strong references

It breaks the strong reference loop between reader and protocol and allows to detect and close the socket if the stream is deleted (garbage collected)
2018-09-12 11:43:04 -07:00
Raymond Hettinger 9dfa0fe587
bpo-34637: Make the *start* argument for *sum()* visible as a keyword argument. (GH-9208) 2018-09-12 10:54:06 -07:00
orlnub123 0fb9fadd3b bpo-34282: Fix Enum._convert shadowing members named _convert (GH-8568)
* Fix enum members getting shadowed by parent attributes
* Move Enum._convert to EnumMeta._convert_
* Deprecate _convert
2018-09-12 10:28:53 -07:00
Ethan Furman f52237400b bpo-33437: add __new__ vs __init__ example (GH-9145)
Improve Enum docs.


https://bugs.python.org/issue33437
2018-09-12 10:00:30 -07:00
Cheryl Sabella 731ff68eee closes bpo-25041: Document AF_PACKET socket address format. (GH-4092) 2018-09-11 17:32:15 -07:00
Sergey Fedoseev 6c7d67ce83 bpo-1621: Avoid signed integer overflow in set_table_resize(). (GH-9059)
Address a C undefined behavior signed integer overflow issue in set object table resizing.  Our -fwrapv compiler flag and practical reasons why sets are unlikely to get this large should mean this was never an issue but it was incorrect code that generates code analysis warnings.

<!-- issue-number: [bpo-1621](https://www.bugs.python.org/issue1621) -->
https://bugs.python.org/issue1621
<!-- /issue-number -->
2018-09-11 16:18:01 -07:00
Max Bélanger 4859ba0d2c closes bpo-31903: Release the GIL when calling into SystemConfiguration (GH-4178) 2018-09-11 16:14:00 -07:00
Sergey Fedoseev ec014a101a bpo-34636: Use fast path for more chars in SRE category macros. (GH-9170)
When handling \s, \d, or \w (and their inverse) escapes in bytes regexes this a small but measurable performance improvement.

<!-- issue-number: [bpo-34636](https://www.bugs.python.org/issue34636) -->
https://bugs.python.org/issue34636
<!-- /issue-number -->
2018-09-11 15:47:59 -07:00
Victor Stinner 998b806366
Revert "bpo-34595: Add %T format to PyUnicode_FromFormatV() (GH-9080)" (GH-9187)
This reverts commit 886483e2b9.
2018-09-12 00:23:25 +02:00
guoci 90fc8980bb closes bpo-31902: Fix the col_offset attribute for ast.Async* nodes to point to the "async" keyword. (GH-4175)
Previously, col_offset points to the keyword after "async".
2018-09-11 14:45:45 -07:00
Oren Milman 24bd50bdcc closes bpo-31608: Fix a crash in methods of a subclass of _collections.deque with a bad __new__(). (GH-3788) 2018-09-11 11:46:55 -07:00
wim glenn 08bcf647d8 bpo-28617 Fixed docs inaccuracies about the types that support membership tests (GH-9086)
<!-- issue-number: [bpo-28617](https://www.bugs.python.org/issue28617) -->
https://bugs.python.org/issue28617
<!-- /issue-number -->
2018-09-11 10:44:52 -07:00
Andrew Svetlov 0baa72f4b2
bpo-34622: Extract asyncio exceptions into a separate module (GH-9141) 2018-09-11 10:13:04 -07:00
Oren Milman 735171e334 closes bpo-29832: Remove "getsockaddrarg" from error messages. (GH-3163) 2018-09-11 09:51:29 -07:00
Zackery Spytz 73870bfeb9 closes bpo-32490: Fix filename duplication in subprocess exception message. (GH-9163)
8621bb5d93 sets the filename in directly in the FileNotFoundError, so we may revert the earlier fix 5f78040057.
2018-09-11 08:54:07 -07:00
Ned Deily 3102e24d83
bpo-34405: Update to OpenSSL 1.1.0i for macOS installer builds (GH-9166) 2018-09-11 08:52:40 -07:00
Benjamin Peterson 5033aa77aa
bpo-34625: Update vendorized expat version to 2.2.6. (GH-9150) 2018-09-10 21:04:00 -07:00
Matthias Bussonnier ffa198c642 bpo-33487: improve BZ2File Deprecation and documentation. (GH-6785)
Emit warning when None passed explicitly, list Python version since
deprecation in warning message and docs.
2018-09-10 18:15:56 -07:00
Lew Kurtz f019579828 bpo-33460: remove ellipsis that look like continuation prompts (GH-7851)
Remove ellipsis that look like continuation prompts,
has a side benefit of putting rest of error message in proper text color.
2018-09-10 18:13:08 -07:00
Gregory P. Smith ce34410b8b
bpo-32270: Don't close stdin/out/err in pass_fds (GH-6242)
When subprocess.Popen() stdin= stdout= or stderr= handles are specified
and appear in pass_fds=, don't close the original fds after dup'ing them.

This implementation and unittest primarily came from @izbyshev (see the PR)

See also b89b52f284

This also removes the old manual p2cread, c2pwrite, and errwrite closing logic
as inheritable flags and _close_open_fds takes care of that properly today without special treatment.

This code is within child_exec() where it is the only thread so there is no
race condition between the dup and _Py_set_inheritable_async_safe call.
2018-09-10 17:46:22 -07:00
Zachary Ware 880d42a3b2
bpo-8110: Refactor platform detection in subprocess (GH-9053)
Check for functionality via imports rather than checking sys.platform
specifically for Windows
2018-09-10 16:16:08 -07:00
Rahul Jha 9430652535 bpo-33217: Raise TypeError for non-Enum lookups in Enums (GH-6651)
* bpo-33217: Raise TypeError for non-Enum lookups in Enums
2018-09-10 11:21:04 -07:00
Matthias Bussonnier 51a4743d19 bpo-33604: Remove deprecated HMAC default value marked for removal in 3.8 (GH-7063)
HMAC's digestmod was deprecated marked for removal, this removes it as planned.
2018-09-10 11:10:01 -07:00
Berker Peksag 9797b7ae44 bpo-26502: Implement FrameSummary.__len__() (GH-8632) 2018-09-10 10:02:33 -07:00
Benjamin Peterson d545869d08
bpo-34588: Fix an off-by-one error in traceback formatting. (GH-9077)
The recursive frame pruning code always undercounted the number of elided frames
by one. That is, in the "[Previous line repeated N more times]" message, N would
always be one too few. Near the recursive pruning cutoff, one frame could be
silently dropped. That situation is demonstrated in the OP of the bug report.

The fix is to start the identical frame counter at 1.
2018-09-10 08:43:10 -07:00
Julien Malard 0afada163c bpo-34421 avoid unicode error in distutils logging (GH-8799)
This caused installation errors in some cases on Windows.
Patch by Julien Malard.
2018-09-08 16:31:26 -04:00
Pablo Aguiar d5fbe9b1a3 bpo-34246: Use no mutable default args in smtplib (GH-8554)
Some methods of the SMTP class use mutable default arguments. Specially
`send_message` is affected as it mutates one of the args by appending items
to it, which has side effects on further calls.
2018-09-07 23:04:48 +01:00
William Grzybowski 28658485a5 bpo-34604: Fix possible mojibake in pwd.getpwnam() and grp.getgrnam() (GH-9098)
Pass the user/group name as Unicode to the formatting function,
instead of always decoding a bytes string from UTF-8.
2018-09-07 19:10:39 +02:00
Victor Stinner 886483e2b9
bpo-34595: Add %T format to PyUnicode_FromFormatV() (GH-9080)
* Add %T format to PyUnicode_FromFormatV(), and so to
  PyUnicode_FromFormat() and PyErr_Format(), to format an object type
  name: equivalent to "%s" with Py_TYPE(obj)->tp_name.
* Replace Py_TYPE(obj)->tp_name with %T format in unicodeobject.c.
* Add unit test on %T format.
* Rename unicode_fromformat_write_cstr() to
  unicode_fromformat_write_utf8(), to make the intent more explicit.
2018-09-07 18:00:58 +02:00
Pablo Galindo 254a4663d8
bpo-20104: Add flag capabilities to posix_spawn (GH-6693)
Implement the "attributes objects" parameter of `os.posix_spawn` to complete the implementation and fully cover the underlying API.
2018-09-07 16:44:24 +01:00
William Grzybowski 23e65b2555 bpo-33625: Release GIL for grp.getgr{nam,gid} and pwd.getpw{nam,uid} (GH-7081)
Release GIL on grp.getgrnam(), grp.getgrgid(), pwd.getpwnam() and
pwd.getpwuid() if reentrant variants of these functions are available.

Patch by William Grzybowski.
2018-09-07 14:06:15 +02:00
jdemeyer 8f735485ac bpo-25750: fix refcounts in type_getattro() (GH-6118)
When calling tp_descr_get(self, obj, type), make sure that
we own a strong reference to "self".
2018-09-07 09:37:00 +02:00
Zackery Spytz b03c2c5190 closes bpo-34594: Don't hardcode errno values in the tests. (GH-9076) 2018-09-06 11:43:30 -07:00
Tal Einat 3e2b29dccc
bpo-30977: make uuid.UUID use __slots__ (GH-9078)
Co-Authored-By: Wouter Bolsterlee.
2018-09-06 14:34:25 +03:00
Erik Janssens 874809ea38 closes bpo-34581 : Conditionalize use of __pragma in Modules/socketmodule.c. (GH-9067) 2018-09-04 23:29:42 -07:00
Alexander Buchkovsky 266f4904a2 bpo-34563: Fix for invalid assert on big output of multiprocessing.Process (GH-9027)
Fix for invalid assert on big output of multiprocessing.Process.
2018-09-04 18:10:28 +02:00
Victor Stinner 39487196c8
bpo-34530: Fix distutils find_executable() (GH-9049)
distutils.spawn.find_executable() now falls back on os.defpath if the
PATH environment variable is not set.
2018-09-04 11:01:09 +02:00
Pablo Galindo ec74d187f5 bpo-33613, test_semaphore_tracker_sigint: fix race condition (#7850)
Fail `test_semaphore_tracker_sigint` if no warnings are expected and one is received.

Fix race condition when the child receives SIGINT before it can register signal handlers for it.

The race condition occurs when the parent calls
`_semaphore_tracker.ensure_running()` (which in turn spawns the
semaphore_tracker using `_posixsubprocess.fork_exec`), the child
registers the signal handlers and the parent tries to kill the child.
What seem to happen is that in some slow systems, the parent sends the
signal to kill the child before the child protects against the signal.
2018-09-04 10:53:54 +02:00
Pablo Galindo e9ba3705de
bpo-33083 - Make math.factorial reject arguments that are not int-like (GH-6149)
math.factorial() was accepting non-integral Decimal instances. This is inconsistent with the actual behaviour for floats, which are not accepted.
2018-09-03 22:20:06 +01:00
Thomas Herzog 2d7102e726 closes bpo-34555: Fix incorrectly nested test for HAVE_LINUX_VM_SOCKETS_H (GH-9016) 2018-09-01 15:13:57 -07:00
Michael Felt 172a71f19b bpo-34558: Add missing parentheses in _aix.py (GH-9017)
p.wait()
2018-08-31 19:31:07 -07:00
Pablo Galindo 95d630e221
bpo-34408: Prevent a null pointer dereference and resource leakage in `PyInterpreterState_New()` (GH-8767)
* A pointer in `PyInterpreterState_New()` could have been `NULL` when being dereferenced.

* Memory was leaked in `PyInterpreterState_New()` when taking some error-handling code path.
2018-08-31 22:49:29 +01:00
Brendan Gerrity aada63b20e bpo-34062: Add missing launcher argument and make behavior consistent between short and long arguments (GH-8827)
Added previously missing "--list" argument.
Made "--list" and "--list-paths" behavior consistent with the corresponding "-0" and "-0p" arguments.
2018-08-31 08:15:42 -07:00
Naris R 1b5f9c9653 bpo-34427: Fix infinite loop when calling MutableSequence.extend() on self (GH-8813) 2018-08-30 09:56:14 -07:00
Christian Heimes e6dac00779
bpo-34542: Update test certs and keys (GH-8997)
Update all test certs and keys to use future proof crypto settings:

* 3072 bit RSA keys
* SHA-256 signature

Signed-off-by: Christian Heimes <christian@python.org>
2018-08-30 07:25:49 +02:00
Victor Stinner c5989cd876
bpo-34523: Py_DecodeLocale() use UTF-8 on Windows (GH-8998)
Py_DecodeLocale() and Py_EncodeLocale() now use the UTF-8 encoding on
Windows if Py_LegacyWindowsFSEncodingFlag is zero.

pymain_read_conf() now sets Py_LegacyWindowsFSEncodingFlag in its
loop, but restore its value at exit.
2018-08-29 19:32:47 +02:00
Victor Stinner b2457efc78
bpo-34523: Add _PyCoreConfig.filesystem_encoding (GH-8963)
_PyCoreConfig_Read() is now responsible to choose the filesystem
encoding and error handler. Using Py_Main(), the encoding is now
chosen even before calling Py_Initialize().

_PyCoreConfig.filesystem_encoding is now the reference, instead of
Py_FileSystemDefaultEncoding, for the Python filesystem encoding.

Changes:

* Add filesystem_encoding and filesystem_errors to _PyCoreConfig
* _PyCoreConfig_Read() now reads the locale encoding for the file
  system encoding.
* PyUnicode_EncodeFSDefault() and PyUnicode_DecodeFSDefaultAndSize()
  now use the interpreter configuration rather than
  Py_FileSystemDefaultEncoding and Py_FileSystemDefaultEncodeErrors
  global configuration variables.
* Add _Py_SetFileSystemEncoding() and _Py_ClearFileSystemEncoding()
  private functions to only modify Py_FileSystemDefaultEncoding and
  Py_FileSystemDefaultEncodeErrors in coreconfig.c.
* _Py_CoerceLegacyLocale() now takes an int rather than
  _PyCoreConfig for the warning.
2018-08-29 13:25:36 +02:00
Victor Stinner 177d921c8c
bpo-34485, Windows: LC_CTYPE set to user preference (GH-8988)
On Windows, the LC_CTYPE is now set to the user preferred locale at
startup: _Py_SetLocaleFromEnv(LC_CTYPE) is now called during the
Python initialization. Previously, the LC_CTYPE locale was "C" at
startup, but changed when calling setlocale(LC_CTYPE, "") or
setlocale(LC_ALL, "").

pymain_read_conf() now also calls _Py_SetLocaleFromEnv(LC_CTYPE) to
behave as _Py_InitializeCore(). Moreover, it doesn't save/restore the
LC_ALL anymore.

On Windows, standard streams like sys.stdout now always use
surrogateescape error handler by default (ignore the locale).
2018-08-29 11:25:15 +02:00
Victor Stinner 315877dc36
bpo-34485: stdout uses surrogateescape on POSIX locale (GH-8986)
Standard streams like sys.stdout now use the "surrogateescape" error
handler, instead of "strict", on the POSIX locale (when the C locale is not
coerced and the UTF-8 Mode is disabled).

Add tests on sys.stdout.errors with LC_ALL=POSIX.
2018-08-29 09:58:12 +02:00
Victor Stinner 9e4994d410
bpo-34485: Enhance init_sys_streams() (GH-8978)
Python now gets the locale encoding with C code to initialize the encoding
of standard streams like sys.stdout. Moreover, the encoding is now
initialized to the Python codec name to get a normalized encoding name and
to ensure that the codec is loaded. The change avoids importing
_bootlocale and _locale modules at startup by default.

When the PYTHONIOENCODING environment variable only contains an encoding,
the error handler is now is now set explicitly to "strict".

Rename also get_default_standard_stream_error_handler() to
get_stdio_errors().

Reduce the buffer to format the "cpXXX" string (Windows locale encoding).
2018-08-28 23:26:33 +02:00
Victor Stinner d500e5307a
bpo-34403: On HP-UX, force ASCII for C locale (GH-8969)
On HP-UX with C or POSIX locale, sys.getfilesystemencoding() now returns
"ascii" instead of "roman8" (when the UTF-8 Mode is disabled and the C locale
is not coerced).

nl_langinfo(CODESET) announces "roman8" whereas it uses the Latin1
encoding in practice.
2018-08-28 17:27:36 +02:00
Victor Stinner 5cb258950c
bpo-34527: POSIX locale enables the UTF-8 Mode (GH-8972)
* The UTF-8 Mode is now also enabled by the "POSIX" locale, not only
  by the "C" locale.
* On FreeBSD, Py_DecodeLocale() and Py_EncodeLocale() now also forces
  the ASCII encoding if the LC_CTYPE locale is "POSIX", not only if
  the LC_CTYPE locale is "C".
* test_utf8_mode.test_cmd_line() checks also that the command line
  arguments are decoded from UTF-8 when the the UTF-8 Mode is enabled
  with POSIX locale or C locale.
2018-08-28 12:35:44 +02:00
Carl Meyer d658deac60 bpo-21145: Add cached_property decorator in functools (#6982)
Robust caching of calculated properties is
harder than it looks at first glance, so add
a solid, well-tested implementation to the
standard library.
2018-08-28 17:11:56 +10:00
Przemysław Spodymek 216b745eaf bpo-33635: Handling Bad file descriptor in Path.is_file and related. (GH-8542) 2018-08-27 14:33:45 -07:00
Michael Felt 7ef1697be5 bpo-34347: Fix test_utf8_mode.test_cmd_line for AIX (GH-8923)
AIX uses ISO-8859-1 encoding for the C locale.
2018-08-27 15:40:17 +02:00
Elias Zamaria 393f1ff62e bpo-32968: Make modulo and floor division involving Fraction and float consistent with other operations (#5956)
Make mixed-type `%` and `//` operations involving `Fraction` and `float` objects behave like all other mixed-type arithmetic operations: first the `Fraction` object is converted to a `float`, then the `float` operation is performed as normal. This fixes some surprising corner cases, like `Fraction('1/3') % inf` giving a NaN.

Thanks Elias Zamaria for the patch.
2018-08-27 07:59:28 +01:00
Michael Felt 89d79b1449 Remove AIX workaround test_subprocess (GH-8939)
An old apparent AIX behavior workaround in test_subprocess's
test_undecodable_env is no longer needed.
2018-08-26 10:29:36 -07:00
Serhiy Storchaka c406d5cd74
bpo-34171: Prevent creating Lib/trace.cover when run the trace module. (GH-8841) 2018-08-25 10:27:55 +03:00
Danish Prakash 09efe49c07 bpo-34426: fix typo (__lltrace__ -> __ltrace__) (GH-8822) 2018-08-24 19:00:25 -07:00
Gregory P. Smith 76be0fffff
bpo-13312: Avoid int underflow in time year. (GH-8912)
Avoids an integer underflow in the time module's year handling code.
2018-08-24 18:08:50 -07:00
Vladimir Matveev 91cb298f81 bpo-6700: Fix inspect.getsourcelines for module level frames/tracebacks (GH-8864) 2018-08-24 17:18:00 +03:00
Paul Ganssle 096329f0b2 bpo-34454: fix .fromisoformat() methods crashing on inputs with surrogate code points (GH-8862)
The current C implementations **crash** if the input includes a surrogate
Unicode code point, which is not possible to encode in UTF-8.

Important notes:

1.  It is possible to pass a non-UTF-8 string as a separator to the
    `.isoformat()` methods.
2.  The pure-Python `datetime.fromisoformat()` implementation accepts
    strings with a surrogate as the separator.

In `datetime.fromisoformat()`, in the special case of non-UTF-8 separators,
this implementation will take a performance hit by making a copy of the
input string and replacing the separator with 'T'.

Co-authored-by: Alexey Izbyshev <izbyshev@ispras.ru>
Co-authored-by: Paul Ganssle <paul@ganssle.io>
2018-08-23 18:06:20 +03:00
Michael Osipov 48ce4897f8 bpo-34412: Make signal.strsignal() work on HP-UX (GH-8786)
Introduce a configure check for strsignal(3) which defines HAVE_STRSIGNAL for
signalmodule.c. Add some common signals on HP-UX. This change applies for
Windows and HP-UX.
2018-08-23 16:27:19 +03:00
Berker Peksag e7d4b2f205
bpo-2122: Make mmap.flush() behave same on all platforms (GH-8692)
Previously, its behavior was platform-dependent and there was no error checking
under Windows.
2018-08-22 21:21:05 +03:00
Alexey Izbyshev cdbf50cba1 bpo-34441: Fix ABC.__subclasscheck__ crash on classes with invalid __subclasses__ (GH-8835)
The missing NULL check was reported by Svace static analyzer.
2018-08-20 23:04:19 +03:00
Serhiy Storchaka 77d5781835
bpo-34318: Convert deprecation warnings to errors in assertRaises() etc. (GH-8623) 2018-08-19 10:00:11 +03:00
Zackery Spytz e349bf2358 bpo-22602: Raise an exception in the UTF-7 decoder for ill-formed sequences starting with "+". (GH-8741)
The UTF-7 decoder now raises UnicodeDecodeError for ill-formed
sequences starting with "+" (as specified in RFC 2152).
2018-08-19 07:43:38 +03:00
davidair 2b32da2fea Improve error message when mock.assert_has_calls fails (GH-8205)
This makes the assertion error message more useful, aiding debugging.

Thanks @davidair!
2018-08-17 12:09:58 -07:00
Christian Heimes 1590c39336
bpo-34391: Fix ftplib test for TLS 1.3 (GH-8787)
Read from data socket to avoid "[SSL] shutdown while in init" exception
during shutdown of the dummy server.

Signed-off-by: Christian Heimes <christian@python.org>


<!-- issue-number: [bpo-34391](https://www.bugs.python.org/issue34391) -->
https://bugs.python.org/issue34391
<!-- /issue-number -->
2018-08-16 19:43:44 +02:00
Steve Dower 864a892af3
bpo-34405: Updated to OpenSSL 1.1.0i for Windows builds. (GH-8775) 2018-08-15 13:29:24 -07:00
Berker Peksag e0b5b2096e
bpo-34384: Fix os.readlink() on Windows (GH-8740)
os.readlink() now accepts path-like and bytes objects on Windows.
Previously, support for path-like and bytes objects was only
implemented on Unix.

This commit also merges Unix and Windows implementations of
os.readlink() in one function and adds basic unit tests to increase
test coverage of the function.
2018-08-15 13:03:41 +03:00
Zackery Spytz 7c4ab2afb1 closes bpo-34400: Fix undefined behavior in parsetok(). (GH-4439)
Avoid undefined pointer arithmetic with NULL.
2018-08-14 23:27:26 -07:00
Christian Heimes 88bfd0bce0
bpo-34399: 2048 bits RSA keys and DH params (#8762)
Downstream vendors have started to deprecate weak keys. Update all RSA keys
and DH params to use at least 2048 bits.

Finite field DH param file use RFC 7919 values, generated with

    certtool --get-dh-params --sec-param=high

Signed-off-by: Christian Heimes <christian@python.org>
2018-08-14 12:54:19 +02:00
Vadim Pushtaev 4d12e4dc28 bpo-34213: Allow dataclasses to work with a field named 'object'. (GH-8452) 2018-08-12 07:46:05 -04:00
Sergey Fedoseev 2fc46979b8 bpo-34151: Improve performance of some list operations (GH-8332) 2018-08-11 21:12:07 +08:00
Paul Price db6075ab3a closes bpo-34377: Update Valgrind suppressions. (GH-8729)
Valgrind isn't seeing PyObject_Free/PyObject_Realloc, but using _PyObject_Free/_PyObject_Realloc works.
2018-08-10 23:41:34 -07:00
Berker Peksag 84a13fbda0
bpo-9372: Deprecate several __getitem__ methods (GH-8609)
The __getitem__ methods of DOMEventStream, FileInput,
and FileWrapper classes ignore their 'index' parameters
and return the next item instead.
2018-08-11 09:05:04 +03:00
Berker Peksag 423d05f6f5
bpo-34333: Fix %-formatting in Path.with_suffix() (GH-8663) 2018-08-11 08:45:06 +03:00
Tal Einat 077059e0f0
bpo-34047: IDLE: fix mousewheel scrolling direction on macOS (GH-8678) 2018-08-10 09:02:08 +03:00
GPery b92c526ed5 closes bpo-34353: Add sockets to stat.filemode fallback python implementation. (GH-8703) 2018-08-09 22:12:08 -07:00
Alex Grönholm cca4eec3c0 bpo-34270: Make it possible to name asyncio tasks (GH-8547)
Co-authored-by: Antti Haapala <antti.haapala@anttipatterns.com>
2018-08-08 17:06:47 -04:00
Berker Peksag e4dcbbd7f4
bpo-18540: Fix EAI_NONAME in imaplib.IMAP4*() (GH-8634) 2018-08-07 05:12:18 +03:00
Anthony Sottile b2499669ef bpo-19891: Ignore error while writing history file (GH-8483) 2018-08-06 17:28:19 +09:00
Tal Einat 87e59ac11e
bpo-33839: refactor IDLE's tooltips & calltips, add docstrings and tests (GH-7683)
* make CallTip and ToolTip sub-classes of a common abstract base class
* remove ListboxToolTip (unused and ugly)
* greatly increase test coverage
* tested on Windows, Linux and macOS
2018-08-05 09:21:08 +03:00
Terry Jan Reedy 9beaef6225
bpo-34275: Make IDLE calltips always visible on Mac. (GH-8639)
Some MacOS-tk combinations need .update_idletasks().
The call is both unneeded and innocuous on Linux and Windows.
Patch by Kevin Waltzer.
2018-08-02 22:50:01 -04:00
Marcel Plch a2fe1e52eb bpo-34097: Add support for zipping files older than 1980-01-01 (GH-8270)
ZipFile can zip files older than 1980-01-01 and newer than 2107-12-31 using
a new strict_timestamps parameter at the cost of setting the timestamp
to the limit.
2018-08-02 15:04:52 +02:00
Tal Einat 10ea9409ce
bpo-34120: fix IDLE freezing after closing dialogs (GH-8603)
Added missing .grab_release() calls to all places where we call .grab_set().
2018-08-02 09:18:29 +03:00
costypetrisor 8ed317f1ca bpo-34113: Fix a crash when using LLTRACE is on (GH-8517)
Fix a crash on negative STACKADJ() when Low-Level trace (LLTRACE) is enabled.
2018-07-31 22:55:14 +02:00
MartinAltmayer 944451cd8d bpo-34263 Cap timeout submitted to epoll/select etc. to one day. (GH-8532) 2018-07-31 10:06:12 -04:00
Raymond Hettinger 9c18b1ae52
bpo-33089: Add math.dist() for computing the Euclidean distance between two points (GH-8561) 2018-07-31 00:45:49 -07:00
Serhiy Storchaka 9d5727326a
bpo-33871: Fix os.sendfile(), os.writev(), os.readv(), etc. (GH-7931)
* Fix integer overflow in os.readv(), os.writev(), os.preadv()
  and os.pwritev() and in os.sendfile() with headers or trailers
  arguments (on BSD-based OSes and MacOS).

* Fix sending the part of the file in os.sendfile() on MacOS.
  Using the trailers argument could cause sending more bytes from
  the input file than was specified.

Thanks Ned Deily for testing on 32-bit MacOS.
2018-07-31 10:24:54 +03:00
Serhiy Storchaka f1d36d8efa
bpo-33729: Fix issues with arguments parsing in hashlib. (GH-8346)
* help(hashlib) didn't work because of incorrect module name in blake2b and
  blake2s classes.
* Constructors blake2*(), sha3_*(), shake_*() and keccak_*() incorrectly
  accepted keyword argument "string" for binary data, but documented as
  accepting the "data" keyword argument. Now this parameter is positional-only.
* Keyword-only parameters in blake2b() and blake2s() were not documented as
  keyword-only.
* Default value for some parameters of blake2b() and blake2s() was None,
  which is not acceptable value.
* The length argument for shake_*.digest() was wrapped out to 32 bits.
* The argument for shake_128.digest() and shake_128.hexdigest() was not
  positional-only as intended.
* TypeError messages for incorrect arguments in all constructors sha3_*(),
  shake_*() and keccak_*() incorrectly referred to sha3_224.

Also made the following enhancements:

* More accurately specified input and result types for strings, bytes and
  bytes-like objects.
* Unified positional parameter names for update() and constructors.
* Improved formatting.
2018-07-31 09:50:16 +03:00
Serhiy Storchaka 4b8a7f51da
Revert "closes bpo-27494: Fix 2to3 handling of trailing comma after a generator expression (GH-3771)" (#8241)
This reverts commit af810b35b4.

This is not valid syntax (see bpo-32012).
2018-07-31 09:34:30 +03:00
Serhiy Storchaka ac20e0f98d
bpo-1617161: Make the hash and equality of methods not depending on the value of self. (GH-7848)
* The hash of BuiltinMethodType instances no longer depends on the hash
  of __self__. It depends now on the hash of id(__self__).
* The hash and equality of ModuleType and MethodWrapperType instances no
  longer depend on the hash and equality of __self__. They depend now on
  the hash and equality of id(__self__).
* MethodWrapperType instances no longer support ordering.
2018-07-31 09:18:24 +03:00
twisteroid ambassador 9045199c5a bpo-33833: Fix ProactorSocketTransport AssertionError (#7893) 2018-07-30 21:58:50 +03:00
Peter Lamut 20678fd874 Add docstrings to public methods from context.c (GH-8531) 2018-07-30 16:15:44 +01:00
Elvis Pranskevichus 22d25085db bpo-34075: Deprecate non-ThreadPoolExecutor in loop.set_default_executor() (GH-8533)
Various asyncio internals expect that the default executor is a
`ThreadPoolExecutor`, so deprecate passing anything else to
`loop.set_default_executor()`.
2018-07-30 12:42:43 +02:00
Mickaël Schoentgen 3f8c6913b8 bpo-34035: Fix several AttributeError in zipfile seek() methods. (GH-8527) 2018-07-29 21:26:52 +03:00
Franz Wöllert d2e902e4fb bpo-31047: Fix ntpath.abspath for invalid paths (GH-8544) 2018-07-29 13:47:09 +01:00
Stéphane Wirtel b7fd73896d bpo-34231: PYTHONBREAKPOINT is not documented on python --help (GH-8475) 2018-07-29 11:27:16 +01:00
Berker Peksag 11eb1a9470 bpo-34251: Restore msilib.Win64 to preserve compatibility (GH-8510) 2018-07-28 16:02:56 -05:00
Raymond Hettinger c6dabe37e3
bpo-33089: Multidimensional math.hypot() (GH-8474) 2018-07-28 07:48:04 -07:00
Dong-hee Na 8fe9eed937 bpo-33476: Fix _header_value_parser when address group is missing final ';' (GH-7484) 2018-07-28 13:55:11 +01:00
Berker Peksag 612dbefe9d
bpo-30722: Fix NEWS entries (GH-8501) 2018-07-27 18:49:58 +03:00
Steve Dower 5473f061f5
bpo-34225: Ensure INCLUDE and LIB directories do not end with a backslash. (GH-8464) 2018-07-26 04:23:10 -07:00
Serhiy Storchaka 7cb7bcff20
bpo-20260: Implement non-bitwise unsigned int converters for Argument Clinic. (GH-8434) 2018-07-26 13:22:16 +03:00
Serhiy Storchaka 323748ad74
bpo-34197: Make _csv.Dialect attributes booleans. (GH-8440)
Attributes skipinitialspace, doublequote and strict are now
booleans instead of integers 0 or 1.
2018-07-26 13:21:09 +03:00
Victor Stinner ecf411c59e
bpo-34170: Enhance _PyCoreConfig_Read() (GH-8468)
* Inline cmdline_get_env_flags() into config_read_env_vars():
  _PyCoreConfig_Read() now reads much more environment variables like
  PYTHONVERBOSE.
* Allow to override faulthandler and allocator even if dev_mode=1.
  PYTHONMALLOC is now the priority over PYTHONDEVMODE.
* Fix _PyCoreConfig_Copy(): copy also install_signal_handlers,
  coerce_c_locale and coerce_c_locale_warn
* _PyCoreConfig.install_signal_handlers default is now 1: install
  signals by default
* Fix also a compiler warning: don't define _PyPathConfig type twice.
2018-07-26 02:37:22 +02:00
chason 48ed88a93b bpo-32663 Make SMTPUTF8SimTests run (#5314)
Enable and fix SMTPUTF8SimTests in test_smtplib.

The tests for SMTPUTF8SimTests in test_smtplib.py were not actually
being run because test_smtplib was still using the 'test_main' pattern,
and the class was never added to test_main.

Additionally, one of the tests needed to be moved to the non-UTF8 server
class because it relies on the server not being UTF-8 compatible (and it
had a bug in in).
2018-07-25 15:01:28 -04:00
Victor Stinner 60b04c9f6f
bpo-34228: Allow PYTHONTRACEMALLOC=0 (GH-8467)
PYTHONTRACEMALLOC=0 environment variable and -X tracemalloc=0 command
line option are now allowed to disable explicitly tracemalloc at
startup.
2018-07-25 19:23:53 +02:00
Ammar Askar 96d1e69a12 bpo-29097: Forego fold detection on windows for low timestamp values (GH-2385)
On Windows, passing a negative value to local results in an OSError because localtime_s on Windows does not support negative timestamps. Unfortunately this means that fold detection for timestamps between 0 and max_fold_seconds will result in this OSError since we subtract max_fold_seconds from the timestamp to detect a fold. However, since we know there haven't been any folds in the interval [0, max_fold_seconds) in any timezone, we can hackily just forego fold detection for this time range on Windows.
2018-07-25 12:54:58 -04:00
Berker Peksag 74102c9a5f
bpo-13041: Use shutil.get_terminal_size() in argparse.HelpFormatter (GH-8459) 2018-07-25 18:23:44 +03:00
Serhiy Storchaka ac0b3c2f4d
bpo-34164: Fix handling of incorrect padding in base64.b32decode(). (GH-8351)
Now base64.Error is always raised instead of UnboundLocalError or
OverflowError.
2018-07-24 12:52:51 +03:00
Serhiy Storchaka 02ec92fa7b
bpo-29209: Remove old-deprecated features in ElementTree. (GH-6769)
Also make getchildren() and getiterator() emitting
a DeprecationWarning instead of PendingDeprecationWarning.
2018-07-24 12:03:34 +03:00
Alexander Marshalov e22072fb11 bpo-34149: Behavior of the min/max with key=None (GH-8328)
Improve consistency with the signature for sorted(), heapq.nsmallest(), heapq.nlargest(), and itertools.groupby().
2018-07-23 20:58:21 -07:00
Serhiy Storchaka a6fdddb7df
bpo-32500: Fix error messages for sequence and mapping C API. (GH-7846)
Fix error messages for PySequence_Size(), PySequence_GetItem(),
PySequence_SetItem() and PySequence_DelItem() called with a mapping
and PyMapping_Size() called with a sequence.
2018-07-23 23:43:42 +03:00
Matěj Cepl caa331d492 bpo-33336, imaplib: Legalize MOVE command (GH-6569)
imaplib now allows MOVE command in IMAP4.uid() (RFC 6851:
IMAP MOVE Extension) and potentially as a name of supported
method of IMAP4 object.
2018-07-23 13:28:54 +02:00
Berker Peksag 7a3056fa7d
bpo-21446: Update reload fixer to use importlib (GH-8391) 2018-07-23 09:49:08 +03:00
Berker Peksag d04f46c59f
bpo-940286: Fix pydoc to show cross refs correctly (GH-8390) 2018-07-23 08:37:47 +03:00
Xtreak 1426daa4fe bpo-34127: Fix grammar in error message with respect to argument count (GH-8395) 2018-07-22 13:13:26 -07:00
jdemeyer 56868f940e bpo-34126: Fix crashes while profiling invalid calls. (GH-8300) 2018-07-21 11:30:59 +03:00
Vinay Sajip 94487d4570
bpo-34011: Update code copying DLLs and init.tcl into venvs. (GH-8253) 2018-07-20 17:07:38 +01:00
Victor Stinner fb47bca9ee
bpo-34008: Allow to call Py_Main() after Py_Initialize() (GH-8043)
Py_Main() can again be called after Py_Initialize(), as in Python
3.6. The new configuration is ignored, except of
_PyMainInterpreterConfig.argv which is used to update sys.argv.
2018-07-20 17:34:23 +02:00
ValeriyaSinevich ce75df3031 bpo-30237: Output error when ReadConsole is canceled by CancelSynchronousIo. (GH-7911) 2018-07-19 15:34:03 -07:00
Serhiy Storchaka 5d4cb54800
bpo-34141: Optimized pickling simple non-recursive values. (GH-8318) 2018-07-18 10:10:49 +03:00
Zackery Spytz 28f07364f0 bpo-34068: _io__IOBase_close_impl could call _PyObject_SetAttrId with an exception set (GH-8282) 2018-07-17 09:31:44 +03:00
Antoine Pitrou 961d54c5c1
bpo-32430: Rename Modules/Setup.dist to Modules/Setup (GH-8229)
bpo-32430: Rename Modules/Setup.dist to Modules/Setup

Remove the necessity to copy the former manually to the latter when updating the local source tree.
2018-07-16 19:03:03 +02:00
Serhiy Storchaka bd47384e07
bpo-24618: Add a check in the code constructor. (GH-8283)
Check that the size of the varnames tuple is enough at least for all arguments.
2018-07-16 09:10:19 +03:00
Benjamin Peterson 15c7b2abdf
bpo-34121: Fix detection of C11 atomic support on clang. (GH-8288) 2018-07-15 17:01:42 -07:00
INADA Naoki 16dfca4d82
bpo-34087: Fix buffer overflow in int(s) and similar functions (GH-8274)
`_PyUnicode_TransformDecimalAndSpaceToASCII()` missed trailing NUL char.
It caused buffer overflow in `_Py_string_to_number_with_underscores()`.

This bug is introduced in 9b6c60cb.
2018-07-14 12:06:43 +09:00
Jason R. Coombs cafaf0447b
bpo-34108: Fix double carriage return in 2to3 on Windows (#8271)
* Add test capturing failure.
* Honor newlines as present in the original file.
2018-07-13 11:26:03 -04:00
Serhiy Storchaka 993030aac5 bpo-34080: Fix a memory leak in the compiler. (GH-8222) 2018-07-11 23:17:53 +02:00
Serhiy Storchaka 504373c59b
bpo-23927: Make getargs.c skipitem() skipping 'w*'. (GH-8192) 2018-07-11 17:41:43 +03:00
Eitan Adler b91a3a0d61 bpo-33648: Remove PY_WARN_ON_C_LOCALE (GH-7114)
This code does not appear to be used anywhere in the python code base.
The use was removed in eb81795d7d.
2018-07-11 20:01:27 +09:00