Commit Graph

24581 Commits

Author SHA1 Message Date
Miss Islington (bot) f16ebcd460
bpo-35193: Fix an off by one error in the RETURN_VALUE case. (GH-10418)
Fix an off by one error in the peephole optimizer when checking for unreachable code beyond a return.

Do a bounds check within find_op so it can return before going past the end as a safety measure.

7db3c48833GH-diff-a33329ae6ae0bb295d742f0caf93c137
introduced this off by one error while fixing another one nearby.

This bug was shipped in all Python 3.6 and 3.7 releases.

The included unittest won't fail unless you do a clang msan build.
(cherry picked from commit 49fa4a9f1e)

Co-authored-by: Gregory P. Smith <greg@krypto.org>
2018-11-08 18:13:14 -08:00
Miss Islington (bot) 9d3658147b
bpo-34966: Improve support of method aliases in pydoc. (GH-9823)
Pydoc now does not duplicate docstrings for aliases of inherited methods.
(cherry picked from commit a44d34e179)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2018-11-07 23:08:05 -08:00
Miss Islington (bot) b097f9fe2b
Fix the construction of subprocess.CalledProcessError in test_venv (GH-10400)
The constructor of subprocess.CalledProcessError in the check_output function had an extra None in it.
(cherry picked from commit b93925047a)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2018-11-07 14:39:21 -08:00
Miss Islington (bot) ca592bcf56
closes bpo-35171: Fix test_TimeRE_recreation_timezone failure on some systems. (GH-10347)
The test depended on '/usr/share/zoneinfo/posixrules' or equivalent
because it set TZ without explicit DST transition rules. At least
on OpenSUSE Tumbleweed that file is linked to '/etc/localtime',
making the test fail with certain local timezones,
such as 'Europe/Moscow' which doesn't have DST transitions since 2011.
(cherry picked from commit f1b9ad3d38)

Co-authored-by: Alexey Izbyshev <izbyshev@ispras.ru>
2018-11-05 20:47:11 -08:00
Miss Islington (bot) 7beb8c54ed
bpo-35133: Fix mistakes when concatenate string literals on different lines. (GH-10284)
Two kind of mistakes:

1. Missed space. After concatenating there is no space between words.

2. Missed comma. Causes unintentional concatenating in a list of strings.
(cherry picked from commit 34fd4c2019)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2018-11-05 06:52:58 -08:00
Miss Islington (bot) 96fb350bfd bpo-21263: Skip test_gdb when python has been compiled with LLVM clang (GH-10318) (GH-10325)
(cherry picked from commit 59668aa8b7)

Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
2018-11-04 16:40:02 -05:00
Miss Islington (bot) 71b6c1af72
bpo-35062: Fix parsing _io.IncrementalNewlineDecoder's *translate* argument. (GH-10217)
_io.IncrementalNewlineDecoder's initializer possibly assigns out-of-range
value to the bitwise struct field.
(cherry picked from commit b08746bfdf)

Co-authored-by: Xiang Zhang <angwerzx@126.com>
2018-10-31 05:36:20 -07:00
Miss Islington (bot) 58f7bf3a89
bpo-34945: Buffer output in test suite only when creating junit file (GH-10204)
After commit d0f49d2f50, the output of the
test suite is always buffered as the test output needs to be included in
the JUnit file in same cases (as when a test fails). This has the
consequence that printing or using debuggers (like pdb) in the test
suite does not result in a good user experience anymore.

This commit modifies the test suite runner so it only captures the test
output when the JUnit file is requested to fix the regression so prints
and debuggers are usable again.
(cherry picked from commit 02277482ea)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2018-10-29 14:05:31 -07:00
Steve Dower a7ffb66395
[3.7] bpo-31047: Fix ntpath.abspath to trim ending separator (GH-10082) 2018-10-25 13:46:23 -04:00
Miss Islington (bot) 178bf58e79
bpo-35028: cgi: Fix max_num_fields off by one error (GH-9973)
https://bugs.python.org/issue35028
(cherry picked from commit b79b5c0949)

Co-authored-by: matthewbelisle-wf <matthew.belisle@workiva.com>
2018-10-23 03:54:52 -07:00
Miss Islington (bot) 313e5015d2
bpo-34482: test datetime classes' handling of non-UTF-8-encodable strings (GH-8878)
(cherry picked from commit 3b0047d8e9)

Co-authored-by: Alexey Izbyshev <izbyshev@ispras.ru>
2018-10-23 00:04:24 -07:00
Miss Islington (bot) 18450be94d
bpo-34454: Clean up datetime.fromisoformat surrogate handling (GH-8959)
* Use _PyUnicode_Copy in sanitize_isoformat_str

* Use repr in fromisoformat error message

This reverses commit 67b74a98b2 per Serhiy Storchaka's suggestion:

     I suggested to use %R in the error message because including the raw
     string can be confusing in the case of empty string, or string
     containing trailing whitespaces, invisible or unprintable characters.

We agree that it is better to change both the C and pure Python versions
to use repr.

* Retain non-sanitized dtstr for error printing

This does not create an extra string, it just holds on to a reference to
the original input string for purposes of creating the error message.

* PEP 7 fixes to from_isoformat

* Separate handling of Unicode and other errors

In the initial implementation, errors other than encoding errors would
both raise an error indicating an invalid format, which would not be
true for errors like MemoryError.

* Drop needs_decref from _sanitize_isoformat_str

Instead _sanitize_isoformat_str returns a new reference, even to the
original string.
(cherry picked from commit 3df85404d4)

Co-authored-by: Paul Ganssle <pganssle@users.noreply.github.com>
2018-10-22 15:35:15 -07:00
Miss Islington (bot) 7f34d55023
bpo-34973: Fix crash in bytes constructor. (GH-9841)
Constructing bytes from mutating list could cause a crash.
(cherry picked from commit 914f9a078f)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2018-10-21 05:55:52 -07:00
Serhiy Storchaka 861d34eee3
bpo-33712: OrderedDict only creates od_fast_nodes cache if needed (GH-7349). (GH-10000)
(cherry picked from commit 6f17e51345)
2018-10-20 11:24:05 +03:00
Miss Islington (bot) dcd56f615e
bpo-34574: Prevent OrderedDict iterators from exhaustion during pickling. (GH-9051)
(cherry picked from commit a5259fb05d)

Co-authored-by: Sergey Fedoseev <fedoseev.sergey@gmail.com>
2018-10-19 22:54:09 -07:00
Miss Islington (bot) 8f53dcdb24
bpo-32890, os: Use errno instead of GetLastError() in execve() and truncate() (GH-5784)
path_error() uses GetLastError() on Windows, but some os functions
are implemented via CRT APIs which report errors via errno.
This may result in raising OSError with invalid error code (such
as zero).

Introduce posix_path_error() function and use it where appropriate.
(cherry picked from commit 834603112e)

Co-authored-by: Alexey Izbyshev <izbyshev@ispras.ru>
2018-10-19 17:46:25 -07:00
Victor Stinner 0f2fc8bee0 bpo-34536: raise error for invalid _missing_ results (GH-9147) (GH-9978)
* raise exception if _missing_ returns None or invalid type
2018-10-19 16:49:30 -07:00
Miss Islington (bot) b9182aa7da bpo-33947: dataclasses no longer can raise RecursionError in repr (GF9916) (#9970)
The reprlib code was copied here instead of importing reprlib. I'm not sure if we really need to avoid the import, but since I expect dataclasses to be more common that reprlib, it seems wise. Plus, the code is small.
(cherry picked from commit dd13c88b53)

Co-authored-by: Srinivas  Thatiparthy (శ్రీనివాస్  తాటిపర్తి) <srinivasreddy@users.noreply.github.com>
2018-10-19 13:28:30 -04:00
Miss Islington (bot) a66f279a13
bpo-34866: Adding max_num_fields to cgi.FieldStorage (GH-9660)
Adding `max_num_fields` to `cgi.FieldStorage` to make DOS attacks harder by
limiting the number of `MiniFieldStorage` objects created by `FieldStorage`.
(cherry picked from commit 209144831b)

Co-authored-by: matthewbelisle-wf <matthew.belisle@workiva.com>
2018-10-19 04:11:16 -07:00
Miss Islington (bot) 5b9b9353de
bpo-35008: Fix possible leaks in Element.__setstate__(). (GH-9924)
C implementation of xml.etree.ElementTree.Element.__setstate__()
leaked references to children when called for already initialized
element.
(cherry picked from commit 6f906b3d72)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2018-10-18 00:17:15 -07:00
Miss Islington (bot) 178d1c0777
bpo-24658: Fix read/write greater than 2 GiB on macOS (GH-1705)
On macOS, fix reading from and writing into a file with a size larger than 2 GiB.
(cherry picked from commit 74a8b6ea7e)

Co-authored-by: Stéphane Wirtel <stephane@wirtel.be>
2018-10-17 23:58:40 -07:00
Stéphane Wirtel 657e3f9a2c [3.7] bpo-23420: Verify the value of '-s' when execute the CLI of cProfile (GH-9925) (GH-9926)
[3.7] [bpo-23420](https://bugs.python.org/issue23420): Verify the value of '-s' when execute the CLI of cProfile (GH-9925)

Verify the value for the parameter '-s' of the cProfile CLI. Patch by Robert
Kuska.

Co-authored-by: Robert Kuska <rkuska@gmail.com>
(cherry picked from commit fcd5e84a51)





https://bugs.python.org/issue23420
2018-10-17 04:48:05 -07:00
Miss Islington (bot) 350aeab812
bpo-34783: Fix test_nonexisting_script() (GH-9896)
Fix test_cmd_line_script.test_nonexisting_script(): the test must not
rely on sys.executable, since main.c uses config->program which can
be different than sys.executable in many cases (for example, on macOS
when using the framework).
(cherry picked from commit ea75187c68)

Co-authored-by: Victor Stinner <vstinner@redhat.com>
2018-10-15 15:30:05 -07:00
Miss Islington (bot) 1370832af2
bpo-34974: Do not replace unexpected errors in bytes() and bytearray(). (GH-9852)
bytes and bytearray constructors converted unexpected exceptions
(e.g. MemoryError and KeyboardInterrupt) to TypeError.
(cherry picked from commit e890421e33)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2018-10-14 14:26:28 -07:00
Miss Islington (bot) b1c800303e
bpo-34941: Fix searching Element subclasses. (GH-9766)
Methods find(), findtext() and findall() of xml.etree.ElementTree.Element
were not able to find chldren which are instances of Element subclasses.
(cherry picked from commit b11c5667f9)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2018-10-14 00:55:49 -07:00
Miss Islington (bot) d07564274a
Add new tests for bytes and bytearray constructors. (GH-9843)
Covered all special cases: bytes, tuple, list, differend
kinds of iterables and iterators.
(cherry picked from commit 1a997eb291)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2018-10-13 11:02:23 -07:00
Miss Islington (bot) 5d8ef8bc3f bpo-34783: Disable test_nonexisting_script for macOS framework builds (GH-9831) (GH-9832)
With macOS framework builds, test case test_nonexisting_script in
test_nonexisting_script fails because the test case assumes that
the file name in sys.executable will appear in the error message.
For macOS framework builds, sys.executable is the file name of the
stub launcher and its file name bears no relationship to the file
name of the actual python executable.  For now, skip the test in
this case.
(cherry picked from commit f6c29a65e2)

Co-authored-by: Ned Deily <nad@python.org>
2018-10-13 00:07:01 -04:00
Miss Islington (bot) c327a5499f
bpo-31516: Skip test_main_thread_during_shutdown() with COUNT_ALLOCS builds. (GH-8052)
(cherry picked from commit 65d2f8c044)

Co-authored-by: Zackery Spytz <zspytz@gmail.com>
2018-10-12 01:49:07 -07:00
Serhiy Storchaka 8b040e5539
[3.7] bpo-34922: Fix integer overflow in the digest() and hexdigest() methods (GH-9751) (GH-9798)
for the SHAKE algorithm in the hashlib module.
(cherry picked from commit 9b8c2e7676)
2018-10-11 08:06:36 +03:00
Miss Islington (bot) 8e23ba021f
bpo-23596: Add unit tests for the command line for the gzip module (GH-9775)
Add unit tests for the command line for the gzip module
(cherry picked from commit 84eec11995)

Co-authored-by: Stéphane Wirtel <stephane@wirtel.be>
2018-10-09 14:42:58 -07:00
Miss Islington (bot) 41e5ec377b
bpo-34769: Thread safety for _asyncgen_finalizer_hook(). (GH-9716)
(cherry picked from commit c880ffe7d2)

Co-authored-by: twisteroid ambassador <twisteroidambassador@users.noreply.github.com>
2018-10-09 09:03:35 -07:00
Miss Islington (bot) 0ce31d340b
bpo-32962: Fix test_gdb failure in debug build with -mcet -fcf-protection -O0 (GH-9656)
When Python is built with the intel control-flow protection flags,
-mcet -fcf-protection, gdb is not able to read the stack without
actually jumping inside the function. This means an extra
'next' command is required to make the $pc (program counter)
enter the function and make the stack of the function exposed to gdb.

Co-Authored-By: Marcel Plch <gmarcel.plch@gmail.com>

(cherry picked from commit 9b7c74ca32)
(cherry picked from commit 79d21331e6)

Co-authored-by: Victor Stinner <vstinner@redhat.com>
2018-10-09 08:21:17 -07:00
Miss Islington (bot) 3b699932e5 bpo-6721: Hold logging locks across fork() (GH-4071) (#9291)
bpo-6721: When os.fork() was called while another thread holds a logging lock, the child process may deadlock when it tries to log.  This fixes that by acquiring all logging locks before fork and releasing them afterwards.

A regression test that fails before this change is included.

Within the new unittest itself: There is a small _potential_ due to mixing of fork and a thread in the child process if the parent's thread happened to hold a non-reentrant library call lock (malloc?) when the os.fork() happens.  buildbots and time will tell if this actually manifests itself in this test or not.  :/  A functionality test that avoids that would be a challenge.

An alternate test that isn't trying to produce the deadlock itself but just checking that the release and acquire calls are made would be the next best alternative if so.
(cherry picked from commit 19003841e9)

Co-authored-by: Gregory P. Smith <greg@krypto.org> [Google]
2018-10-07 00:09:59 -07:00
Miss Islington (bot) 1a2189353f bpo-34334: Don't log traceback twice in QueueHandler (GH-9537) (GH-9581)
(cherry picked from commit d345bb4d9b)

Co-authored-by: Cheryl Sabella <cheryl.sabella@gmail.com>
2018-10-07 00:33:37 -04:00
Miss Islington (bot) 453b3b0e87 bpo-34909: keep searching mixins until base class is found (GH-9737) (GH-9738)
(cherry picked from commit cd45385ffa)
2018-10-06 00:43:20 -07:00
Elvis Pranskevichus a67bd53d3f [3.7] bpo-34872: Fix self-cancellation in C implementation of asyncio.Task (GH-9679) (GH-9691)
The C implementation of asyncio.Task currently fails to perform the
cancellation cleanup correctly in the following scenario.

    async def task1():
        async def task2():
            await task3     # task3 is never cancelled

        asyncio.current_task().cancel()
        await asyncio.create_task(task2())

The actuall error is a hardcoded call to `future_cancel()` instead of
calling the `cancel()` method of a future-like object.

Thanks to Vladimir Matveev for noticing the code discrepancy and to
Yury Selivanov for coming up with a pathological scenario..
(cherry picked from commit 548ce9dedd)

Co-authored-by: Elvis Pranskevichus <elvis@magic.io>


https://bugs.python.org/issue34872
2018-10-03 08:49:00 -07:00
Miss Islington (bot) 97f998a4df bpo-34172: multiprocessing.Pool leaks resources after being deleted (GH-8450) (GH-9676)
Fix a reference issue inside multiprocessing.Pool that caused the pool to remain alive if it was deleted without being closed or terminated explicitly.
(cherry picked from commit 97bfe8d3eb)

Co-authored-by: tzickel <tzickel@users.noreply.github.com>
2018-10-02 23:17:04 +02:00
Miss Islington (bot) cd4dd9374d
closes bpo-34868: Improve error message with '_' is combined with an invalid type specifier. (GH-9666)
(cherry picked from commit cbda8fc5d7)

Co-authored-by: Benjamin Peterson <benjamin@python.org>
2018-10-01 22:12:02 -07:00
Miss Islington (bot) 0f161b3079
bpo-34854: Fix compiling string annotations containing lambdas. (GH-9645)
* Compiling a string annotation containing a lambda with keyword-only
argument without default value caused a crash.

* Remove the final "*" (it is incorrect syntax) in the representation of
lambda without *args and keyword-only arguments when compile from AST.

* Improve the representation of lambda without arguments.
(cherry picked from commit 2a2940e5c3)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2018-09-30 11:19:15 -07:00
Miss Islington (bot) 7e35081bc8
bpo-34736: improve error message for invalid length b64decode inputs (GH-9563)
Improvements:
1. Include the number of valid data characters in the error message.
2. Mention "number of data characters" rather than "length".

https://bugs.python.org/issue34736
(cherry picked from commit 1fba2ffc37)

Co-authored-by: Tal Einat <taleinat+github@gmail.com>
2018-09-27 23:12:54 -07:00
Miss Islington (bot) addef07ca7
Fix tests in test_socket to use correctly CMSG_LEN (GH-9594)
After some failures in AMD64 FreeBSD CURRENT Debug 3.x buildbots
regarding tests in test_socket that are using
testFDPassSeparateMinSpace(), FreeBDS revision 337423 was pointed
out to be the reason the test started to fail.

A close examination of the manpage for cmsg_space(3) reveals that
the number of file descriptors needs to be taken into account when
using CMSG_LEN().

This commit fixes tests in test_socket to use correctly CMSG_LEN, taking
into account the number of FDs.
(cherry picked from commit 7291108d88)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2018-09-27 06:30:47 -07:00
Miss Islington (bot) 12e3e80241
bpo-34320: Fix dict(o) didn't copy order of dict subclass (GH-8624)
When dict subclass overrides order (`__iter__()`, `keys()`, and `items()`), `dict(o)`
should use it instead of dict ordering.

https://bugs.python.org/issue34320
(cherry picked from commit 2aaf98c16a)

Co-authored-by: INADA Naoki <methane@users.noreply.github.com>
2018-09-25 21:17:52 -07:00
Miss Islington (bot) ef1173ab14
bpo-33937: Catch ENOMEM error in test_socket (GH-9557)
Fix test_socket.SendmsgSCTPStreamTest: catch ENOMEM error.
testSendmsgTimeout() and testSendmsgDontWait() randomly fail on
Travis CI with: "OSError: [Errno 12] Cannot allocate memory".
(cherry picked from commit 46f40be8b9)

Co-authored-by: Victor Stinner <vstinner@redhat.com>
2018-09-25 08:49:28 -07:00
Miss Islington (bot) 7a26222d7c
bpo-34783: Add test_cmd_line_script.test_nonexisting_script() (GH-9535)
Make sure that "./python script.py" does not crash if the script
file doesn't exist.
(cherry picked from commit a46467ff19)

Co-authored-by: Victor Stinner <vstinner@redhat.com>
2018-09-24 08:28:23 -07:00
Miss Islington (bot) 657fdf2eba bpo-34582: Update syntax of Azure Pipelines builds (GH-9521) 2018-09-24 08:43:33 -04:00
Christian Heimes 394e55a927 [3.7] bpo-17239: Disable external entities in SAX parser (GH-9217) (GH-9511)
The SAX parser no longer processes general external entities by default
to increase security. Before, the parser created network connections
to fetch remote files or loaded local files from the file system for DTD
and entities.

Signed-off-by: Christian Heimes <christian@python.org>

https://bugs.python.org/issue17239.
(cherry picked from commit 17b1d5d4e3)

Co-authored-by: Christian Heimes <christian@python.org>



https://bugs.python.org/issue17239
2018-09-24 05:38:37 -07:00
Christian Heimes 2756ef3165 [3.7] bpo-34670: Add TLS 1.3 post handshake auth (GH-9460) (GH-9505)
Add SSLContext.post_handshake_auth and
SSLSocket.verify_client_post_handshake for TLS 1.3 post-handshake
authentication.

Signed-off-by: Christian Heimes <christian@python.org>q

https://bugs.python.org/issue34670.
(cherry picked from commit 9fb051f032)

Co-authored-by: Christian Heimes <christian@python.org>



https://bugs.python.org/issue34670
2018-09-23 00:22:52 -07:00
Ethan Furman 0c076caaa8
[3.7] bpo-29577: Enum: mixin classes don't mix well with already mixed Enums (GH-9328) (GH-9486)
* bpo-29577: allow multiple mixin classes
2018-09-21 22:26:32 -07:00
Miss Islington (bot) c00f7037df
bpo-34759: Fix error handling in ssl 'unwrap()' (GH-9468)
OpenSSL follows the convention that whenever you call a function, it
returns an error indicator value; and if this value is negative, then
you need to go look at the actual error code to see what happened.

Commit c6fd1c1c3a introduced a small mistake in
_ssl__SSLSocket_shutdown_impl: instead of checking whether the error
indicator was negative, it started checking whether the actual error
code was negative, and it turns out that the error codes are never
negative. So the effect was that 'unwrap()' lost the ability to raise
SSL errors.

https://bugs.python.org/issue34759
(cherry picked from commit c0da582b22)

Co-authored-by: Nathaniel J. Smith <njs@pobox.com>
2018-09-21 22:00:42 -07:00
Miss Islington (bot) e5fde1f992
bpo-34537: Fix test_gdb:test_strings with LC_ALL=C (GH-9483)
We cannot simply call locale.getpreferredencoding() here,
as GDB might have been linked against a different version
of Python with a different encoding and coercion policy
with respect to PEP 538 and PEP 540.

Thanks to Victor Stinner for a hint on how to fix this.
(cherry picked from commit 7279b5125e)

Co-authored-by: Elvis Pranskevichus <elvis@magic.io>
2018-09-21 18:29:34 -07:00