Commit Graph

22894 Commits

Author SHA1 Message Date
Daniel Fortunov 2a16eea71f bpo-36582: Make collections.UserString.encode() return bytes, not str (GH-13138) 2019-08-27 21:38:09 -07:00
Christian Heimes 98d90f745d
bpo-37951: Lift subprocess's fork() restriction (GH-15544) 2019-08-27 23:36:56 +02:00
Ethan Furman 3224e1a6bb
Add Florian Ernst to ACKS (GH-15524) 2019-08-27 14:18:05 -07:00
vrajivk 8bf5fef873 bpo-36205: Fix the rusage implementation of time.process_time() (GH-15538) 2019-08-27 00:13:12 -04:00
Inada Naoki fae0ed5099
bpo-37328: remove deprecated HTMLParser.unescape (GH-14186)
It is deprecated since Python 3.4.
2019-08-27 11:48:06 +09:00
Batuhan Taşkaya d3cc189b17 bpo-37925: Mention --embed in python-config usage (GH-15458) 2019-08-26 23:45:36 +02:00
Pradyun Gedam 10c452b894 bpo-37664: Update ensurepip bundled wheels, again (GH-15483)
/cc @ambv since this needs to be included in 3.8 -- see https://github.com/pypa/pip/issues/6885.

Sorry about the last minute PR!


https://bugs.python.org/issue37664



Automerge-Triggered-By: @zooba
2019-08-26 11:19:30 -07:00
Pablo Galindo 0e4ea16336
bpo-37947: Adjust correctly the recursion level in symtable for named expressions (GH-15499) 2019-08-26 15:52:25 +01:00
Andrew Svetlov 1c06009986
bpo-34679: Restore instantiation Windows IOCP event loop from non-main thread (#15492)
* Restore running proactor event loop from non-main thread

Co-Authored-By: Kyle Stanley <aeros167@gmail.com>
2019-08-26 12:51:08 +03:00
Forest Gregg 998cf1f03a bpo-27575: port set intersection logic into dictview intersection (GH-7696) 2019-08-26 00:17:43 -07:00
Serhiy Storchaka c3ea41e9bf
bpo-36917: Add default implementation of ast.NodeVisitor.visit_Constant(). (GH-15490)
It emits a deprecation warning and calls corresponding method
visit_Num(), visit_Str(), etc.
2019-08-26 10:13:19 +03:00
Dong-hee Na 44cd86bbdd bpo-37805: Add tests for json.dump(..., skipkeys=True) (GH-15489)
https://bugs.python.org/issue37805



Automerge-Triggered-By: @methane
2019-08-26 00:04:35 -07:00
Terry Jan Reedy 1039f39c9c
bpo-37824: Properly handle user input warnings in IDLE shell. (GH-15500)
Cease turning SyntaxWarnings into SyntaxErrors.
2019-08-26 02:27:56 -04:00
Flavian Hautbois da27d9b9dc bpo-29553: Fix ArgumentParser.format_usage() for mutually exclusive groups (GH-14976)
Co-authored-by: Andrew Nester <andrew.nester.dev@gmail.com>
2019-08-25 22:06:45 +03:00
Nick Coghlan 5dbe0f59b7
bpo-37757: Disallow PEP 572 cases that expose implementation details (GH-15131)
- drop TargetScopeError in favour of raising SyntaxError directly
  as per the updated PEP 572
- comprehension iteration variables are explicitly local, but
  named expression targets in comprehensions are nonlocal or
  global. Raise SyntaxError as specified in PEP 572
- named expression targets in the outermost iterable of a
  comprehension have an ambiguous target scope. Avoid resolving
  that question now by raising SyntaxError. PEP 572
  originally required this only for cases where the bound name
  conflicts with the iteration variable in the comprehension,
  but CPython can't easily restrict the exception to that case
  (as it doesn't know the target variable names when visiting
  the outermost iterator expression)
2019-08-25 23:45:40 +10:00
Zackery Spytz ce6a070414 bpo-34880: Add the LOAD_ASSERTION_ERROR opcode. (GH-15073)
Fix assert statement misbehavior if AssertionError is shadowed.
2019-08-25 12:44:09 +03:00
Tal Einat d4b4c00b57
bpo-37929: IDLE: avoid Squeezer-related config dialog crashes (GH-15452)
These were caused by keeping around a reference to the Squeezer
instance and calling it's load_font() upon config changes, which
sometimes happened even if the shell window no longer existed.

This change completely removes that mechanism, instead having the
editor window properly update its width attribute, which can then
be used by Squeezer.
2019-08-25 08:52:58 +03:00
Raymond Hettinger aef9ad82f7
bpo-37942: Improve argument clinic float converter (GH-15470) 2019-08-24 19:10:39 -07:00
Berker Peksag 805f8f9afe bpo-19072: Make @classmethod support chained decorators (GH-8405) 2019-08-24 15:37:25 -07:00
Terry Jan Reedy 0dfc025ccc
Fix typo and rearrange words in IDLE news item (GH-15471)
Redo of #15402 by GeeTransit and Kyle Stanley.
2019-08-24 17:32:33 -04:00
Terry Jan Reedy fa7a6f5e35
Revert "(TH-15402)" (#15468)
This reverts commit a50f0a4c2b.
2019-08-24 14:10:04 -04:00
Dong-hee Na 8ad22a4226 bpo-37798: Test both Python and C versions in test_statistics.py (GH-15453) 2019-08-24 10:51:20 -07:00
Greg Price 5e63ab05f1 bpo-37812: Convert CHECK_SMALL_INT macro to a function so the return is explicit. (GH-15216) 2019-08-24 10:19:37 -07:00
GeeTransit a50f0a4c2b (TH-15402) 2019-08-24 13:01:36 -04:00
shireenrao a4e2991bdc bpo-37772: fix zipfile.Path.iterdir() outputs (GH-15170)
* fix Path._add_implied_dirs to include all implied directories

* fix Path._add_implied_dirs to include all implied directories

* Optimize code by using sets instead of lists

* 📜🤖 Added by blurb_it.

* fix Path._add_implied_dirs to include all implied directories

* Optimize code by using sets instead of lists

* 📜🤖 Added by blurb_it.

* Add tests to zipfile.Path.iterdir() fix

* Update test for zipfile.Path.iterdir()

* remove whitespace from test file

* Rewrite NEWS blurb to describe the user-facing impact and avoid implementation details.

* remove redundant [] within set comprehension

* Update to use unique_everseen to maintain order and other suggestions in review

* remove whitespace and add back add_dirs in tests

* Add new standalone function parents using posixpath to get parents of a directory

* removing whitespace (sorry)

* Remove import pathlib from zipfile.py

* Rewrite _parents as a slice on a generator of the ancestry of a path.

* Remove check for '.' and '/', now that parents no longer returns those.

* Separate calculation of implied dirs from adding those

* Re-use _implied_dirs in tests for generating zipfile with dir entries.

* Replace three fixtures (abcde, abcdef, abde) with one representative example alpharep.

* Simplify implementation of _implied_dirs by collapsing the generation of parent directories for each name.
2019-08-24 11:26:41 -04:00
Serhiy Storchaka ef61c524dd
bpo-37830: Fix compilation of break and continue in finally. (GH-15320)
Fix compilation of "break" and "continue" in the
"finally" block when the corresponding "try" block
contains "return" with a non-constant value.
2019-08-24 13:11:52 +03:00
Serhiy Storchaka e9c90aa431
bpo-29535: Remove promize about hash randomization of datetime objects. (GH-15269) 2019-08-24 12:49:27 +03:00
Dong-hee Na 0a18ee4be7 bpo-37798: Add C fastpath for statistics.NormalDist.inv_cdf() (GH-15266) 2019-08-23 15:20:30 -07:00
Zackery Spytz 5be666010e bpo-37549: os.dup() fails for standard streams on Windows 7 (GH-15389) 2019-08-23 11:38:41 -07:00
Raymond Hettinger 8f080b0995
bpo-26589: Add http status code 451 (GH-15413) 2019-08-23 10:19:15 -07:00
Victor Stinner 120b707a6d
bpo-36763: PyConfig_Read() handles PySys_AddXOption() (GH-15431)
PyConfig_Read() is now responsible to handle early calls to
PySys_AddXOption() and PySys_AddWarnOption().

Options added by PySys_AddXOption() are now handled the same way than
PyConfig.xoptions and command line -X options.

For example, PySys_AddXOption(L"faulthandler") enables faulthandler
as expected.
2019-08-23 18:03:08 +01:00
Stefan Behnel b5d3ceea48
bpo-14465: Add an indent() function to xml.etree.ElementTree to pretty-print XML trees (GH-15200) 2019-08-23 16:44:25 +02:00
Victor Stinner c48682509d
bpo-37926: Fix PySys_SetArgvEx(0, NULL, 0) crash (GH-15415)
empty_argv is no longer static in Python 3.8, but it is declared in
a temporary scope, whereas argv keeps a reference to it.
empty_argv memory (allocated on the stack) is reused by
make_sys_argv() code which is inlined when using gcc -O3.

Define empty_argv in PySys_SetArgvEx() body, to ensure
that it remains valid for the whole lifetime of
the PySys_SetArgvEx() call.
2019-08-23 11:04:16 +01:00
Pablo Galindo 4be11c009a bpo-37915: Fix comparison between tzinfo objects and timezone objects (GH-15390)
https://bugs.python.org/issue37915



Automerge-Triggered-By: @pablogsal
2019-08-22 12:24:25 -07:00
Raymond Hettinger d0cdeaab76
bpo-32554: Deprecate hashing arbitrary types in random.seed() (GH-15382) 2019-08-22 09:19:36 -07:00
Steve Dower 7ebdda0dbe
bpo-36311: Fixes decoding multibyte characters around chunk boundaries and improves decoding performance (GH-15083) 2019-08-21 16:22:33 -07:00
bsiem df0c21ff46 bpo-37482: Fix email address name with encoded words and special chars (GH-14561)
Special characters in email address header display names are normally
put within double quotes. However, encoded words (=?charset?x?...?=) are
not allowed withing double quotes. When the header contains a word with
special characters and another word that must be encoded, the first one
must also be encoded.

In the next example, the display name in the From header is quoted and
therefore the comma is allowed; in the To header, the comma is not
within quotes and not encoded, which is not allowed and therefore
rejected by some mail servers.

From: "Foo Bar, France" <foo@example.com>
To: Foo Bar, =?utf-8?q?Espa=C3=B1a?= <foo@example.com>





https://bugs.python.org/issue37482
2019-08-21 16:00:39 -07:00
Brett Cannon 48ede6b8f6
bpo-37663: have venv activation scripts all consistently use __VENV_PROMPT__ for prompt customization (GH-14941)
The activation scripts generated by venv were inconsistent in how they changed the shell's prompt. Some used `__VENV_PROMPT__` exclusively, some used `__VENV_PROMPT__` if it was set even though by default `__VENV_PROMPT__` is always set and the fallback matched the default, and one ignored `__VENV_PROMPT__` and used `__VENV_NAME__` instead (and even used a differing format to the default prompt). This change now has all activation scripts use `__VENV_PROMPT__` only and relies on the fact that venv sets that value by default.

The color of the customization is also now set in fish to the blue from the Python logo for as hex color support is built into that shell (much like PowerShell where the built-in green color is used).
2019-08-21 15:58:01 -07:00
Steve Dower df2d4a6f3d
bpo-37834: Normalise handling of reparse points on Windows (GH-15231)
bpo-37834: Normalise handling of reparse points on Windows
* ntpath.realpath() and nt.stat() will traverse all supported reparse points (previously was mixed)
* nt.lstat() will let the OS traverse reparse points that are not name surrogates (previously would not traverse any reparse point)
* nt.[l]stat() will only set S_IFLNK for symlinks (previous behaviour)
* nt.readlink() will read destinations for symlinks and junction points only

bpo-1311: os.path.exists('nul') now returns True on Windows
* nt.stat('nul').st_mode is now S_IFCHR (previously was an error)
2019-08-21 15:27:33 -07:00
Steve Dower 75e064962e
bpo-9949: Enable symlink traversal for ntpath.realpath (GH-15287) 2019-08-21 13:43:06 -07:00
Anthony Sottile e1c638da6a Fix difflib `?` hint in diff output when dealing with tabs (#15201) 2019-08-21 13:59:25 -05:00
Jordon Xu 20f59fe1f7 bpo-37751: Fix codecs.lookup() normalization (GH-15092)
Fix codecs.lookup() to normalize the encoding name the same way
than encodings.normalize_encoding(), except that codecs.lookup()
also converts the name to lower case.
2019-08-21 14:26:20 +01:00
Victor Stinner d8c5adf6f8
bpo-37851: faulthandler allocates its stack on demand (GH-15358)
The faulthandler module no longer allocates its alternative stack at
Python startup. Now the stack is only allocated at the first
faulthandler usage.

faulthandler no longer ignores memory allocation failure when
allocating the stack. sigaltstack() failure now raises an OSError
exception, rather than being ignored.

The alternative stack is no longer used if sigaction() is
not available. In practice, sigaltstack() should only be available
when sigaction() is avaialble, so this change should have no effect
in practice.

faulthandler.dump_traceback_later() internal locks are now only
allocated at the first dump_traceback_later() call, rather than
always being allocated at Python startup.
2019-08-21 13:40:42 +01:00
Victor Stinner de2d9eed8b
bpo-37531: Enhance regrtest multiprocess timeout (GH-15345)
* Write a message when killing a worker process
* Put a timeout on the second popen.communicate() call
  (after killing the process)
* Put a timeout on popen.wait() call
* Catch popen.kill() and popen.wait() exceptions
2019-08-21 10:59:20 +01:00
Greg Price 9ece4a5057 Unmark files as executable that can't actually be executed. (GH-15353)
There are plenty of legitimate scripts in the tree that begin with a
`#!`, but also a few that seem to be marked executable by mistake.

Found them with this command -- it gets executable files known to Git,
filters to the ones that don't start with a `#!`, and then unmarks
them as executable:

    $ git ls-files --stage \
      | perl -lane 'print $F[3] if (!/^100644/)' \
      | while read f; do
          head -c2 "$f" | grep -qxF '#!' \
          || chmod a-x "$f"; \
        done

Looking at the list by hand confirms that we didn't sweep up any
files that should have the executable bit after all.  In particular

 * The `.psd` files are images from Photoshop.

 * The `.bat` files sure look like things that can be run.
   But we have lots of other `.bat` files, and they don't have
   this bit set, so it must not be needed for them.



Automerge-Triggered-By: @benjaminp
2019-08-20 21:53:59 -07:00
Zackery Spytz 46a7564578 bpo-32793: Fix a duplicate debug message in smtplib (GH-15341)
_get_socket() already prints a debug message for the host and port.



https://bugs.python.org/issue32793



Automerge-Triggered-By: @maxking
2019-08-20 10:52:25 -07:00
Joannah Nanjekye 9e66aba999 bpo-15913: Implement PyBuffer_SizeFromFormat() (GH-13873)
Implement PyBuffer_SizeFromFormat() function (previously
documented but not implemented): call struct.calcsize().
2019-08-20 15:46:36 +01:00
Eric V. Smith b0f4dab873
bpo-37868: Improve is_dataclass for instances. (GH-15325) 2019-08-20 01:40:28 -04:00
Antoine d3c8d73514 Minor documentation fixes on library/enum (GH-15234)
* Minor documentation fixes on library/enum
2019-08-19 18:41:31 -07:00
Victor Stinner d11c2c6077
Revert "bpo-37788: Fix a reference leak if a thread is not joined (GH-15228)" (GH-15338)
This reverts commit d3dcc92778.
2019-08-20 00:47:07 +01:00
Victor Stinner d3dcc92778
bpo-37788: Fix a reference leak if a thread is not joined (GH-15228)
Add threading.Thread.__del__() method to ensure that the thread state
lock is removed from the _shutdown_locks list when a thread
completes.
2019-08-19 23:37:17 +01:00
shireenrao 24fe46081b bpo-36266: Add module name in ImportError when DLL not found on Windows (GH-15180) 2019-08-17 13:50:39 -07:00
Jeroen Demeyer 0567786d26 bpo-37540: vectorcall: keyword names must be strings (GH-14682)
The fact that keyword names are strings is now part of the vectorcall and `METH_FASTCALL` protocols. The biggest concrete change is that `_PyStack_UnpackDict` now checks that and raises `TypeError` if not.

CC @markshannon @vstinner 


https://bugs.python.org/issue37540
2019-08-16 03:41:27 -07:00
Raymond Hettinger f3cb68f2e4
bpo-37863: Optimize Fraction.__hash__() (#15298) 2019-08-15 20:58:26 -07:00
Jeroen Demeyer 37806f404f bpo-37207: enable vectorcall for type.__call__ (GH-14588)
Base PR for other PRs that want to play with `type.__call__` such as #13930 and #14589.

The author is really @markshannon I just made the PR.


https://bugs.python.org/issue37207



Automerge-Triggered-By: @encukou
2019-08-15 08:49:46 -07:00
Victor Stinner ac827edc49
bpo-21131: Fix faulthandler.register(chain=True) stack (GH-15276)
faulthandler now allocates a dedicated stack of SIGSTKSZ*2 bytes,
instead of just SIGSTKSZ bytes. Calling the previous signal handler
in faulthandler signal handler uses more than SIGSTKSZ bytes of stack
memory on some platforms.
2019-08-14 23:35:27 +02:00
Artem Khramov 2814620657 bpo-37811: FreeBSD, OSX: fix poll(2) usage in sockets module (GH-15202)
FreeBSD implementation of poll(2) restricts the timeout argument to be
either zero, or positive, or equal to INFTIM (-1).

Unless otherwise overridden, socket timeout defaults to -1. This value
is then converted to milliseconds (-1000) and used as argument to the
poll syscall. poll returns EINVAL (22), and the connection fails.

This bug was discovered during the EINTR handling testing, and the
reproduction code can be found in
https://bugs.python.org/issue23618 (see connect_eintr.py,
attached). On GNU/Linux, the example runs as expected.

This change is trivial:
If the supplied timeout value is negative, truncate it to -1.
2019-08-14 23:21:48 +02:00
Tal Einat 71662dc2f1 bpo-37849: IDLE: fix completion window positioning above line (GH-15267) 2019-08-14 13:06:06 -04:00
Victor Stinner b0c8369c60
bpo-37531: Fix regrtest timeout for subprocesses (GH-15072)
Co-Authored-By: Joannah Nanjekye <joannah.nanjekye@ibm.com>
2019-08-14 14:18:51 +02:00
Victor Stinner 077af8c2c9
bpo-37738: Fix curses addch(str, color_pair) (GH-15071)
Fix the implementation of curses addch(str, color_pair): pass the
color pair to setcchar(), instead of always passing 0 as the color
pair.
2019-08-14 12:31:43 +02:00
Ngalim Siregar 38c7199beb bpo-37256: Wording in Request class docs (#14792)
* bpo-37256: Wording in Request class docs

* 📜🤖 Added by blurb_it.

* Update Misc/NEWS.d/next/Documentation/2019-07-16-14-48-12.bpo-37256.qJTrBb.rst

Co-Authored-By: Kyle Stanley <aeros167@gmail.com>
2019-08-13 18:10:58 -07:00
Hai Shi 82642a052d bpo-37689: add Path.is_relative_to() method (GH-14982) 2019-08-13 21:54:02 +02:00
Derek Keeler 732775d6be bpo-37354: Make Powershell Activate.ps1 script static to allow for signing (GH-14967)
- Remove use of replacement text in the script
- Make use of the pyvenv.cfg file for prompt value.
- Add parameters to allow more flexibility
- Make use of the current path, and assumptions about where env puts things, to compensate
- Make the script a bit more 'idiomatic' Powershell
- Add script documentation (Get-Help .\.venv\Scripts\Activate.ps1 shows PS help page now
2019-08-12 13:06:02 -07:00
Dong-hee Na 44046fe4fc bpo-37804: Remove the deprecated method threading.Thread.isAlive() (GH-15225) 2019-08-12 19:41:08 +02:00
Raymond Hettinger f03b4c8a48
bpo-37819: Add Fraction.as_integer_ratio() (GH-15212) 2019-08-11 14:40:59 -07:00
Abhilash Raj 09a1872a80 bpo-32178: Fix IndexError trying to parse 'To' header starting with ':'. (GH-15044)
This should fix the IndexError trying to retrieve `DisplayName.display_name` and `DisplayName.value` when the `value` is basically an empty string.




https://bugs.python.org/issue32178
2019-08-11 13:45:09 -07:00
Gregory P. Smith b4be87a04a bpo-32912: Revert SyntaxWarning on invalid escape sequences. (GH-15195)
DeprecationWarning will continue to be emitted for invalid escape
sequences in string and bytes literals just as it did in 3.7.

SyntaxWarning may be emitted in the future. But per mailing list
discussion, we don't yet know when because we haven't settled on how to
do so in a non-disruptive manner.

(Applies 4c5b6bac24 to the master branch).
(This is https://github.com/python/cpython/pull/15142 for master/3.9)


https://bugs.python.org/issue32912



Automerge-Triggered-By: @gpshead
2019-08-10 00:19:07 -07:00
Ngalim Siregar 92c7e30adf bpo-37642: Update acceptable offsets in timezone (GH-14878)
This fixes an inconsistency between the Python and C implementations of
the datetime module. The pure python version of the code was not
accepting offsets greater than 23:59 but less than 24:00. This is an
accidental legacy of the original implementation, which was put in place
before tzinfo allowed sub-minute time zone offsets.

GH-14878
2019-08-09 10:22:16 -04:00
shireenrao f4e725f224 bpo-25172: Raise appropriate ImportError msg when crypt module used on Windows (GH-15149) 2019-08-08 21:02:49 +01:00
Serhiy Storchaka 662db125cd
bpo-37685: Fixed __eq__, __lt__ etc implementations in some classes. (GH-14952)
They now return NotImplemented for unsupported type of the other operand.
2019-08-08 08:42:54 +03:00
aiudirog 4c69be22df bpo-34775: Return NotImplemented in PurePath division. (GH-9509) 2019-08-08 08:41:10 +03:00
Steve Dower 87ce9588ce
bpo-37778: Fixes the icons used for file associations to the Microsoft Store package (GH-15150) 2019-08-07 10:50:17 -07:00
Steve Dower 1fab9cbfba
bpo-37734: Fix use of registry values to launch Python from Microsoft Store app (GH-15146) 2019-08-07 10:49:40 -07:00
sweeneyde e9cbcd0018 bpo-37004: Documented asymmetry of string arguments in difflib.SequenceMatcher for ratio method (GH-13482)
https://bugs.python.org/issue37004
2019-08-06 21:37:08 -07:00
Sergey Fedoseev 3e41f3cabb bpo-34488: optimize BytesIO.writelines() (GH-8904)
Avoid the creation of unused int object for each line.
2019-08-07 09:38:31 +09:00
Raymond Hettinger 4f9ffc9d1a
bpo-37759: First round of major edits to Whatsnew 3.8 (GH-15127) 2019-08-05 13:33:19 -07:00
Terry Jan Reedy 1213123005
bpo-36419: IDLE - Refactor autocompete and improve testing. (#15121) 2019-08-04 19:48:52 -04:00
Timothy Hopper d748a80855 Adds Tim Hopper and Dan Lidral-Porter to ACKS (GH-15101)
Tim and Dan were authors for GH-11847
2019-08-04 14:23:29 -07:00
Terry Jan Reedy 14070299cd
bpo-37748: Re-order the Run menu. (GH-15115)
Put the most common choice, Run Module, at the top.
2019-08-04 16:45:15 -04:00
Raymond Hettinger b1c8ec010f
bpo-28292: Mark calendar.py helper functions as private. (GH-15113) 2019-08-04 13:14:03 -07:00
Serhiy Storchaka 18b711c5a7
bpo-37648: Fixed minor inconsistency in some __contains__. (GH-14904)
The collection's item is now always at the left and
the needle is on the right of ==.
2019-08-04 14:12:48 +03:00
Serhiy Storchaka 17e52649c0
bpo-37685: Fixed comparisons of datetime.timedelta and datetime.timezone. (GH-14996)
There was a discrepancy between the Python and C implementations.

Add singletons ALWAYS_EQ, LARGEST and SMALLEST in test.support
to test mixed type comparison.
2019-08-04 12:38:46 +03:00
Michael Felt 9670ce76b8 bpo-18049: Define THREAD_STACK_SIZE for AIX to pass default recursion limit test (GH-15081)
* Define THREAD_STACK_SIZE for AIX to pass default recursion limit test
2019-08-03 08:12:26 +02:00
Ngalim Siregar c5fa44944e bpo-37444: Update differing exception between builtins and importlib (GH-14869)
Imports now raise `TypeError` instead of `ValueError` for relative import failures. This makes things consistent between `builtins.__import__` and `importlib.__import__` as well as using a more natural import for the failure.


https://bugs.python.org/issue37444



Automerge-Triggered-By: @brettcannon
2019-08-02 22:46:02 -07:00
Timothy Hopper 7ea9a85f13 bpo-20523: pdb searches for .pdbrc in ~ instead of $HOME (GH-11847)
Previously pdb checked the $HOME environmental variable
to find the user .pdbrc. If $HOME is not set, the user
.pdbrc would not be found.

Change pdb to use `os.path.expanduser('~')` to determine
the user's home directory. Thus, if $HOME is not set (as
in tox or on Windows), os.path.expanduser('~') falls
back on other techniques for locating the user's home
directory.

This follows pip's implementation for loading .piprc.

Co-authored-by: Dan Lidral-Porter <dlp@aperiodic.org>
2019-08-02 15:20:14 -07:00
Greg Bowser 8fbece135d bpo-36590: Add Bluetooth RFCOMM and support for Windows. (GH-12767)
Support for RFCOMM, L2CAP, HCI, SCO is based on the BTPROTO_* macros
being defined. Winsock only supports RFCOMM, even though it has a
BTHPROTO_L2CAP macro. L2CAP support would build on windows, but not
necessarily work.

This also adds some basic unittests for constants (all of which existed
prior to this commit, just not on windows) and creating sockets.

pair: Nate Duarte <slacknate@gmail.com>
2019-08-02 13:29:52 -07:00
Vinay Sajip cb65b3a4f4
bpo-37742: Return the root logger when logging.getLogger('root') is c… (#15077)
* bpo-37742: Return the root logger when logging.getLogger('root') is called.

* Added type check guard on logger name in logging.getLogger() and refined a test.
2019-08-02 16:53:00 +01:00
Joannah Nanjekye 854d0a4b98 bpo-36487: Make C-API docs clear about what the main interpreter is. (gh-12666) 2019-08-02 09:50:22 -06:00
tmblweed 4b3e975923 bpo-16970: Adding error message for invalid args (GH-14844)
BPO -16970: Adding error message for invalid args

Applied the patch argparse-v2 patch issue 16970, ran patch check and the test suite, test_argparse with 0 errors


https://bugs.python.org/issue16970
2019-08-01 21:57:13 -07:00
mental 2491134029 bpo-37726: Prefer argparse over getopt in stdlib tutorial (#15052) 2019-08-01 07:17:30 -07:00
Bernhard M. Wiedemann 0d30ae1a03 bpo-36302: Sort list of sources (GH-12341)
When building packages (e.g. for openSUSE Linux)
(random) filesystem order of input files
influences ordering of functions in the output .so files.
Thus without the patch, builds (in disposable VMs) would usually differ.

Without this patch, all callers have to be patched individually
https://github.com/dugsong/libdnet/pull/42
https://github.com/sass/libsass-python/pull/212
https://github.com/tahoe-lafs/pycryptopp/pull/41
https://github.com/yt-project/yt/pull/2206
https://github.com/pyproj4/pyproj/pull/142
https://github.com/pytries/datrie/pull/49
https://github.com/Roche/pyreadstat/pull/37
but that is an infinite effort.

See https://reproducible-builds.org/ for why this matters.
2019-08-01 15:18:03 +02:00
Ronald Oussoren 1a057bab0f
bpo-18049: Sync thread stack size to main thread size on macOS (GH-14748)
This changeset increases the default size of the stack
for threads on macOS to the size of the stack
of the main thread and reenables the relevant
recursion test.
2019-08-01 07:43:07 +02:00
yannvgn 9f55551f3d bpo-37723: Fix performance regression on regular expression parsing. (GH-15030)
Improve performance of sre_parse._uniq function.
2019-07-31 21:50:39 +03:00
Anthony Sottile c9345e382c bpo-37695: Correct unget_wch error message. (GH-14986) 2019-07-31 15:11:24 +03:00
karl ding 31c4fd2a10 bpo-37085: Expose SocketCAN bcm_msg_head flags (#13646)
Expose the CAN_BCM SocketCAN constants used in the bcm_msg_head struct
flags (provided by <linux/can/bcm.h>) under the socket library.

This adds the following constants with a CAN_BCM prefix:

  * SETTIMER
  * STARTTIMER
  * TX_COUNTEVT
  * TX_ANNOUNCE
  * TX_CP_CAN_ID
  * RX_FILTER_ID
  * RX_CHECK_DLC
  * RX_NO_AUTOTIMER
  * RX_ANNOUNCE_RESUME
  * TX_RESET_MULTI_IDX
  * RX_RTR_FRAME
  * CAN_FD_FRAME

The CAN_FD_FRAME flag was introduced in the 4.8 kernel, while the other
ones were present since SocketCAN drivers were mainlined in 2.6.25. As
such, it is probably unnecessary to guard against these constants being
missing.
2019-07-31 10:47:16 +02:00
Terry Jan Reedy d04f8907ba
IDLE: changelog correction and addition (GH-15042) 2019-07-31 01:35:30 -04:00
Min ho Kim c4cacc8c5e Fix typos in comments, docs and test names (#15018)
* Fix typos in comments, docs and test names

* Update test_pyparse.py

account for change in string length

* Apply suggestion: splitable -> splittable

Co-Authored-By: Terry Jan Reedy <tjreedy@udel.edu>

* Apply suggestion: splitable -> splittable

Co-Authored-By: Terry Jan Reedy <tjreedy@udel.edu>

* Apply suggestion: Dealloccte -> Deallocate

Co-Authored-By: Terry Jan Reedy <tjreedy@udel.edu>

* Update posixmodule checksum.

* Reverse idlelib changes.
2019-07-30 18:16:13 -04:00
Neil Schemenauer 52a48e62c6
bpo-37707: Exclude expensive unit tests from PGO task (GH-15009)
Mark some individual tests to skip when --pgo is used.  The tests
marked increase the PGO task time significantly and likely don't
help improve optimization of the final executable.
2019-07-30 11:08:18 -07:00
Victor Stinner 27eeaf0f2c
bpo-37704: Remove Tools/scripts/h2py.py (GH-15000)
Use cffi to access a C API in Python.
2019-07-30 17:45:09 +02:00
Marco Paolini 8a758f5b99 bpo-37587: Make json.loads faster for long strings (GH-14752)
When scanning the string, most characters are valid, so
checking for invalid characters first means never needing
to check the value of strict on valid strings, and only
needing to check it on invalid characters when doing
non-strict parsing of invalid strings.

This provides a measurable reduction in per-character
processing time (~11% in the pre-merge patch testing).
2019-07-31 00:16:34 +10:00
Pablo Galindo 9211e2fd81 bpo-37268: Add deprecation notice and a DeprecationWarning for the parser module (GH-15017)
Deprecate the parser module and add a deprecation warning triggered on import and a warning block in the documentation.





https://bugs.python.org/issue37268



Automerge-Triggered-By: @pablogsal
2019-07-30 04:04:01 -07:00
Anthony Sottile 5b94f3578c Fix `SyntaxError` indicator printing too many spaces for multi-line strings (GH-14433) 2019-07-29 14:59:13 +01:00
Jason R. Coombs 049460da9c
bpo-37697: Sync with importlib_metadata 0.19 (#14993)
* bpo-37697: Sync with importlib_metadata 0.19

* Run make regen-importlib

* 📜🤖 Added by blurb_it.
2019-07-28 14:59:24 -04:00
Terry Jan Reedy b222955355
bpo-37692: Improve highlight config sample (#14983)
Use an example shell interaction in the sample and better labels for shell elements.
2019-07-28 12:04:31 -04:00
Raymond Hettinger 6b5f1b496f
bpo-37691: Let math.dist() accept sequences and iterables for coordinates (GH-14975) 2019-07-27 14:04:29 -07:00
Tal Einat 3221a63c69 bpo-37628: Fix IDLE config sample sizes (#14958)
The boxes for the font and highlight samples are now constrained by the overall config dialog size.  They gain scrollbars when the when a large font size makes the samples too large for the box.
2019-07-27 12:57:48 -04:00
Derek Keeler 91e4957509 bpo-32910: Remove implementation detail in venv documentation. (GH-14968) 2019-07-26 14:57:11 -07:00
Inada Naoki 3e54b57531
bpo-37340: remove free_list for bound method objects (GH-14232) 2019-07-26 15:05:50 +09:00
Flavian Hautbois 76b645124b bpo-29446: tkinter 'import *' only imports what it should (GH-14864)
Add __all__ to tkinter.__init__ and submodules.  Replace 'import *'
with explicit imports in some submodules.
2019-07-25 21:30:33 -04:00
Markus Mohrhard 898318b53d bpo-37502: handle default parameter for buffers argument of pickle.loads correctly (GH-14593) 2019-07-25 18:00:34 +02:00
Benjamin Peterson 93e8aa62cf
closes bpo-37675: Use pkgutil.iter_modules to find fixers in a package rather than listdir. (14942) 2019-07-24 16:38:50 -07:00
Steve Dower 123536fdab
bpo-37672: Switch Windows Store package to use pip.ini for user mode (GH-14939) 2019-07-24 15:13:22 -07:00
Pradyun Gedam 5380def826 bpo-37664: Update bundled pip to 19.2.1 and setuptools to 41.0.1 (GH-14934) 2019-07-24 12:08:02 -07:00
Jeroen Demeyer 151b91dfd2 bpo-29548: deprecate PyEval_Call* functions (GH-14804) 2019-07-24 21:02:49 +09:00
Tal Einat 7123ea009b
bpo-17535: IDLE editor line numbers (GH-14030) 2019-07-23 15:22:11 +03:00
Neil Schemenauer 4e16a4a311
bpo-36044: Reduce number of unit tests run for PGO build (GH-14702)
Reduce the number of unit tests run for the PGO generation task.  This
speeds up the task by a factor of about 15x.  Running the full unit test
suite is slow.  This change may result in a slightly less optimized build
since not as many code branches will be executed.  If you are willing to
wait for the much slower build, the old behavior can be restored using
'./configure [..] PROFILE_TASK="-m test --pgo-extended"'.  We make no
guarantees as to which PGO task set produces a faster build.  Users who
care should run their own relevant benchmarks as results can depend on
the environment, workload, and compiler tool chain.
2019-07-22 12:54:25 -07:00
Xtreak 7397cda997 bpo-21478: Record calls to parent when autospecced objects are used as child with attach_mock (GH 14688)
* Clear name and parent of mock in autospecced objects used with attach_mock

* Add NEWS entry

* Fix reversed order of comparison

* Test child and standalone function calls

* Use a helper function extracting mock to avoid code duplication and refactor tests.
2019-07-22 08:38:22 +01:00
Terry Jan Reedy 4214f1ec3b
bpo-37627: Add acknowledgment (#14883) 2019-07-21 16:26:24 -04:00
Min ho Kim 96e12d5f4f Fix typos in docs, comments and test assert messages (#14872) 2019-07-21 16:12:33 -04:00
Ngalim Siregar 35b87e6001 bpo-37627: Initialize IDLE Custom Run dialog with previous entries (#14870)
Repeat the command line arguments most recently entered before so the user can edit them.
2019-07-21 11:37:28 -04:00
Raymond Hettinger 02c91f59b6
bpo-36324: Make internal attributes for statistics.NormalDist() private. (GH-14871)
* Make internals private

* Finish making mu and sigma private

* Add missing __hash__() method

* Add blurb
2019-07-21 00:34:47 -07:00
Tal Einat e0a1f8fb5c bpo-33610: IDLE's code-context always shows current context immediately (GH-14821)
Eliminate delay of up to 100ms and accompanying visual artifact.
Fix bug of never showing context when hide and show.
2019-07-18 16:03:18 -04:00
Walter Dörwald 323842c279 bpo-34443: Use __qualname__ instead of __name__ in enum exception messages. (GH-14809)
* Use __qualname__ instead of __name__ in enum exception messages.
2019-07-18 11:37:13 -07:00
Terry Jan Reedy 1b38922434
bpo-36390: Gather IDLE Format menu functions into format.py (#14827)
Add two indent spec methods from editor and Rstrip to existing file.
Tests are not added for indent methods because they need change
in lights of 3.x's prohibition on mixing tabs and spaces.
2019-07-17 20:48:36 -04:00
jpic 8cb65d1381 bpo-34155: Dont parse domains containing @ (GH-13079)
Before:
    
        >>> email.message_from_string('From: a@malicious.org@important.com', policy=email.policy.default)['from'].addresses
        (Address(display_name='', username='a', domain='malicious.org'),)
    
        >>> parseaddr('a@malicious.org@important.com')
        ('', 'a@malicious.org')
    
    After:
    
        >>> email.message_from_string('From: a@malicious.org@important.com', policy=email.policy.default)['from'].addresses
        (Address(display_name='', username='', domain=''),)
    
        >>> parseaddr('a@malicious.org@important.com')
        ('', 'a@')




https://bugs.python.org/issue34155
2019-07-17 14:54:25 -07:00
Abhilash Raj 719a062bcb Fix IndexError when parsing unexpectedly ending quoted-string. (GH-14813)
This exception was caused because the input ended unexpectedly with only one
single quote instead of a pair with some value inside it.
2019-07-17 09:48:52 -07:00
Abhilash Raj a4a994bd3e bpo-37461: Fix infinite loop in parsing of specially crafted email headers (GH-14794)
* bpo-37461: Fix infinite loop in parsing of specially crafted email headers.

Some crafted email header would cause the get_parameter method to run in an
infinite loop causing a DoS attack surface when parsing those headers. This
patch fixes that by making sure the DQUOTE character is handled to prevent
going into an infinite loop.
2019-07-17 09:44:27 -07:00
Cheryl Sabella 82494aa6d9 bpo-36390: IDLE: Combine region formatting methods. (GH-12481)
Rename paragraph.py to format.py and add region formatting methods
from editor.py.  Add tests for the latter.
2019-07-17 09:44:44 -04:00
Inada Naoki fb26504d14
bpo-37543: optimize pymalloc (#14674)
PyObject_Malloc() and PyObject_Free() inlines pymalloc_alloc and
pymalloc_free partially.
But when PGO is not used, compiler don't know where is the hot part
in pymalloc_alloc and pymalloc_free.
2019-07-17 21:23:57 +09:00
Tal Einat 7036e1de3a
bpo-37530: simplify, optimize and clean up IDLE code context (GH-14675)
* Only create CodeContext instances for "real" editors windows, but
  not e.g. shell or output windows.
* Remove configuration update Tk event fired every second, by having
  the editor window ask its code context widget to update when
  necessary, i.e. upon font or highlighting updates.
* When code context isn't being shown, avoid having a Tk event fired
  every 100ms to check whether the code context needs to be updated.
* Use the editor window's getlineno() method where applicable.
* Update font of the code context widget before the main text widget
2019-07-17 11:15:53 +03:00
Cheryl Sabella f8d4cc7dbb bpo-27452: IDLE: Cleanup config.py code (GH-14577) 2019-07-16 16:58:25 -04:00
Paul Ganssle f69d5c6198 Fix infinite loop in email folding logic (GH-12732)
As far as I can tell, this infinite loop would be triggered if:

1. The value being folded contains a single word (no spaces) longer than
   max_line_length
2. The max_line_length is shorter than the encoding's name + 9
   characters.

bpo-36564: https://bugs.python.org/issue36564
2019-07-16 10:50:01 -07:00
Victor Stinner e8692818af
bpo-37468: make install no longer install wininst-*.exe files (GH-14511)
make install no longer installs "wininst-*.exe" files used by
distutils bdist_wininst: bdist_wininst only works on Windows.
2019-07-15 17:18:42 +02:00
Giovanni Cappellotto 52693c10e8 bpo-37284: Add note to sys.implementation doc (GH-14328)
Add a brief note to indicate that any new required attributes must go through the PEP process.





https://bugs.python.org/issue37284
2019-07-15 07:37:09 -07:00
Pablo Galindo cd6e83b481 bpo-37593: Swap the positions of posonlyargs and args in the constructor of ast.parameters nodes (GH-14778)
https://bugs.python.org/issue37593
2019-07-14 16:32:18 -07:00
Sergey Fedoseev 1c5e68e714 bpo-34749: Improved performance of binascii.a2b_base64(). (GH-9444)
https://bugs.python.org/issue34749
2019-07-14 05:15:32 -07:00
Zac Hatfield-Dodds dffca9e925 bpo-26967: fix flag grouping with allow_abbrev=False (GH-14316)
The `allow_abbrev` option for ArgumentParser is documented and intended to disable support for unique prefixes of --options, which may sometimes be ambiguous due to deferred parsing.

However, the initial implementation also broke parsing of grouped short flags, such as `-ab` meaning `-a -b` (or `-a=b`).  Checking the argument for a leading `--` before rejecting it fixes this.

This was prompted by pytest-dev/pytest#5469, so a backport to at least 3.8 would be great 😄  
And this is my first PR to CPython, so please let me know if I've missed anything!


https://bugs.python.org/issue26967
2019-07-13 22:35:58 -07:00
Sviatoslav Sydorenko e44184749c bpo-30088: Document that existing dir structure isn't verified by mailbox.Maildir (GH-1163)
Hi,

I've faced an issue w/ `mailbox.Maildir()`. The case is following:
1. I create a folder with `tempfile.TemporaryDirectory()`, so it's empty
2. I pass that folder path as an argument when instantiating `mailbox.Maildir()`
3. Then I receive an exception happening because "there's no such file or directory" (namely `cur`, `tmp` or `new`) during interaction with Maildir

**Expected result:** subdirs are created during `Maildir()` instance creation.

**Actual result:** subdirs are assumed as existing which leads to exceptions during use.

**Workaround:** remove the actual dir before passing the path to `Maildir()`. It will be created automatically with all subdirs needed.

**Fix:** This PR. Basically it adds creation of subdirs regardless of whether the base dir existed before.


https://bugs.python.org/issue30088
2019-07-13 07:47:14 -07:00
Jeroen Demeyer ed184c06e2 bpo-37358: Use vectorcall for functools.partial (GH-14284)
https://bugs.python.org/issue37358
2019-07-13 07:39:18 -07:00
Giovanni Cappellotto dc3f99fa77 bpo-37548: Document range of atan, acos and asin (GH-14717) 2019-07-13 14:59:55 +01:00
Xtreak e6b46aafad bpo-37579: Improve equality behavior for pure Python datetime and time (GH-14726)
Returns NotImplemented for timedelta and time in __eq__ for different types in Python implementation, which matches the C implementation.

This also adds tests to enforce that these objects will fall back to the right hand side's __eq__ and/or __ne__ implementation.

bpo-37579
2019-07-13 15:22:21 +02:00
Minmin Gong 05f2d84cae bpo-28269: Replace strcasecmp with system function _stricmp. (GH-13095) 2019-07-13 16:11:43 +03:00
gescheit b9a0376b0d closes bpo-37347: Fix refcount problem in sqlite3. (GH-14268) 2019-07-12 20:15:48 -07:00
Brett Cannon 0827064c95 bpo-37521: No longer treat insertion into sys.modules as optional in importlib examples (GH-14723)
Fix importlib examples to insert any newly created modules via importlib.util.module_from_spec() immediately into sys.modules instead of after calling loader.exec_module().

Thanks to Benjamin Mintz for finding the bug.


https://bugs.python.org/issue37521
2019-07-12 15:35:34 -07:00
Jakub Kulík 4737265622 bpo-37558: Shared memory tests are failing due to double slashes (GH-14703)
With the addition of shared memory into Python 3.8, we now have three tests failing on Solaris, namely `test_multiprocessing_fork`, `test_multiprocessing_forkserver` and `test_multiprocessing_spawn`. The reason seems to be incorrect name handling which results in two slashes being prepended.


https://bugs.python.org/issue37558
2019-07-11 11:04:09 -07:00
Jeroen Demeyer 59ad110d7a bpo-37547: add _PyObject_CallMethodOneArg (GH-14685) 2019-07-11 17:59:05 +09:00
Neil Schemenauer 5d25f2b703
bpo-37537: Compute allocated blocks in _Py_GetAllocatedBlocks() (#14680)
Keeping an account of allocated blocks slows down _PyObject_Malloc()
and _PyObject_Free() by a measureable amount.  Have
_Py_GetAllocatedBlocks() iterate over the arenas to sum up the
allocated blocks for pymalloc.
2019-07-10 12:04:16 -07:00
Victor Stinner 91b4f7ab7f
bpo-37526: Add support.catch_threading_exception() (GH-14664)
Context manager catching threading.Thread exception using
threading.excepthook.
2019-07-09 13:00:23 +02:00
Carl Friedrich Bolz-Tereick 110a47c4f4 bpo-18374: fix wrong col_offset of some ast.BinOp instances (GH-14607)
Nested BinOp instances (e.g. a+b+c) had a wrong col_offset for the
second BinOp (e.g. 2 instead of 0 in the example). Fix it by using the
correct st node to copy the line and col_offset from in ast.c.
2019-07-08 22:17:56 +01:00
Victor Stinner e676244235
bpo-37421: test_concurrent_futures stops ForkServer (GH-14643)
test_concurrent_futures now explicitly stops the ForkServer instance
if it's running.
2019-07-08 10:49:11 +02:00
Jeroen Demeyer 762f93ff2e bpo-37337: Add _PyObject_CallMethodNoArgs() (GH-14267) 2019-07-08 17:19:25 +09:00
Jason R. Coombs 38f44b4a4a
bpo-37520: Correct behavior for zipfile.Path.parent (GH-14638)
* bpo-37520: Correct behavior for zipfile.Path.parent

* 📜🤖 Added by blurb_it.
2019-07-07 17:37:50 -04:00
Kyle Stanley 0717b4d9b3 bpo-37478: Specify possible exceptions for os.chdir() (GH-14611) 2019-07-06 21:20:15 -04:00
Terry Jan Reedy 6f2a8c0857
bpo-37456: Slash ('/') is now part of syntax. (GH-14627) 2019-07-06 18:13:02 -04:00
Terry Jan Reedy f8709e804d
bpo-37487: Fix PyList_GetItem index description. (GH-14623)
0 is a legal index.
2019-07-06 17:40:27 -04:00
Tal Einat fcf1d003bf
bpo-26806: add 30 to the recursion limit in IDLE's shell (GH-13944)
This is done to compensate for the extra stack frames added by
IDLE itself, which cause problems when setting the recursion limit
to low values.

This wraps sys.setrecursionlimit() and sys.getrecursionlimit()
as invisibly as possible.
2019-07-06 15:35:24 +03:00
Terry Jan Reedy 45bc61b971
bpo-37149: Replace dead link for online Tkinter reference (GH-14616)
Also fix a name misspelling.
2019-07-06 02:31:54 -04:00
Min ho Kim f7d72e48fb Fix some typos (GH-14435) 2019-07-05 14:39:32 -07:00
Victor Stinner 8fbeb14312
bpo-37421: multiprocessing tests now stop ForkServer (GH-14601)
multiprocessing tests now stop the ForkServer instance if it's
running: close the "alive" file descriptor to ask the server to stop
and then remove its UNIX address.
2019-07-05 16:15:39 +02:00
Jeroen Demeyer 0d722f3cd6 bpo-36974: separate vectorcall functions for each calling convention (GH-13781) 2019-07-05 14:48:24 +02:00
Victor Stinner 1da4462765
bpo-37481: Deprecate distutils bdist_wininst command (GH-14553)
The distutils bdist_wininst command is now deprecated, use
bdist_wheel (wheel packages) instead.
2019-07-05 10:44:12 +02:00
thatneat 2f19e82fbe bpo-37479: on Enum subclasses with mixins, __format__ uses overridden __str__ (GH-14545)
* bpo-37479: on Enum subclasses with mixins, __format__ uses overridden __str__
2019-07-04 11:28:37 -07:00
Jeroen Demeyer 196a530e00 bpo-37483: add _PyObject_CallOneArg() function (#14558) 2019-07-04 19:31:34 +09:00
Victor Stinner 9d40554e0d
bpo-37421: Fix multiprocessing get_temp_dir() finalizer (GH-14572)
Fix multiprocessing.util.get_temp_dir() finalizer: clear also the
'tempdir' configuration of the current process, so next call to
get_temp_dir() will create a new temporary directory, rather than
reusing the removed temporary directory.
2019-07-04 12:28:55 +02:00
Kyle Stanley 56ec4f1fde bpo-19696: Replace deprecated method in "test_import_pkg.py" (GH-14466)
Replacing the deprecated method "random.choose" to "random.choice" was technically not part of the original issue. However, it was discussed in the talk page and involved one of the files being moved. I assumed this was too minor to justify the creation of a separate issue. 

Also, I added my name to the contributors list in Misc/ACKS. This will be my third PR to cpython, forgot to do it in the previous ones. 



https://bugs.python.org/issue19696
2019-07-03 11:22:40 -07:00
Victor Stinner 74c9dd5777
bpo-37421: Fix test_distutils.test_build_ext() (GH-14564)
test_distutils.test_build_ext() is now able to remove the temporary
directory on Windows: don't import the newly built C extension ("xx")
in the current process, but test it in a separated process.
2019-07-03 11:12:27 +02:00
Victor Stinner 684cb47fff
bpo-37421: test_concurrent_futures cleans up multiprocessing (GH-14563)
test_concurrent_futures now cleans up multiprocessing to remove
immediately temporary directories created by
multiprocessing.util.get_temp_dir().

The test now uses setUpModule() and tearDownModule().
2019-07-03 11:10:30 +02:00
Victor Stinner b71d8d6795
bpo-37421: test_winconsoleio doesn't leak temp file anymore (GH-14562)
test_winconsoleio doesn't leak a temporary file anymore: use
tempfile.TemporaryFile() to remove it when the test completes.
2019-07-03 11:09:56 +02:00
Christian Heimes 477b1b2576 bpo-37463: match_hostname requires quad-dotted IPv4 (GH-14499)
ssl.match_hostname() no longer accepts IPv4 addresses with additional text
after the address and only quad-dotted notation without trailing
whitespaces. Some inet_aton() implementations ignore whitespace and all data
after whitespace, e.g. '127.0.0.1 whatever'.

Short notations like '127.1' for '127.0.0.1' were already filtered out.

The bug was initially found by Dominik Czarnota and reported by Paul Kehrer.

Signed-off-by: Christian Heimes <christian@python.org>



https://bugs.python.org/issue37463
2019-07-02 11:39:42 -07:00
Victor Stinner 7cb9204ee1
bpo-37421: urllib.request tests call urlcleanup() (GH-14529)
urllib.request tests now call urlcleanup() to remove temporary files
created by urlretrieve() tests and to clear the _opener global
variable set by urlopen() and functions calling indirectly urlopen().

regrtest now checks if urllib.request._url_tempfiles and
urllib.request._opener are changed by tests.
2019-07-02 14:50:19 +02:00
Victor Stinner 039fb49c18
bpo-37421: multiprocessing tests call _run_finalizers() (GH-14527)
multiprocessing tests now call explicitly _run_finalizers() to remove
immediately temporary directories created by
multiprocessing.util.get_temp_dir().
2019-07-02 13:32:32 +02:00
Jakub Kulík 61bf97e916 bpo-37335, test_c_locale_coercion: Remove unnecessary code (GH-14447)
Python initialization now ensures that sys stream encoding
names are always normalized by codecs.lookup(encoding).name.
Simplify test_c_locale_coercion: it doesn't have to normalize
encoding names anymore.
2019-07-02 12:48:28 +02:00
Ned Deily 5bbbc733e6
bpo-34602: Avoid failures setting macOS stack resource limit (GH-14546)
Under some conditions the earlier fix for bpo-18075, "Infinite recursion
tests triggering a segfault on Mac OS X", now causes failures on macOS
when attempting to change stack limit with resource.setrlimit
resource.RLIMIT_STACK, like regrtest does when running the test suite.
The reverted change had specified a non-default stack size when linking
the python executable on macOS.  As of macOS 10.14.4, the previous
code causes a hard failure when running tests, although similar
failures had been seen under some conditions under some earlier
systems.  Reverting the change to the interpreter stack size at link
time helped for release builds but caused some tests to fail when
built --with-pydebug.  Try the opposite approach: continue to build
the interpreter with an increased stack size on macOS and remove
the failing setrlimit call in regrtest initialization.  This will
definitely avoid the resource.RLIMIT_STACK error and should have
no, or fewer, side effects.
2019-07-02 03:12:18 -04:00
Steve Dower e226e83d36
bpo-37363: Add audit events on startup for the run commands (GH-14524) 2019-07-01 16:03:53 -07:00
Victor Stinner 36242fd871
bpo-36763: Add PyConfig_SetWideStringList() (GH-14444) 2019-07-01 19:13:50 +02:00
Victor Stinner e21b45a8e7
bpo-37472: Remove Lib/test/outstanding_bugs.py (GH-14516) 2019-07-01 18:35:07 +02:00
Victor Stinner f9b7457bd7
bpo-37467: Fix PyErr_Display() for bytes filename (GH-14504)
Fix sys.excepthook() and PyErr_Display() if a filename is a bytes
string. For example, for a SyntaxError exception where the filename
attribute is a bytes string.

Cleanup also test_sys:

* Sort imports.
* Rename numruns global var to INTERN_NUMRUNS.
* Add DisplayHookTest and ExceptHookTest test case classes.
* Don't save/restore sys.stdout and sys.displayhook using
  setUp()/tearDown(): do it in each test method.
* Test error case (call hook with no argument) after the success case.
2019-07-01 16:51:18 +02:00
Miro Hrončok 72cd653c4e bpo-10945: Drop support for bdist_wininst on non-Windows systems (GH-14506)
bdist_wininst depends on MBCS codec, unavailable on non-Windows,
and bdist_wininst have not worked since at least Python 3.2, possibly
never on Python 3.

Here we document that bdist_wininst is only supported on Windows,
and we mark it unsupported otherwise to skip tests.

Distributors of Python 3 can now safely drop the bdist_wininst .exe files
without the need to skip bdist_wininst related tests.
2019-07-01 14:12:40 +02:00
Pablo Galindo 4a2edc34a4 bpo-37221: Add PyCode_NewWithPosOnlyArgs to be used internally and set PyCode_New as a compatibility wrapper (GH-13959)
Add PyCode_NewEx to be used internally and set PyCode_New as a compatibility wrapper
2019-07-01 12:35:05 +02:00
Christian Heimes d1bd6e79da bpo-37440: Enable TLS 1.3 post-handshake auth in http.client (GH-14448)
Post-handshake authentication is required for conditional client cert authentication with TLS 1.3.


https://bugs.python.org/issue37440
2019-06-30 23:32:24 -07:00
Christian Heimes f0f5930ac8 bpo-37428: Don't set PHA verify flag on client side (GH-14421)
SSLContext.post_handshake_auth = True no longer sets
SSL_VERIFY_POST_HANDSHAKE verify flag for client connections. Although the
option is documented as ignored for clients, OpenSSL implicitly enables cert
chain validation when the flag is set.

Signed-off-by: Christian Heimes <christian@python.org>



https://bugs.python.org/issue37428
2019-06-30 23:29:17 -07:00
Zackery Spytz c2cda638d6 bpo-37199: Fix test failures when IPv6 is unavailable or disabled (#14480) 2019-06-30 18:24:43 +03:00
Andrew Svetlov 0d671c04c3
bpo-35621: Support running subprocesses in asyncio when loop is executed in non-main thread (GH-14344) 2019-06-30 12:54:59 +03:00
Steve Dower 9048c49322
bpo-37369: Fix initialization of sys members when launched via an app container (GH-14428)
sys._base_executable is now always defined on all platforms, and can be overridden through configuration.
Also adds test.support.PythonSymlink to encapsulate platform-specific logic for symlinking sys.executable
2019-06-29 10:34:11 -07:00
Brian Quinlan 242c26f53e
bpo-31783: Fix a race condition creating workers during shutdown (#13171)
* bpo-31783: Fix a race condition while creating workers during interpreter shutdown

* 📜🤖 Added by blurb_it.
2019-06-28 11:54:52 -07:00
Ruslan Kuprieiev 042821ae3c bpo-37380: subprocess: don't use _active on win (GH-14360)
As noted by @eryksun in [1] and [2], using _cleanup and _active(in
__del__) is not necessary on Windows, since:

> Unlike Unix, a process in Windows doesn't have to be waited on by
> its parent to avoid a zombie. Keeping the handle open will actually
> create a zombie until the next _cleanup() call, which may be never
> if Popen() isn't called again.

This patch simply defines `subprocess._active` as `None`, for which we already
have the proper logic in place in `subprocess.Popen.__del__`, that prevents it
from trying to append the process to the `_active`. This patch also defines
`subprocess._cleanup` as a noop for Windows.

[1] https://bugs.python.org/issue37380#msg346333
[2] https://bugs.python.org/issue36067#msg336262

Signed-off-by: Ruslan Kuprieiev <ruslan@iterative.ai>
2019-06-28 18:12:16 +02:00
Jeroen Demeyer b1263d5a60 bpo-37337: Add _PyObject_VectorcallMethod() (GH-14228) 2019-06-28 18:49:00 +09:00
Benjamin Peterson 3b03b09fc9
closes bpo-37437: Update vendorized expat to 2.2.7. (GH-14436) 2019-06-27 20:54:44 -07:00
Brandt Bucher 45a30af109 closes bpo-37420: Handle errors during iteration in os.sched_setaffinity. (GH-14414) 2019-06-27 09:10:57 -07:00
Victor Stinner 4c26abd14f
bpo-37421: Fix test_shutil: don't leak temporary files (GH-14416)
* Fix typo in supports_file2file_sendfile(); ensure that dst is
  removed
* Fix test_copytree_custom_copy_function(): remove dst tree.
  Use support.rmtree() rather than shutil.rmtree() to remove
  temporary directories: support tries harder.
2019-06-27 01:39:53 +02:00
Carl Bordum Hansen 06a8916cf4 bpo-37376: pprint support for SimpleNamespace (GH-14318)
https://bugs.python.org/issue37376
2019-06-26 16:13:18 -07:00
Abhilash Raj 7213df7bbf bpo-29412: Fix indexError when parsing a header value ending unexpectedly (GH-14387)
* patched string index out of range error in get_word function of _header_value_parser.py and created tests in test__header_value_parser.py for CFWS.
* Raise HeaderParseError instead of continuing when parsing a word.
2019-06-26 13:13:02 -07:00
Brandt Bucher 2a7d596f27 bpo-37417: Fix error handling in bytearray.extend. (GH-14407) 2019-06-26 22:06:18 +03:00
Victor Stinner 5150d32792
bpo-37411: Rewrite test_wsgiref.testEnviron() (GH-14394)
Fix test_wsgiref.testEnviron() to no longer depend on the environment
variables (don't fail if "X" variable is set).

testEnviron() now overrides os.environ to get a deterministic
environment. Test full TestHandler.environ content: not only a few
selected variables.
2019-06-26 18:16:35 +02:00
Serhiy Storchaka 2d88e63bfc
bpo-37163: Make the obj argument of dataclasses.replace() a positional-only. (GH-14390) 2019-06-26 19:07:44 +03:00
Victor Stinner 69150669f2
bpo-37414: Remove sys.callstats() (GH-14398)
Remove the undocumented sys.callstats() function. Since Python 3.7,
it was deprecated and always returned None. It required a special
build option CALL_PROFILE which was already removed in Python 3.7.
2019-06-26 18:01:10 +02:00
Victor Stinner 689830ee62
bpo-37412: os.getcwdb() now uses UTF-8 on Windows (GH-14396)
The os.getcwdb() function now uses the UTF-8 encoding on Windows,
rather than the ANSI code page: see PEP 529 for the rationale. The
function is no longer deprecated on Windows.

os.getcwd() and os.getcwdb() now detect integer overflow on memory
allocations. On Unix, these functions properly report MemoryError on
memory allocation failure.
2019-06-26 17:31:12 +02:00
Victor Stinner c6a2320e87
bpo-37406: sqlite3 raises TypeError for wrong operation type (GH-14386)
The sqlite3 module now raises TypeError, rather than ValueError, if
operation argument type is not str: execute(), executemany() and
calling a connection.
2019-06-26 03:16:24 +02:00
Victor Stinner 22eb689cf3
bpo-37388: Development mode check encoding and errors (GH-14341)
In development mode and in debug build, encoding and errors arguments
are now checked on string encoding and decoding operations. Examples:
open(), str.encode() and bytes.decode().

By default, for best performances, the errors argument is only
checked at the first encoding/decoding error, and the encoding
argument is sometimes ignored for empty strings.
2019-06-26 00:51:05 +02:00
Abhilash Raj 02257012f6 bpo-33972: Fix EmailMessage.iter_attachments raising AttributeError. (GH-14119)
* bpo-33972: Fix EmailMessage.iter_attachments raising AttributeError.

When certain malformed messages have content-type set to 'mulitpart/*' but
still have a single part body, iter_attachments can raise AttributeError. This
patch fixes it by returning a None value instead when the body is single part.
2019-06-25 10:03:19 -07:00
Victor Stinner d7c87d982d
bpo-37400: Fix test_os.test_chown() (GH-14374)
Use os.getgroups() rather than grp.getgrall() to get groups.
Rename also the test to test_chown_gid().
2019-06-25 17:06:24 +02:00
Victor Stinner 3939c321c9
bpo-20443: _PyConfig_Read() gets the absolute path of run_filename (GH-14053)
Python now gets the absolute path of the script filename specified on
the command line (ex: "python3 script.py"): the __file__ attribute of
the __main__ module, sys.argv[0] and sys.path[0] become an absolute
path, rather than a relative path.

* Add _Py_isabs() and _Py_abspath() functions.
* _PyConfig_Read() now tries to get the absolute path of
  run_filename, but keeps the relative path if _Py_abspath() fails.
* Reimplement os._getfullpathname() using _Py_abspath().
* Use _Py_isabs() in getpath.c.
2019-06-25 15:02:43 +02:00
Serhiy Storchaka 894263ba80
bpo-24214: Fixed the UTF-8 and UTF-16 incremental decoders. (GH-14304)
* The UTF-8 incremental decoders fails now fast if encounter
  a sequence that can't be handled by the error handler.
* The UTF-16 incremental decoders with the surrogatepass error
  handler decodes now a lone low surrogate with final=False.
2019-06-25 11:54:18 +03:00
Raymond Hettinger 1791128677
bpo-36546: Mark first argument as position only (GH-14363) 2019-06-25 04:39:22 +02:00
Pablo Galindo 3f5b9088b0
bpo-37394: Fix pure Python implementation of the queue module (GH-14351) 2019-06-25 02:53:30 +01:00
Victor Stinner 36456df138
bpo-37392: Remove sys.setcheckinterval() (GH-14355)
Remove sys.getcheckinterval() and sys.setcheckinterval() functions.
They were deprecated since Python 3.2. Use sys.getswitchinterval()
and sys.setswitchinterval() instead.

Remove also check_interval field of the PyInterpreterState structure.
2019-06-25 03:01:08 +02:00
David K. Hess 9fc720e5e4 bpo-4963: Fix for initialization and non-deterministic behavior issues in mimetypes (GH-3062) 2019-06-24 16:46:59 -07:00
animalize 8bd2872adb bpo-25361: Enable SSE2 instructions for Windows 32-bit build (GH-12438) 2019-06-24 16:43:26 -07:00
animalize 7fd2ba354e bpo-35360: Update Windows builds to use SQLite 3.28.0 (GH-14179) 2019-06-24 16:22:14 -07:00
ziheng 6ffd9b05df bpo-32627: Fix compile error when conflicting `_uuid` headers included (GH-11751) 2019-06-24 13:59:50 -04:00
Steve Dower 60419a7e96
bpo-37363: Add audit events for a range of modules (GH-14301) 2019-06-24 08:42:54 -07:00
Jeroen Demeyer a8b27e623d bpo-36974: inherit tp_vectorcall_offset unconditionally (GH-13858) 2019-06-24 12:41:05 +02:00
Victor Stinner 47fbc4e45b
bpo-37359: Add --cleanup option to python3 -m test (GH-14332)
* regrtest: Add --cleanup option to remove "test_python_*" directories
  of previous failed test jobs.
* Add "make cleantest" to run "python3 -m test --cleanup".
2019-06-24 12:03:00 +02:00
Gabe Appleton 2ac3bab2a6 bpo-37345: Add formal UDPLITE support (GH-14258)
At the moment you can definitely use UDPLITE sockets on Linux systems, but it would be good if this support were formalized such that you can detect support at runtime easily.

At the moment, to make and use a UDPLITE socket requires something like the following code:

```
>>> import socket
>>> a = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, 136)
>>> b = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, 136)
>>> a.bind(('localhost', 44444))
>>> b.sendto(b'test'*256, ('localhost', 44444))
>>> b.setsockopt(136, 10, 16)
>>> b.sendto(b'test'*256, ('localhost', 44444))
>>> b.setsockopt(136, 10, 32)
>>> b.sendto(b'test'*256, ('localhost', 44444))
>>> b.setsockopt(136, 10, 64)
>>> b.sendto(b'test'*256, ('localhost', 44444))
```

If you look at this through Wireshark, you can see that the packets are different in that the checksums and checksum coverages change.

With the pull request that I am submitting momentarily, you could do the following code instead:

```
>>> import socket
>>> a = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDPLITE)
>>> b = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDPLITE)
>>> a.bind(('localhost', 44444))
>>> b.sendto(b'test'*256, ('localhost', 44444))
>>> b.set_send_checksum_coverage(16)
>>> b.sendto(b'test'*256, ('localhost', 44444))
>>> b.set_send_checksum_coverage(32)
>>> b.sendto(b'test'*256, ('localhost', 44444))
>>> b.set_send_checksum_coverage(64)
>>> b.sendto(b'test'*256, ('localhost', 44444))
```

One can also detect support for UDPLITE just by checking

```
>>> hasattr(socket, 'IPPROTO_UDPLITE')
```


https://bugs.python.org/issue37345
2019-06-24 02:58:56 -07:00
Inada Naoki 770847a7db
bpo-37348: optimize decoding ASCII string (GH-14283)
`_PyUnicode_Writer` is a relatively complex structure.  Initializing it is significant overhead when decoding short ASCII string.
2019-06-24 12:30:24 +09:00
Jörn Heissler c8a35417db bpo-35224: Reverse evaluation order of key: value in dict comprehensions (GH-14139)
… as proposed in PEP 572; key is now evaluated before value.





https://bugs.python.org/issue35224
2019-06-22 07:40:55 -07:00
Steve Dower 184f3d4f39 bpo-37364: Use io.open_code() to read .pth files (GH-14299)
https://bugs.python.org/issue37364
2019-06-21 15:16:46 -07:00
Steve Dower f5690925df bpo-37351: Removes libpython38.a from standard Windows distribution (#14276) 2019-06-21 14:28:46 -07:00
Victor Stinner e56a123fd0
bpo-37362: test_gdb now ignores stderr (GH-14287)
test_gdb no longer fails if it gets an "unexpected" message on
stderr: it now ignores stderr. The purpose of test_gdb is to test
that python-gdb.py commands work as expected, not to test gdb.
2019-06-21 23:17:30 +02:00
Joannah Nanjekye a0d73a143a bpo-30202 : Update test.test_importlib.test_abc to test find_spec() (GH-12847) 2019-06-21 11:17:00 -07:00
Zackery Spytz 08286d52b2 bpo-37316: mmap.mmap() passes the wrong variable to PySys_Audit() (GH-14152)
Also, add a missing call to va_end() in PySys_Audit().
2019-06-21 08:31:59 -07:00
Michael Felt 08970cb03c bpo-36210: update optional extension handling for AIX (GH-12202)
* Switch to officially supported curses from 3rd-party ASIS supported ncurses
* stop saying optional modules osaudiodev and spwd are missing on AIX

Patch by M.Felt
2019-06-21 23:58:00 +10:00
Eric V. Smith 0c48618cc0
Fix bpo number in News file. (GH-14260) 2019-06-20 04:22:28 -04:00
Terry Jan Reedy 54cf2e0780
bpo-37325: Fix focus traversal for 2 IDLE dialogs (#14209)
Tab now moves focus across and down for Help Source and Custom Run.
2019-06-18 17:08:24 -04:00
Paul Ganssle d287215df5
Add pganssle to CODEOWNERS and ACKS (GH-14138)
Also adds abalkin to CODEOWNERS for date and time related files.
2019-06-18 19:57:45 +01:00
Mike Gleen 6b9c204ee7 bpo-34903: Document that some strptime formats only require 1 digit (GH-14149)
For datetime.datetime.strptime(), the leading zero for some two-digit formats is optional.

This adds a footnote to the strftime/strptime documentation to reflect this fact, and adds some tests to ensure that it is true.

bpo-34903
2019-06-18 19:14:57 +01:00
animalize d8f336fdc1 bpo-35360: Update macOS installer to use SQLite 3.28.0 (GH-14180) 2019-06-18 05:25:19 -04:00
Ned Deily f3fb8393e3
bpo-34631: Updated OpenSSL to 1.1.1c in macOS installer (GH-14187) 2019-06-18 04:17:33 -04:00
Cheryl Sabella 201bc2d18b bpo-5680: IDLE: Customize running a module (GH-13763)
The initialize options are 1) add command line options, which are appended to sys.argv as if passed on a real command line, and 2) skip the shell restart. The customization dialog is accessed by a new entry on the Run menu.
2019-06-17 22:24:10 -04:00
Victor Stinner 343ed0ffe0
bpo-37189: Export old PyRun_XXX() functions (#14142)
Many PyRun_XXX() functions like PyRun_String() were no longer
exported in libpython38.dll by mistake. Export them again to fix the
ABI compatibiliy.
2019-06-18 00:15:13 +02:00
Victor Stinner ac7b1a3f32
bpo-37320: Remove openfp() of aifc, sunau and wave (GH-14169)
aifc.openfp() alias to aifc.open(), sunau.openfp() alias to
sunau.open(), and wave.openfp() alias to wave.open() have been
removed. They were deprecated since Python 3.7.
2019-06-18 00:00:24 +02:00
Terry Jan Reedy 8fac122109
bpo-37321: Edit IDLE subprocess connection error messages. (#14170)
Mainly, add a doc reference to message in pyshell.
2019-06-17 17:23:28 -04:00
Tal Einat 5bff3c86ab bpo-37039: Make IDLE's Zoom Height adjust to users' screens (GH-13678)
Measure required height by quickly maximizing once per screen.
A search for a better method failed.
2019-06-17 15:41:00 -04:00
Steve Dower a268edd6a4
bpo-34631: Updated OpenSSL to 1.1.1c in Windows installer (GH-14163) 2019-06-17 11:36:08 -07:00
Cooper Lees 4acdbf11b1 bpo-34556: Add --upgrade-deps to venv module (#13100)
Add --upgrade-deps to venv module
- This allows for pip + setuptools to be automatically upgraded to the latest version on PyPI
- Update documentation to represent this change

bpo-34556: Add --upgrade to venv module
2019-06-17 19:18:13 +01:00
Vinay Sajip ca7b504a4d
bpo-37111: Add 'encoding' and 'errors' parameters to logging.basicCon… (GH-14008) 2019-06-17 17:40:52 +01:00
Serhiy Storchaka 231aad3849
bpo-37315: Deprecate accepting floats in math.factorial(). (GH-14147) 2019-06-17 16:57:27 +03:00
Victor Stinner 2ff58a24e8
bpo-37194: Add a new public PyObject_CallNoArgs() function (GH-13890)
Add a new public PyObject_CallNoArgs() function to the C API: call a
callable Python object without any arguments.

It is the most efficient way to call a callback without any argument.
On x86-64, for example, PyObject_CallFunctionObjArgs(func, NULL)
allocates 960 bytes on the stack per call, whereas
PyObject_CallNoArgs(func) only allocates 624 bytes per call.

It is excluded from stable ABI 3.8.

Replace private _PyObject_CallNoArg() with public
PyObject_CallNoArgs() in C extensions: _asyncio, _datetime,
_elementtree, _pickle, _tkinter and readline.
2019-06-17 14:27:23 +02:00
Victor Stinner 8bf08ee45b
bpo-37312: Remove _dummy_thread and dummy_threading modules (GH-14143)
Remove _dummy_thread and dummy_threading modules. These modules
were deprecated since Python 3.7 which requires threading support.
2019-06-17 14:18:43 +02:00
Jeroen Demeyer 2e9954d347 bpo-36922: use Py_TPFLAGS_METHOD_DESCRIPTOR in lookup_maybe_method() (GH-13865) 2019-06-17 20:53:20 +09:00
Zackery Spytz 28fca0c422 bpo-37267: Do not check for FILE_TYPE_CHAR in os.dup() on Windows (GH-14051)
On Windows, os.dup() no longer creates an inheritable fd when handling a
character file.
2019-06-17 09:17:14 +02:00
Hai Shi c83356cae2 closes bpo-37300: Remove unnecessary Py_XINCREF in classobject.c. (GH-14120) 2019-06-16 13:19:19 -07:00
Jeroen Demeyer 5600b5e1b2 bpo-28805: document METH_FASTCALL (GH-14079) 2019-06-17 02:03:22 +09:00
Rémi Lapeyre 8047e0e1c6 bpo-35922: Fix RobotFileParser when robots.txt has no relevant crawl delay or request rate (GH-11791)
Co-Authored-By: Tal Einat <taleinat+github@gmail.com>
2019-06-16 09:48:57 +03:00
Michael Felt 3a1d50e7e5 bpo-28009: Fix uuid SkipUnless logic to be based on platform programs capable of introspection (GH-12777)
uuid could try fallback methods that had no chance of working on a particular
platform, and this could cause spurious test failures, as well as degraded
performance as fallback options were tried and failed.

This fixes both the uuid module and its test's SkipUnless logic to use a
prefiltered list of techniques that may at least potentially work on that platform.

Patch by Michael Felt (aixtools).
2019-06-16 01:52:29 +10:00
Andrew Svetlov 0237265e82
Use threadpool for reading from file in sendfile fallback mode (#14076) 2019-06-15 14:05:35 +03:00
Andrew Svetlov ef2152354f
bpo-37279: Fix asyncio sendfile support when extra data are sent in fallback mode. (GH-14075) 2019-06-15 14:05:08 +03:00
Victor Stinner 066e5b1a91
bpo-37266: Daemon threads are now denied in subinterpreters (GH-14049)
In a subinterpreter, spawning a daemon thread now raises an
exception. Daemon threads were never supported in subinterpreters.
Previously, the subinterpreter finalization crashed with a Pyton
fatal error if a daemon thread was still running.

* Add _thread._is_main_interpreter()
* threading.Thread.start() now raises RuntimeError if the thread is a
  daemon thread and the method is called from a subinterpreter.
* The _thread module now uses Argument Clinic for the new function.
* Use textwrap.dedent() in test_threading.SubinterpThreadingTests
2019-06-14 18:55:22 +02:00
Zackery Spytz 9765efcb39 bpo-19865: ctypes.create_unicode_buffer() supports non-BMP strings on Windows (GH-14081) 2019-06-14 17:53:59 +02:00
Andrew Svetlov f0749da9a5 bpo-35998: Avoid TimeoutError in test_asyncio: test_start_tls_server_1() (GH-14080) 2019-06-14 17:26:24 +02:00
Victor Stinner 07559450b2
bpo-37278: Fix test_asyncio ProactorLoopCtrlC (GH-14074)
Join the thread to prevent leaking a running thread and leaking a
reference.

Cleanup also the test:

* asyncioWindowsProactorEventLoopPolicy became the default policy,
  there is no need to set it manually.
* Only start the thread once the loop is running.
* Use a shorter sleep in the thread (100 ms rather than 1 sec).
* Use close_loop(loop) rather than loop.close().
* Use longer variable names.
2019-06-14 13:02:51 +02:00
Pablo Galindo 05f8318655
bpo-37269: Correctly optimise conditionals with constant booleans (GH-14071)
Fix a regression introduced by af8646c805 that was causing code of the form:

if True and False:
   do_something()

to be optimized incorrectly, eliminating the block.
2019-06-14 06:54:53 +01:00
Michael Felt d0eeb936d8 bpo-37077: Add native thread ID (TID) for AIX (GH-13624)
This is the followup  for issue36084



https://bugs.python.org/issue37077
2019-06-13 15:34:46 -07:00
Pablo Galindo 3498c642f4
bpo-37213: Handle negative line deltas correctly in the peephole optimizer (GH-13969)
The peephole optimizer was not optimizing correctly bytecode after negative deltas were introduced. This is due to the fact that some special values (255) were being searched for in both instruction pointer delta and line number deltas.
2019-06-13 19:16:22 +01:00
Victor Stinner 6d22cc8e90
bpo-37261: Fix support.catch_unraisable_exception() (GH-14052)
The __exit__() method of test.support.catch_unraisable_exception
context manager now ignores unraisable exception raised when clearing
self.unraisable attribute.
2019-06-13 14:44:54 +02:00
Victor Stinner 63ab4ba07b
bpo-37210: Fix pure Python pickle when _pickle is unavailable (GH-14016)
Allow pure Python implementation of pickle to work
even when the C _pickle module is unavailable.

Fix test_pickle when _pickle is missing: declare PyPicklerHookTests
outside "if has_c_implementation:" block.
2019-06-13 13:58:51 +02:00
Jeroen Demeyer b4b814b398 bpo-37231: optimize calls of special methods (GH-13973) 2019-06-13 18:26:44 +09:00
Jeffrey Kintscher 8725c83ed5 bpo-35070: test_getgrouplist may fail on macOS if too many groups (GH-13071) 2019-06-13 03:01:29 -04:00
Tim Peters d1c85a27ea
bpo-37257: obmalloc: stop simple arena thrashing (#14039)
GH-14039:  allow (no more than) one wholly empty arena on the usable_arenas list.

This prevents thrashing in some easily-provoked simple cases that could end up creating and destroying an arena on each loop iteration in client code.   Intuitively, if the only arena on the list becomes empty, it makes scant sense to give it back to the system unless we know we'll never need another free pool again before another arena frees a pool.  If the latter obtains, then - yes - this will "waste" an arena.
2019-06-12 22:41:03 -05:00
Victor Stinner 468e5fec8a
bpo-36402: Fix threading._shutdown() race condition (GH-13948)
Fix a race condition at Python shutdown when waiting for threads.
Wait until the Python thread state of all non-daemon threads get
deleted (join all non-daemon threads), rather than just wait until
Python threads complete.

* Add threading._shutdown_locks: set of Thread._tstate_lock locks
  of non-daemon threads used by _shutdown() to wait until all Python
  thread states get deleted. See Thread._set_tstate_lock().
* Add also threading._shutdown_locks_lock to protect access to
  threading._shutdown_locks.
* Add test_finalization_shutdown() test.
2019-06-13 01:30:17 +02:00
Paul Monson b4c7defe58 bpo-36779: time.tzname returns empty string on Windows if default cod… (GH-13073)
Calling setlocale(LC_CTYPE, "") on a system where GetACP() returns CP_UTF8 results in empty strings in _tzname[].

This causes time.tzname to be an empty string.
I have reported the bug to the UCRT team and will follow up, but it will take some time get a fix into production.

In the meantime one possible workaround is to temporarily change the locale by calling setlocale(LC_CTYPE, "C") before calling _tzset and restore the current locale after if the GetACP() == CP_UTF8 or CP_UTF7

@zooba 


https://bugs.python.org/issue36779
2019-06-12 16:13:27 -07:00
Victor Stinner 95f61c8b16
bpo-37069: regrtest uses sys.unraisablehook (GH-13759)
regrtest now uses sys.unraisablehook() to mark a test as "environment
altered" (ENV_CHANGED) if it emits an "unraisable exception".
Moreover, regrtest logs a warning in this case.

Use "python3 -m test --fail-env-changed" to catch unraisable
exceptions in tests.
2019-06-13 01:09:04 +02:00
David Carlier 5287022eee bpo-37160: Thread native ID NetBSD support (GH-13835) 2019-06-12 17:37:56 +02:00
Guido van Rossum 10b55c1643 bpo-35766: Change format for feature_version to (major, minor) (GH-13992)
(A single int is still allowed, but undocumented.)



https://bugs.python.org/issue35766
2019-06-11 17:23:12 -07:00
Andrew Svetlov 65aa64fae8 bpo-36607: Eliminate RuntimeError raised by asyncio.all_tasks() (GH-13971)
If internal tasks weak set is changed by another thread during iteration.



https://bugs.python.org/issue36607
2019-06-11 08:27:30 -07:00
Raymond Hettinger 1f11cf9521
bpo-37219: Remove erroneous optimization for differencing an empty set (GH-13965) 2019-06-11 01:15:24 -07:00
Victor Stinner 4f6f7c5a61
bpo-18748: Fix _pyio.IOBase destructor (closed case) (GH-13952)
_pyio.IOBase destructor now does nothing if getting the closed
attribute fails to better mimick _io.IOBase finalizer.
2019-06-11 02:49:06 +02:00
Christian Heimes 8a8b59c979 bpo-37215: Fix dtrace issue introduce by bpo-36842 (GH-13940)
Signed-off-by: Christian Heimes <christian@python.org>


https://bugs.python.org/issue37215
2019-06-10 08:19:48 -07:00
Marcin Niemira 45a14942c9 bpo-11122: fix hardcoded path checking for rpmbuild in bdist_rpm.py (GH-10594) 2019-06-08 17:05:05 -04:00
Raymond Hettinger e119b3d136
bpo-37178: Allow a one argument form of math.perm() (GH-13905) 2019-06-08 08:58:11 -07:00
Rémi Lapeyre 8cc605acdd bpo-34886: Fix subprocess.run handling of exclusive arguments (GH-11727)
Fix an unintended ValueError from :func:`subprocess.run` when checking for
conflicting `input` and `stdin` or `capture_output` and `stdout` or `stderr` args
when they were explicitly provided but with `None` values within a passed in
`**kwargs` dict rather than as passed directly by name.
2019-06-08 07:56:24 -07:00
Philipp A d407d2a726 bpo-37173: Show passed class in inspect.getfile error (GH-13861)
Currently, inspect.getfile(str) will report nonsense:

```pytb
>>> inspect.getfile(str)
TypeError: <module 'builtins' (built-in)> is a built-in class
```

This fixes that


https://bugs.python.org/issue37173
2019-06-08 05:05:46 -07:00
zygocephalus 03d5831a2d bpo-37150: Throw ValueError if FileType class object was passed in add_argument (GH-13805)
There is a possibility that someone (like me) accidentally will omit parentheses with `FileType` arguments after `FileType`, and parser will contain wrong file until someone will try to use it. 

Example:
```python
parser = argparse.ArgumentParser()
parser.add_argument('-x', type=argparse.FileType)
```


https://bugs.python.org/issue37150
2019-06-07 13:08:36 -07:00
Victor Stinner 3bf0f3ad20
bpo-37169: Rewrite _PyObject_IsFreed() unit tests (GH-13888)
Replace two Python function calls with a single one to ensure that no
memory allocation is done between the invalid object is created and
when _PyObject_IsFreed() is called.
2019-06-07 16:22:21 +02:00
Tal Einat 554450fb4e
bpo-37177: make IDLE's search dialogs transient (GH-13869)
This avoids the search dialogs being hidden behind the editor window.
2019-06-07 08:54:40 +03:00
Zackery Spytz dc2476500d bpo-37170: Fix the cast on error in PyLong_AsUnsignedLongLongMask() (GH-13860) 2019-06-06 22:39:23 +02:00
websurfer5 f6713e84af bpo-36520: Email header folded incorrectly (#13608)
* bpo-36520: reset the encoded word offset when starting a new
line during an email header folding operation

* 📜🤖 Added by blurb_it.

* bpo-36520: add an additional test case, and provide descriptive
comments for the test_folding_with_utf8_encoding_* tests

* bpo-36520: fix whitespace issue

* bpo-36520: changes per reviewer request -- remove extraneous
backslashes; add whitespace between terminating quotes and
line-continuation backslashes; use "bpo-" instead of
"issue #" in comments
2019-06-06 12:53:27 -07:00
Raymond Hettinger e985804207
bpo-37165: Convert _count_elements to the argument clinic (GH-13848) 2019-06-05 16:05:25 -07:00
Ashwin Ramaswami c4c15ed7a2 bpo-35551: encodings update (GH-11446) 2019-06-05 18:18:06 -04:00
Abhilash Raj 66c4f3f38b bpo-21315: Fix parsing of encoded words with missing leading ws. (#13425)
* bpo-21315: Fix parsing of encoded words with missing leading ws.

Because of missing leading whitespace, encoded word would get parsed as
unstructured token. This patch fixes that by looking for encoded words when
splitting tokens with whitespace.

Missing trailing whitespace around encoded word now register a defect
instead.

Original patch suggestion by David R. Murray on bpo-21315.
2019-06-05 09:56:33 -07:00
Raymond Hettinger 6c01ebcc0d
bpo-37158: Simplify and speed-up statistics.fmean() (GH-13832) 2019-06-05 07:39:38 -07:00
Zackery Spytz 9aa78566fb bpo-34767: Do not always create a collections.deque() in asyncio.Lock() (GH-13834)
https://bugs.python.org/issue34767
2019-06-05 02:33:26 -07:00
Terry Jan Reedy 949fe976d5
bpo-35763: Make IDLE calltip note about '/' less obtrusive (GH-13791)
Add it to the end of the first line if there is room.  Tests were reworked.
2019-06-04 21:55:37 -04:00
Zachary Ware 19a1e1eb86 bpo-34282: Remove deprecated enum _convert method (GH-13823) 2019-06-04 23:03:10 +02:00
Łukasz Langa 23d7ce7471 Python 3.8.0b1
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE4/8oOcBIslwITevpsmmV4xAlBWgFAlz2r2cACgkQsmmV4xAl
 BWhasQ/9EDLL+KtFUdi3iMr7+C1UN/2u2v3OnOsZQA0WE7TfEebbf5GS4baN/pHk
 PslVY6kRqRfQIxjemXFpq52mn9+3SUE4m3avkgnUZMu8XEaUczM4IdcizGMO3qNr
 c1iz6BraJo8gZ1g+dNcRbEc89V9e1SzrBZOR9MXGjxxo0HHEPdjW9ptg1BtHq9To
 n4iivdQSnnxzP7D7SpItWPL/gFv90GBbXaTRerGUlmj/uwnBhzHB1fi4r+WDn8yR
 4AiFVSlUFpTXYOiH5A3MYQtLq8kH9AAmo6Bmj/73NvChxv7GV5v968+tlEWEK+/h
 UtTUEw8L/oSH74m8wd0q0tQdTe8VJgtwKyR6h5GLL8TLa73nN/jK4Koqirpc8mpK
 DcHVLpxHQ/yhPtAcQXp81qqJSbhnPOvbRjNQP5bHcNIl8p+OsF7N9z/tu6gmMorO
 k+f2OCOC0ZsG/EL7DPz6kG6+Cy5NhA+/IJ9I3Ayk6AsSjbzqtgai3bKKVF3s27r8
 OIZprMAVKsmsPpEfuSG8rshKxmLH5/n7YuwSuMmWXv06QspfG43zxp1VYq94zBoI
 wpBy8QxBwef7oDpMI+WgE/g8Prj1NmQuIy7co/pZDsOHeQWr96z90BcpSX8XKHKT
 9W829jFKoQEnGnuFgoUxacR8a4bQKVla5mVFvjUtGjb9GS9kJkk=
 =V2hy
 -----END PGP SIGNATURE-----
gpgsig -----BEGIN PGP SIGNATURE-----
 
 iQJEBAABCAAuFiEE4/8oOcBIslwITevpsmmV4xAlBWgFAlz2zD8QHGx1a2FzekBs
 YW5nYS5wbAAKCRCyaZXjECUFaIw/EACXkzdCOq5D/itTvFElnNWjlon8ot4toVEf
 zw4isSYiYS4B3dBI6+VyCZWNxR5F6fwfyhPLHf7CzPn+7Yp5+dX9//x0HqhNLQCY
 PmwuYG/4nWcww8UhUweGVI1u8Fjrw+LzLJPVyhLQia/TXUBfCFcggS2xo9S6mSsA
 isJr9RaCDaoit6ZjFIuo+sOn2YTx2eYBe6LlhlSx9zz0ayyy+ADqPeUq/hoEFDPD
 EXUOwsLJPo/vu7sFqKOhHBWtMCcnvkkbiBLrAmHS2vhKjl22zVez0lUiVxLUlhZT
 WcmfRkCtoBKu2rYbm85pt08fgF/CnHq9oO57yDBZ4j/Lv0HqQkp+/yZRzWqSSThw
 vw4z4Ur3l06ZMxsTH+Dd/xjqc41AlpedJNhuAgOf0URrbl4H48y5U6AG1vZWrpAk
 wUsTF1QGXL1SAam2pjI2cTpjRaCh3RK2wf0KPewxUpcvcRcGtGAbq+EFHHOFg2Lm
 AcxAuAEUXKcyJQnLtFiIOWJFttaRzAB6zq8sEL0kS1BfAEspqTk+ZdnDY1ajppqu
 7sb3/TGrxa+38Th5iVvUehHGQqrQYLORVn0w8PKUXunbUbv6AYoDJHKxiOUO4JsX
 L5jZEPjTeG0BjntTnvVYdgqPXAVbJr76LmlCPkPcmSruvXrfZP1OOi+ybepwbTpD
 8pw88nSTuA==
 =zOCJ
 -----END PGP SIGNATURE-----

Merge tag 'v3.8.0b1'

Python 3.8.0b1
2019-06-04 21:53:26 +02:00
Abhilash Raj aa79707262 bpo-30835: email: Fix AttributeError when parsing invalid CTE (GH-13598)
* bpo-30835: email: Fix AttributeError when parsing invalid Content-Transfer-Encoding

Parsing an email containing a multipart Content-Type, along with a
Content-Transfer-Encoding containing an invalid (non-ASCII-decodable) byte
will fail. email.feedparser.FeedParser._parsegen() gets the header and
attempts to convert it to lowercase before comparing it with the accepted
encodings, but as the header contains an invalid byte, it's returned as a
Header object rather than a str.

Cast the Content-Transfer-Encoding header to a str to avoid this.

Found using the AFL fuzzer.

Reported-by: Daniel Axtens <dja@axtens.net>
Signed-off-by: Andrew Donnellan <andrew@donnellan.id.au>

* Add email and NEWS entry for the bugfix.
2019-06-04 11:00:47 -07:00
Łukasz Langa 3b5deb0116
Python 3.8.0b1 2019-06-04 19:44:34 +02:00
Abhilash Raj 46d88a1131 bpo-35805: Add parser for Message-ID email header. (GH-13397)
* bpo-35805: Add parser for Message-ID header.

This parser is based on the definition of Identification Fields from RFC 5322
Sec 3.6.4.

This should also prevent folding of Message-ID header using RFC 2047 encoded
words and hence fix bpo-35805.

* Prevent folding of non-ascii message-id headers.
* Add fold method to MsgID token to prevent folding.
2019-06-04 10:41:34 -07:00
Victor Stinner bc6469f79c
bpo-37153: test_venv.test_mutiprocessing() calls pool.terminate() (GH-13816)
test_venv.test_mutiprocessing() now explicitly calls pool.terminate()
to wait until the pool completes.
2019-06-04 19:03:13 +02:00
Łukasz Langa 7f9a2ae780
Revert "bpo-34037, asyncio: add BaseEventLoop.wait_executor_on_close (GH-13786)" (#13802)
This reverts commit 0f0a30f4da.
2019-06-04 13:03:20 +02:00
Victor Stinner 0fd2c300c2
Revert "bpo-36818: Add PyInterpreterState.runtime field. (gh-13129)" (GH-13795)
This reverts commit 396e0a8d9d.
2019-06-04 03:15:09 +02:00
Andrew Svetlov 9535aff942 Revert "bpo-35621: Support running subprocesses in asyncio when loop is executed in non-main thread (#13630)" (GH-13793)
https://bugs.python.org/issue35621
2019-06-03 17:09:19 -07:00
Victor Stinner 0f0a30f4da
bpo-34037, asyncio: add BaseEventLoop.wait_executor_on_close (GH-13786)
Add BaseEventLoop.wait_executor_on_close attribute: true by default.

loop.close() now waits for the default executor to finish by default.
Set loop.wait_executor_on_close attribute to False to not wait for
the executor.
2019-06-03 23:31:04 +02:00
Christian Heimes 78c7d52779
bpo-37120: Add SSLContext.num_tickets (GH-13719)
Signed-off-by: Christian Heimes <christian@python.org>
2019-06-03 21:00:10 +02:00
Christian Heimes 47eb223406 bpo-36868: Fix what's new for SSLContext.hostname_checks_common_name (GH-13248)
What's new now mentions SSLContext.hostname_checks_common_name instead of SSLContext.host_flags.


https://bugs.python.org/issue36868
2019-06-03 11:51:27 -07:00
Christian Heimes 06651ee418
bpo-37081: Test with OpenSSL 1.1.1c (GH-13631)
Signed-off-by: Christian Heimes <christian@python.org>
2019-06-03 20:10:19 +02:00
Victor Stinner e225bebc14
Revert "bpo-33608: Factor out a private, per-interpreter _Py_AddPendingCall(). (gh-13714)" (GH-13780)
This reverts commit 6a150bcaeb.
2019-06-03 18:14:24 +02:00
David Carlier 0b9956e916 bpo-37087: Adding native ID support for OpenBSD (GH-13654) 2019-06-03 17:43:33 +02:00
Inada Naoki 91234a1636
bpo-26219: per opcode cache for LOAD_GLOBAL (GH-12884)
This patch implements per opcode cache mechanism, and use it in
only LOAD_GLOBAL opcode.

Based on Yury's opcache3.patch in bpo-26219.
2019-06-03 21:30:58 +09:00
Ned Deily 0288dd6a51
bpo-36231: Support building on macOS without /usr/include (GH-13773) 2019-06-03 06:34:48 -04:00
Pablo Galindo 8565f6b6db
bpo-35814: Allow unpacking in r.h.s of annotated assignment expressions (GH-13760) 2019-06-03 08:34:20 +01:00
Augusto Hack 01ee12ba35 bpo-33569 Preserve type information with dataclasses.InitVar (GH-8927) 2019-06-02 22:14:48 -04:00
Victor Stinner 0025350294
bpo-37069: tests use catch_unraisable_exception() (GH-13762)
Modify test_coroutines, test_cprofile, test_generators, test_raise,
test_ssl and test_yield_from to use
support.catch_unraisable_exception() rather than
support.captured_stderr().

test_thread: remove test_save_exception_state_on_error() which is now
updated. test_unraisable_exception() checks that sys.unraisablehook()
is called to handle _thread.start_new_thread() exception.

test_cprofile now rely on unittest for test discovery: replace
support.run_unittest() with unittest.main().
2019-06-03 03:51:43 +02:00
Matthias Bussonnier 13136e83a6 bpo-36896: Clarify that some types constructors are unstable (GH-13271) 2019-06-03 02:43:21 +02:00
Michele Angrisano aca273e240 bpo-37014: Update docstring and Documentation of fileinput.FileInput(). (GH-13545)
* bpo-37014: Update docstring and Documentation of fileinput.FileInput()

* Explain the behavior of fileinput.FileInput() when reading stdin.

* Update blurb.

* bpo-37014: Fix typo in the docstring and documentation.
2019-06-02 23:01:49 +02:00
Cheryl Sabella 6bdc4dee01 bpo-35610: IDLE - Replace .context_use_ps1 with .prompt_last_line (GH-11307)
Changes in bpo- 31858 made the less informative 'context_use_ps1' redundant.
2019-06-02 14:56:47 -04:00
Pablo Galindo 7ffcf848df
bpo-37126: Allow structseq objects to be tracked by the GC (GH-13729) 2019-06-02 15:45:13 +01:00
Andrew Svetlov 13ed07998a
bpo-35621: Support running subprocesses in asyncio when loop is executed in non-main thread (#13630) 2019-06-02 13:56:38 +03:00
Mark Dickinson c52996785a
bpo-36027: Extend three-argument pow to negative second argument (GH-13266) 2019-06-02 10:24:06 +01:00
Serhiy Storchaka 5ae299ac78
bpo-37128: Add math.perm(). (GH-13731) 2019-06-02 11:16:49 +03:00
Anthony Sottile b7daabd711 Improve version added references in `typing` module docs (GH-13457) 2019-06-02 01:13:25 +01:00
Eric Snow 6a150bcaeb
bpo-33608: Factor out a private, per-interpreter _Py_AddPendingCall(). (gh-13714) 2019-06-01 15:39:46 -06:00
Serhiy Storchaka bdbad71b9d
bpo-20092. Use __index__ in constructors of int, float and complex. (GH-13108) 2019-06-02 00:05:48 +03:00
Cheryl Sabella 1a4d9ffa1a bpo-32411: IDLE: Remove line number sort in browser.py (#5011)
Insertion in line order makes sorting keys by line order unneeded.
2019-06-01 17:03:22 -04:00
Sergey Fedoseev e5f6207ba6 bpo-34303: Micro-optimizations in functools.reduce() (GH-8598) 2019-06-01 13:32:17 -07:00
Pablo Galindo cd74e66a8c
bpo-37122: Make co->co_argcount represent the total number of positonal arguments in the code object (GH-13726) 2019-06-01 18:08:04 +01:00
Bar Harel 6b282e1887 bpo-36813: Fix QueueListener to call task_done() upon termination. (GH-13113)
Fixed QueueListener in order to avoid random deadlocks.
Unable to add regression tests atm due to time constraints, will add it in a bit.
Regarding implementation, although it's nested, it does not cause performance issues whatsoever, and does not call task_done() in case of an exception (which is the right thing to do IMHO).


https://bugs.python.org/issue36813
2019-06-01 02:19:09 -07:00
Yash Aggarwal 4a686504eb bpo-35431: Implemented math.comb (GH-11414) 2019-06-01 00:21:27 -07:00
Mario Corchero 354227a1e9 Add option to trace to run modules (GH-5134)
Adds a new option in trace that allows tracing runnable modules. It is
exposed as `--module module_name` as `-m` is already in use for another
argument.
2019-06-01 14:49:09 +10:00
Eric Snow 396e0a8d9d
bpo-36818: Add PyInterpreterState.runtime field. (gh-13129)
https://bugs.python.org/issue36818
2019-05-31 21:16:47 -06:00
Tim Peters 1c263e39c4
bpo-37029: keep usable_arenas in sorted order without searching (#13612)
This adds a vector of "search fingers" so that usable_arenas can be kept in sorted order (by number of free pools) via constant-time operations instead of linear search.

This should reduce worst-case time for reclaiming a great many objects from O(A**2) to O(A), where A is the number of arenas.  See bpo-37029.
2019-05-31 21:16:04 -05:00
Zackery Spytz 549e55a308 bpo-12202: Properly check MsiSummaryInfoGetProperty() calls in msilib (GH-13711) 2019-06-01 03:16:20 +03:00
Ammar Askar a6ec1ce1ac bpo-33361: Fix bug with seeking in StreamRecoders (GH-8278) 2019-05-31 22:44:00 +03:00
Pablo Galindo aac4d0342c
bpo-26826: Expose copy_file_range in the os module (GH-7255) 2019-05-31 19:39:47 +01:00
Zackery Spytz c8d5bf6c3f bpo-12639: msilib.Directory.start_component() fails if *keyfile* is not None (GH-13688)
msilib.Directory.start_component() was passing an extra argument to CAB.gen_id().
2019-05-31 09:43:13 -07:00
Akshay Sharma 4612671df2 bpo-25735: math.factorial doc should mention integer return type (GH-6420) 2019-05-31 12:41:17 -04:00
Christian Heimes 8cbb5b6625
bpo-26835: Add file sealing constants to fcntl (GH-13694)
Co-authored-by: nanjekyejoannah <nanjekyejoannah@gmail.com>
2019-05-31 18:32:33 +02:00
Zackery Spytz c7f803b08e bpo-36379: __ipow__ must be a ternaryfunc, not a binaryfunc (GH-13546)
If a type's __ipow__ method was implemented in C, attempting to use
the *modulo* parameter would cause crashes.


https://bugs.python.org/issue36379
2019-05-31 02:46:36 -07:00
Christian Heimes c7f7069e77
bpo-34271: Add ssl debugging helpers (GH-10031)
The ssl module now can dump key material to a keylog file and trace TLS
protocol messages with a tracing callback. The default and stdlib
contexts also support SSLKEYLOGFILE env var.

The msg_callback and related enums are private members. The feature
is designed for internal debugging and not for end users.

Signed-off-by: Christian Heimes <christian@python.org>
2019-05-31 11:44:05 +02:00
Serhiy Storchaka e9b51c0ad8
bpo-26660, bpo-35144: Fix permission errors in TemporaryDirectory cleanup. (GH-10320)
TemporaryDirectory.cleanup() failed when non-writeable or non-searchable
files or directories were created inside a temporary directory.
2019-05-31 11:30:37 +03:00
Serhiy Storchaka 38ab7d4721
bpo-31829: Make protocol 0 pickles be loadable in text mode in Python 2. (GH-11859)
Escape ``\r``, ``\0`` and ``\x1a`` (end-of-file on Windows) in Unicode strings.
2019-05-31 11:29:39 +03:00
Serhiy Storchaka 14a0e16c88
bpo-36548: Improve the repr of re flags. (GH-12715) 2019-05-31 10:39:47 +03:00
Ivan Levkivskyi 4c23aff065
bpo-29262: Add get_origin() and get_args() introspection helpers to typing (GH-13685)
This is an old feature request that appears from time to time. After a year of experimenting with various introspection capabilities in `typing_inspect` on PyPI, I propose to add these two most commonly used functions: `get_origin()` and `get_args()`. These are essentially thin public wrappers around private APIs: `__origin__` and `__args__`.

As discussed in the issue and on the typing tracker, exposing some public helpers instead of `__origin__` and `__args__` directly will give us more flexibility if we will decide to update the internal representation, while still maintaining backwards compatibility.

The implementation is very simple an is essentially a copy from `typing_inspect` with one exception: `ClassVar` was special-cased in `typing_inspect`, but I think this special-casing doesn't really help and only makes things more complicated.
2019-05-31 00:10:07 +01:00
Matthias Bussonnier eea47e0939 bpo-36953: Delay removal of ABCs from collections. (GH-13409)
Bump the removal to 3.9, indicate collections.abc available since 3.3,
replace version-changed directive to deprecated-removed.


https://bugs.python.org/issue36953
2019-05-30 14:45:48 -07:00
xdegaye 5437ccca14 bpo-36342: Fix test_multiprocessing in test_venv (GH-12513)
when platform lacks a functioning sem_open implementation



https://bugs.python.org/issue36342
2019-05-30 14:42:29 -07:00
Alex Grönholm 98ef92002e bpo-36999: Add asyncio.Task.get_coro() (GH-13680)
https://bugs.python.org/issue36999
2019-05-30 08:30:09 -07:00
Zackery Spytz 249b7d59d8 bpo-20602: Do not clear sys.flags and sys.float_info during shutdown (GH-8096)
There is no need to clear these immutable objects during shutdown.
2019-05-30 13:08:24 +02:00
Christian Heimes 6eb814b8ce
bpo-37098: Skip memfd_create test before Linux 3.17 (GH-13677) 2019-05-30 11:27:06 +02:00
Giampaolo Rodola 413d955f8e
bpo-36610: shutil.copyfile(): use sendfile() on Linux only (GH-13675)
...and avoid using it on Solaris as it can raise EINVAL if offset is equal or bigger than the size of the file
2019-05-30 14:05:41 +08:00
Ying Wang a16387ab2d bpo-24564: shutil.copystat(): ignore EINVAL on os.setxattr() (GH-13369) 2019-05-30 11:25:31 +08:00
Zackery Spytz 8f96c9f8ed bpo-37007: Implement socket.if_nametoindex(), if_indextoname() and if_nameindex() on Windows (GH-13522) 2019-05-29 14:02:37 -07:00
Antoine Pitrou ada319bb6d
bpo-32388: Remove cross-version binary compatibility requirement in tp_flags (GH-4944)
It is now allowed to add new fields at the end of the PyTypeObject struct without having to allocate a dedicated compatibility flag in tp_flags.

This will reduce the risk of running out of bits in the 32-bit tp_flags value.
2019-05-29 22:12:38 +02:00
Zackery Spytz 43fdbd2729 bpo-26836: Add os.memfd_create() (#13567)
* bpo-26836: Add os.memfd_create()

* Use the glibc wrapper for memfd_create()

Co-Authored-By: Christian Heimes <christian@python.org>

* Fix deletions caused by autoreconf.

* Use MFD_CLOEXEC as the default value for *flags*.

* Add memset_s to configure.ac.

* Revert memset_s changes.

* Apply the requested changes.

* Tweak the docs.
2019-05-29 21:57:03 +02:00
Gregory P. Smith 0c2f930564
bpo-22385: Support output separators in hex methods. (#13578)
* bpo-22385: Support output separators in hex methods.

Also in binascii.hexlify aka b2a_hex.

The underlying implementation behind all hex generation in CPython uses the
same pystrhex.c implementation.  This adds support to bytes, bytearray,
and memoryview objects.

The binascii module functions exist rather than being slated for deprecation
because they return bytes rather than requiring an intermediate step through a
str object.

This change was inspired by MicroPython which supports sep in its binascii
implementation (and does not yet support the .hex methods).

https://bugs.python.org/issue22385
2019-05-29 11:46:58 -07:00
Jeroen Demeyer aacc77fbd7 bpo-36974: implement PEP 590 (GH-13185)
Co-authored-by: Jeroen Demeyer <J.Demeyer@UGent.be>
Co-authored-by: Mark Shannon <mark@hotpy.org>
2019-05-29 20:31:52 +02:00
Anthony Sottile d30da5dd9a bpo-36983: Fix typing.__all__ and add test for exported names (GH-13456)
https://bugs.python.org/issue36983
2019-05-29 11:19:37 -07:00
Andrew Svetlov 4dd3e3f9bb bpo-32972: Async test case (GH-13386)
Add explicit `asyncSetUp` and `asyncTearDown` methods.
The rest is the same as for #13228

`AsyncTestCase` create a loop instance for every test for the sake of test isolation.
Sometimes a loop shared between all tests can speed up tests execution time a lot but it requires control of closed resources after every test finish. Basically, it requires nested supervisors support that was discussed with @1st1 many times. Sorry, asyncio supervisors have no chance to land on Python 3.8.

The PR intentionally does not provide API for changing the used event loop or getting the test loop: use `asyncio.set_event_loop_policy()` and `asyncio.get_event_loop()` instead.

The PR adds four overridable methods to base `unittest.TestCase` class:
```
    def _callSetUp(self):
        self.setUp()

    def _callTestMethod(self, method):
        method()

    def _callTearDown(self):
        self.tearDown()

    def _callCleanup(self, function, /, *args, **kwargs):
        function(*args, **kwargs)
```
It allows using asyncio facilities with minimal influence on the unittest code.

The last but not least: the PR respects contextvars. The context variable installed by `asyncSetUp` is available on test, `tearDown` and a coroutine scheduled by `addCleanup`.


https://bugs.python.org/issue32972
2019-05-29 02:33:59 -07:00
Bo Bayles ca80495592 bpo-22454: Add shlex.join() (the opposite of shlex.split()) (GH-7605) 2019-05-29 09:06:11 +01:00
依云 744c08a9c7 bpo-35246: fix support for path-like args in asyncio subprocess (GH-13628)
Drop isinstance checks from create_subprocess_exec function and let
subprocess module do them.

https://bugs.python.org/issue35246


https://bugs.python.org/issue35246
2019-05-28 23:50:59 -07:00
Victor Stinner 8b09500345
bpo-37076: _thread.start_new_thread() calls _PyErr_WriteUnraisableMsg() (GH-13617)
_thread.start_new_thread() now logs uncaught exception raised by the
function using sys.unraisablehook(), rather than sys.excepthook(), so
the hook gets access to the function which raised the exception.
2019-05-29 02:57:56 +02:00
Guido van Rossum 77f0ed7a42
bpo-37072: Fix crash in PyAST_FromNodeObject() when flags is NULL (#13634)
I'm confident that this fixes the reported crash. flags=NULL is treated as using the latest minor version.

https://bugs.python.org/issue37072
2019-05-28 16:44:58 -07:00
Dino Viehland 415406999d
bpo-37001: Makes symtable.symtable have parity with compile for input (#13483)
* Makes symtable.symtable have parity for accepted datatypes
for source code as compile()

* Add NEWS blurb
2019-05-28 16:21:17 -07:00
Francisco Facioni ab0716ed1e bpo-22102: Fixes zip files with disks set to 0 (GH-5985) 2019-05-28 19:15:11 -04:00
Serhiy Storchaka 9e3c452639
bpo-31961: Fix support of path-like executables in subprocess. (GH-5914) 2019-05-28 22:49:35 +03:00
Joannah Nanjekye 2e33ecd7c9 bpo-22640: Add silent mode to py_compile.compile() (GH-12976) 2019-05-28 19:29:04 +03:00
Zackery Spytz 3c8724fc60 bpo-33407: Implement Py_DEPRECATED() on MSVC (GH-8980) 2019-05-28 17:16:33 +02:00
Victor Stinner 17a5588740
bpo-33725: multiprocessing uses spawn by default on macOS (GH-13603)
On macOS, the multiprocessing module now uses the "spawn" start
method by default.
2019-05-28 16:02:50 +02:00
Zackery Spytz 05f16416d9 bpo-26423: Fix possible overflow in wrap_lenfunc() (GH-13606)
Fix possible overflow in wrap_lenfunc() when
sizeof(long) < sizeof(Py_ssize_t) (e.g., 64-bit Windows).
2019-05-28 14:55:28 +02:00
Mario Corchero 04530812e9 bpo-32299: Return patched dict when using patch.dict as a context manager (GH-11062) 2019-05-28 08:53:30 -04:00
Jeroen Demeyer eb65e2443a bpo-36922: implement PEP-590 Py_TPFLAGS_METHOD_DESCRIPTOR (GH-13338)
Co-authored-by: Mark Shannon <mark@hotpy.org>
2019-05-28 14:42:53 +02:00
Inada Naoki 9a7e5b1b42
bpo-35279: reduce default max_workers of ThreadPoolExecutor (GH-13618) 2019-05-28 21:02:52 +09:00
Andrew Svetlov bafd4b5ac8 bpo-29883: Asyncio proactor udp (GH-13440)
Follow-up for #1067


https://bugs.python.org/issue29883
2019-05-28 02:52:15 -07:00
Ivan Levkivskyi 74d7f76e2c
bpo-37058: PEP 544: Add Protocol to typing module (GH-13585)
I tried to get rid of the `_ProtocolMeta`, but unfortunately it didn'y work. My idea to return a generic alias from `@runtime_checkable` made runtime protocols unpickleable. I am not sure what is worse (a custom metaclass or having some classes unpickleable), so I decided to stick with the status quo (since there were no complains so far). So essentially this is a copy of the implementation in `typing_extensions` with two modifications:
* Rename `@runtime` to `@runtime_checkable` (plus corresponding updates).
* Allow protocols that extend `collections.abc.Iterable` etc.
2019-05-28 08:40:15 +01:00
Matthias Bussonnier 3880f263d2 bpo-36933: Remove sys.set_coroutine_wrapper (marked for removal in 3.8) (GH-13577)
It has been documented as deprecated and to be removed in 3.8; 

From a comment on another thread – which I can't find ; leave get_coro_wrapper() for now, but always return `None`.


https://bugs.python.org/issue36933
2019-05-28 00:10:59 -07:00
Xtreak 436c2b0d67 bpo-36996: Handle async functions when mock.patch is used as a decorator (GH-13562)
Return a coroutine while patching async functions with a decorator. 

Co-authored-by: Andrew Svetlov <andrew.svetlov@gmail.com>


https://bugs.python.org/issue36996
2019-05-28 00:07:38 -07:00
sbstp f0d4c64019 bpo-36686: Improve the documentation of the std* params in loop.subprocess_exec (GH-13586)
https://bugs.python.org/issue36686
2019-05-27 16:51:19 -07:00
Victor Stinner a3568417c4
bpo-37054, _pyio: Fix BytesIO and TextIOWrapper __del__() (GH-13601)
Fix destructor _pyio.BytesIO and _pyio.TextIOWrapper: initialize
their _buffer attribute as soon as possible (in the class body),
because it's used by __del__() which calls close().
2019-05-28 01:44:21 +02:00
Victor Stinner cd590a7ced
bpo-1230540: Add threading.excepthook() (GH-13515)
Add a new threading.excepthook() function which handles uncaught
Thread.run() exception. It can be overridden to control how uncaught
exceptions are handled.

threading.ExceptHookArgs is not documented on purpose: it should not
be used directly.

* threading.excepthook() and threading.ExceptHookArgs.
* Add _PyErr_Display(): similar to PyErr_Display(), but accept a
  'file' parameter.
* Add _thread._excepthook(): C implementation of the exception hook
  calling _PyErr_Display().
* Add _thread._ExceptHookArgs: structseq type.
* Add threading._invoke_excepthook_wrapper() which handles the gory
  details to ensure that everything remains alive during Python
  shutdown.
* Add unit tests.
2019-05-28 00:39:52 +02:00
Andrew Svetlov 23b4b697e5 bpo-36889: Merge asyncio streams (GH-13251)
https://bugs.python.org/issue36889
2019-05-27 12:56:22 -07:00
Eric V. Smith 6f6ff8a565
bpo-37050: Remove expr_text from FormattedValue ast node, use Constant node instead (GH-13597)
When using the "=" debug functionality of f-strings, use another Constant node (or a merged constant node) instead of adding expr_text to the FormattedValue node.
2019-05-27 15:31:52 -04:00
Zackery Spytz 02db696732 bpo-32941: Add madvise() for mmap objects (GH-6172)
Allow mmap objects to access the madvise() system call.
2019-05-27 18:48:16 +02:00
Victor Stinner 331a6a56e9
bpo-36763: Implement the PEP 587 (GH-13592)
* Add a whole new documentation page:
  "Python Initialization Configuration"
* PyWideStringList_Append() return type is now PyStatus,
  instead of int
* PyInterpreterState_New() now calls PyConfig_Clear() if
  PyConfig_InitPythonConfig() fails.
* Rename files:

  * Python/coreconfig.c => Python/initconfig.c
  * Include/cpython/coreconfig.h => Include/cpython/initconfig.h
  * Include/internal/: pycore_coreconfig.h => pycore_initconfig.h

* Rename structures

  * _PyCoreConfig => PyConfig
  * _PyPreConfig => PyPreConfig
  * _PyInitError => PyStatus
  * _PyWstrList => PyWideStringList

* Rename PyConfig fields:

  * use_module_search_paths => module_search_paths_set
  * module_search_path_env => pythonpath_env

* Rename PyStatus field: _func => func
* PyInterpreterState: rename core_config field to config
* Rename macros and functions:

  * _PyCoreConfig_SetArgv() => PyConfig_SetBytesArgv()
  * _PyCoreConfig_SetWideArgv() => PyConfig_SetArgv()
  * _PyCoreConfig_DecodeLocale() => PyConfig_SetBytesString()
  * _PyInitError_Failed() => PyStatus_Exception()
  * _Py_INIT_ERROR_TYPE_xxx enums => _PyStatus_TYPE_xxx
  * _Py_UnixMain() => Py_BytesMain()
  * _Py_ExitInitError() => Py_ExitStatusException()
  * _Py_PreInitializeFromArgs() => Py_PreInitializeFromBytesArgs()
  * _Py_PreInitializeFromWideArgs() => Py_PreInitializeFromArgs()
  * _Py_PreInitialize() => Py_PreInitialize()
  * _Py_RunMain() => Py_RunMain()
  * _Py_InitializeFromConfig() => Py_InitializeFromConfig()
  * _Py_INIT_XXX() => _PyStatus_XXX()
  * _Py_INIT_FAILED() => _PyStatus_EXCEPTION()

* Rename 'err' PyStatus variables to 'status'
* Convert RUN_CODE() macro to config_run_code() static inline function
* Remove functions:

  * _Py_InitializeFromArgs()
  * _Py_InitializeFromWideArgs()
  * _PyInterpreterState_GetCoreConfig()
2019-05-27 16:39:22 +02:00
Yury Selivanov 8cd5165ba0 bpo-37027: Return a proxy socket object from transp.get_extra_info('socket') (GH-13530)
Return a safe to use proxy socket object from `transport.get_extra_info('socket')`




https://bugs.python.org/issue37027
2019-05-27 06:57:19 -07:00
Rémi Lapeyre 674ee12600 bpo-35397: Remove deprecation and document urllib.parse.unwrap (GH-11481) 2019-05-27 09:43:45 -04:00
Andrew Svetlov 1f39c28e48 bpo-37035: Don't log OSError (GH-13548)
https://bugs.python.org/issue37035
2019-05-27 06:28:34 -07:00
Xtreak ff6b2e66b1 bpo-37047: Refactor AsyncMock setup logic for autospeccing (GH-13574)
Handle late binding and attribute access in unittest.mock.AsyncMock
setup for autospeccing.
2019-05-27 14:56:23 +02:00
Yury Selivanov 431b540bf7
bpo-32528: Make asyncio.CancelledError a BaseException. (GH-13528)
This will address the common mistake many asyncio users make:
an "except Exception" clause breaking Tasks cancellation.

In addition to this change, we stop inheriting asyncio.TimeoutError
and asyncio.InvalidStateError from their concurrent.futures.*
counterparts.  There's no point for these exceptions to share the
inheritance chain.

In 3.9 we'll focus on implementing supervisors and cancel scopes,
which should allow better handling of all exceptions, including
SystemExit and KeyboardInterrupt
2019-05-27 14:45:12 +02:00
Yury Selivanov 16cefb0bc7
bpo-37028: asyncio REPL; activated via 'python -m asyncio'. (GH-13472)
This makes it easy to play with asyncio APIs with simply
using async/await in the REPL.
2019-05-27 13:42:29 +02:00
Raymond Hettinger b821868e6d
bpo-36772 Allow lru_cache to be used as decorator without making a function call (GH-13048) 2019-05-26 11:27:35 -07:00
Chih-Hsuan Yen aaf47caf35 bpo-37053: handle strings like u"bar" correctly in Tools/parser/unparse.py (GH-13583)
Constant.kind is added in https://bugs.python.org/issue36280.
Current possible values for Constant.kind are "u" or None.

For r'bar' and b'bar', Constant.kind value is None, so there's no need
for special handling.


https://bugs.python.org/issue37053
2019-05-26 10:08:19 -07:00
Antoine Pitrou 91f4380ced
bpo-36785: PEP 574 implementation (GH-7076) 2019-05-26 17:10:09 +02:00
Julien Palard 180dc1b0f4 bpo-28866: No type cache for types with specialized mro, invalidation is hard. (#13157)
* No type cache for types with specialized mro, invalidation is hard.

* FIX: Don't disable method cache custom types that do not implement mro().

* fixing implem.

* Avoid storing error flags, also decref.

* news entry

* Clear as soon as we're getting an error.

* FIX: Reference leak.
2019-05-26 15:25:47 +01:00
Ivan Levkivskyi 135c6a56e5
bpo-37049: PEP 589: Add TypedDict to typing module (GH-13573)
The implementation is straightforward and essentially is just copied from `typing_extensions`.
2019-05-26 09:39:24 +01:00
Ivan Levkivskyi b891c465bb
bpo-37046: PEP 586: Add Literal to typing module (#13572)
The implementation is straightforward and essentially is just copied from `typing_extensions`.
2019-05-26 09:37:48 +01:00
Ivan Levkivskyi f367242d10
bpo-37045: PEP 591: Add final qualifiers to typing module (GH-13571)
The implementation is straightforward, it just mimics `ClassVar` (since the latter is also a name/access qualifier, not really a type). Also it is essentially copied from `typing_extensions`.
2019-05-26 09:37:07 +01:00
Michael J. Sullivan 47dd2f9fd8 bpo-37017: PyObject_CallMethodObjArgs uses LOAD_METHOD optimization (GH-13516)
Update PyObject_CallMethodObjArgs and _PyObject_CallMethodIdObjArgs
to use _PyObject_GetMethod to avoid creating a bound method object
in many cases.

On a microbenchmark of PyObject_CallMethodObjArgs calling a method on
an interpreted Python class, this optimization resulted in a 1.7x
speedup.
2019-05-26 16:23:33 +09:00
Terry Jan Reedy 81bb97df61
bpo-37038: Make idlelib.run runnable; add test clause (GH-13560) 2019-05-24 21:59:53 -04:00
Jason R. Coombs 1bbf7b661f bpo-34632: Add importlib.metadata (GH-12547)
Add importlib.metadata module as forward port of the standalone importlib_metadata.
2019-05-24 16:59:01 -07:00
Victor Stinner a9f05d69cc
bpo-37032: Add CodeType.replace() method (GH-13542) 2019-05-24 23:57:23 +02:00
Victor Stinner 1c9debd236
bpo-35907: Fix typo in the NEWS entry (GH-13559) 2019-05-24 23:06:25 +02:00
Rémi Lapeyre 458560347f bpo-36969: Make PDB args command display positional only arguments (GH-13459) 2019-05-24 21:44:31 +01:00
Victor Stinner deffee5774
bpo-35907: Clarify the NEWS entry (GH-13523) 2019-05-24 22:06:32 +02:00
Berker Peksag 14738ff83d
bpo-8138: Initialize wsgiref's SimpleServer as single-threaded (GH-12977) 2019-05-24 20:24:42 +03:00
E. M. Bray c994c8fc19 bpo-21536: On Cygwin, C extensions must be linked with libpython (GH-13549)
It is also possible to link against a library or executable with a
statically linked libpython, but not both with the same DLL.  In fact
building a statically linked python is currently broken on Cygwin
for other (related) reasons.

The same problem applies to other POSIX-like layers over Windows
(MinGW, MSYS) but Python's build system does not seem to attempt
to support those platforms at the moment.
2019-05-24 17:33:47 +02:00
Dan Rose 2a37f8f55b bpo-36045: builtins.help() now prefixes `async` for async functions (GH-12010)
Previously, it was hard to tell whether a function should be awaited. It was also incorrect (per PEP 484) to put this in the type hint for coroutine functions. Added this info to the output of builtins.help and pydoc.


https://bugs.python.org/issue36045
2019-05-24 04:38:01 -07:00
Cheryl Sabella c95c93d4eb
bpo-20285: Improve help docs for object (GH-4759) 2019-05-24 06:43:29 -04:00
Matěj Cepl 608876b6b1 bpo-23395: Fix PyErr_SetInterrupt if the SIGINT signal is ignored or not handled (GH-7778)
``_thread.interrupt_main()`` now avoids setting the Python error status if the ``SIGINT`` signal is ignored or not handled by Python.
2019-05-23 22:30:00 +02:00
Steve Dower b82e17e626
bpo-36842: Implement PEP 578 (GH-12613)
Adds sys.audit, sys.addaudithook, io.open_code, and associated C APIs.
2019-05-23 08:45:22 -07:00
Damien Nadé 394119afc6 bpo-37008: make mock_open handle able to honor next() (GH-13492)
I've reported the issue on https://bugs.python.org/issue37008 and now I'm trying to bring a solution to this minor issue.

I think it could be trivially backported to 3.7 branch.


https://bugs.python.org/issue37008
2019-05-23 03:03:25 -07:00
David Carlier 51aa35e9e1 bpo-33164: update blake2 implementation (GH-6286) 2019-05-23 13:32:44 +09:00
Victor Stinner 0a8e57248b
bpo-36721: Add --embed option to python-config (GH-13500)
To embed Python into an application, a new --embed option must be
passed to "python3-config --libs --embed" to get "-lpython3.8" (link
the application to libpython). To support both 3.8 and older, try
"python3-config --libs --embed" first and fallback to "python3-config
--libs" (without --embed) if the previous command fails.

Add a pkg-config "python-3.8-embed" module to embed Python into an
application: "pkg-config python-3.8-embed --libs" includes
"-lpython3.8".  To support both 3.8 and older, try "pkg-config
python-X.Y-embed --libs" first and fallback to "pkg-config python-X.Y
--libs" (without --embed) if the previous command fails (replace
"X.Y" with the Python version).

On the other hand, "pkg-config python3.8 --libs" no longer contains
"-lpython3.8". C extensions must not be linked to libpython (except
on Android, case handled by the script); this change is backward
incompatible on purpose.

"make install" now also installs "python-3.8-embed.pc".
2019-05-23 03:30:23 +02:00
Batuhan Taşkaya ef5bb25e2d bpo-27737: Allow whitespace only headers encoding (#13478) 2019-05-22 21:13:16 -04:00
Victor Stinner df22c03b93
bpo-36829: PyErr_WriteUnraisable() normalizes exception (GH-13507)
PyErr_WriteUnraisable() now creates a traceback object if there is no
current traceback. Moreover, call PyErr_NormalizeException() and
PyException_SetTraceback() to normalize the exception value. Ignore
silently any error.
2019-05-23 01:00:58 +02:00
Victor Stinner e4d300e07c
bpo-36829: Add test.support.catch_unraisable_exception() (GH-13490)
* Copy test_exceptions.test_unraisable() to
  test_sys.UnraisableHookTest().
* Use catch_unraisable_exception() in test_coroutines,
  test_exceptions, test_generators.
2019-05-22 23:44:02 +02:00
Sean 904e34d4e6 bpo-24882: Let ThreadPoolExecutor reuse idle threads before creating new thread (#6375)
* Fixes issue 24882

* Add news file entry for change.

* Change test_concurrent_futures.ThreadPoolShutdownTest

Adjust the shutdown test so that, after submitting three jobs
to the executor, the test checks for less than three threads,
instead of looking for exactly three threads.

If idle threads are being recycled properly, then we should have
less than three threads.

* Switched idle count to semaphor, Updated tests

As suggested by reviewer tomMoral, swapped lock-protected counter
with a semaphore to track the number of unused threads.

Adjusted test_threads_terminate to wait for completiton of the
previous future before submitting a new one (and checking the
number of threads used).

Also added a new test to confirm the thread pool can be saturated.

* Updates tests as requested by pitrou.

* Correct minor whitespace error.

* Make test_saturation faster
2019-05-22 23:29:58 +02:00
Sam Martin 2a3a2ece50 bpo-33110: Catch errors raised when running add_done_callback on already completed futures (GH-13141)
Wrap the callback call within the `add_done_callback` function within concurrent.futures, in order to behave in an identical manner to callbacks added to a running future are triggered once it has completed.
2019-05-22 23:29:02 +02:00
Michael J. Sullivan d8a82e2897 bpo-36878: Only allow text after `# type: ignore` if first character ASCII (GH-13504)
This disallows things like `# type: ignoreé`, which seems wrong.

Also switch to using Py_ISALNUM for the alnum check, for consistency
with other code (and maybe correctness re: locale issues?).


https://bugs.python.org/issue36878
2019-05-22 13:43:36 -07:00
Victor Stinner 0c2b6a3943
bpo-35907, CVE-2019-9948: urllib rejects local_file:// scheme (GH-13474)
CVE-2019-9948: Avoid file reading as disallowing the unnecessary URL
scheme in URLopener().open() and URLopener().retrieve()
of urllib.request.

Co-Authored-By: SH <push0ebp@gmail.com>
2019-05-22 22:15:01 +02:00
Jake Tesler b121f63155 bpo-36084: Add native thread ID (TID) to threading.Thread (GH-13463)
Add native thread ID (TID) to threading.Thread objects
(supported platforms: Windows, FreeBSD, Linux, macOS).
2019-05-22 17:43:16 +02:00
Jelle Zijlstra b3be407288 bpo-33482: fix codecs.StreamRecoder.writelines (GH-6779)
A very simple fix. I found this while writing typeshed stubs for StreamRecoder.


https://bugs.python.org/issue33482
2019-05-22 08:18:26 -07:00
Michael J. Sullivan 933e1509ec bpo-36878: Track extra text added to 'type: ignore' in the AST (GH-13479)
GH-13238 made extra text after a # type: ignore accepted by the parser.
This finishes the job and actually plumbs the extra text through the
parser and makes it available in the AST.
2019-05-22 15:54:20 +01:00
Paul Dagnelie 4c7a46eb3c bpo-36972: Add SupportsIndex (GH-13448)
In order to support typing checks calling hex(), oct() and bin() on user-defined classes, a SupportIndex protocol is required. The ability to check these at runtime would be good to add for completeness sake. This is pretty much just a copy of SupportsInt with the names tweaked.
2019-05-22 15:23:01 +01:00
Marcel Plch 33e71e01e9 bpo-31862: Port binascii to PEP 489 multiphase initialization (GH-4108) 2019-05-22 13:51:25 +02:00
Jeroen Demeyer 77aa396bb9 bpo-36907: fix refcount bug in _PyStack_UnpackDict() (GH-13381) 2019-05-22 13:09:35 +02:00
Victor Stinner ef9d9b6312
bpo-36829: Add sys.unraisablehook() (GH-13187)
Add new sys.unraisablehook() function which can be overridden to
control how "unraisable exceptions" are handled. It is called when an
exception has occurred but there is no way for Python to handle it.
For example, when a destructor raises an exception or during garbage
collection (gc.collect()).

Changes:

* Add an internal UnraisableHookArgs type used to pass arguments to
  sys.unraisablehook.
* Add _PyErr_WriteUnraisableDefaultHook().
* The default hook now ignores exception on writing the traceback.
* test_sys now uses unittest.main() to automatically discover tests:
  remove test_main().
* Add _PyErr_Init().
* Fix PyErr_WriteUnraisable(): hold a strong reference to sys.stderr
  while using it
2019-05-22 11:28:22 +02:00
Batuhan Taşkaya 7abf8c6081 bpo-25652: Fix __rmod__ of UserString (GH-13326)
The ``__rmod__`` method of ``collections.UserString`` class had a bug that made it unusable. 


https://bugs.python.org/issue25652
2019-05-21 13:27:36 -07:00
Matthias Bussonnier 565b4f1ac7 bpo-34616: Add PyCF_ALLOW_TOP_LEVEL_AWAIT to allow top-level await (GH-13148)
Co-Authored-By: Yury Selivanov <yury@magic.io>
2019-05-21 16:12:02 -04:00
Batuhan Taşkaya aa32a7e111 bpo-23378: Add an extend action to argparse (GH-13305)
Add an extend action to argparse


https://bugs.python.org/issue23378
2019-05-21 10:47:42 -07:00
Jörg Stucke d5c120f7eb bpo-36035: fix Path.rglob for broken links (GH-11988)
Links creating an infinite symlink loop would raise an exception.
2019-05-21 19:44:40 +02:00
Max Bernstein ccb7ca728e bpo-36929: Modify io/re tests to allow for missing mod name (#13392)
* bpo-36929: Modify io/re tests to allow for missing mod name

For a vanishingly small number of internal types, CPython sets the
tp_name slot to mod_name.type_name, either in the PyTypeObject or the
PyType_Spec. There are a few minor places where this surfaces:

* Custom repr functions for those types (some of which ignore the
  tp_name in favor of using a string literal, such as _io.TextIOWrapper)
* Pickling error messages

The test suite only tests the former. This commit modifies the test
suite to allow Python implementations to omit the module prefix.

https://bugs.python.org/issue36929
2019-05-21 10:09:21 -07:00
Lihua Zhao 4fb1502189 bpo-36648: fix mmap issue for VxWorks (GH-12394)
The mmap module set MAP_SHARED flag when map anonymous memory, however VxWorks
only support MAP_PRIVATE when map anonymous memory, this commit clear MAP_SHARED
and set MAP_PRIVATE.
2019-05-21 12:50:14 +02:00
pxinwr f2d7ac7e5b bpo-31904: Add posix module support for VxWorks (GH-12118) 2019-05-21 12:46:37 +02:00
Victor Stinner d12e75734d
Revert "bpo-36084: Add native thread ID to threading.Thread objects (GH-11993)" (GH-13458)
This reverts commit 4959c33d25.
2019-05-21 12:44:57 +02:00
Erik Janssens 925af1d99b bpo-36965: Fix includes in main.c on Windows with non-MSC compilers (GH-13421)
Include windows.h rather than crtdbg.h to get STATUS_CONTROL_C_EXIT constant.
Moreover, include windows.h on Windows, not only when MSC is used.
2019-05-21 12:11:11 +02:00
Rémi Lapeyre bf457c7d82 bpo-36969: Make PDB args command display keyword only arguments (GH-13452) 2019-05-20 23:17:30 +01:00
Matthias Bussonnier 1a3faf9d97 bpo-36952: Remove the bufsize parameter in fileinput.input(). (GH-13400)
This parameter is marked as deprecated since 3.6 and for removal in 3.8.
It already had no effects.
2019-05-20 23:44:11 +03:00
Batuhan Taşkaya 4011d865d0 bpo-23896: Add a grammar where exec isn't a stmt (#13272)
https://bugs.python.org/issue23896
2019-05-20 13:27:10 -07:00
Thomas Moreau c09a9f56c0 bpo-36888: Add multiprocessing.parent_process() (GH-13247) 2019-05-20 21:37:05 +02:00
Batuhan Taşkaya 5ae1c84bcd bpo-36949: Implement __repr__ on WeakSet (GH-13415) 2019-05-20 10:01:07 -07:00
Lisa Roach 77b3b7701a
bpo-26467: Adds AsyncMock for asyncio Mock library support (GH-9296) 2019-05-20 09:19:53 -07:00
Geoff Shannon 522ccef869 bpo-22865: Expand on documentation for the pty.spawn function (GH-11980) 2019-05-20 17:06:16 +02:00
Niklas Fiekas 9932fd91e8 bpo-35721: Close socket pair if Popen in _UnixSubprocessTransport fails (GH-11553)
This slightly expands an existing test case `test_popen_error` to trigger a `ResourceWarning` and fixes it.


https://bugs.python.org/issue35721
2019-05-20 05:02:16 -07:00
Terry Jan Reedy 6d965b39b7
bpo-36958: In IDLE, print exit message (GH-13435)
Print any argument other than None or int passed to SystemExit
or sys.exit().
2019-05-19 22:52:22 -04:00
Lysandros Nikolaou d673810b9d bpo-35252: Remove FIXME from test_functools (GH-10551) 2019-05-19 15:11:20 -07:00
Berker Peksag 7c59362a15
bpo-29183: Fix double exceptions in wsgiref.handlers.BaseHandler (GH-12914) 2019-05-19 18:56:15 +03:00
Bar Harel f4e1babf44 bpo-27141: Fix collections.UserList and UserDict shallow copy. (GH-4094) 2019-05-19 16:57:13 +03:00
Xtreak c661b30f89 bpo-36948: Fix NameError in urllib.request.URLopener.retrieve (GH-13389) 2019-05-19 16:40:05 +03:00
Jelle Zijlstra 9892f454d1 bpo-33519: clarify that .copy() is not part of the MutableSequence ABC (GH-6965) 2019-05-18 20:17:56 -04:00
Batuhan Taşkaya fa19a25c23 Add support for PEP572 in ast_unparse.c (GH-13337) 2019-05-18 23:10:20 +01:00
Anthony Sottile abea73bf4a bpo-2180: Treat line continuation at EOF as a `SyntaxError` (GH-13401)
This makes the parser consistent with the tokenize module (already the case
in `pypy`).

sample
------

```python
x = 5\
```

before
------

```console
$ python3 t.py
$ python3 -mtokenize t.py
t.py:2:0: error: EOF in multi-line statement
```

after
-----

```console
$ ./python t.py
  File "t.py", line 3
    x = 5\

         ^
SyntaxError: unexpected EOF while parsing
$ ./python -m tokenize t.py
t.py:2:0: error: EOF in multi-line statement
```



https://bugs.python.org/issue2180
2019-05-18 11:27:16 -07:00
Mark Dickinson 73934b9da0
bpo-36887: add math.isqrt (GH-13244)
* Add math.isqrt function computing the integer square root.

* Code cleanup: remove redundant comments, rename some variables.

* Tighten up code a bit more; use Py_XDECREF to simplify error handling.

* Update Modules/mathmodule.c

Co-Authored-By: Serhiy Storchaka <storchaka@gmail.com>

* Update Modules/mathmodule.c

Use real argument clinic type instead of an alias

Co-Authored-By: Serhiy Storchaka <storchaka@gmail.com>

* Add proof sketch

* Updates from review.

* Correct and expand documentation.

* Fix bad reference handling on error; make some variables block-local; other tidying.

* Style and consistency fixes.

* Add missing error check; don't try to DECREF a NULL a

* Simplify some error returns.

* Another two test cases:

- clarify that floats are rejected even if they happen to be
  squares of small integers
- TypeError beats ValueError for a negative float

* Documentation and markup improvements; thanks Serhiy for the suggestions!

* Cleaner Misc/NEWS entry wording.

* Clean up (with one fix) to the algorithm explanation and proof.
2019-05-18 12:29:50 +01:00
Toshio Kuratomi e7b1136ec3 Document a workaround for a curses bug (GH-13209) 2019-05-17 15:54:02 -07:00
Victor Stinner 12083284c5
bpo-36763: _Py_RunMain() doesn't call Py_Exit() anymore (GH-13390)
Py_Main() and _Py_RunMain() now return the exitcode rather than
calling Py_Exit(exitcode) when calling PyErr_Print() if the current
exception type is SystemExit.

* Add _Py_HandleSystemExit().
* Add pymain_exit_err_print().
* Add pymain_exit_print().
2019-05-17 23:05:29 +02:00
Edison A 98ff4d5fb6 bpo-36782: Created C API wrappers and added missing tests for functions in the PyDateTimeAPI. (#13088)
* created a c API wrapper for pyDate_FromDate and added the test

* 📜🤖 Added by blurb_it.

* fixed auto-alignment by vscode

* made changes as per PEP7

* Update 2019-05-04-21-25-19.bpo-36782.h3oPIb.rst

* Refactored code as per requested changes

* Remove Whitespace to Fix failed travis build

* Update 2019-05-04-21-25-19.bpo-36782.h3oPIb.rst

* Add a new line at end of ACKS

* Added C API function for PyDateTime_FromDateAndTime

* Added a test for the C API wrapper of PyDateTime_FromDateAndTime

* Added C API function for PyDateTime_FromDateAndTime

* Added a test for the C API wrapper of PyDateTime_FromDateAndTimeAndFold

* Remove Whitespace using patchcheck

* Added a C API function for PyTime_FromTime

* Added a test for the C API wrapper of PyTime_FromTime

* Added a C API function for PyTime_FromTimeAndFold

* Added a test for the C API wrapper of PyTime_FromTimeAndFold

* Added a C API function for PyDelta_FromDSU

* Added a test for the C API wrapper of PyDelta_FromDSU

* Refactor code, re-edit lines longer than 80 chars

* Fix Whitespace issues in DatetimeTester

* List all tests that were added in this PR

* Update 2019-05-04-21-25-19.bpo-36782.h3oPIb.rst

* Reformat code as per PEP7 guidelines

* Remove unused varibles from another function

* Added specific tests for the Fold Attribute

* Update 2019-05-04-21-25-19.bpo-36782.h3oPIb.rst

* Reformat code according to requested changes

* Reformat code to PEP7 Guidelines

* Reformat code to PEP7 Guidelines

* Re-add name to blurb

* Added a backtick to blurb file

* Update 2019-05-04-21-25-19.bpo-36782.h3oPIb.rst

* Remove the need to initialize mandatory parameters

* Make the macro parameter mandatory

* Re-arrange the order of unit-test args

* Removed the need to initialize macro

change all the int macro = 0 to int macro; now that macro is required

Co-Authored-By: Paul Ganssle <pganssle@users.noreply.github.com>

* Removed the need to initialize macro

change all the `int macro = 0` to `int macro`; now that macro is required

Co-Authored-By: Paul Ganssle <pganssle@users.noreply.github.com>

* Removed the need to initialize macro

change all the `int macro = 0` to `int macro`; now that macro is required

Co-Authored-By: Paul Ganssle <pganssle@users.noreply.github.com>

* Removed the need to initialize macro

change all the `int macro = 0` to `int macro`; now that macro is required

Co-Authored-By: Paul Ganssle <pganssle@users.noreply.github.com>

* Removed the need to initialize macro

change all the `int macro = 0` to `int macro`; now that macro is required

Co-Authored-By: Paul Ganssle <pganssle@users.noreply.github.com>

* Removed the need to initialize macro

change all the `int macro = 0` to `int macro`; now that macro is required

Co-Authored-By: Paul Ganssle <pganssle@users.noreply.github.com>
2019-05-17 22:28:42 +02:00
Abhilash Raj feac6cd775 bpo-33524: Fix the folding of email header when max_line_length is 0 or None (#13391)
and there are non-ascii characters in the header.
2019-05-17 15:28:44 -04:00
Pablo Galindo af8646c805
bpo-1875: Raise SyntaxError in invalid blocks that will be optimised away (GH-13332)
Move the check for dead conditionals (if 0) to the peephole optimizer
and make sure that the code block is still compiled to report any
existing syntax errors within.
2019-05-17 11:37:08 +01:00
Erwan Le Pape ac8eb8f36b bpo-35545: Fix asyncio discarding IPv6 scopes (GH-11271)
This PR proposes a solution to [bpo-35545](https://bugs.python.org/issue35545) by adding an optional `flowinfo` and `scopeid` to `asyncio.base_events._ipaddr_info` to carry the full address information into `_ipaddr_info` and avoid discarding IPv6 specific information.

Changelog entry & regression tests to come.


https://bugs.python.org/issue35545
2019-05-17 01:28:39 -07:00
Zackery Spytz 14514d9084 bpo-36946: Fix possible signed integer overflow when handling slices. (GH-13375)
The final addition (cur += step) may overflow, so use size_t for "cur".
"cur" is always positive (even for negative steps), so it is safe to use
size_t here.

Co-Authored-By: Martin Panter <vadmium+py@gmail.com>
2019-05-17 10:13:03 +03:00
Pablo Galindo aee19f54f6
bpo-36751: Undeprecate getfullargspec (GH-13245) 2019-05-16 21:08:15 +01:00
Andrew Svetlov 68b34a7204 bpo-36921: Deprecate @coroutine for sake of async def (GH-13346)
The second attempt. Now deprecate `@coroutine` only, keep `yield from fut` as is.


https://bugs.python.org/issue36921
2019-05-16 07:52:10 -07:00