Commit Graph

23294 Commits

Author SHA1 Message Date
Brandt Bucher 8bae21962b
bpo-39868: Update Language Reference for PEP 572. (#18793) 2020-03-05 21:19:22 -08:00
Victor Stinner f7b5d419bf
bpo-39855: Fix test_subprocess if nobody user doesn't exist (GH-18781)
test_subprocess.test_user() now skips the test on an user name if the
user name doesn't exist. For example, skip the test if the user
"nobody" doesn't exist on Linux.
2020-03-05 14:28:40 +01:00
Gregory P. Smith 85cf1d514b
bpo-13487: Use sys.modules.copy() in inspect.getmodule() for thread safety. (GH-18786)
`list(sys.modules.items())` was apparently not immune to "dictionary
changed size during iteration" errors.

Tested internally using an integration test that has run into this a couple of times in the past two years.  With this patch applied, the test is no longer flaky.
2020-03-04 16:45:22 -08:00
Victor Stinner 942f7a2dea
bpo-39674: Revert "bpo-37330: open() no longer accept 'U' in file mode (GH-16959)" (GH-18767)
This reverts commit e471e72977.

The mode will be removed from Python 3.10.
2020-03-04 18:50:22 +01:00
Batuhan Taşkaya d82e469048
bpo-39639: Remove the AST "Suite" node and associated code (GH-18513)
The AST "Suite" node is no longer used and it can be removed from the ASDL definition and related structures (compiler, visitors, ...).

Co-Authored-By: Victor Stinner <vstinner@python.org>
Co-authored-by: Brett Cannon <54418+brettcannon@users.noreply.github.com>
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-03-04 16:16:46 +00:00
Victor Stinner 1ec63b6203
bpo-39763: distutils.spawn now uses subprocess (GH-18743)
Reimplement distutils.spawn.spawn() function with the subprocess
module.

setup.py now uses a basic implementation of the subprocess module if
the subprocess module is not available: before required C extension
modules are built.
2020-03-04 14:50:19 +01:00
l0rb 22a9a546ff
bpo-39826: add getConnection() hook to logging HTTPHandler (GH-18745) 2020-03-04 10:49:51 +00:00
Brandt Bucher be501ca241
bpo-39702: Relax grammar restrictions on decorators (PEP 614) (GH-18570) 2020-03-03 14:25:44 -08:00
MojoVampire 469325c30e
bpo-35712: Make using NotImplemented in a boolean context issue a deprecation warning (GH-13195) 2020-03-03 20:50:17 +02:00
Armin Rigo 6daa37fd42
bpo-38091: Import deadlock detection causes deadlock (GH-17518)
Automerge-Triggered-By: @brettcannon
2020-03-02 17:37:25 -08:00
Steve Dower ce3a498408
bpo-38597: Never statically link extension initialization code on Windows (GH-18724) 2020-03-03 00:04:11 +00:00
Pablo Galindo 0c2b509f9d
bpo-39778: Don't traverse weak-reference lists OrderedDict's tp_traverse and tp_clear (GH-18749)
Objects do not own weak references to them directly through the __weakref__ list so these
do not need to be traversed by the GC.
2020-03-02 23:12:54 +00:00
Stefan Krah b3b9ade4a3
bpo-39776: Lock ++interp->tstate_next_unique_id. (GH-18746) (#18746)
- Threads created by PyGILState_Ensure() could have a duplicate tstate->id.
2020-03-02 21:22:36 +01:00
Lidi Zheng 4482337dec
bpo-39764: Make Task.get_stack accept ag_frame (#18669)
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2020-03-02 14:45:54 +02:00
Erlend Egeberg Aasland 1382c3289b
bpo-38380: Update macOS & Windows builds to SQLite v3.31.1 (GH-18678)
Automerge-Triggered-By: @zooba
2020-03-02 04:25:10 -08:00
Inada Naoki 2110551761
bpo-39775: inspect: Change Signature.parameters back to OrderedDict. (GH-18684) 2020-03-02 18:54:48 +09:00
Mark Dickinson 9f1cb1bb49
Fix misleading statement about mixed-type numeric comparisons (GH-18615) 2020-03-02 08:57:27 +00:00
Serhiy Storchaka 28d0bcac8b
bpo-38913: Fix segfault in Py_BuildValue("(s#O)", ...) if entered with exception raised. (GH-18656) 2020-03-02 08:42:39 +02:00
Chris A 2565edec2c
bpo-38971: Open file in codecs.open() closes if exception raised. (GH-17666)
Open issue in the BPO indicated a desire to make the implementation of
codecs.open() at parity with io.open(), which implements a try/except to
assure file stream gets closed before an exception is raised.
2020-03-02 08:39:50 +02:00
Shantanu 4edc95cf0a
bpo-39495: Remove default value from C impl of TreeBuilder.start (GH-18275) 2020-03-02 08:33:24 +02:00
Hakan Çelik 217dce9ee6
bpo-39815: add cached_property to all (GH-18726)
Automerge-Triggered-By: @pablogsal
2020-03-01 13:01:34 -08:00
Thomas Moreau 0e89076247
bpo-39678: refactor queue manager thread (GH-18551) 2020-03-01 21:49:14 +01:00
Batuhan Taşkaya 185903de12
bpo-39520: Fix un-parsing of ext slices with no dimensions (GH-18304) 2020-03-01 20:07:22 +00:00
Vlad Emelianov 768d739c1c
bpo-38641: Add lib2to3 support for starred expressions in return/yield statements (GH-16994) 2020-03-01 19:59:26 +00:00
Stephen Balousek 5e260e0fde
bpo-39548: Fix handling of 'WWW-Authenticate' header for Digest Auth (GH-18338)
* bpo-39548: Fix handling of 'WWW-Authenticate' header for Digest authentication

 - The 'qop' value in the 'WWW-Authenticate' header is optional. The
   presence of 'qop' in the header should be checked before its value
   is parsed with 'split'.

Signed-off-by: Stephen Balousek <stephen@balousek.net>

* bpo-39548: Fix handling of 'WWW-Authenticate' header for Digest authentication

 - Add NEWS item

Signed-off-by: Stephen Balousek <stephen@balousek.net>

* Update Misc/NEWS.d/next/Library/2020-02-06-05-33-52.bpo-39548.DF4FFe.rst

Co-Authored-By: Brandt Bucher <brandtbucher@gmail.com>

Co-authored-by: Brandt Bucher <brandtbucher@gmail.com>
2020-02-29 12:31:58 -08:00
Stefan Krah 815280eb16
bpo-39794: Add --without-decimal-contextvar (#18702) 2020-02-29 19:43:42 +01:00
Jason R. Coombs 0aeab5c438
bpo-39667: Sync zipp 3.0 (GH-18540)
* bpo-39667: Improve pathlib.Path compatibility on zipfile.Path and correct performance degradation as found in zipp 3.0

* 📜🤖 Added by blurb_it.

* Update docs for new zipfile.Path.open

* Rely on dict, faster than OrderedDict.

* Syntax edits on docs

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2020-02-29 10:34:11 -06:00
Henry Harutyunyan dc04a0571e
bpo-37534: Allow adding Standalone Document Declaration when generating XML documents (GH-14912) 2020-02-29 09:22:19 +01:00
Gregory P. Smith 02673352b5
bpo-39769: Fix compileall ddir for subpkgs. (GH-18676)
Fix compileall.compile_dir() ddir= behavior on sub-packages.

Fixes compileall.compile_dir's ddir parameter and compileall command
line flag `-d` to no longer write the wrong pathname to the generated
pyc file for submodules beneath the root of the directory tree being
compiled.  This fixes a regression introduced with Python 3.5.

Also marks the _new_ in 3.9 from PR #16012 parameters to compile_dir as keyword only (as that is the only way they will be used) and fixes an omission of them in one place from the docs.
2020-02-28 17:28:37 -08:00
Steve Dower 03153dd145
bpo-39789: Update Windows release build machines to VS 2019 (GH-18695)
Also fixes some potential Nuget build issues.
2020-02-29 00:21:46 +00:00
Shantanu c2f7eb254b
bpo-39718: add TYPE_IGNORE, COLONEQUAL to py38 changes in token (GH-18598) 2020-02-28 18:25:36 -05:00
Terry Jan Reedy 916895f939
bpo-13790: Change 'string' to 'specification' in format doc (GH-18690) 2020-02-28 14:59:16 -05:00
Terry Jan Reedy c705fd1e89
bpo-39781: Do not jump when select in IDLE codecontext (GH-18683)
Previously, the button-up part of selecting with a mouse was treated as a click
that meant 'jump' to this line, which modified the context and undid the selection
2020-02-28 13:22:55 -05:00
Markus Mohrhard 374d998b50
bpo-39609: set the thread_name_prefix for the default asyncio executor (GH-18458)
Just a small debugging improvement to identify the asyncio executor threads.
2020-02-27 12:01:47 -08:00
Inada Naoki 02a4d57263
bpo-39087: Optimize PyUnicode_AsUTF8AndSize() (GH-18327)
Avoid using temporary bytes object.
2020-02-27 13:48:59 +09:00
opavlyuk 21da76d1f1
bpo-34788: Add support for scoped IPv6 addresses (GH-13772)
Automerge-Triggered-By: @asvetlov
2020-02-26 06:33:57 -08:00
Łukasz Langa 9e52330415 Python 3.9.0a4
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE4/8oOcBIslwITevpsmmV4xAlBWgFAl5VnLMACgkQsmmV4xAl
 BWgOrxAAhpv1CJtFDT0st4Apksq6mZNvZ/Fn03wiKjoR88qGMQXlHLF0uI48Rh0Z
 AuxfzBu/PsuDo7JHCeMG4sEsaJ1OSD7nl6Xq4Q8EWFiiP6QwMQuS8yxQgTi8Vonf
 EjujegOT4HpD9gAnw8esjFK+ln89kaeTYWoUi62f9mEi3EMw9wFBt0JfZ0SWkQvf
 tuJ401zstVTNv2D0GV1l10QAKDEzjjxj/oelEGM1urxMAbv/Ztls6SEpCEvMQp3c
 cUP1GaHJcu1tQIXCDvUWeLDdEb+uW5mDzbRPTbMSwqcnTvr1HHdzsNMlJTwKcjpV
 wgJ88TraN9oonvv63oJ//38I1zxh+WnZHLc2jlh4aksLz6J98W/2rw0UT4LOoPdR
 6cdjL4jSkFAZqJb5+gEQ6iQBEF7Wz7FpPlaRjrX4ARJxj2EkvvYScyLqVfH5BFzu
 JrSR+FSfsXc1FpiQFmZXN2Fx+ci5WmJ2yzMV0kEriHdjRh+BMXNJEZUNJr5T1Ep/
 3DNTcnNSSlt8B+N4/lRaI+cgvtvIv5o0EdRKxXrchmsQz9gUb358Sid2XltQJZPC
 4baiv0/8mN8olljcVo4XkMASCx4g0u14mFtpOLOk43EJAmINTryGSrc2rAZ7vHDk
 QpiuFn16VROGHN0/1yAOSnSJT6t6r5OSz7DfYOiz+m55/Ve3pSc=
 =vwQR
 -----END PGP SIGNATURE-----
gpgsig -----BEGIN PGP SIGNATURE-----
 
 iQJEBAABCAAuFiEE4/8oOcBIslwITevpsmmV4xAlBWgFAl5VvAsQHGx1a2FzekBs
 YW5nYS5wbAAKCRCyaZXjECUFaKNdD/42ChJD3lR8atOaDMyioxF3gGqSmFZpizKC
 nuum61SbBlr3O2vbxVLROI49PZyZnCNyu2kxBAxnRtWabvCOWkpMNWxePQH2UTyI
 70upGno+buof3cVZIb4NZyqjkoQ9N2RZX95t9TC0vp/avT/cEf9/pRRFFSHfZhQm
 1awVZDXcuJzrgfswTKas9CbnjMWNcdjxXTtDRV1RBZ/72HnhPFM7uBYsQZSD8NeG
 674vsavGFToldZwFK1seYtSwzFoQpSiSKgkJUZvLpBUoqSxzfhu0K8FqmJCyWL1F
 IA/ORKKSrVxpspNqgqaMZjs722Nl7nYQM9SAB9X4tM07lopVt/hgCyv+ylsg26CN
 gqgypNLOLgj9EMODeEec04BPcCIjln3ASKyuO/HfOtH6zoUw+B+L7n/cPCC537Pv
 g4MwNOlTvWGjl3Ucfr5x4aoOo0vbfHG1WNbecq1+iPHxowMJMVbWiSUcjQmDn08p
 kjkztn1zCI9amrK/gNEBPRLdrxy614M1YZI8oD52OFFH1jpzq6zXKljX4T5VdUw2
 YGmvVnJyUPf3ovMqIoZSRVIO2N1tG02Dxm4H65XnO/zWkPPiDNxnJluY1ihzT1bP
 KHztUPVUgeQ4uHu267TqsYEAzv+lOL2OkA/4NcXzuBCD2GsoQTakBr3CLhC8BVht
 LegRXHDDmQ==
 =A6+h
 -----END PGP SIGNATURE-----

Merge tag 'v3.9.0a4'

Python 3.9.0a4
2020-02-26 01:30:02 +01:00
Łukasz Langa 6e02691f30
Python 3.9.0a4 2020-02-25 22:06:39 +01:00
Steve Dower d644891970
bpo-38403: Update nuspec file for deprecated field and git repository (GH-18657) 2020-02-25 20:07:00 +00:00
Brandt Bucher eb8ac57af2
bpo-36144: Dictionary Union (PEP 584) (#12088) 2020-02-24 19:47:34 -08:00
Berker Peksag ba22e8f174
bpo-30566: Fix IndexError when using punycode codec (GH-18632)
Trying to decode an invalid string with the punycode codec
shoud raise UnicodeError.
2020-02-25 06:19:03 +03:00
Daniel Hahler 4015d1cda3
bpo-39649: Remove obsolete check for `__args__` in bdb.Bdb.format_stack_entry (GH-18531)
Appears to be obsolete since 75bb54c3d8.

Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
2020-02-23 22:14:53 -05:00
Hakan Çelik aea045adb8
bpo-39654: Update pyclbr doc to reflect additional information returned (GH-18528)
Full nested function and class info makes it a module browser.

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
2020-02-23 21:00:40 -05:00
Antoine Pitrou 9f37872e30
bpo-39681: Fix C pickle regression with minimal file-like objects (#18592)
Fix a regression where the C pickle module wouldn't allow unpickling from a
file-like object that doesn't expose a readinto() method.
2020-02-23 23:33:53 +01:00
Serhiy Storchaka 559e7f165a
bpo-39648: Expand math.gcd() and math.lcm() to handle multiple arguments. (GH-18604)
* bpo-39648: Expand math.gcd() and math.lcm() to handle multiple arguments.

* Simplify fast path.

* Difine lcm() without arguments returning 1.

* Apply suggestions from code review

Co-Authored-By: Mark Dickinson <dickinsm@gmail.com>

Co-authored-by: Mark Dickinson <dickinsm@gmail.com>
2020-02-23 11:21:29 +00:00
ananthan-123 fbe2e0bb8a
bpo-17422: Language reference should specify restrictions on class namespace (#18559)
The language reference now specifies restrictions on class namespaces.  Adapted from a patch by Ethan Furman.
2020-02-22 09:56:01 -08:00
Yonatan Goldschmidt 1c56f8ffad
bpo-39382: Avoid dangling object use in abstract_issubclass() (GH-18530)
Hold reference of __bases__ tuple until tuple item is done with, because by
dropping the reference the item may be destroyed.
2020-02-22 15:11:48 +02:00
Berker Peksag fd5116c0e7
bpo-35950: Raise UnsupportedOperation in BufferedReader.truncate() (GH-18586)
The truncate() method of io.BufferedReader() should raise
UnsupportedOperation when it is called on a read-only
io.BufferedReader() instance.





https://bugs.python.org/issue35950



Automerge-Triggered-By: @methane
2020-02-21 09:57:26 -08:00
Christopher Hunt c2ac4cf040
bpo-35727: Use exit code 0 on sys.exit() in multiprocessing.Process. (GH-11538) 2020-02-21 10:33:04 +01:00
alclarks d4331c56b4
bpo-9495: avoid confusing chained exception in argparse test (GH-17120) 2020-02-21 10:48:36 +02:00
ananthan-123 f2ee21d858
bpo-39479:Add math.lcm() function: Least Common Multiple (#18547)
* Update math.rst

* Update math.rst

* updated whats new

* Update test_math.py

* Update mathmodule.c

* Update mathmodule.c.h

* Update ACKS

* 📜🤖 Added by blurb_it.

* Update 3.9.rst

* Update 2020-02-18-12-37-16.bpo-39479.j3UcCq.rst

* Update math.rst

* Update 2020-02-18-12-37-16.bpo-39479.j3UcCq.rst

* Update test_math.py

* Update ACKS

* Update mathmodule.c.h

* Update mathmodule.c

* Update mathmodule.c.h

* Update mathmodule.c.h

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2020-02-19 18:21:37 +00:00
Victor Stinner 4dee92b0ad
Revert "bpo-38691 Added a switch to ignore PYTHONCASEOK when -E or -I flags passed (#18314)" (GH-18553)
This reverts commit d83b6600b2.
2020-02-19 14:23:47 +01:00
ananthan-123 ab6423fe2d
bpo-39572: Document ’total’ flag of TypedDict (GH-18554) 2020-02-18 20:33:05 -08:00
Cheryl Sabella a4ba8a3983
Include subsections in TOC for PDF version of docs. (GH-9629) 2020-02-19 00:01:15 +01:00
Victor Stinner af5ee3ff61
bpo-39674: Revert "bpo-25988: Do not expose abstract collection classes in the collections module. (GH-10596)" (GH-18545)
This reverts commit ef092fe990.

Update collections __getattr__() and documentation to defer aliases
removal to Python 3.10.
2020-02-18 16:28:53 +01:00
Petr Viktorin 6e35da9763
bpo-37207: Use vectorcall for range() (GH-18464)
This continues the `range()` part of #13930. The complete pull request is stalled on discussions around dicts, but `range()` should not be controversial. (And I plan to open PRs for other parts if this is merged.)
On top of Mark's change, I unified `range_new` and `range_vectorcall`, which had a lot of duplicate code.


https://bugs.python.org/issue37207
2020-02-18 07:13:17 -08:00
Hai Shi 5d38517aa1
bpo-1635741: Port _bz2 extension module to multiphase initialization(PEP 489) (GH-18050)
https://bugs.python.org/issue1635741
2020-02-18 03:17:38 -08:00
Kyle Meyer 8edfc47bae
bpo-39546: argparse: Honor allow_abbrev=False for specified prefix_chars (GH-18337)
When `allow_abbrev` was first added, disabling the abbreviation of
long options broke the grouping of short flags ([bpo-26967](https://bugs.python.org/issue26967)).  As a fix,
b1e4d1b603 (contained in v3.8) ignores `allow_abbrev=False` for a
given argument string if the string does _not_ start with "--"
(i.e. it doesn't look like a long option).

This fix, however, doesn't take into account that long options can
start with alternative characters specified via `prefix_chars`,
introducing a regression: `allow_abbrev=False` has no effect on long
options that start with an alternative prefix character.

The most minimal fix would be to replace the "starts with --" check
with a "starts with two prefix_chars characters".  But
`_get_option_tuples` already distinguishes between long and short
options, so let's instead piggyback off of that check by moving the
`allow_abbrev` condition into `_get_option_tuples`.





https://bugs.python.org/issue39546
2020-02-18 01:48:57 -08:00
Cheryl Sabella ffda25f6b8
bpo-39663: IDLE: Add additional tests for pyparse (GH-18536)
Test when find_good_parse_start should return 0.

Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
2020-02-17 21:47:52 -05:00
Hai Shi 4c1b6a6f4f
bpo-1635741: Port _abc extension to multiphase initialization (PEP 489) (GH-18030) 2020-02-17 14:50:35 +01:00
Hai Shi 7d7956833c
bpo-1635741: Port _contextvars module to multiphase initialization (PEP 489) (GH-18374) 2020-02-17 14:49:26 +01:00
Hai Shi b2b6e27bca
bpo-1635741: Port _crypt extension module to multiphase initialization (PEP 489) (GH-18404) 2020-02-17 10:11:34 +01:00
idomic d83b6600b2
bpo-38691 Added a switch to ignore PYTHONCASEOK when -E or -I flags passed (#18314)
* Hard reset + cherry piciking the changes.

* 📜🤖 Added by blurb_it.

* Added @vstinner News

* Update Misc/NEWS.d/next/Library/2020-02-11-13-01-38.bpo-38691.oND8Sk.rst

Co-Authored-By: Victor Stinner <vstinner@python.org>

* Hard reset to master

* Hard reset to master + latest changes

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
2020-02-17 10:05:11 +01:00
Thomas Moreau a5cbab552d
bpo-39104: Fix hanging ProcessPoolExecutor on shutdown nowait with pickling failure (GH-17670)
As reported initially by @rad-pat in #6084, the following script causes a deadlock.

```
from concurrent.futures import ProcessPoolExecutor


class ObjectWithPickleError():
    """Triggers a RuntimeError when sending job to the workers"""

    def __reduce__(self):
        raise RuntimeError()


if __name__ == "__main__":
    e = ProcessPoolExecutor()
    f = e.submit(id, ObjectWithPickleError())
    e.shutdown(wait=False)
    f.result()  # Deadlock on get
```

This is caused by the fact that the main process is closing communication channels that might be necessary to the `queue_management_thread` later. To avoid this, this PR let the `queue_management_thread` manage all the closing.



https://bugs.python.org/issue39104



Automerge-Triggered-By: @pitrou
2020-02-16 10:09:26 -08:00
Vinay Sajip 1ed61617a4
bpo-12915: Add pkgutil.resolve_name (GH-18310) 2020-02-14 22:02:13 +00:00
Dong-hee Na 9aeb0ef930
bpo-39573: Update clinic to use Py_IS_TYPE() function (GH-18507) 2020-02-14 08:50:19 +01:00
Ian Norton a9edf44a2d
closes bpo-39619 Fix os.chroot on HP-UX 11.31 (GH-18495)
Setting `-D_XOPEN_SOURCE=700` on HP-UX causes system functions such as chroot to be undefined.  This change stops `_XOPEN_SOURCE` begin set on HP-UX

Co-authored-by: Benjamin Peterson <benjamin@python.org>
2020-02-13 19:09:11 -08:00
Vlad Emelianov 10e87e5ef4
bpo-39627: Fix TypedDict totality check for inherited keys (#18503)
(Adapted from https://github.com/python/typing/pull/700)
2020-02-13 11:53:29 -08:00
Dong-hee Na d905df766c
bpo-39573: Add Py_IS_TYPE() function (GH-18488)
Co-Author: Neil Schemenauer <nas-github@arctrix.com>
2020-02-13 18:37:17 +01:00
Nathaniel J. Smith 925dc7fb1d
bpo-39606: allow closing async generators that are already closed (GH-18475)
The fix for [bpo-39386](https://bugs.python.org/issue39386) attempted to make it so you couldn't reuse a
agen.aclose() coroutine object. It accidentally also prevented you
from calling aclose() at all on an async generator that was already
closed or exhausted. This commit fixes it so we're only blocking the
actually illegal cases, while allowing the legal cases.

The new tests failed before this patch. Also confirmed that this fixes
the test failures we were seeing in Trio with Python dev builds:
  https://github.com/python-trio/trio/pull/1396


https://bugs.python.org/issue39606
2020-02-13 00:15:38 -08:00
Saiyang Gou 7514f4f625
bpo-39184: Add audit events to functions in `fcntl`, `msvcrt`, `os`, `resource`, `shutil`, `signal`, `syslog` (GH-18407) 2020-02-13 07:47:42 +00:00
Victor Stinner e9e7d284c4
bpo-35081: Move dtoa.h header to the internal C API (GH-18489)
Move the dtoa.h header file to the internal C API as pycore_dtoa.h:
it only contains private functions (prefixed by "_Py").

The math and cmath modules must now be compiled with the
Py_BUILD_CORE macro defined.
2020-02-12 22:54:42 +01:00
Victor Stinner 45876a90e2
bpo-35081: Move bytes_methods.h to the internal C API (GH-18492)
Move the bytes_methods.h header file to the internal C API as
pycore_bytes_methods.h: it only contains private symbols (prefixed by
"_Py"), except of the PyDoc_STRVAR_shared() macro.
2020-02-12 22:32:34 +01:00
Serhiy Storchaka 6e619c48b8
bpo-39474: Fix AST pos for expressions like (a)(b), (a)[b] and (a).b. (GH-18477) 2020-02-12 22:37:49 +02:00
William Chargin 674935b8ca
bpo-18819: tarfile: only set device fields for device files (GH-18080)
The GNU docs describe the `devmajor` and `devminor` fields of the tar
header struct only in the context of character and block special files,
suggesting that in other cases they are not populated. Typical utilities
behave accordingly; this patch teaches `tarfile` to do the same.
2020-02-12 11:56:02 -08:00
Victor Stinner 4fac7ed43e
bpo-21016: pydoc and trace use sysconfig (GH-18476)
bpo-21016, bpo-1294959: The pydoc and trace modules now use the
sysconfig module to get the path to the Python standard library, to
support uncommon installation path like /usr/lib64/python3.9/ on
Fedora.

Co-Authored-By: Jan Matějek <jmatejek@suse.com>
2020-02-12 13:02:29 +01:00
Serhiy Storchaka 8c579b1cc8
bpo-32856: Optimize the assignment idiom in comprehensions. (GH-16814)
Now `for y in [expr]` in comprehensions is as fast as a simple
assignment `y = expr`.
2020-02-12 12:18:59 +02:00
Serhiy Storchaka 0cc6b5e559
bpo-39219: Fix SyntaxError attributes in the tokenizer. (GH-17828)
* Always set the text attribute.
* Correct the offset attribute for non-ascii sources.
2020-02-12 12:17:00 +02:00
Serhiy Storchaka f4f445b693
bpo-39567: Add audit for os.walk(), os.fwalk(), Path.glob() and Path.rglob(). (GH-18372) 2020-02-12 12:11:34 +02:00
Jason R. Coombs e5bd73632e
bpo-39595: Improve zipfile.Path performance (#18406)
* Improve zipfile.Path performance on zipfiles with a large number of entries.

* 📜🤖 Added by blurb_it.

* Add bpo to blurb

* Sync with importlib_metadata 1.5 (6fe70ca)

* Update blurb.

* Remove compatibility code

* Add stubs module, omitted from earlier commit

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2020-02-11 21:58:47 -05:00
Victor Stinner f3e7ea5b8c
bpo-39500: Document PyUnicode_IsIdentifier() function (GH-18397)
PyUnicode_IsIdentifier() does not call Py_FatalError() anymore if the
string is not ready.
2020-02-11 14:29:33 +01:00
Hai Shi 1ea45ae257
bpo-1635741: Port _codecs extension module to multiphase initialization (PEP 489) (GH-18065)
https://bugs.python.org/issue1635741
2020-02-11 03:16:38 -08:00
Victor Stinner 038770edc4
bpo-38325: Skip non-BMP tests of test_winconsoleio (GH-18448)
Skip tests on non-BMP characters of test_winconsoleio.
2020-02-11 00:58:23 +01:00
Wellington Pardim 6c9974e12c
bpo-39369 Doc: Update mmap readline method documentation (GH-17957)
* Update mmap readline method documentation

Update mmap `readline` method description. The fact that the `readline` method does update the file position should not be ignored since this might give the impression for the programmer that it doesn't update it.

* 📜🤖 Added by blurb_it.

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2020-02-10 12:13:41 -08:00
Victor Stinner ed335cf53b
bpo-39600, IDLE: Remove duplicated font names (GH-18430)
In the font configuration window, remove duplicated font names.
2020-02-10 11:41:26 -08:00
Hugo van Kemenade 29b3fc0a18
bpo-39586: Deprecate distutils bdist_msi command (GH-18415) 2020-02-10 14:26:40 +01:00
sweeneyde c6dedde160
bpo-39590: make deque.__contains__ and deque.count hold strong references (GH-18421) 2020-02-09 00:16:43 -08:00
Lysandros Nikolaou d2e1098641
bpo-39579: Fix Attribute end_col_offset to point at the current node (GH-18405) 2020-02-07 15:36:32 -08:00
Victor Stinner dc7a50d73a
bpo-39350: Fix fractions for int subclasses (GH-18375)
Fix regression in fractions.Fraction if the numerator and/or the
denominator is an int subclass. The math.gcd() function is now
used to normalize the numerator and denominator. math.gcd() always
return a int type. Previously, the GCD type depended on numerator
and denominator.
2020-02-07 23:42:51 +01:00
Michael Felt de6f38db48
bpo-39502: Fix 64-bit Python PyTime_localtime() on AIX (GH-18285)
Fix time.localtime() on 64-bit AIX to support years before 1902 and after 2038.
2020-02-07 18:56:16 +01:00
Victor Stinner b10dc3e7a1
bpo-39573: Add Py_SET_SIZE() function (GH-18400)
Add Py_SET_SIZE() function to set the size of an object.
2020-02-07 12:05:12 +01:00
Victor Stinner d2ec81a8c9
bpo-39573: Add Py_SET_TYPE() function (GH-18394)
Add Py_SET_TYPE() function to set the type of an object.
2020-02-07 09:17:07 +01:00
Victor Stinner c86a11221d
bpo-39573: Add Py_SET_REFCNT() function (GH-18389)
Add a Py_SET_REFCNT() function to set the reference counter of an
object.
2020-02-07 01:24:29 +01:00
Sebastian Berg 427c84f13f
bpo-39274: Ensure Fraction.__bool__() returns a bool (GH-18017)
Some numerator types used (specifically NumPy) decides to not
return a Python boolean for the "a != b" operation. Using the equivalent
call to bool() guarantees a bool return also for such types.
2020-02-06 15:54:05 +01:00
Petr Viktorin 3f563cea56
bpo-39245: Make Vectorcall C API public (GH-17893)
* Add backcompat defines and move non-limited API declaration to cpython/

This partially reverts commit 2ff58a24e8
which added PyObject_CallNoArgs to the 3.9+ stable ABI. This should not
be done; there are enough other call APIs in the stable ABI to choose from.

* Adjust documentation

Mark all newly public functions as added in 3.9.
Add a note about the 3.8 provisional names.
Add notes on public API.

* Put PyObject_CallNoArgs back in the limited API

* Rename PyObject_FastCallDict to PyObject_VectorcallDict
2020-02-06 15:48:27 +01:00
Serhiy Storchaka 54b4f14712
bpo-38149: Call sys.audit() only once per call for glob.glob(). (GH-18360) 2020-02-06 10:26:37 +02:00
Shantanu 8b6f6526f8
bpo-39559: Remove unused, undocumented argument from uuid.getnode (GH-18369) 2020-02-05 22:43:09 +02:00
Victor Stinner 509dd90f46
bpo-39542: Convert PyType_Check() to static inline function (GH-18364)
Convert PyType_HasFeature(), PyType_Check() and PyType_CheckExact()
macros to static inline functions.
2020-02-05 14:24:17 +01:00
Victor Stinner f58bd7c169
bpo-39542: Make PyObject_INIT() opaque in limited C API (GH-18363)
In the limited C API, PyObject_INIT() and PyObject_INIT_VAR() are now
defined as aliases to PyObject_Init() and PyObject_InitVar() to make
their implementation opaque. It avoids to leak implementation details
in the limited C API.

Exclude the following functions from the limited C API, move them
from object.h to cpython/object.h:

* _Py_NewReference()
* _Py_ForgetReference()
* _PyTraceMalloc_NewReference()
* _Py_GetRefTotal()
2020-02-05 13:12:19 +01:00
Victor Stinner 0fa4f43db0
bpo-39542: Exclude trashcan from the limited C API (GH-18362)
Exclude trashcan mechanism from the limited C API: it requires access to
PyTypeObject and PyThreadState structure fields, whereas these structures
are opaque in the limited C API.

The trashcan mechanism never worked with the limited C API. Move it
from object.h to cpython/object.h.
2020-02-05 12:23:27 +01:00
Zackery Spytz b439a715cb
bpo-39553: Delete HAVE_SXS protected code (GH-18356)
https://bugs.python.org/issue39553



Automerge-Triggered-By: @zooba
2020-02-04 19:13:00 -08:00
Jakub Stasiak cf5b109dbb
bpo-39491: Merge PEP 593 (typing.Annotated) support (#18260)
* bpo-39491: Merge PEP 593 (typing.Annotated) support

PEP 593 has been accepted some time ago. I got a green light for merging
this from Till, so I went ahead and combined the code contributed to
typing_extensions[1] and the documentation from the PEP 593 text[2].

My changes were limited to:

* removing code designed for typing_extensions to run on older Python
  versions
* removing some irrelevant parts of the PEP text when copying it over as
  documentation and otherwise changing few small bits to better serve
  the purpose
* changing the get_type_hints signature to match reality (parameter
  names)

I wasn't entirely sure how to go about crediting the authors but I used
my best judgment, let me know if something needs changing in this
regard.

[1] 8280de241f/typing_extensions/src_py3/typing_extensions.py
[2] 17710b8798/pep-0593.rst
2020-02-04 17:10:19 -08:00
Anthony Shaw 89ae20b30e
bpo-39185 Add the d[etailed] and q[uiet] verbosity levels for msbuild (GH-17791) 2020-02-05 11:30:19 +11:00
Saiyang Gou 95f6001021
bpo-39184: Add audit events to command execution functions in os and pty modules (GH-17824) 2020-02-05 11:15:00 +11:00
Philipp Gesang cb1c0746f2
closes bpo-39510: Fix use-after-free in BufferedReader.readinto() (GH-18295)
When called on a closed object, readinto() segfaults on account
of a write to a freed buffer:

    ==220553== Process terminating with default action of signal 11 (SIGSEGV): dumping core
    ==220553==  Access not within mapped region at address 0x2A
    ==220553==    at 0x48408A0: memmove (vg_replace_strmem.c:1272)
    ==220553==    by 0x58DB0C: _buffered_readinto_generic (bufferedio.c:972)
    ==220553==    by 0x58DCBA: _io__Buffered_readinto_impl (bufferedio.c:1053)
    ==220553==    by 0x58DCBA: _io__Buffered_readinto (bufferedio.c.h:253)

Reproducer:

    reader = open ("/dev/zero", "rb")
    _void  = reader.read (42)
    reader.close ()
    reader.readinto (bytearray (42)) ### BANG!

The problem exists since 2012 when commit dc469454ec added code
to free the read buffer on close().

Signed-off-by: Philipp Gesang <philipp.gesang@intra2net.com>
2020-02-04 13:25:16 -08:00
Stefan Behnel 9538bc9185
bpo-39432: Implement PEP-489 algorithm for non-ascii "PyInit_*" symbol names in distutils (GH-18150)
Make it export the correct init symbol also on Windows.



https://bugs.python.org/issue39432
2020-02-04 07:24:30 -08:00
Eddie Elizondo 4590f72259
bpo-38076 Clear the interpreter state only after clearing module globals (GH-18039)
Currently, during runtime destruction, `_PyImport_Cleanup` is clearing the interpreter state before clearing out the modules themselves. This leads to a segfault on modules that rely on the module state to clear themselves up.

For example, let's take the small snippet added in the issue by @DinoV :
```
import _struct

class C:
    def __init__(self):
        self.pack = _struct.pack
    def __del__(self):
        self.pack('I', -42)

_struct.x = C()
```

The module `_struct` uses the module state to run `pack`. Therefore, the module state has to be alive until after the module has been cleared out to successfully run `C.__del__`. This happens at line 606, when `_PyImport_Cleanup` calls `_PyModule_Clear`. In fact, the loop that calls `_PyModule_Clear` has in its comments: 

> Now, if there are any modules left alive, clear their globals to minimize potential leaks.  All C extension modules actually end up here, since they are kept alive in the interpreter state.

That means that we can't clear the module state (which is used by C Extensions) before we run that loop.

Moving `_PyInterpreterState_ClearModules` until after it, fixes the segfault in the code snippet.

Finally, this updates a test in `io` to correctly assert the error that it now throws (since it now finds the io module state). The test that uses this is: `test_create_at_shutdown_without_encoding`. Given this test is now working is a proof that the module state now stays alive even when `__del__` is called at module destruction time. Thus, I didn't add a new tests for this.


https://bugs.python.org/issue38076
2020-02-04 02:29:25 -08:00
Victor Stinner c6e5c1123b
bpo-39489: Remove COUNT_ALLOCS special build (GH-18259)
Remove:

* COUNT_ALLOCS macro
* sys.getcounts() function
* SHOW_ALLOC_COUNT code in listobject.c
* SHOW_TRACK_COUNT code in tupleobject.c
* PyConfig.show_alloc_count field
* -X showalloccount command line option
* @test.support.requires_type_collecting decorator
2020-02-03 15:17:15 +01:00
Steve Cirelli 032de7324e
bpo-39450 Stripped whitespace before parsing the docstring in TestCase.shortDescription (GH-18175) 2020-02-03 07:06:50 +00:00
Pierre Glaser 0f2f35e15f
bpo-39492: Fix a reference cycle between reducer_override and a Pickler instance (GH-18266)
This also needs a backport to 3.8


https://bugs.python.org/issue39492



Automerge-Triggered-By: @pitrou
2020-02-02 10:55:21 -08:00
Kyle Stanley 339fd46cb7
bpo-39349: Add *cancel_futures* to Executor.shutdown() (GH-18057) 2020-02-02 13:49:00 +01:00
Andrew Svetlov 90d9ba6ef1
bpo-34793: Drop old-style context managers in asyncio.locks (GH-17533) 2020-02-01 13:12:52 +02:00
Victor Stinner 4d96b4635a
bpo-39511: PyThreadState_Clear() calls on_delete (GH-18296)
PyThreadState.on_delete is a callback used to notify Python when a
thread completes. _thread._set_sentinel() function creates a lock
which is released when the thread completes. It sets on_delete
callback to the internal release_sentinel() function. This lock is
known as Threading._tstate_lock in the threading module.

The release_sentinel() function uses the Python C API. The problem is
that on_delete is called late in the Python finalization, when the C
API is no longer fully working.

The PyThreadState_Clear() function now calls the
PyThreadState.on_delete callback. Previously, that happened in
PyThreadState_Delete().

The release_sentinel() function is now called when the C API is still
fully working.
2020-02-01 02:30:25 +01:00
Zackery Spytz bfdeaa37b3
bpo-38792: Remove IDLE shell calltip before new prompt. (#17150)
Previously, a calltip might be left after SyntaxError, KeyboardInterrupt, or Shell Restart.

Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
Co-authored-by: Tal Einat <taleinat+github@gmail.com>
2020-01-30 20:55:42 -05:00
Victor Stinner c232c9110c
bpo-39502: Skip test_zipfile.test_add_file_after_2107() on AIX (GH-18282)
Skip test_zipfile.test_add_file_after_2107() if time.localtime()
fails with OverflowError. It is the case on AIX 6.1 for example.
2020-01-30 15:47:53 +01:00
Dong-hee Na 8d49f7ceb4
bpo-39434: Improve float __floordiv__ performance and error message (GH-18147) 2020-01-30 13:23:15 +00:00
Victor Stinner c38fd0df2b
bpo-39353: binascii.crc_hqx() is no longer deprecated (GH-18276)
The binascii.crc_hqx() function is no longer deprecated.
2020-01-30 09:56:40 +01:00
Shantanu 2e6569b669
bpo-39493: Fix definition of IO.closed in typing.py (#18265) 2020-01-29 18:52:36 -08:00
Carl Friedrich Bolz-Tereick a327677905
bpo-39485: fix corner-case in method-detection of mock (GH-18252)
Replace check for whether something is a method in the mock module. The
previous version fails on PyPy, because there no method wrappers exist
(everything looks like a regular Python-defined function). Thus the
isinstance(getattr(result, '__get__', None), MethodWrapperTypes) check
returns True for any descriptor, not just methods.

This condition could also return erroneously True in CPython for
C-defined descriptors.

Instead to decide whether something is a method, just check directly
whether it's a function defined on the class. This passes all tests on
CPython and fixes the bug on PyPy.
2020-01-29 15:43:37 +00:00
Joannah Nanjekye e1e80002e2
bpo-39153: Clarify C API *SetItem refcounting semantics (GH-18220)
Some of the *SetItem methods in the C API steal a reference to the
given value. This annotates the better behaved ones to assure the
reader that these are not the ones with the inconsistent behaviour.

* 📜🤖 Added by blurb_it.

* make docs consistent with signature

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2020-01-29 21:20:53 +10:00
Bruce Merry d07d9f4c43
bpo-36051: Drop GIL during large bytes.join() (GH-17757)
Improve multi-threaded performance by dropping the GIL in the fast path
of bytes.join. To avoid increasing overhead for small joins, it is only
done if the output size exceeds a threshold.
2020-01-29 16:09:24 +09:00
Steve Dower 6a65eba44b
bpo-39401: Avoid unsafe DLL load on Windows 7 and earlier (GH-18231)
As Windows 7 is not supported by Python 3.9, we just replace the dynamic load with a static import. Backports will have a different fix to ensure they continue to behave the same.
2020-01-29 13:46:33 +11:00
David Carlier aabdeb766b bpo-38960: DTrace build fix for FreeBSD. (GH-17451)
DTrace build fix for FreeBSD.

- allowing passing an extra flag as it need to define the arch size.
- casting some probe's arguments.
2020-01-28 13:53:32 +01:00
Rémi Lapeyre 2cca8efe46 bpo-36350: inspect: Replace OrderedDict with dict. (GH-12412) 2020-01-28 21:47:03 +09:00
Adam Meily 0be3246d4f bpo-39439: Fix multiprocessing spawn path in a venv on Windows (GH-18158) 2020-01-28 21:34:23 +11:00
Zackery Spytz 13c1c3556f bpo-39393: Misleading error message on dependent DLL resolution failure (GH-18093) 2020-01-28 20:42:43 +11:00
Christoph Reiter c45a2aa9e2 bpo-38883: Don't use POSIX `$HOME` in `pathlib.Path.home/expanduser` on Windows (GH-17961)
In bpo-36264 os.path.expanduser was changed to ignore HOME on Windows.

Path.expanduser/home still honored HOME despite being documented as behaving the same
as os.path.expanduser. This makes them also ignore HOME so that both implementations
behave the same way again.
2020-01-28 20:41:50 +11:00
Terry Jan Reedy 2824c45a0a
bpo-39392: Turtle overlap fill depends on OS (#18223)
Whether or not overlap regions for self-intersecting polygons
or multiple shapes are filled depends on the operating system graphics,
typeof overlap, and number of overlaps.
2020-01-27 18:41:18 -05:00
Cheryl Sabella dd023ad161 bpo-30780: Add IDLE configdialog tests (#3592)
Expose dialog buttons to test code and complete their test coverage.
Complete test coverage for highlights and keys tabs.

Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
2020-01-27 17:15:56 -05:00
Dong-hee Na 4dbf2d8c67 bpo-39453: Make list.__contains__ hold strong references to avoid crashes (GH-18181) 2020-01-27 15:02:23 +00:00
Mark Shannon 8a4cd700a7
bpo-39320: Handle unpacking of **values in compiler (GH-18141)
* Add DICT_UPDATE and DICT_MERGE bytecodes. Use them for ** unpacking.

* Remove BUILD_MAP_UNPACK and BUILD_MAP_UNPACK_WITH_CALL, as they are now unused.

* Update magic number for ** unpacking opcodes.

* Update dis.rst to incorporate new bytecodes.

* Add blurb entry.
2020-01-27 09:57:45 +00:00
Karthikeyan Singaravelan 72b1004657 bpo-25597: Ensure wraps' return value is used for magic methods in MagicMock (#16029) 2020-01-27 06:48:15 +00:00
加和 4515a590a4 Fix linecache.py add lazycache to __all__ and use dict.clear to clear the cache (GH-4641) 2020-01-25 21:07:40 -05:00
Vegard Stikbakke aef7dc8987 bpo-38932: Mock fully resets child objects on reset_mock(). (GH-17409) 2020-01-25 15:44:46 +00:00
Łukasz Langa b07ead3411 Python 3.9.0a3
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE4/8oOcBIslwITevpsmmV4xAlBWgFAl4rXBoACgkQsmmV4xAl
 BWid+xAAmRi6rMnpTrJG6V4herWLg4tQg4LaghxXPHtAfcZCgYkbA3kI4nq7SoKy
 7XnJRX6Uk3FhT2rzM2+sg+sPDMNz5ki+G6L0rYrmouZSy+y7/KDBTzh7u8nnAZD1
 t72Zl3LyWFgeg58MVld5LWoCt7+ywAfIAEbeYAFonH4yY1MqK8Q5+kbNtuMBar81
 S44i/UIjVD1GJTzxxDqWU4B0BmCGizenBiyssAwTbM556dzjR7tDqWhDfiXcMLgW
 7Wq5sj96GjNOrNTbtPikiu3sl97hHlJgZEel/vBq2RNArEspRa0F3uqNYyaRedHr
 F9XDlsXLdj+18w1tG1HejZiGquW9y55vexEtDp2e2FXl9bHm5n7D8PUX8rCuw6li
 lR2W+oB0i8/Jd1R8GymJ4RfHrh7jHoTCskcLp8BrBlFBU0d3DAoYFT9rA5GSj3dx
 MIQgA2KbDJEYU7i0/uLLqK6L37PD8NaAPhdVjUN1B2qJ47RrC3yc5tPHIuyiBMYu
 M2PW8LNhg/+ZIQjznSam3xiLDzEv/TeHA0YgfdIaXgdtybFSGIUWMxCGQjNtzrHS
 u4WjsQy9yFAtF+mgPKqXBI+coBtLUhvRfOGHwhnbjkao7i65ztZC2VX/e2CfquO/
 DDJZoiGS0wGbDJfeApNaT7/Kac2uGgvYfx1foGReK3fqRqLl8aU=
 =xsba
 -----END PGP SIGNATURE-----
gpgsig -----BEGIN PGP SIGNATURE-----
 
 iQJEBAABCAAuFiEE4/8oOcBIslwITevpsmmV4xAlBWgFAl4sSAgQHGx1a2FzekBs
 YW5nYS5wbAAKCRCyaZXjECUFaGJkD/9ktOT+LKyW8+9nKRkVNHy4+PRGdKA/7O7y
 q8qieymxaJoq2sFKQmgBvDTEh/S6Z75BlsHiTucpDnFe+WVwLJ79g0w9ehbzWx8g
 XjNn1ONvsd/sDgUxZ3YiYy6uMbrCMPmcb0VPdlVW1l55uCDccNkh94Sa0lDEYnJo
 +Xr6kDd+dyrF6XswyfDSSYriUNHODfT4aezFbS/EAaD3uX+hqtEW+kl1C3unbN5H
 xxqe986NYGX5Yd7sIr9tqVBz4m0gQLDXX1i10pZFi+4pqog/ZV6XQbW+RSjr3SuU
 FktleKz5erwY2fEz9KHRjonpPDPwEnWOnnn+MNPDECuIVyHswkAidUS69DkzfUpf
 us1RrUoOmzi3cBxhrPCLVMnFFKpdxigiKWVa1CVNEhr7nhLLT3JPdfjXY/XEezW3
 i9xJkCaxJUw7xrt808O1MHhkKAC9VFk/s2YDpmhCcmwfF8IR2K4L0XpQRG4+dfwz
 WFPA8xQC2W+BHsHQoXW4XFZJQaCDmXZX2c0SOjl2fwZUyi2h3vvqJPfDxtjIRltN
 TqAZJuaZgJNhtvK9R2BIY6Xr72owczdM8z35Je44K6oiUgFYs4UN9U+6D4sdRkSc
 YBDgRmnGHTb07VWTWxyuzkXmUDiQ458KpdA9xDE47463v+NqkD9scR3EgX/e6OTR
 0+EgUU6dYg==
 =MyW2
 -----END PGP SIGNATURE-----

Merge tag 'v3.9.0a3'

Python 3.9.0a3
2020-01-25 14:52:06 +01:00
Matthew Kokotovich 62865f4532 bpo-39082: Allow AsyncMock to correctly patch static/class methods (GH-18116) 2020-01-25 10:17:47 +00:00
Cheryl Sabella d0d9fa8c5e bpo-39388: IDLE: Fix bug when cancelling out of configdialog (GH-18068)
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
2020-01-25 04:00:54 -05:00
Łukasz Langa c33378df39
Python 3.9.0a3 2020-01-24 22:05:07 +01:00
Serhiy Storchaka 9017e0bd5e bpo-39430: Fix race condition in lazy imports in tarfile. (GH-18161)
Use `from ... import ...` to ensure module is fully loaded before accessing its attributes.
2020-01-24 09:55:52 -08:00
mbarkhau 88704334e5 bpo-39390 shutil: fix argument types for ignore callback (GH-18122) 2020-01-24 15:51:16 +01:00
Karthikeyan Singaravelan 66b00a9d3a bpo-38473: Handle autospecced functions and methods used with attach_mock (GH-16784) 2020-01-24 13:14:29 +00:00
Victor Stinner b8d1262e8a
bpo-39395: putenv() and unsetenv() always available (GH-18135)
The os.putenv() and os.unsetenv() functions are now always available.

On non-Windows platforms, Python now requires setenv() and unsetenv()
functions to build.

Remove putenv_dict from posixmodule.c: it's not longer needed.
2020-01-24 14:05:48 +01:00
Victor Stinner 161e7b36b1
bpo-39413: Implement os.unsetenv() on Windows (GH-18163)
The os.unsetenv() function is now also available on Windows.
2020-01-24 11:53:44 +01:00
Mario Corchero e131c9720d Fix `mock.patch.dict` to be stopped with `mock.patch.stopall` (#17606)
As the function was not registering in the active patches, the mocks
started by `mock.patch.dict` were not being stopped when
`mock.patch.stopall` was being called.
2020-01-24 08:38:32 +00:00
Pablo Galindo 99e6c260d6
bpo-17005: Add a class to perform topological sorting to the standard library (GH-11583)
Co-Authored-By: Tim Peters <tim.peters@gmail.com>
2020-01-23 15:29:52 +00:00
Pablo Galindo 79f89e6e5a
bpo-39421: Fix posible crash in heapq with custom comparison operators (GH-18118)
* bpo-39421: Fix posible crash in heapq with custom comparison operators

* fixup! bpo-39421: Fix posible crash in heapq with custom comparison operators

* fixup! fixup! bpo-39421: Fix posible crash in heapq with custom comparison operators
2020-01-23 14:07:05 +00:00
Mark Shannon 13bc13960c
bpo-39320: Handle unpacking of *values in compiler (GH-17984)
* Add three new bytecodes: LIST_TO_TUPLE, LIST_EXTEND, SET_UPDATE. Use them to implement star unpacking expressions.

* Remove four bytecodes BUILD_LIST_UNPACK, BUILD_TUPLE_UNPACK, BUILD_SET_UNPACK and  BUILD_TUPLE_UNPACK_WITH_CALL opcodes as they are now unused.

* Update magic number and dis.rst for new bytecodes.
2020-01-23 09:25:17 +00:00
Zackery Spytz 2e43b64c94 bpo-39050: The Help button in IDLE's config menu works again (GH-17611)
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
2020-01-22 22:54:30 -05:00
William Woodruff dd754caf14 bpo-29435: Allow is_tarfile to take a filelike obj (GH-18090)
`is_tarfile()` now supports `name` being a file or file-like object.
2020-01-22 18:24:16 -08:00
Pablo Galindo 41f0ef6abb bpo-39427: Document -X opt options in the CLI --help and the man page (GH-18131)
https://bugs.python.org/issue39427



Automerge-Triggered-By: @pablogsal
2020-01-22 17:03:04 -08:00
Dino Viehland 9b6fec4651
bpo-39336: Allow packages to not let their child modules be set on them (#18006)
* bpo-39336: Allow setattr to fail on modules which aren't assignable

When attaching a child module to a package if the object in sys.modules raises an AttributeError (e.g. because it is immutable) it causes the whole import to fail.  This now allows immutable packages to exist and an ImportWarning is reported and the AttributeError exception is ignored.
2020-01-22 16:42:38 -08:00
Alex Rebert d3ae95e1e9 bpo-35182: fix communicate() crash after child closes its pipes (GH-17020) (GH-18117)
When communicate() is called in a loop, it crashes when the child process
has already closed any piped standard stream, but still continues to be running

Co-authored-by: Andriy Maletsky <andriy.maletsky@gmail.com>
2020-01-22 15:28:31 -08:00
Dong-hee Na 1f0f102dec bpo-39366: Remove xpath() and xgtitle() methods of NNTP (GH-18035) 2020-01-23 00:59:43 +03:00
Victor Stinner b477d19a6b
bpo-39406: Implement os.putenv() with setenv() if available (GH-18128)
If setenv() C function is available, os.putenv() is now implemented
with setenv() instead of putenv(), so Python doesn't have to handle
the environment variable memory.
2020-01-22 22:48:16 +01:00
Victor Stinner b73dd02ea7
Revert "bpo-39413: Implement os.unsetenv() on Windows (GH-18104)" (GH-18124)
This reverts commit 56cd3710a1.
2020-01-22 21:11:17 +01:00
Victor Stinner beea26b57e
bpo-39353: Deprecate the binhex module (GH-18025)
Deprecate binhex4 and hexbin4 standards. Deprecate the binhex module
and the following binascii functions:

* b2a_hqx(), a2b_hqx()
* rlecode_hqx(), rledecode_hqx()
* crc_hqx()
2020-01-22 20:44:22 +01:00
Victor Stinner 56cd3710a1
bpo-39413: Implement os.unsetenv() on Windows (GH-18104)
The os.unsetenv() function is now also available on Windows.

It is implemented with SetEnvironmentVariableW(name, NULL).
2020-01-21 16:13:09 +01:00
William Chargin eab3b3f1c6 bpo-39389: gzip: fix compression level metadata (GH-18077)
As described in RFC 1952, section 2.3.1, the XFL (eXtra FLags) byte of a
gzip member header should indicate whether the DEFLATE algorithm was
tuned for speed or compression ratio. Prior to this patch, archives
emitted by the `gzip` module always indicated maximum compression.
2020-01-21 13:25:24 +02:00
Victor Stinner 85ead4fc62
bpo-39396: Fix math.nextafter(-0.0, +0.0) on AIX 7.1 (GH-18094)
Move also math.nextafter() on math.ulp() tests from IsCloseTests to
MathTests.
2020-01-21 11:14:10 +01:00
Cheryl Sabella ec64640a2c bpo-32989: IDLE - fix bad editor call of pyparse method (GH-5968)
Fix comments and add tests for editor newline_and_indent_event method.
Remove unused None default for function parameter of pyparse find_good_parse_start method
and code triggered by that default.

Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
2020-01-21 05:11:26 -05:00
Andrew Svetlov a96e06db77
bpo-39386: Prevent double awaiting of async iterator (GH-18081) 2020-01-21 00:49:30 +02:00
Andrew Svetlov 2c49becc69
Fix asyncio.get_event_loop() documentation (GH-18051)
Mention that the function implicitly creates new event loop only if called from the main thread.
2020-01-21 00:46:38 +02:00
Inada Naoki 5492bfcefe
bpo-39377: json: Remove the encoding option. (GH-18075) 2020-01-20 13:54:00 +09:00
Inada Naoki e96d954527
bpo-38536: locale: Remove trailing space in formatted currency (GH-16864) 2020-01-20 12:45:50 +09:00
Zackery Spytz d8ef64422a bpo-35561: Supress valgrind false alarm on epoll_ctl(event) (GH-18060)
Update Misc/valgrind-python.supp to suppress the false alarm.
2020-01-19 23:38:37 +01:00
Pablo Galindo cd7db76a63
bpo-39372: Clean header files of declared interfaces with no implementations (GH-18037)
The public API symbols being removed are:

_PyBytes_InsertThousandsGroupingLocale, _PyBytes_InsertThousandsGrouping, _Py_InitializeFromArgs, _Py_InitializeFromWideArgs, _PyFloat_Repr, _PyFloat_Digits,
_PyFloat_DigitsInit, PyFrame_ExtendStack, _PyAIterWrapper_Type, PyNullImporter_Type, PyCmpWrapper_Type, PySortWrapper_Type, PyNoArgsFunction.
2020-01-18 03:14:59 +00:00
Victor Stinner 9baf242fc7
bpo-39357: Remove buffering parameter of bz2.BZ2File (GH-18028)
Remove the buffering parameter of bz2.BZ2File. Since Python 3.0, it
was ignored and using it was emitting a DeprecationWarning. Pass an
open file object to control how the file is opened.

The compresslevel parameter becomes keyword-only.
2020-01-16 15:33:30 +01:00
Victor Stinner 4691a2f2a2
bpo-39350: Remove deprecated fractions.gcd() (GH-18021)
Remove fractions.gcd() function, deprecated since Python 3.5
(bpo-22486): use math.gcd() instead.
2020-01-16 11:02:51 +01:00
Victor Stinner 210c19e3c5
bpo-39351: Remove base64.encodestring() (GH-18022)
Remove base64.encodestring() and base64.decodestring(), aliases
deprecated since Python 3.1: use base64.encodebytes() and
base64.decodebytes() instead.
2020-01-16 10:24:16 +01:00
Daniel Olshansky 01602ae403 bpo-37958: Adding get_profile_dict to pstats (GH-15495)
pstats is really useful or profiling and printing the output of the execution of some block of code, but I've found on multiple occasions when I'd like to access this output directly in an easily usable dictionary on which I can further analyze or manipulate.

The proposal is to add a function called get_profile_dict inside of pstats that'll automatically return this data the data in an easily accessible dict.

The output of the following script:

```
import cProfile, pstats
import pprint
from pstats import func_std_string, f8

def fib(n):
    if n == 0:
        return 0
    if n == 1:
        return 1
    return fib(n-1) + fib(n-2)

pr = cProfile.Profile()
pr.enable()
fib(5)
pr.create_stats()

ps = pstats.Stats(pr).sort_stats('tottime', 'cumtime')

def get_profile_dict(self, keys_filter=None):
    """
        Returns a dict where the key is a function name and the value is a dict
        with the following keys:
            - ncalls
            - tottime
            - percall_tottime
            - cumtime
            - percall_cumtime
            - file_name
            - line_number

        keys_filter can be optionally set to limit the key-value pairs in the
        retrieved dict.
    """
    pstats_dict = {}
    func_list = self.fcn_list[:] if self.fcn_list else list(self.stats.keys())

    if not func_list:
        return pstats_dict

    pstats_dict["total_tt"] = float(f8(self.total_tt))
    for func in func_list:
        cc, nc, tt, ct, callers = self.stats[func]
        file, line, func_name = func
        ncalls = str(nc) if nc == cc else (str(nc) + '/' + str(cc))
        tottime = float(f8(tt))
        percall_tottime = -1 if nc == 0 else float(f8(tt/nc))
        cumtime = float(f8(ct))
        percall_cumtime = -1 if cc == 0 else float(f8(ct/cc))
        func_dict = {
            "ncalls": ncalls,
            "tottime": tottime, # time spent in this function alone
            "percall_tottime": percall_tottime,
            "cumtime": cumtime, # time spent in the function plus all functions that this function called,
            "percall_cumtime": percall_cumtime,
            "file_name": file,
            "line_number": line
        }
        func_dict_filtered = func_dict if not keys_filter else { key: func_dict[key] for key in keys_filter }
        pstats_dict[func_name] = func_dict_filtered

    return pstats_dict

pp = pprint.PrettyPrinter(depth=6)
pp.pprint(get_profile_dict(ps))
```

will produce:

```
{"<method 'disable' of '_lsprof.Profiler' objects>": {'cumtime': 0.0,
                                                      'file_name': '~',
                                                      'line_number': 0,
                                                      'ncalls': '1',
                                                      'percall_cumtime': 0.0,
                                                      'percall_tottime': 0.0,
                                                      'tottime': 0.0},
 'create_stats': {'cumtime': 0.0,
                  'file_name': '/usr/local/Cellar/python/3.7.4/Frameworks/Python.framework/Versions/3.7/lib/python3.7/cProfile.py',
                  'line_number': 50,
                  'ncalls': '1',
                  'percall_cumtime': 0.0,
                  'percall_tottime': 0.0,
                  'tottime': 0.0},
 'fib': {'cumtime': 0.0,
         'file_name': 'get_profile_dict.py',
         'line_number': 5,
         'ncalls': '15/1',
         'percall_cumtime': 0.0,
         'percall_tottime': 0.0,
         'tottime': 0.0},
 'total_tt': 0.0}
 ```

 As an example, this can be used to generate a stacked column chart using various visualization tools which will assist in easily identifying program bottlenecks.



https://bugs.python.org/issue37958



Automerge-Triggered-By: @gpshead
2020-01-15 14:51:54 -08:00
Victor Stinner e85a305503
bpo-38630: Fix subprocess.Popen.send_signal() race condition (GH-16984)
On Unix, subprocess.Popen.send_signal() now polls the process status.
Polling reduces the risk of sending a signal to the wrong process if
the process completed, the Popen.returncode attribute is still None,
and the pid has been reassigned (recycled) to a new different
process.
2020-01-15 17:38:55 +01:00
Hai Shi ed154c387e bpo-1635741: Port _json extension module to multiphase initialization (PEP 489) (GH-17835) 2020-01-15 17:32:51 +01:00
Dong-hee Na 65a5ce247f bpo-39329: Add timeout parameter for smtplib.LMTP constructor (GH-17998) 2020-01-14 22:42:09 +01:00
Vinay Sajip 7d6378051f
bpo-38901: Allow setting a venv's prompt to the basename of the current directory. (GH-17946)
When a prompt value of '.' is specified, os.path.basename(os.getcwd()) is used to
configure the prompt for the created venv.
2020-01-14 20:49:30 +00:00
Václav Bartoš f04750bb7a bpo-38361: syslog: fixed making default "ident" from sys.argv[0] (GH-16557)
The default value of "ident" parameter should be sys.argv[0] with leading path
components stripped, but it contained the last slash, i.e. '/program' instead
of 'program'.

BPO issue: https://bugs.python.org/issue38361


https://bugs.python.org/issue38361
2020-01-14 09:57:04 -08:00
Pablo Galindo a2ec3f07f7
bpo-39322: Add gc.is_finalized to check if an object has been finalised by the gc (GH-17989) 2020-01-14 12:06:45 +00:00
Géry Ogam 1d1b97ae64 bpo-39048: Look up __aenter__ before __aexit__ in async with (GH-17609)
* Reorder the __aenter__ and __aexit__ checks for async with
* Add assertions for async with body being skipped
* Swap __aexit__ and __aenter__ loading in the documentation
2020-01-14 21:58:29 +10:00
Mark Shannon 9af0e47b17
bpo-39156: Break up COMPARE_OP into four logically distinct opcodes. (GH-17754)
Break up COMPARE_OP into four logically distinct opcodes:
* COMPARE_OP for rich comparisons
* IS_OP for 'is' and 'is not' tests
* CONTAINS_OP for 'in' and 'is not' tests
* JUMP_IF_NOT_EXC_MATCH for checking exceptions in 'try-except' statements.
2020-01-14 10:12:45 +00:00
Dong-hee Na 62e3973395 bpo-39259: smtp.SMTP/SMTP_SSL now reject timeout = 0 (GH-17958) 2020-01-14 08:49:59 +01:00
Anthony Shaw 2de064e630 bpo-39160 Align the verbs, grammar and defaults for `./configure --help` (GH-17747) 2020-01-14 01:40:10 -05:00
Dong-hee Na a190e2ade1 bpo-39259: ftplib.FTP/FTP_TLS now reject timeout = 0 (GH-17959) 2020-01-13 20:34:34 +01:00
Julien Danjou 3430c55417 bpo-39164: Add private _PyErr_GetExcInfo() function (GH-17752)
This adds a new function named _PyErr_GetExcInfo() that is a variation of the
original PyErr_GetExcInfo() taking a PyThreadState as its first argument.
That function allows to retrieve the exceptions information of any Python
thread -- not only the current one.
2020-01-13 17:30:14 +01:00
Victor Stinner 0b2ab21956
bpo-39310: Add math.ulp(x) (GH-17965)
Add math.ulp(): return the value of the least significant bit
of a float.
2020-01-13 12:44:35 +01:00
Philip McMahon b2b4a51f74 bpo-32021: Support brotli .br encoding in mimetypes (#12200)
Add support for brotli encoding in the encoding_map.
2020-01-12 14:31:49 -08:00
Batuhan Taşkaya 61b14151cc bpo-39313: Add an option to RefactoringTool for using exec as a function (GH-17967)
https://bugs.python.org/issue39313


Automerge-Triggered-By: @pablogsal
2020-01-12 14:13:31 -08:00
Batuhan Taşkaya 6680f4a9f5 bpo-3530: Add advice on when to correctly use fix_missing_locations in the AST docs (GH-17172)
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-01-12 20:38:53 +00:00
Guðni Natan Gunnarsson 9f3fc6c5b4 bpo-38293: Allow shallow and deep copying of property objects (GH-16438)
Copying property objects results in a TypeError. Steps to reproduce:

```
>>> import copy
>>> obj = property()
>>> copy.copy(obj)
````

This affects both shallow and deep copying.  
My idea for a fix is to add property objects to the list of "atomic" objects in the copy module.
These already include types like functions and type objects.

I also added property objects to the unit tests test_copy_atomic and test_deepcopy_atomic. This is my first PR, and it's highly likely I've made some mistake, so please be kind :)


https://bugs.python.org/issue38293
2020-01-12 09:41:49 -08:00
Victor Stinner 100fafcf20
bpo-39288: Add math.nextafter(x, y) (GH-17937)
Return the next floating-point value after x towards y.
2020-01-12 02:15:42 +01:00
Dong-hee Na 1b335ae281 bpo-39259: nntplib.NNTP/NNTP_SSL now reject timeout = 0 (GH-17936)
nntplib.NNTP and nntplib.NNTP_SSL now raise a ValueError
if the given timeout for their constructor is zero to
prevent the creation of a non-blocking socket.
2020-01-11 18:39:15 +01:00
Jason R. Coombs 136735c1a2
bpo-39297: Update for importlib_metadata 1.4. (GH-17947)
* bpo-39297: Update for importlib_metadata 1.4. Includes performance updates.

* 📜🤖 Added by blurb_it.

* Update blurb

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2020-01-11 10:37:28 -05:00
Dong-hee Na abdc634f33 bpo-39200: Correct the error message for min/max builtin function (GH-17814)
Correct the error message when calling the min() or max() with
no arguments.
2020-01-10 17:31:43 +01:00
Dong-hee Na c39b52f152 bpo-39259: poplib now rejects timeout = 0 (GH-17912)
poplib.POP3 and poplib.POP3_SSL now raise a ValueError
if the given timeout for their constructor is zero to
prevent the creation of a non-blocking socket.
2020-01-10 15:34:05 +01:00
Pablo Galindo 4c53e63cc9 bpo-39166: Fix trace of last iteration of async for loops (#17800) 2020-01-10 09:24:22 +00:00
Guido van Rossum a796d8ef9d bpo-39235: Fix end location for genexp in call args (GH-17925)
The fix changes copy_location() to require an extra node from which to extract the end location, and fixing all 5 call sites.


https://bugs.python.org/issue39235
2020-01-09 11:18:47 -08:00
An Long 5907e61a8d bpo-35292: Avoid calling mimetypes.init when http.server is imported (GH-17822) 2020-01-08 10:28:14 -08:00
Dong-hee Na 2e6a8efa83 bpo-39242: Updated the Gmane domain into news.gmane.io (GH-17903) 2020-01-08 16:29:34 +01:00
Dong-hee Na b821173b54 bpo-38871: Fix lib2to3 for filter-based statements that contain lambda (GH-17780)
Correctly parenthesize filter-based statements that contain lambda
expressions in lib2to3.
2020-01-07 18:30:54 +01:00
Dong-hee Na 13a7ee8d62 bpo-38615: Add timeout parameter for IMAP4 and IMAP4_SSL constructor (GH-17203)
imaplib.IMAP4 and imaplib.IMAP4_SSL now have an 
optional *timeout* parameter for their constructors.
Also, the imaplib.IMAP4.open() method now has an optional *timeout* parameter
with this change. The overridden methods of imaplib.IMAP4_SSL and
imaplib.IMAP4_stream were applied to this change.
2020-01-07 18:28:10 +01:00
Derek Brown 950c6795aa bpo-39198: Ensure logging global lock is released on exception in isEnabledFor (GH-17689) 2020-01-07 16:40:23 +00:00
Victor Stinner 5b23f7618d
bpo-39239: epoll.unregister() no longer ignores EBADF (GH-17882)
The select.epoll.unregister() method no longer ignores the EBADF
error.
2020-01-07 15:00:02 +01:00