Commit Graph

105681 Commits

Author SHA1 Message Date
Victor Stinner c9a413ede4 bpo-38321: Fix PyCStructUnionType_update_stgdict() warning (GH-16492)
bpo-22273, bpo-38321: Fix following warning:

    modules\_ctypes\stgdict.c(704):
    warning C4244: 'initializing': conversion from 'Py_ssize_t' to 'int', possible loss of data
2019-09-30 16:49:47 +01:00
Victor Stinner efe74b6369
bpo-38321: Fix _asynciomodule.c compiler warning (GH-16493)
bpo-38248, bpo-38321: Fix warning:

    modules\_asynciomodule.c(2667):
    warning C4102: 'set_exception': unreferenced label

The related goto has been removed by
commit edad4d89e3.
2019-09-30 16:41:34 +02:00
Victor Stinner dec39716ca
bpo-38322: Fix gotlandmark() of PC/getpathp.c (GH-16489)
Write the filename into a temporary buffer instead of reusing prefix.
The problem is that join() modifies prefix inplace. If prefix is not
normalized, join() can make prefix shorter and so gotlandmark()
does modify prefix instead of returning it unmodified.
2019-09-30 14:49:34 +02:00
Victor Stinner 89f8177dcf
bpo-38304: Fix PyConfig usage in python_uwp.cpp (GH-16487)
* Set PyPreConfig.struct_size and PyConfig.struct_size as required by
  the API.
* PyPreConfig_InitPythonConfig() can now fail: check PyStatus result.
2019-09-30 12:52:29 +02:00
Yury Selivanov fc4a044a3c
bpo-30773: Fix ag_running; prohibit running athrow/asend/aclose in parallel (#7468) 2019-09-29 22:59:11 -07:00
Yury Selivanov 6758e6e12a
bpo-38242: Revert "bpo-36889: Merge asyncio streams (GH-13251)" (#16482)
See https://bugs.python.org/issue38242 for more details
2019-09-29 21:59:55 -07:00
Lisa Roach 3667e1ee6c
bpo-38163: Child mocks detect their type as sync or async (GH-16471) 2019-09-29 21:56:47 -07:00
Giampaolo Rodola 5bcc6d89bc
bpo-37096: Add large-file tests for modules using sendfile(2) (GH-13676) 2019-09-30 12:51:55 +08:00
Lisa Roach 25e115ec00
bpo-38161: Removes _AwaitEvent from AsyncMock. (GH-16443) 2019-09-29 21:01:28 -07:00
Victor Stinner fb4ae152a9
bpo-38317: Fix PyConfig.warnoptions priority (GH-16478)
Fix warnings options priority: PyConfig.warnoptions has the highest
priority, as stated in the PEP 587.

* Document options order in PyConfig.warnoptions documentation.
* Make PyWideStringList_INIT macro private: replace "Py" prefix
  with "_Py".
* test_embed: add test_init_warnoptions().
2019-09-30 01:40:17 +02:00
Andrew Svetlov 58498bc717
bpo-38019: correctly handle pause/resume reading of closed asyncio unix pipe (GH-16472) 2019-09-29 15:00:35 +03:00
Lisa Roach 9a7d951950
bpo-38108: Makes mock objects inherit from Base (GH-16060) 2019-09-28 18:42:44 -07:00
Brandt Bucher f185a73249 bpo-38310: Predict BUILD_MAP_UNPACK_WITH_CALL -> CALL_FUNCTION_EX. (GH-16467) 2019-09-28 17:12:49 -07:00
Pascal Chambon c5a7e0ce19 bpo-37408: Precise that Tarfile "format" argument only concerns writing. (GH-14389) 2019-09-28 08:04:44 -07:00
T. Wouters c8165036f3 bpo-38115: Deal with invalid bytecode offsets in lnotab (GH-16079)
Document that lnotab can contain invalid bytecode offsets (because of
terrible reasons that are difficult to fix). Make dis.findlinestarts()
ignore invalid offsets in lnotab. All other uses of lnotab in CPython
(various reimplementations of addr2line or line2addr in Python, C and gdb)
already ignore this, because they take an address to look for, instead.

Add tests for the result of dis.findlinestarts() on wacky constructs in
test_peepholer.py, because it's the easiest place to add them.
2019-09-28 07:49:15 -07:00
Jason R. Coombs 7774d7831e
bpo-38216, bpo-36274: Allow subclasses to separately override validation and encoding behavior (GH-16448)
* bpo-38216: Allow bypassing input validation

* bpo-36274: Also allow the URL encoding to be overridden.

* bpo-38216, bpo-36274: Add tests demonstrating a hook for overriding validation, test demonstrating override encoding, and a test to capture expectation of the interface for the URL.

* Call with skip_host to avoid tripping on the host checking in the URL.

* Remove obsolete comment.

* Make _prepare_path_encoding its own attr.

This makes overriding just that simpler.

Also, don't use the := operator to make backporting easier.

* Add a news entry.

* _prepare_path_encoding -> _encode_prepared_path()

* Once again separate the path validation and request encoding, drastically simplifying the behavior. Drop the guarantee that all processing happens in _prepare_path.
2019-09-28 08:32:01 -04:00
Victor Stinner 441b10cf28
bpo-38304: Add PyConfig.struct_size (GH-16451)
Add a new struct_size field to PyPreConfig and PyConfig structures to
allow to modify these structures in the future without breaking the
backward compatibility.

* Replace private _config_version field with public struct_size field
  in PyPreConfig and PyConfig.
* Public PyPreConfig_InitIsolatedConfig() and
  PyPreConfig_InitPythonConfig()
  return type becomes PyStatus, instead of void.
* Internal _PyConfig_InitCompatConfig(),
  _PyPreConfig_InitCompatConfig(), _PyPreConfig_InitFromConfig(),
  _PyPreConfig_InitFromPreConfig() return type becomes PyStatus,
  instead of void.
* Remove _Py_CONFIG_VERSION
* Update the Initialization Configuration documentation.
2019-09-28 04:28:35 +02:00
Jesús Cea 52d1b86bde
bpo-38301: In Solaris family, we must be sure to use '-D_REENTRANT' (#16446) 2019-09-28 03:44:32 +02:00
Dong-hee Na e8650a4f8c bpo-38243, xmlrpc.server: Escape the server_title (GH-16373)
Escape the server title of xmlrpc.server.DocXMLRPCServer
when rendering the document page as HTML.
2019-09-27 21:59:37 +02:00
bariod dd6117c6d7 Fix typo in the "Porting to Python 3.8" section. (GH-16435) 2019-09-27 21:01:33 +03:00
Serhiy Storchaka 5d6f5b6293
bpo-32820: Simplify __format__ implementation for ipaddress. (GH-16378)
Also cache the compiled RE for parsing the format specifier.
2019-09-27 20:02:58 +03:00
Eric Snow 6693f730e0
bpo-38187: Fix a refleak in Tools/c-analyzer. (gh-16304)
The "Slot" helper (descriptor) is leaking references due to its caching mechanism. The change includes a partial fix to Slot, but also adds Variable.storage to replace the problematic use of Slot.

https://bugs.python.org/issue38187
2019-09-27 15:53:34 +01:00
Christian Heimes 9055815809 bpo-38270: More fixes for strict crypto policy (GH-16418)
test_hmac and test_hashlib test built-in hashing implementations and
OpenSSL-based hashing implementations. Add more checks to skip OpenSSL
implementations when a strict crypto policy is active.

Use EVP_DigestInit_ex() instead of EVP_DigestInit() to initialize the
EVP context. The EVP_DigestInit() function clears alls flags and breaks
usedforsecurity flag again.

Signed-off-by: Christian Heimes <christian@python.org>



https://bugs.python.org/issue38270
2019-09-27 06:03:53 -07:00
Ammar Askar 5faff977ad bpo-38206: Clarify tp_dealloc requirements for heap allocated types. (GH-16248)
As mentioned in the bpo ticket, this mistake came up on two reviews:
- https://github.com/python/cpython/pull/16127#pullrequestreview-288312751
- https://github.com/python/cpython/pull/16071#pullrequestreview-287819525

Would be nice to have it documented in a more permanent place than 3.8's whatsnew entry.


https://bugs.python.org/issue38206



Automerge-Triggered-By: @encukou
2019-09-27 04:11:27 -07:00
HongWeipeng 6ce03ec627 cleanup ababstractproperty in typing.py (GH-16432) 2019-09-27 08:54:26 +01:00
Michael Felt 0bcbfa43d5 bpo-28009: Fix uuid.uuid1() and uuid.get_node() on AIX (GH-8672) 2019-09-26 22:43:15 +03:00
Christian Heimes 9f77268f90
bpo-38275: Fix test_ssl issue caused by GH-16386 (#16428)
Check presence of SSLContext.minimum_version to make tests pass with
old versions of OpenSSL.

Signed-off-by: Christian Heimes <christian@python.org>
2019-09-26 18:23:17 +02:00
Christian Heimes df6ac7e2b8 bpo-38275: Skip ssl tests for disabled versions (GH-16386)
test_ssl now handles disabled TLS/SSL versions better. OpenSSL's crypto
policy and run-time settings are recognized and tests for disabled versions
are skipped.

Signed-off-by: Christian Heimes <christian@python.org>



https://bugs.python.org/issue38275
2019-09-26 08:02:59 -07:00
Victor Stinner 64b4a3a2de
bpo-38239: Fix test_gdb for Link Time Optimization (LTO) (GH-16422) 2019-09-26 16:54:13 +02:00
Victor Stinner 12f2f177fc
bpo-38234: Py_Initialize() sets global path configuration (GH-16421)
* Py_InitializeFromConfig() now writes PyConfig path configuration to
  the global path configuration (_Py_path_config).
* Add test_embed.test_get_pathconfig().
* Fix typo in _PyWideStringList_Join().
2019-09-26 15:51:50 +02:00
Petr Viktorin 3d984a1fd0
compileall tests: Use shorter name for long_path test (GH-16419)
Apparently, the path needs to be limited to 260 characters on
(some versions of) Windows.
2019-09-26 15:16:32 +02:00
Serhiy Storchaka 4f2eac04e4
bpo-38223: Reorganize test_shutil. (GH-16281)
* Group tests for specific functions and groups of related functions
into separate classes.
* Clean up creating and cleaning up temporary directories.
* Simplify and make more robust monkey patching of shutil.open.
2019-09-26 13:15:08 +03:00
Petr Viktorin 4267c989e7
bpo-38112: compileall: Skip long path path on Windows if the path can't be created (GH-16414)
This avoids the buildbot failure on Windows:
```
FileNotFoundError: [WinError 206] The filename or extension is too long: 'd:\\temp\\tmp5r3z438t\\long\\1\\2\\3\\4\\5\\6\\7\\8\\9\\10\\11\\12\\13\\14\\15\\16\\17\\18\\19\\20\\21\\22\\23\\24\\25\\26\\27\\28\\29\\30\\31\\32\\33\\34\\35\\36\\37\\38\\39\\40\\41\\42\\43\\44\\45\\46\\47\\48\\49\\50\\51\\52\\53\\54\\55\\56\\57\\58\\59\\60\\61\\62\\63\\64\\65\\66\\67\\68\\69\\70\\71\\72\\73\\74\\75\\76\\77\\78'
```
Creates a path that's long but avoids OS restrictions.

https://bugs.python.org/issue38112
2019-09-26 11:53:51 +02:00
Prateek Nayak a7414571a1 bpo-38244: [Documentation] Fix unreliable link to sever in ftplib.rst (#16319) 2019-09-26 17:22:31 +08:00
Mohammad Dehghan 1ad7be2f16 bpo-38130: Fix error in explaining when an exception is re-raised (GH-16016)
Co-Authored-By: Ashwin Ramaswami <aramaswamis@gmail.com>
2019-09-26 11:09:05 +02:00
Kyle Stanley 6a10d59541 Doc: Several fixes and improvements for 3.9 whatsnew (GH-16375) 2019-09-26 09:06:46 +02:00
Stéphane Wirtel 5d326abf2c
Doc: Use the `with` statement in the first example of the ftplib doc. (GH-16271) 2019-09-26 09:01:18 +02:00
Lumír 'Frenzy' Balhar 8e7bb991de bpo-38112: Compileall improvements (GH-16012)
* Raise the limit of maximum path depth to actual  recursion limit

* Add posibilities to adjust a path compiled in .pyc  file.

Now, you can:
- Strip a part of path from a beggining of path into compiled file
   example "-s /test /test/build/real/test.py" → "build/real/test.py"
- Append some new path to a beggining of path into compiled file
   example "-p /boo real/test.py" → "/boo/real/test.py"

You can also use both options in the same time. In that case,
striping is done before appending.

* Add a possibility to specify multiple optimization levels

Each optimization level then leads to separated compiled file.
Use `action='append'` instead of `nargs='+'` for the -o option.
Instead of `-o 0 1 2`, specify `-o 0 -o 1 -o 2`. It's more to type,
but much more explicit.

* Add a symlinks limitation feature

This feature allows us to limit byte-compilation of symbolic
links if they are pointing outside specified dir (build root
for example).
2019-09-26 08:28:26 +02:00
Benjamin Peterson 52b9408038
closes bpo-38174: Update vendored expat library to 2.2.8. (GH-16346)
Fixes CVE-2019-15903. See full changelog at https://github.com/libexpat/libexpat/blob/R_2_2_8/expat/Changes.
2019-09-25 21:33:58 -07:00
Victor Stinner 49d99f01e6
bpo-38234: Fix test_embed.test_init_setpath_config() on FreeBSD (GH-16406)
Explicitly preinitializes with a Python preconfiguration to avoid
Py_SetPath() implicit preinitialization with a compat
preconfiguration.

Fix also test_init_setpath() and test_init_setpythonhome() on macOS:
use self.test_exe as the executable (and base_executable), rather
than shutil.which('python3').
2019-09-26 04:01:49 +02:00
Victor Stinner 88feaecd46
bpo-38234: Complete init config documentation (GH-16404) 2019-09-26 03:15:07 +02:00
Victor Stinner 8bf39b606e
bpo-38234: Add test_init_setpath_config() to test_embed (GH-16402)
* Add test_embed.test_init_setpath_config(): test Py_SetPath()
  with PyConfig.
* test_init_setpath() and test_init_setpythonhome() no longer call
  Py_SetProgramName(), but use the default program name.
* _PyPathConfig: isolated, site_import  and base_executable
  fields are now only available on Windows.
* If executable is set explicitly in the configuration, ignore
  calculated base_executable: _PyConfig_InitPathConfig() copies
  executable to base_executable.
* Complete path config documentation.
2019-09-26 02:22:35 +02:00
Christian Heimes df69e75edc
bpo-38142: Updated _hashopenssl.c to be PEP 384 compliant (#16071)
* Updated _hashopenssl.c to be PEP 384 compliant
* Remove refleak test from test_hashlib. The updated type no longer accepts random arguments to __init__.
2019-09-25 23:03:30 +02:00
Vinay Sajip cc28ed2421
bpo-22273: Removed temporary test skipping on PPC platforms. (GH-16399) 2019-09-25 20:57:20 +01:00
Christian Heimes bfd0c963d8 bpo-38271: encrypt private key test files with AES256 (GH-16385)
The private keys for test_ssl were encrypted with 3DES in traditional
PKCS#5 format. 3DES and the digest algorithm of PKCS#5 are blocked by
some strict crypto policies. Use PKCS#8 format with AES256 encryption
instead.

Signed-off-by: Christian Heimes <christian@python.org>



https://bugs.python.org/issue38271



Automerge-Triggered-By: @tiran
2019-09-25 08:55:02 -07:00
Serhiy Storchaka 543a3951a1
bpo-38005: Remove support of string argument in InterpreterID(). (GH-16227)
Make negative interpreter id to raise ValueError instead of RuntimeError.
2019-09-25 18:35:57 +03:00
Raymond Hettinger 15ccc4fac0
bpo-38255: super() can search attributes as well as methods (GH-16368)
Improvement suggested by Géry Ogam.
2019-09-25 08:13:19 -07:00
Victor Stinner 00508a7407
bpo-38234: Fix test_embed pathconfig tests (GH-16390)
bpo-38234: On macOS and FreeBSD, the temporary directory can be
symbolic link. For example, /tmp can be a symbolic link to /var/tmp.
Call realpath() to resolve all symbolic links.
2019-09-25 16:30:36 +02:00
Christian Heimes c64a1a61e6 bpo-38270: Check for hash digest algorithms and avoid MD5 (GH-16382)
Make it easier to run and test Python on systems with restrict crypto policies:

* add requires_hashdigest to test.support to check if a hash digest algorithm is available and working
* avoid MD5 in test_hmac
* replace MD5 with SHA256 in test_tarfile
* mark network tests that require MD5 for MD5-based digest auth or CRAM-MD5


https://bugs.python.org/issue38270
2019-09-25 07:30:20 -07:00
Vinay Sajip 417089e88b
bpo-22273: Re-enabled ctypes test on ARM machines. (GH-16388) 2019-09-25 15:05:55 +01:00