Commit Graph

21661 Commits

Author SHA1 Message Date
Steve Dower 28f6cb34f6
bpo-35683: Improve Azure Pipelines steps (GH-11493) 2019-01-22 10:49:52 -08:00
Victor Stinner bf4ac2d2fd
bpo-35713: Rework Python initialization (GH-11647)
* The PyByteArray_Init() and PyByteArray_Fini() functions have been
  removed. They did nothing since Python 2.7.4 and Python 3.2.0, were
  excluded from the limited API (stable ABI), and were not
  documented.
* Move "_PyXXX_Init()" and "_PyXXX_Fini()" declarations from
  Include/cpython/pylifecycle.h to
  Include/internal/pycore_pylifecycle.h. Replace
  "PyAPI_FUNC(TYPE)" with "extern TYPE".
* _PyExc_Init() now returns an error on failure rather than calling
  Py_FatalError(). Move macros inside _PyExc_Init() and undefine them
  when done. Rewrite macros to make them look more like statement:
  add ";" when using them, add "do { ... } while (0)".
* _PyUnicode_Init() now returns a _PyInitError error rather than call
  Py_FatalError().
* Move stdin check from _PySys_BeginInit() to init_sys_streams().
* _Py_ReadyTypes() now returns a _PyInitError error rather than
  calling Py_FatalError().
2019-01-22 17:39:03 +01:00
Lucas Cimon 35ca1820e1 bpo-35720: Fixing a memory leak in pymain_parse_cmdline_impl() (GH-11528)
When the loop in the pymain_read_conf function in this same file
calls pymain_init_cmdline_argv() a 2nd time, the pymain->command
buffer of wchar_t is overriden and the previously allocated memory
is never freed.
2019-01-22 17:15:01 +01:00
Ivan Levkivskyi 9932a22897
bpo-33416: Add end positions to Python AST (GH-11605)
The majority of this PR is tediously passing `end_lineno` and `end_col_offset` everywhere. Here are non-trivial points:
* It is not possible to reconstruct end positions in AST "on the fly", some information is lost after an AST node is constructed, so we need two more attributes for every AST node `end_lineno` and `end_col_offset`.
* I add end position information to both CST and AST.  Although it may be technically possible to avoid adding end positions to CST, the code becomes more cumbersome and less efficient.
* Since the end position is not known for non-leaf CST nodes while the next token is added, this requires a bit of extra care (see `_PyNode_FinalizeEndPos`). Unless I made some mistake, the algorithm should be linear.
* For statements, I "trim" the end position of suites to not include the terminal newlines and dedent (this seems to be what people would expect), for example in
  ```python
  class C:
      pass

  pass
  ```
  the end line and end column for the class definition is (2, 8).
* For `end_col_offset` I use the common Python convention for indexing, for example for `pass` the `end_col_offset` is 4 (not 3), so that `[0:4]` gives one the source code that corresponds to the node.
* I added a helper function `ast.get_source_segment()`, to get source text segment corresponding to a given AST node. It is also useful for testing.

An (inevitable) downside of this PR is that AST now takes almost 25% more memory. I think however it is probably justified by the benefits.
2019-01-22 11:18:22 +00:00
Minmin Gong 7a2368063f bpo-35758: Fix building on ARM + MSVC (gh-11531)
* Disable x87 control word for non-x86 targets

On msvc, x87 control word is only available on x86 target. Need to disable it for other targets to prevent compiling problems.

* Include immintrin.h on x86 and x64 only

Immintrin.h is only available on x86 and x64. Need to disable it for other targets to prevent compiling problems.
2019-01-21 21:49:40 +01:00
Victor Stinner b2385458ce
bpo-35772: Fix test_tarfile on ppc64 (GH-11606)
Fix sparse file tests of test_tarfile on ppc64le with the tmpfs
filesystem.

Fix the function testing if the filesystem supports sparse files:
create a file which contains data and "holes", instead of creating a
file which contains no data.

tmpfs effective block size is a page size (tmpfs lives in the page
cache). RHEL uses 64 KiB pages on aarch64, ppc64 and ppc64le, only
s390x and x86_64 use 4 KiB pages, whereas the test punch holes of
4 KiB.

test.pythoninfo: Add resource.getpagesize().
2019-01-21 10:24:12 +01:00
Pablo Galindo 222d303ade bpo-20239: Allow repeated deletion of unittest.mock.Mock attributes (#11057)
* Allow repeated deletion of unittest.mock.Mock attributes

* fixup! Allow repeated deletion of unittest.mock.Mock attributes

* fixup! fixup! Allow repeated deletion of unittest.mock.Mock attributes
2019-01-21 08:57:46 +00:00
Marc Schlaich b2dc4a3313 bpo-35699: fix distuils cannot detect Build Tools 2017 anymore (GH-11495) 2019-01-20 13:47:42 -05:00
Anthony Sottile 7417622617 bpo-35733: Make isinstance(ast.Constant(boolean), ast.Num) be false. (GH-11547) 2019-01-18 21:30:28 +02:00
Terry Jan Reedy 39ed289a35
bpo-35770: IDLE macosx deletes Options => Configure IDLE. (GH-11614)
It previously deleted Window => Zoom Height by mistake.
(Zoom Height is now on the Options menu).  On Mac, the settings
dialog is accessed via Preferences on the IDLE menu.
2019-01-18 14:00:45 -05:00
Ashwin Ramaswami 478f829132 bpo-21257: document http.client.parse_headers (GH-11443)
Document http.client.parse_headers
2019-01-18 08:49:16 -07:00
Christian Heimes 34de2d312b bpo-35045: Accept TLSv1 default in min max test (GH-11510)
Make ssl tests less strict and also accept TLSv1 as system default. The
changes unbreaks test_min_max_version on Fedora 29.

Signed-off-by: Christian Heimes <christian@python.org>
2019-01-18 16:09:30 +01:00
Terry Jan Reedy a902239f22
bpo-35769: Change IDLE's name for new files from 'Untitled' to 'untitled' (GH-11602)
'Untitled' violates the PEP 8 standard for .py files
2019-01-18 02:09:53 -05:00
Serhiy Storchaka 3bcbedc9f1
bpo-34850: Emit a warning for "is" and "is not" with a literal. (GH-9642) 2019-01-18 07:47:48 +02:00
Timo Furrer 17f05bbc78 bpo-33687: Fix call to os.chmod() in uu.decode() (GH-7282) 2019-01-17 17:15:53 +03:00
Dong-hee Na 89669ffe10 bpo-35283: Add deprecation warning for Thread.isAlive (GH-11454)
Add a deprecated warning for the threading.Thread.isAlive() method.
2019-01-17 13:14:45 +01:00
Joannah Nanjekye 92b8322e7e bpo-35674: Add os.posix_spawnp() (GH-11554)
Add a new os.posix_spawnp() function.
2019-01-16 14:29:26 +01:00
Victor Stinner 9daecf37a5
bpo-35537: subprocess uses os.posix_spawn in some cases (GH-11452)
The subprocess module can now use the os.posix_spawn() function
in some cases for better performance. Currently, it is only used on macOS
and Linux (using glibc 2.24 or newer) if all these conditions are met:

* executable path contains a directory
* close_fds=False
* preexec_fn, pass_fds, cwd, stdin, stdout, stderr
  and start_new_session parameters are not set

Co-authored-by: Joannah Nanjekye <nanjekyejoannah@gmail.com>
2019-01-16 00:02:35 +01:00
Christian Heimes a37f52436f bpo-35746: Fix segfault in ssl's cert parser (GH-11569)
Fix a NULL pointer deref in ssl module. The cert parser did not handle CRL
distribution points with empty DP or URI correctly. A malicious or buggy
certificate can result into segfault.

Signed-off-by: Christian Heimes <christian@python.org>



https://bugs.python.org/issue35746
2019-01-15 14:47:42 -08:00
Victor Stinner c9f872b0bd
bpo-23846: Fix ProactorEventLoop._write_to_self() (GH-11566)
asyncio.ProactorEventLoop now catchs and logs send errors when the
self-pipe is full: BaseProactorEventLoop._write_to_self() now catchs
and logs OSError exceptions, as done by
BaseSelectorEventLoop._write_to_self().
2019-01-15 13:58:38 +01:00
Victor Stinner b1e45739d8
bpo-34323: Enhance IocpProactor.close() log (GH-11555)
IocpProactor.close() now uses time to decide when to log: wait 1
second before the first log, then log every second. Log also the
number of seconds since close() was called.
2019-01-15 11:48:00 +01:00
Serhiy Storchaka efcf82f945
bpo-35619: Improve support of custom data descriptors in help() and pydoc. (GH-11366) 2019-01-15 10:53:18 +02:00
MichaelSaah 454b3d4ea2 bpo-35066: _dateime.datetime.strftime copies trailing '%' (GH-10692)
Previously, calling the strftime() method on a datetime object with a
trailing '%' in the format string would result in an exception. However,
this only occured when the datetime C module was being used; the python
implementation did not match this behavior. Datetime is now PEP-399
compliant, and will not throw an exception on a trailing '%'.
2019-01-14 11:23:39 +01:00
Tal Einat 39a33e9927
bpo-35196: Optimize Squeezer's write() interception (GH-10454)
The new functionality of Squeezer.reload() is also tested, along with some general
re-working of the tests in test_squeezer.py.
2019-01-13 17:01:50 +02:00
Anthony Sottile 995d9b9297 bpo-16806: Fix `lineno` and `col_offset` for multi-line string tokens (GH-10021) 2019-01-13 13:05:13 +09:00
Serhiy Storchaka d586ccb04f
bpo-35552: Fix reading past the end in PyUnicode_FromFormat() and PyBytes_FromFormat(). (GH-11276)
Format characters "%s" and "%V" in PyUnicode_FromFormat() and "%s" in PyBytes_FromFormat()
no longer read memory past the limit if precision is specified.
2019-01-12 10:30:35 +02:00
Serhiy Storchaka f1ec3cefad
bpo-35634: Raise an error when first passed kwargs contains duplicated keys. (GH-11438) 2019-01-12 10:12:24 +02:00
Serhiy Storchaka 58159ef856
bpo-35494: Improve syntax error messages for unbalanced parentheses in f-string. (GH-11161) 2019-01-12 09:46:50 +02:00
Serhiy Storchaka 44cc4822bb
bpo-33817: Fix _PyBytes_Resize() for empty bytes object. (GH-11516)
Add also tests for PyUnicode_FromFormat() and PyBytes_FromFormat()
with empty result.
2019-01-12 09:22:29 +02:00
Serhiy Storchaka d0d3e99120
bpo-35719: Optimize multi-argument math functions. (GH-11527)
Use the fast call convention for math functions atan2(),
copysign(), hypot() and remainder() and inline unpacking
arguments. This sped up them by 1.3--2.5 times.
2019-01-12 08:26:34 +02:00
Eric Snow fdf282d609
bpo-35423: Stop using the "pending calls" machinery for signals. (gh-10972)
This change separates the signal handling trigger in the eval loop from the "pending calls" machinery. There is no semantic change and the difference in performance is insignificant.

The change makes both components less confusing. It also eliminates the risk of changes to the pending calls affecting signal handling. This is particularly relevant for some upcoming pending calls changes I have in the works.
2019-01-11 14:26:55 -07:00
Michael Felt a909460a09 bpo-34569: Fix subinterpreter 32-bit ABI, pystate.c/_new_long_object() (gh-9127)
This fixes ShareableTypeTests.test_int() in Lib/test/test__xxsubinterpreters.py.
2019-01-11 11:17:03 -07:00
Victor Stinner 5485085b32
bpo-32710: Fix _overlapped.Overlapped memory leaks (GH-11489)
Fix memory leaks in asyncio ProactorEventLoop on overlapped operation
failures.

Changes:

* Implement the tp_traverse slot in the _overlapped.Overlapped type
  to help to break reference cycles and identify referrers in the
  garbage collector.
* Always clear overlapped on failure: not only set type to
  TYPE_NOT_STARTED, but release also resources.
2019-01-11 14:35:14 +01:00
Bo Bayles bab4bbb4c9 bpo-32146: Add documentation about frozen executables on Unix (GH-5850) 2019-01-10 18:51:28 +01:00
Joannah Nanjekye 572168a016 bpo-35702: Add new identifier time.CLOCK_UPTIME_RAW for macOS 10.12 (GH-11503) 2019-01-10 17:56:38 +01:00
Terry Jan Reedy ee65594367
bpo-35641: Move IDLE blurb to IDLE directory (#11479) 2019-01-09 10:44:07 -05:00
Sanyam Khurana cbb1645993 bpo-24746: Avoid stripping trailing whitespace in doctest fancy diff (#10639) 2019-01-09 05:38:38 -08:00
Victor Stinner a234e14839
bpo-32710: Fix leak in Overlapped_WSASend() (GH-11469)
Fix a memory leak in asyncio in the ProactorEventLoop when ReadFile()
or WSASend() overlapped operation fail immediately: release the
internal buffer.
2019-01-08 14:23:09 +01:00
Steve Dower 872bd2b57c
bpo-35596: Use unchecked PYCs for the embeddable distro to avoid zipimport restrictions (GH-11465)
Also adds extra steps to the CI build for Windows on Azure Pipelines to validate that the various layouts at least execute.
2019-01-08 02:38:01 -08:00
Vladimir Matveev c24c6c2c93 bpo-35568: add 'raise_signal' function (GH-11335)
As in title, expose C `raise` function as `raise_function` in `signal` module. Also drop existing `raise_signal` in `_testcapi` module and replace all usages with new function.


https://bugs.python.org/issue35568
2019-01-08 01:58:25 -08:00
Victor Stinner 80fda712c8
bpo-35682: Fix _ProactorBasePipeTransport._force_close() (GH-11462)
bpo-32622, bpo-35682: Fix asyncio.ProactorEventLoop.sendfile(): don't
attempt to set the result of an internal future if it's already done.

Fix asyncio _ProactorBasePipeTransport._force_close(): don't set the
result of _empty_waiter if it's already done.
2019-01-08 02:46:59 +01:00
Gregory Szorc fbf50683b3 bpo-35642: Remove asynciomodule.c from pythoncore.vcxproj (GH-11410)
This module is built by _asyncio.vcxproj and does not need to be included in pythoncore.
2019-01-07 17:27:18 -08:00
Victor Stinner ddd7c422fd
bpo-33717: pythoninfo logs information of all clocks (GH-11460)
test.pythoninfo now logs information of all clocks, not only time.time() and
time.perf_counter().
2019-01-08 01:27:27 +01:00
Victor Stinner df8e1fb4e3
bpo-32710: test_asyncio: test_sendfile reset policy (GH-11461)
test_asyncio/test_sendfile.py now resets the event loop policy using
tearDownModule() as done in other tests, to prevent a warning when
running tests on Windows.
2019-01-07 23:55:57 +01:00
Raymond Hettinger 2d53bed79c
bpo-35664: Optimize operator.itemgetter (GH-11435) 2019-01-07 09:38:41 -07:00
Xtreak 3f7983a25a bpo-35560: Remove assertion from format(float, "n") (GH-11288)
Fix an assertion error in format() in debug build for floating point
formatting with "n" format, zero padding and small width. Release build is
not impacted. Patch by Karthikeyan Singaravelan.
2019-01-07 16:09:14 +01:00
Cheryl Sabella 11303dd603 bpo-35660: Fix imports in idlelib.window (#11434)
* bpo-35660: IDLE: Remove * import from window.py

* sys was being imported through the *, so also added an import sys.

* Update 2019-01-04-19-14-29.bpo-35660.hMxI7N.rst

Anyone who wants details can check the issue, where I added the point about the sys import bug.
2019-01-06 15:55:52 -05:00
Anthony Shaw 83da926b89 bpo-35488: Add tests for ** glob matching in pathlib (GH-11171) 2019-01-06 12:31:29 -08:00
Emmanuel Arias ab54b9a130 bpo-35641: IDLE - format calltip properly when no docstring (GH-11415) 2019-01-03 02:47:58 -05:00
Terry Jan Reedy aff0adabf3
bpo-33987: IDLE - use ttk Frame for ttk widgets (GH-11395) 2019-01-02 22:04:06 -05:00