The majority of this PR is tediously passing `end_lineno` and `end_col_offset` everywhere. Here are non-trivial points:
* It is not possible to reconstruct end positions in AST "on the fly", some information is lost after an AST node is constructed, so we need two more attributes for every AST node `end_lineno` and `end_col_offset`.
* I add end position information to both CST and AST. Although it may be technically possible to avoid adding end positions to CST, the code becomes more cumbersome and less efficient.
* Since the end position is not known for non-leaf CST nodes while the next token is added, this requires a bit of extra care (see `_PyNode_FinalizeEndPos`). Unless I made some mistake, the algorithm should be linear.
* For statements, I "trim" the end position of suites to not include the terminal newlines and dedent (this seems to be what people would expect), for example in
```python
class C:
pass
pass
```
the end line and end column for the class definition is (2, 8).
* For `end_col_offset` I use the common Python convention for indexing, for example for `pass` the `end_col_offset` is 4 (not 3), so that `[0:4]` gives one the source code that corresponds to the node.
* I added a helper function `ast.get_source_segment()`, to get source text segment corresponding to a given AST node. It is also useful for testing.
An (inevitable) downside of this PR is that AST now takes almost 25% more memory. I think however it is probably justified by the benefits.
* Disable x87 control word for non-x86 targets
On msvc, x87 control word is only available on x86 target. Need to disable it for other targets to prevent compiling problems.
* Include immintrin.h on x86 and x64 only
Immintrin.h is only available on x86 and x64. Need to disable it for other targets to prevent compiling problems.
Fix sparse file tests of test_tarfile on ppc64le with the tmpfs
filesystem.
Fix the function testing if the filesystem supports sparse files:
create a file which contains data and "holes", instead of creating a
file which contains no data.
tmpfs effective block size is a page size (tmpfs lives in the page
cache). RHEL uses 64 KiB pages on aarch64, ppc64 and ppc64le, only
s390x and x86_64 use 4 KiB pages, whereas the test punch holes of
4 KiB.
test.pythoninfo: Add resource.getpagesize().
It previously deleted Window => Zoom Height by mistake.
(Zoom Height is now on the Options menu). On Mac, the settings
dialog is accessed via Preferences on the IDLE menu.
Make ssl tests less strict and also accept TLSv1 as system default. The
changes unbreaks test_min_max_version on Fedora 29.
Signed-off-by: Christian Heimes <christian@python.org>
The subprocess module can now use the os.posix_spawn() function
in some cases for better performance. Currently, it is only used on macOS
and Linux (using glibc 2.24 or newer) if all these conditions are met:
* executable path contains a directory
* close_fds=False
* preexec_fn, pass_fds, cwd, stdin, stdout, stderr
and start_new_session parameters are not set
Co-authored-by: Joannah Nanjekye <nanjekyejoannah@gmail.com>
Fix a NULL pointer deref in ssl module. The cert parser did not handle CRL
distribution points with empty DP or URI correctly. A malicious or buggy
certificate can result into segfault.
Signed-off-by: Christian Heimes <christian@python.org>
https://bugs.python.org/issue35746
asyncio.ProactorEventLoop now catchs and logs send errors when the
self-pipe is full: BaseProactorEventLoop._write_to_self() now catchs
and logs OSError exceptions, as done by
BaseSelectorEventLoop._write_to_self().
IocpProactor.close() now uses time to decide when to log: wait 1
second before the first log, then log every second. Log also the
number of seconds since close() was called.
Previously, calling the strftime() method on a datetime object with a
trailing '%' in the format string would result in an exception. However,
this only occured when the datetime C module was being used; the python
implementation did not match this behavior. Datetime is now PEP-399
compliant, and will not throw an exception on a trailing '%'.
Format characters "%s" and "%V" in PyUnicode_FromFormat() and "%s" in PyBytes_FromFormat()
no longer read memory past the limit if precision is specified.
Use the fast call convention for math functions atan2(),
copysign(), hypot() and remainder() and inline unpacking
arguments. This sped up them by 1.3--2.5 times.
This change separates the signal handling trigger in the eval loop from the "pending calls" machinery. There is no semantic change and the difference in performance is insignificant.
The change makes both components less confusing. It also eliminates the risk of changes to the pending calls affecting signal handling. This is particularly relevant for some upcoming pending calls changes I have in the works.
Fix memory leaks in asyncio ProactorEventLoop on overlapped operation
failures.
Changes:
* Implement the tp_traverse slot in the _overlapped.Overlapped type
to help to break reference cycles and identify referrers in the
garbage collector.
* Always clear overlapped on failure: not only set type to
TYPE_NOT_STARTED, but release also resources.
As in title, expose C `raise` function as `raise_function` in `signal` module. Also drop existing `raise_signal` in `_testcapi` module and replace all usages with new function.
https://bugs.python.org/issue35568
bpo-32622, bpo-35682: Fix asyncio.ProactorEventLoop.sendfile(): don't
attempt to set the result of an internal future if it's already done.
Fix asyncio _ProactorBasePipeTransport._force_close(): don't set the
result of _empty_waiter if it's already done.
test_asyncio/test_sendfile.py now resets the event loop policy using
tearDownModule() as done in other tests, to prevent a warning when
running tests on Windows.
Fix an assertion error in format() in debug build for floating point
formatting with "n" format, zero padding and small width. Release build is
not impacted. Patch by Karthikeyan Singaravelan.
* bpo-35660: IDLE: Remove * import from window.py
* sys was being imported through the *, so also added an import sys.
* Update 2019-01-04-19-14-29.bpo-35660.hMxI7N.rst
Anyone who wants details can check the issue, where I added the point about the sys import bug.
* bpo-35588: Implement mod and divmod operations for Fraction type by spelling out the numerator/denominator calculation, instead of instantiating and normalising Fractions along the way. This speeds up '%' and divmod() by 2-3x.
* bpo-35588: Also reimplement Fraction.__floordiv__() using integer operations to make it ~4x faster.
* Improve code formatting.
Co-Authored-By: scoder <stefan_ml@behnel.de>
* bpo-35588: Fix return type of divmod(): the result of the integer division should be an integer.
* bpo-35588: Further specialise __mod__() and inline the original helper function _flat_divmod() since it's no longer reused.
* bpo-35588: Add some tests with large numerators and/or denominators.
* bpo-35588: Use builtin "divmod()" function for implementing __divmod__() in order to simplify the implementation, even though performance results are mixed.
* Rremove accidentally added empty line.
* bpo-35588: Try to provide more informative output on test failures.
* bpo-35588: Improve wording in News entry.
Co-Authored-By: scoder <stefan_ml@behnel.de>
* Remove stray space.
Ivan Levkivskyi
Minmin Gong
Victor Stinner
Pablo Galindo
Marc Schlaich
Anthony Sottile
Terry Jan Reedy
Ashwin Ramaswami
Christian Heimes
Serhiy Storchaka
Timo Furrer
Dong-hee Na
Joannah Nanjekye
MichaelSaah
Tal Einat
Eric Snow
Michael Felt
Bo Bayles
Sanyam Khurana
Steve Dower
Vladimir Matveev
Gregory Szorc
Raymond Hettinger
Xtreak
Cheryl Sabella
Anthony Shaw
Emmanuel Arias
Mickaël Schoentgen
Stefan Behnel