* Properly decref on _pylong import error.
* Improve the error message on _pylong TypeError.
* Fix the assertion error in pydebug builds to be a TypeError.
* Tie the return value comments together.
These are minor followups to issues not caught among the reviewers on
https://github.com/python/cpython/pull/96673.
Add Python implementations of certain longobject.c functions. These use
asymptotically faster algorithms that can be used for operations on
integers with many digits. In those cases, the performance overhead of
the Python implementation is not significant since the asymptotic
behavior is what dominates runtime. Functions provided by this module
should be considered private and not part of any public API.
Co-author: Tim Peters <tim.peters@gmail.com>
Co-author: Mark Dickinson <dickinsm@gmail.com>
Co-author: Bjorn Martinsson
It had to live as a global outside of PyConfig for stable ABI reasons in
the pre-3.12 backports.
This removes the `_Py_global_config_int_max_str_digits` and gets rid of
the equivalent field in the internal `struct _is PyInterpreterState` as
code can just use the existing nested config struct within that.
Adds tests to verify unique settings and configs in subinterpreters.
A regression would still absolutely fail and even a flaky pass isn't
harmful as it'd fail most of the time across our N system test runs.
Windows has a low resolution timer and CI systems are prone to odd
timing so this just gives more leeway to avoid flakiness.
Converting a large enough `int` to a decimal string raises `ValueError` as expected. However, the raise comes _after_ the quadratic-time base-conversion algorithm has run to completion. For effective DOS prevention, we need some kind of check before entering the quadratic-time loop. Oops! =)
The quick fix: essentially we catch _most_ values that exceed the threshold up front. Those that slip through will still be on the small side (read: sufficiently fast), and will get caught by the existing check so that the limit remains exact.
The justification for the current check. The C code check is:
```c
max_str_digits / (3 * PyLong_SHIFT) <= (size_a - 11) / 10
```
In GitHub markdown math-speak, writing $M$ for `max_str_digits`, $L$ for `PyLong_SHIFT` and $s$ for `size_a`, that check is:
$$\left\lfloor\frac{M}{3L}\right\rfloor \le \left\lfloor\frac{s - 11}{10}\right\rfloor$$
From this it follows that
$$\frac{M}{3L} < \frac{s-1}{10}$$
hence that
$$\frac{L(s-1)}{M} > \frac{10}{3} > \log_2(10).$$
So
$$2^{L(s-1)} > 10^M.$$
But our input integer $a$ satisfies $|a| \ge 2^{L(s-1)}$, so $|a|$ is larger than $10^M$. This shows that we don't accidentally capture anything _below_ the intended limit in the check.
<!-- gh-issue-number: gh-95778 -->
* Issue: gh-95778
<!-- /gh-issue-number -->
Co-authored-by: Gregory P. Smith [Google LLC] <greg@krypto.org>
Integer to and from text conversions via CPython's bignum `int` type is not safe against denial of service attacks due to malicious input. Very large input strings with hundred thousands of digits can consume several CPU seconds.
This PR comes fresh from a pile of work done in our private PSRT security response team repo.
Signed-off-by: Christian Heimes [Red Hat] <christian@python.org>
Tons-of-polishing-up-by: Gregory P. Smith [Google] <greg@krypto.org>
Reviews via the private PSRT repo via many others (see the NEWS entry in the PR).
<!-- gh-issue-number: gh-95778 -->
* Issue: gh-95778
<!-- /gh-issue-number -->
I wrote up [a one pager for the release managers](https://docs.google.com/document/d/1KjuF_aXlzPUxTK4BMgezGJ2Pn7uevfX7g0_mvgHlL7Y/edit#). Much of that text wound up in the Issue. Backports PRs already exist. See the issue for links.
This avoids possible buffer overreads when int(), float(), compile(), exec()
and eval() are passed bytes-like objects. Similar code is removed from the
complex() constructor, where it was not reachable.
Patch by John Leitch, Serhiy Storchaka and Martin Panter.
I have compared output between pre- and post-patch runs of these tests
to make sure there's nothing missing and nothing broken, on both
Windows and Linux. The only differences I found were actually tests
that were previously *not* run.
types. Added a new API function, PyUnicode_TransformDecimalToASCII(),
which transforms non-ASCII decimal digits in a Unicode string to their
ASCII equivalents.
svn+ssh://pythondev@svn.python.org/python/trunk
................
r62425 | andrew.kuchling | 2008-04-21 03:45:57 +0200 (Mon, 21 Apr 2008) | 1 line
Comment typo
................
r62426 | mark.dickinson | 2008-04-21 03:55:50 +0200 (Mon, 21 Apr 2008) | 2 lines
Silence 'r may be used uninitialized' compiler warning.
................
r62427 | andrew.kuchling | 2008-04-21 04:08:00 +0200 (Mon, 21 Apr 2008) | 1 line
Markup fix
................
r62428 | andrew.kuchling | 2008-04-21 04:08:13 +0200 (Mon, 21 Apr 2008) | 1 line
Wording changes
................
r62429 | andrew.kuchling | 2008-04-21 04:14:24 +0200 (Mon, 21 Apr 2008) | 1 line
Add various items
................
r62434 | thomas.heller | 2008-04-21 15:46:55 +0200 (Mon, 21 Apr 2008) | 1 line
Fix typo.
................
r62435 | david.goodger | 2008-04-21 16:40:22 +0200 (Mon, 21 Apr 2008) | 1 line
corrections ("reStructuredText" is one word)
................
r62436 | david.goodger | 2008-04-21 16:43:33 +0200 (Mon, 21 Apr 2008) | 1 line
capitalization
................
r62441 | gregory.p.smith | 2008-04-21 19:46:40 +0200 (Mon, 21 Apr 2008) | 2 lines
explicitly flush after the ... since there wasn't a newline
................
r62444 | jeroen.ruigrok | 2008-04-21 22:15:39 +0200 (Mon, 21 Apr 2008) | 2 lines
Windows x64 also falls under VER_PLATFORM_WIN32_NT.
................
r62446 | gregory.p.smith | 2008-04-21 23:31:08 +0200 (Mon, 21 Apr 2008) | 3 lines
If sys.stdin is not a tty, fall back to default_getpass after printing
a warning instead of failing with a termios.error.
................
r62447 | mark.dickinson | 2008-04-22 00:32:24 +0200 (Tue, 22 Apr 2008) | 8 lines
test_math and test_cmath are failing on the FreeBSD 6.2 trunk buildbot,
apparently because tanh(-0.) loses the sign of zero on that platform.
If true, this is a bug in FreeBSD.
Added a configure test to verify this. I still need to figure out
how best to deal with this failure.
................
r62448 | amaury.forgeotdarc | 2008-04-22 00:35:30 +0200 (Tue, 22 Apr 2008) | 7 lines
Issue 2665: On Windows, sys.stderr does not contain a valid file when running without a console.
It seems to work, but will fail at the first flush.
This causes IDLE to crash when too many warnings are printed.
Will backport.
................
r62450 | benjamin.peterson | 2008-04-22 00:57:00 +0200 (Tue, 22 Apr 2008) | 2 lines
Fix Sphinx warnings
................
r62451 | mark.dickinson | 2008-04-22 02:54:27 +0200 (Tue, 22 Apr 2008) | 3 lines
Make configure test for tanh(-0.) == -0. committed in r62447 actually
work. (The test wasn't properly linked with libm. Sigh.)
................
r62452 | benjamin.peterson | 2008-04-22 04:16:03 +0200 (Tue, 22 Apr 2008) | 2 lines
Various io doc updates
................
r62453 | neal.norwitz | 2008-04-22 07:07:47 +0200 (Tue, 22 Apr 2008) | 1 line
Add Thomas Lee
................
r62454 | gregory.p.smith | 2008-04-22 10:08:41 +0200 (Tue, 22 Apr 2008) | 8 lines
Major improvements:
* Default to using /dev/tty for the password prompt and input before
falling back to sys.stdin and sys.stderr.
* Use sys.stderr instead of sys.stdout.
* print the 'password may be echoed' warning to stream used to display
the prompt rather than always sys.stderr.
* warn() with GetPassWarning when input may be echoed.
................
r62455 | gregory.p.smith | 2008-04-22 10:11:33 +0200 (Tue, 22 Apr 2008) | 2 lines
update the getpass entry
................
r62463 | amaury.forgeotdarc | 2008-04-22 23:14:41 +0200 (Tue, 22 Apr 2008) | 5 lines
Issue #2670: urllib2.build_opener() failed when two handlers
derive the same default base class.
Will backport.
................
r62465 | skip.montanaro | 2008-04-23 00:45:09 +0200 (Wed, 23 Apr 2008) | 3 lines
Factor in documentation changes from issue 1753732.
................
r62466 | gregory.p.smith | 2008-04-23 03:06:42 +0200 (Wed, 23 Apr 2008) | 2 lines
syntax fixup
................
r62469 | benjamin.peterson | 2008-04-23 22:38:06 +0200 (Wed, 23 Apr 2008) | 2 lines
#2673 Fix example typo in optparse docs
................
r62474 | martin.v.loewis | 2008-04-24 11:50:50 +0200 (Thu, 24 Apr 2008) | 2 lines
Add Guilherme Polo.
................
r62476 | martin.v.loewis | 2008-04-24 15:16:36 +0200 (Thu, 24 Apr 2008) | 3 lines
Remove Py_Refcnt, Py_Type, Py_Size, as they were added only
for backwards compatibility, yet 2.5 did not have them at all.
................
r62477 | martin.v.loewis | 2008-04-24 15:17:24 +0200 (Thu, 24 Apr 2008) | 2 lines
Fix typo.
................
r62478 | martin.v.loewis | 2008-04-24 15:18:03 +0200 (Thu, 24 Apr 2008) | 2 lines
Add Jesus Cea.
................
r62480 | amaury.forgeotdarc | 2008-04-24 20:07:05 +0200 (Thu, 24 Apr 2008) | 4 lines
Issue2681: the literal 0o8 was wrongly accepted, and evaluated as float(0.0).
This happened only when 8 is the first digit.
Credits go to Lukas Meuser.
................
r62485 | amaury.forgeotdarc | 2008-04-24 22:10:26 +0200 (Thu, 24 Apr 2008) | 5 lines
Disable gc when running test_trace, or we may record the __del__ of collected objects.
See http://mail.python.org/pipermail/python-checkins/2008-April/068633.html
the extra events perfectly match several calls to socket._fileobject.__del__()
................
r62492 | neal.norwitz | 2008-04-25 05:40:17 +0200 (Fri, 25 Apr 2008) | 1 line
Fix typo (now -> no)
................
r62497 | armin.rigo | 2008-04-25 11:35:18 +0200 (Fri, 25 Apr 2008) | 2 lines
A new crasher.
................
r62498 | thomas.heller | 2008-04-25 17:44:16 +0200 (Fri, 25 Apr 2008) | 1 line
Add from_buffer and from_buffer_copy class methods to ctypes types.
................
r62500 | mark.dickinson | 2008-04-25 18:59:09 +0200 (Fri, 25 Apr 2008) | 3 lines
Issue 2635: fix bug in the fix_sentence_endings option to textwrap.fill.
................
r62507 | benjamin.peterson | 2008-04-25 23:43:56 +0200 (Fri, 25 Apr 2008) | 2 lines
Allow test_import to work when it is invoked directly
................
r62513 | georg.brandl | 2008-04-26 20:31:07 +0200 (Sat, 26 Apr 2008) | 2 lines
#2691: document PyLong (s)size_t APIs, patch by Alexander Belopolsky.
................
r62514 | georg.brandl | 2008-04-26 20:32:17 +0200 (Sat, 26 Apr 2008) | 2 lines
Add missing return type to dealloc.
................
r62516 | alexandre.vassalotti | 2008-04-27 02:52:24 +0200 (Sun, 27 Apr 2008) | 2 lines
Fixed URL of PEP 205 in weakref's module docstring.
................
r62521 | georg.brandl | 2008-04-27 11:39:59 +0200 (Sun, 27 Apr 2008) | 2 lines
#2677: add note that not all functions may accept keyword args.
................
r62531 | georg.brandl | 2008-04-27 19:38:55 +0200 (Sun, 27 Apr 2008) | 2 lines
Use correct XHTML tags.
................
r62535 | benjamin.peterson | 2008-04-27 20:14:39 +0200 (Sun, 27 Apr 2008) | 2 lines
#2700 Document PyNumber_ToBase
................
r62545 | skip.montanaro | 2008-04-27 22:53:57 +0200 (Sun, 27 Apr 2008) | 1 line
minor wording changes, rewrap a few lines
................
r62546 | kurt.kaiser | 2008-04-27 23:07:41 +0200 (Sun, 27 Apr 2008) | 7 lines
Home / Control-A toggles between left margin and end of leading white
space. Patch 1196903 Jeff Shute.
M idlelib/PyShell.py
M idlelib/EditorWindow.py
M idlelib/NEWS.txt
................
r62548 | kurt.kaiser | 2008-04-27 23:38:05 +0200 (Sun, 27 Apr 2008) | 2 lines
Improved AutoCompleteWindow logic. Patch 2062 Tal Einat.
................
r62549 | kurt.kaiser | 2008-04-27 23:52:19 +0200 (Sun, 27 Apr 2008) | 4 lines
Autocompletion of filenames now support alternate separators, e.g. the
'/' char on Windows. Patch 2061 Tal Einat.
................
r62550 | skip.montanaro | 2008-04-28 00:49:56 +0200 (Mon, 28 Apr 2008) | 6 lines
A few small changes:
* The only exception we should catch when trying to import cStringIO is an
ImportError.
* Delete the function signatures embedded in the mk*temp docstrings.
* The tempdir global variable was initialized twice.
................
r62551 | skip.montanaro | 2008-04-28 00:52:02 +0200 (Mon, 28 Apr 2008) | 4 lines
Wrap some long paragraphs and include the default values for optional
function parameters.
................
r62553 | skip.montanaro | 2008-04-28 04:57:23 +0200 (Mon, 28 Apr 2008) | 7 lines
Minor cleanups:
* Avoid creating unused local variables where we can. Where we can't prefix
the unused variables with '_'.
* Avoid shadowing builtins where it won't change the external interface of a
function.
* Use None as default path arg to readmodule and readmodule_ex.
................
r62554 | skip.montanaro | 2008-04-28 04:59:45 +0200 (Mon, 28 Apr 2008) | 6 lines
Correct documentation to match implementation: "Class" instead of
"class_descriptor", "Function" instead of "function_descriptor". Note
default path value for readmodule*. Wrap some long paragraphs. Don't
mention 'inpackage' which isn't part of the public API.
................
r62555 | brett.cannon | 2008-04-28 05:23:50 +0200 (Mon, 28 Apr 2008) | 5 lines
Fix a bug introduced by the warnings rewrite where tracebacks were being
improperly indented.
Closes issue #2699.
................
r62556 | skip.montanaro | 2008-04-28 05:25:37 +0200 (Mon, 28 Apr 2008) | 2 lines
Wrap some long lines.
................
r62557 | skip.montanaro | 2008-04-28 05:27:53 +0200 (Mon, 28 Apr 2008) | 6 lines
Get rid of _test(), _main(), _debug() and _check(). Tests are no longer
needed (better set available in Lib/test/test_robotparser.py). Clean up a
few PEP 8 nits (compound statements on a single line, whitespace around
operators).
................
r62558 | brett.cannon | 2008-04-28 06:50:06 +0200 (Mon, 28 Apr 2008) | 3 lines
Rename the test_traceback_print() function to traceback_print() to prevent
test_capi from automatically calling the function.
................
r62559 | georg.brandl | 2008-04-28 07:16:30 +0200 (Mon, 28 Apr 2008) | 2 lines
Fix markup.
................
r62569 | amaury.forgeotdarc | 2008-04-28 23:07:06 +0200 (Mon, 28 Apr 2008) | 5 lines
test_sundry performs minimal tests (a simple import...) on modules that are not tested otherwise.
Some of them now have tests and can be removed.
Only 70 to go...
................
r62574 | andrew.kuchling | 2008-04-29 04:03:54 +0200 (Tue, 29 Apr 2008) | 1 line
Strip down SSL docs; I'm not managing to get test programs working, so I'll just give a minimal description
................
r62577 | martin.v.loewis | 2008-04-29 08:10:53 +0200 (Tue, 29 Apr 2008) | 2 lines
Add Rodrigo and Heiko.
................
r62593 | nick.coghlan | 2008-04-30 16:23:36 +0200 (Wed, 30 Apr 2008) | 1 line
Update command line usage documentation to reflect 2.6 changes (also includes some minor cleanups). Addresses TODO list issue 2258
................
r62595 | andrew.kuchling | 2008-04-30 18:19:55 +0200 (Wed, 30 Apr 2008) | 1 line
Typo fix
................
r62604 | benjamin.peterson | 2008-04-30 23:03:58 +0200 (Wed, 30 Apr 2008) | 2 lines
make test_support's captured_output a bit more robust when exceptions happen
................
r62605 | georg.brandl | 2008-04-30 23:08:42 +0200 (Wed, 30 Apr 2008) | 2 lines
#1748: use functools.wraps instead of rolling own metadata update.
................
r62606 | benjamin.peterson | 2008-04-30 23:25:55 +0200 (Wed, 30 Apr 2008) | 2 lines
Remove some from __future__ import with_statements
................
r62608 | benjamin.peterson | 2008-05-01 00:03:36 +0200 (Thu, 01 May 2008) | 2 lines
Fix typo in whatsnew
................
r62616 | georg.brandl | 2008-05-01 20:24:32 +0200 (Thu, 01 May 2008) | 2 lines
Fix synopsis.
................
r62626 | brett.cannon | 2008-05-02 04:25:09 +0200 (Fri, 02 May 2008) | 6 lines
Fix a backwards-compatibility mistake where a new optional argument for
warnings.showwarning() was being used. This broke pre-existing replacements for
the function since they didn't support the extra argument.
Closes issue 2705.
................
r62627 | gregory.p.smith | 2008-05-02 09:26:52 +0200 (Fri, 02 May 2008) | 20 lines
This should fix issue2632. A long description of the two competing
problems is in the bug report (one old, one recently introduced trying
to fix the old one). In short:
buffer data during socket._fileobject.read() and readlines() within a
cStringIO object instead of a [] of str()s returned from the recv()
call.
This prevents excessive memory use due to the size parameter being
passed to recv() being grossly larger than the actual size of the data
returned *and* prevents excessive cpu usage due to looping in python
calling recv() with a very tiny size value if min() is used as the
previous memory-use bug "fix" did.
It also documents what the socket._fileobject._rbufsize member is
actually used for.
This is a candidate for back porting to 2.5.
................
r62636 | mark.hammond | 2008-05-02 14:48:15 +0200 (Fri, 02 May 2008) | 2 lines
#2581: Vista UAC/elevation support for bdist_wininst
................
r62638 | facundo.batista | 2008-05-02 19:39:00 +0200 (Fri, 02 May 2008) | 3 lines
Fixed some test structures. Thanks Mark Dickinson.
................
r62644 | ronald.oussoren | 2008-05-02 21:45:11 +0200 (Fri, 02 May 2008) | 7 lines
Fix for issue #2573: Can't change the framework name on OS X builds
This introduces a new configure option: --with-framework-name=NAME
(defaulting to 'Python'). This allows you to install several copies
of the Python framework with different names (such as a normal build
and a debug build).
................
r62645 | ronald.oussoren | 2008-05-02 21:58:56 +0200 (Fri, 02 May 2008) | 2 lines
Finish fix for issue2573, previous patch was incomplete.
................
r62647 | martin.v.loewis | 2008-05-02 23:30:20 +0200 (Fri, 02 May 2008) | 13 lines
Merged revisions 62263-62646 via svnmerge from
svn+ssh://pythondev@svn.python.org/sandbox/trunk/2to3/lib2to3
........
r62470 | david.wolever | 2008-04-24 02:11:07 +0200 (Do, 24 Apr 2008) | 3 lines
Fixed up and applied the patch for #2431 -- speeding up 2to3 with a lookup table.
........
r62646 | martin.v.loewis | 2008-05-02 23:29:27 +0200 (Fr, 02 Mai 2008) | 2 lines
Fix whitespace.
........
................
r62648 | ronald.oussoren | 2008-05-02 23:42:35 +0200 (Fri, 02 May 2008) | 4 lines
Fix for #1905: PythonLauncher not working correctly on OSX 10.5/Leopard
This fixes both Python Launchar and the terminalcommand module.
................
r62651 | ronald.oussoren | 2008-05-02 23:54:56 +0200 (Fri, 02 May 2008) | 2 lines
Fix for issue #2520 (cannot import macerrors)
................
r62652 | benjamin.peterson | 2008-05-03 00:12:58 +0200 (Sat, 03 May 2008) | 2 lines
capitalization nit for reStructuredText
................
r62653 | brett.cannon | 2008-05-03 03:02:41 +0200 (Sat, 03 May 2008) | 2 lines
Fix some indentation errors.
................
r62656 | brett.cannon | 2008-05-03 05:19:39 +0200 (Sat, 03 May 2008) | 6 lines
Fix the C implementation of 'warnings' to infer the filename of the module that
raised an exception properly when __file__ is not set, __name__ == '__main__',
and sys.argv[0] is a false value.
Closes issue2743.
................
r62661 | amaury.forgeotdarc | 2008-05-03 14:21:13 +0200 (Sat, 03 May 2008) | 8 lines
In test_io, StatefulIncrementalDecoderTest was not part of the test suite.
And of course, the test failed:
a bytearray was used without reason in io.TextIOWrapper.tell().
The difference is that iterating over bytes (i.e. str in python2.6) returns 1-char bytes,
whereas bytearrays yield integers.
This code should still work with python3.0
................
r62663 | benjamin.peterson | 2008-05-03 17:56:42 +0200 (Sat, 03 May 2008) | 2 lines
The compiling struct is now passed around to all AST helpers (see issue 2720)
................
r62680 | benjamin.peterson | 2008-05-03 23:35:18 +0200 (Sat, 03 May 2008) | 2 lines
Moved testing of builtin types out of test_builtin and into type specific modules
................
r62686 | mark.dickinson | 2008-05-04 04:25:46 +0200 (Sun, 04 May 2008) | 4 lines
Make sure that Context traps and flags dictionaries have values 0 and 1
(as documented) rather than True and False.
................
r62687 | benjamin.peterson | 2008-05-04 05:05:49 +0200 (Sun, 04 May 2008) | 2 lines
Fix typo in whatsnew
................
r62696 | georg.brandl | 2008-05-04 11:15:04 +0200 (Sun, 04 May 2008) | 2 lines
#2752: wrong meaning of '' for socket host.
................
r62699 | christian.heimes | 2008-05-04 13:50:53 +0200 (Sun, 04 May 2008) | 1 line
Added note that Python requires at least Win2k SP4
................
r62700 | gerhard.haering | 2008-05-04 14:59:57 +0200 (Sun, 04 May 2008) | 3 lines
SQLite requires 64-bit integers in order to build. So the whole HAVE_LONG_LONG
#ifdefing was useless.
................
r62701 | gerhard.haering | 2008-05-04 15:15:12 +0200 (Sun, 04 May 2008) | 3 lines
Applied sqliterow-richcmp.diff patch from Thomas Heller in Issue2152. The
sqlite3.Row type is now correctly hashable.
................
r62702 | gerhard.haering | 2008-05-04 15:42:44 +0200 (Sun, 04 May 2008) | 5 lines
Implemented feature request 2157: Converter names are cut off at '('
characters. This avoids the common case of something like 'NUMBER(10)' not
being parsed as 'NUMBER', like expected. Also corrected the docs about
converter names being case-sensitive. They aren't any longer.
................
r62703 | georg.brandl | 2008-05-04 17:45:05 +0200 (Sun, 04 May 2008) | 2 lines
#2757: Remove spare newline.
................
r62711 | benjamin.peterson | 2008-05-04 21:10:02 +0200 (Sun, 04 May 2008) | 2 lines
Fix typo in bugs.rst
................
Gregory P. Smith
Neil Schemenauer
Mark Dickinson
Gregory P. Smith
Zackery Spytz
Serhiy Storchaka
Brett Cannon
Martin Panter
Zachary Ware
Andrew Svetlov
Mark Dickinson
Alexander Belopolsky
Ezio Melotti
Benjamin Peterson
Martin v. Löwis
Christian Heimes