Commit Graph

192 Commits

Author SHA1 Message Date
Armin Rigo 7ccbca93a2 Forward-port of r52136,52138: a review of overflow-detecting code.
* unified the way intobject, longobject and mystrtoul handle
  values around -sys.maxint-1.

* in general, trying to entierely avoid overflows in any computation
  involving signed ints or longs is extremely involved.  Fixed a few
  simple cases where a compiler might be too clever (but that's all
  guesswork).

* more overflow checks against bad data in marshal.c.

* 2.5 specific: fixed a number of places that were still confusing int
  and Py_ssize_t.  Some of them could potentially have caused
  "real-world" breakage.

* list.pop(x): fixing overflow issues on x was messy.  I just reverted
  to PyArg_ParseTuple("n"), which does the right thing.  (An obscure
  test was trying to give a Decimal to list.pop()... doesn't make
  sense any more IMHO)

* trying to write a few tests...
2006-10-04 12:17:45 +00:00
Neal Norwitz 8a87f5d37e Patch #1538606, Patch to fix __index__() clipping.
I modified this patch some by fixing style, some error checking, and adding
XXX comments.  This patch requires review and some changes are to be expected.
I'm checking in now to get the greatest possible review and establish a
baseline for moving forward.  I don't want this to hold up release if possible.
2006-08-12 17:03:09 +00:00
Neal Norwitz c09efa8444 Move the initialization of size_a down below the check for a being NULL.
Reported by Klocwork #106
2006-07-23 07:53:14 +00:00
Neal Norwitz ef02b9e144 a & b were dereffed above, so they are known to be valid pointers.
z is known to be NULL, nothing to DECREF.

Reported by Klockwork, #107.
2006-07-16 02:00:32 +00:00
Tim Peters 9faa3eda6b PyLong_FromString(): Continued fraction analysis (explained in
a new comment) suggests there are almost certainly large input
integers in all non-binary input bases for which one Python digit
too few is initally allocated to hold the final result.  Instead
of assert-failing when that happens, allocate more space.  Alas,
I estimate it would take a few days to find a specific such case,
so this isn't backed up by a new test (not to mention that such
a case may take hours to run, since conversion time is quadratic
in the number of digits, and preliminary attempts suggested that
the smallest such inputs contain at least a million digits).
2006-05-30 15:53:34 +00:00
Tim Peters d89fc22dc6 Patch #1494387: SVN longobject.c compiler warnings
The SIGCHECK macro defined here has always been bizarre, but
it apparently causes compiler warnings on "Sun Studio 11".
I believe the warnings are bogus, but it doesn't hurt to make
the macro definition saner.

Bugfix candidate (but I'm not going to bother).
2006-05-25 22:28:46 +00:00
Bob Ippolito a85bf202ac Faster path for PyLong_FromLongLong, using PyLong_FromLong algorithm 2006-05-25 18:20:23 +00:00
Tim Peters da53afa1b0 A new table to help string->integer conversion was added yesterday to
both mystrtoul.c and longobject.c.  Share the table instead.  Also
cut its size by 64 entries (they had been used for an inscrutable
trick originally, but the code no longer tries to use that trick).
2006-05-25 17:34:03 +00:00
Tim Peters 696cf43b58 Heavily fiddled variant of patch #1442927: PyLong_FromString optimization.
``long(str, base)`` is now up to 6x faster for non-power-of-2 bases.  The
largest speedup is for inputs with about 1000 decimal digits.  Conversion
from non-power-of-2 bases remains quadratic-time in the number of input
digits (it was and remains linear-time for bases 2, 4, 8, 16 and 32).

Speedups at various lengths for decimal inputs, comparing 2.4.3 with
current trunk.  Note that it's actually a bit slower for 1-digit strings:

  len  speedup
 ----  -------
   1     -4.5%
   2      4.6%
   3      8.3%
   4     12.7%
   5     16.9%
   6     28.6%
   7     35.5%
   8     44.3%
   9     46.6%
  10     55.3%
  11     65.7%
  12     77.7%
  13     73.4%
  14     75.3%
  15     85.2%
  16    103.0%
  17     95.1%
  18    112.8%
  19    117.9%
  20    128.3%
  30    174.5%
  40    209.3%
  50    236.3%
  60    254.3%
  70    262.9%
  80    295.8%
  90    297.3%
 100    324.5%
 200    374.6%
 300    403.1%
 400    391.1%
 500    388.7%
 600    440.6%
 700    468.7%
 800    498.0%
 900    507.2%
1000    501.2%
2000    450.2%
3000    463.2%
4000    452.5%
5000    440.6%
6000    439.6%
7000    424.8%
8000    418.1%
9000    417.7%
2006-05-24 21:10:40 +00:00
Neal Norwitz c6a989ac3a Fix problems found by Coverity.
longobject.c: also fix an ssize_t problem
  <a> could have been NULL, so hoist the size calc to not use <a>.

_ssl.c: under fail: self is DECREF'd, but it would have been NULL.

_elementtree.c: delete self if there was an error.

_csv.c: I'm not sure if lineterminator could have been anything other than
a string.  However, other string method calls are checked, so check this
one too.
2006-05-10 06:57:58 +00:00
Skip Montanaro 429433b30b C++ compiler cleanup: bunch-o-casts, plus use of unsigned loop index var in a couple places 2006-04-18 00:35:43 +00:00
Thomas Wouters 9cb28bea04 Fix int() and long() to repr() their argument when formatting the exception,
to avoid confusing situations like:

>>> int("")
ValueError: invalid literal for int():
>>> int("2\n\n2")
ValueError: invalid literal for int(): 2

2

Also report the base used, to avoid:

ValueError: invalid literal for int(): 2

They now report:

>>> int("")
ValueError: invalid literal for int() with base 10: ''
>>> int("2\n\n2")
ValueError: invalid literal for int() with base 10: '2\n\n2'
>>> int("2", 2)
ValueError: invalid literal for int() with base 2: '2'

(Reporting the base could be avoided when base is 10, which is the default,
but hrm.) Another effect of these changes is that the errormessage can be
longer; before, it was cut off at about 250 characters. Now, it can be up to
four times as long, as the unrepr'ed string is cut off at 200 characters,
instead.

No tests were added or changed, since testing for exact errormsgs is (pardon
the pun) somewhat errorprone, and I consider not testing the exact text
preferable. The actually changed code is tested frequent enough in the
test_builtin test as it is (120 runs for each of ints and longs.)
2006-04-11 23:50:33 +00:00
Anthony Baxter 377be11ee1 More C++-compliance. Note especially listobject.c - to get C++ to accept the
PyTypeObject structures, I had to make prototypes for the functions, and
move the structure definition ahead of the functions. I'd dearly like a better
way to do this - to change this would make for a massive set of changes to
the codebase.

There's still some warnings - this is purely to get rid of errors first.
2006-04-11 06:54:30 +00:00
Martin v. Löwis 0bc2ab9a20 Patch #837242: id() for large ptr should return a long. 2006-04-10 20:28:17 +00:00
Martin v. Löwis c48c8db110 Add PY_SSIZE_T_MIN, as suggested by Ralf W. Grosse-Kunstleve. 2006-04-05 18:21:17 +00:00
Georg Brandl 347b30042b Remove unnecessary casts in type object initializers. 2006-03-30 11:57:00 +00:00
Armin Rigo 12bec1b985 fix a comment. 2006-03-28 19:27:56 +00:00
Guido van Rossum 38fff8c4e4 Checking in the code for PEP 357.
This was mostly written by Travis Oliphant.
I've inspected it all; Neal Norwitz and MvL have also looked at it
(in an earlier incarnation).
2006-03-07 18:50:55 +00:00
Hye-Shik Chang 4af5c8cee4 SF #1444030: Fix several potential defects found by Coverity.
(reviewed by Neal Norwitz)
2006-03-07 15:39:21 +00:00
Martin v. Löwis 15e62742fa Revert backwards-incompatible const changes. 2006-02-27 16:46:16 +00:00
Martin v. Löwis 18e165558b Merge ssize_t branch. 2006-02-15 17:27:45 +00:00
Thomas Wouters 553489ab1d As discussed on python-dev, silence three gcc-4.0.x warnings, using assert()
to protect against actual uninitialized usage.

Objects/longobject.c: In function ‘PyLong_AsDouble’:
Objects/longobject.c:655: warning: ‘e’ may be used uninitialized in this function

Objects/longobject.c: In function ‘long_true_divide’:
Objects/longobject.c:2263: warning: ‘aexp’ may be used uninitialized in this function
Objects/longobject.c:2263: warning: ‘bexp’ may be used uninitialized in this function
2006-02-01 21:32:04 +00:00
Jeremy Hylton af68c874a6 Add const to several API functions that take char *.
In C++, it's an error to pass a string literal to a char* function
without a const_cast().  Rather than require every C++ extension
module to put a cast around string literals, fix the API to state the
const-ness.

I focused on parts of the API where people usually pass literals:
PyArg_ParseTuple() and friends, Py_BuildValue(), PyMethodDef, the type
slots, etc.  Predictably, there were a large set of functions that
needed to be fixed as a result of these changes.  The most pervasive
change was to make the keyword args list passed to
PyArg_ParseTupleAndKewords() to be a const char *kwlist[].

One cast was required as a result of the changes:  A type object
mallocs the memory for its tp_doc slot and later frees it.
PyTypeObject says that tp_doc is const char *; but if the type was
created by type_new(), we know it is safe to cast to char *.
2005-12-10 18:50:16 +00:00
Tim Peters de7990b8af SF bug #1238681: freed pointer is used in longobject.c:long_pow().
In addition, long_pow() skipped a necessary (albeit extremely unlikely
to trigger) error check when converting an int modulus to long.

Alas, I was unable to write a test case that crashed due to either
cause.

Bugfix candidate.
2005-07-17 23:45:23 +00:00
Raymond Hettinger 3296e696db SF bug #1224347: int/long unification and hex()
Hex longs now print with lowercase letters like their int counterparts.
2005-06-29 23:29:56 +00:00
Brett Cannon c3647ac93e Make subclasses of int, long, complex, float, and unicode perform type
conversion using the proper magic slot (e.g., __int__()).  Also move conversion
code out of PyNumber_*() functions in the C API into the nb_* function.

Applied patch #1109424.  Thanks Walter Doewald.
2005-04-26 03:45:26 +00:00
Martin v. Löwis 6ce7ed23d0 Revert previous checkin on getargs 'L' code. Try to convert all
numbers in PyLong_AsLongLong, and update test suite accordingly.
Backported to 2.4.
2005-03-03 12:26:35 +00:00
Martin v. Löwis 729d47db09 Patch #1024670: Support int objects in PyLong_AsUnsignedLong[Mask]. 2004-09-20 06:17:46 +00:00
Tim Peters cd97da3b1d long_pow(): Fix more instances of leaks in error cases.
Bugfix candidate -- although long_pow() is so different now I doubt a
patch would apply to 2.3.
2004-08-30 02:58:59 +00:00
Tim Peters 47e52ee0c5 SF patch 936813: fast modular exponentiation
This checkin is adapted from part 2 (of 3) of Trevor Perrin's patch set.

BACKWARD INCOMPATIBILITY:  SHIFT must now be divisible by 5.  AFAIK,
nobody will care.  long_pow() could be complicated to worm around that,
if necessary.

long_pow():
  - BUGFIX:  This leaked the base and power when the power was negative
    (and so the computation delegated to float pow).
  - Instead of doing right-to-left exponentiation, do left-to-right.  This
    is more efficient for small bases, which is the common case.
  - In addition, if the exponent is large (more than FIVEARY_CUTOFF
    digits), precompute [a**i % c for i in range(32)], and go left to
    right 5 bits at a time.
l_divmod():
  - The signature changed so that callers who don't want the quotient,
    or don't want the remainder, can pass NULL in the slot they don't
    want.  This saves them from having to declare a vrbl for unwanted
    stuff, and remembering to decref it.
long_mod(), long_div(), long_classic_div():
  - Adjust to new l_divmod() signature, and simplified as a result.
2004-08-30 02:44:38 +00:00
Tim Peters 0973b99e1c SF patch 936813: fast modular exponentiation
This checkin is adapted from part 1 (of 3) of Trevor Perrin's patch set.

x_mul()
  - sped a little by optimizing the C
  - sped a lot (~2X) if it's doing a square; note that long_pow() squares
    often
k_mul()
  - more cache-friendly now if it's doing a square
KARATSUBA_CUTOFF
  - boosted; gradeschool mult is quicker now, and it may have been too low
    for many platforms anyway
KARATSUBA_SQUARE_CUTOFF
  - new
  - since x_mul is a lot faster at squaring now, the point at which
    Karatsuba pays for squaring is much higher than for general mult
2004-08-29 22:16:50 +00:00
Raymond Hettinger f466793fcc SF patch 703666: Several objects don't decref tmp on failure in subtype_new
Submitted By: Christopher A. Craig

Fillin some missing decrefs.
2003-06-28 20:04:25 +00:00
Tim Peters c7bc0b98e7 SF patch 730594: assert from longobject.c, line 1215.
Some version of gcc in the "RTEMS port running on the Coldfire (m5200)
processor" generates bad code for a loop in long_from_binary_base(),
comparing the wrong half of an int to a short.  The patch changes the
decl of the short temp to be an int temp instead.  This "simplifies"
the code enough that gcc no longer blows it.
2003-05-05 20:39:43 +00:00
Jeremy Hylton fc61f9a36e Silence compiler warnings in VC 7. 2003-05-01 21:31:53 +00:00
Thomas Heller a4ea603b05 SF # 595026: support for masks in getargs.c.
New functions:
  unsigned long PyInt_AsUnsignedLongMask(PyObject *);
  unsigned PY_LONG_LONG) PyInt_AsUnsignedLongLongMask(PyObject *);
  unsigned long PyLong_AsUnsignedLongMask(PyObject *);
  unsigned PY_LONG_LONG) PyLong_AsUnsignedLongLongMask(PyObject *);

New and changed format codes:

b unsigned char 0..UCHAR_MAX
B unsigned char none **
h unsigned short 0..USHRT_MAX
H unsigned short none **
i int INT_MIN..INT_MAX
I * unsigned int 0..UINT_MAX
l long LONG_MIN..LONG_MAX
k * unsigned long none
L long long LLONG_MIN..LLONG_MAX
K * unsigned long long none

Notes:

* New format codes.

** Changed from previous "range-and-a-half" to "none"; the
range-and-a-half checking wasn't particularly useful.

New test test_getargs2.py, to verify all this.
2003-04-17 18:55:45 +00:00
Martin v. Löwis b9a0f91218 Rename LONG_LONG to PY_LONG_LONG. Fixes #710285. 2003-03-29 10:06:18 +00:00
Neal Norwitz d5a65a77cf Fix SF bug #689659, 64-bit int and long hash keys incompatible
On a 64-bit machine, a dictionary could contain duplicate int/long keys
if the value was > 2**32.
2003-02-23 23:11:41 +00:00
Guido van Rossum 004a65c9b1 _PyLong_Sign(): remove an assert that needed a variable ndigits that
wasn't used outside the assert (and hence caused a compiler warning
about an unused variable in NDEBUG mode).  The assert wasn't very
useful any more.

_PyLong_NumBits(): moved the calculation of ndigits after asserting
that v != NULL.
2003-02-03 15:28:19 +00:00
Tim Peters 1a3b19a6e9 long_from_binary_base(): Sped this a little by computing the # of bits
needed outside the first loop.
2003-02-02 17:33:53 +00:00
Tim Peters efb9625b81 Tightened a too-generous assert. 2003-02-02 08:05:32 +00:00
Tim Peters bf2674be0e long(string, base) now takes time linear in len(string) when base is a
power of 2.  Enabled the tail end of test_long() in pickletester.py
because it no longer takes forever when run from test_pickle.py.
2003-02-02 07:51:32 +00:00
Tim Peters ee1a53cbb1 cPickle.c: Full support for the new LONG1 and LONG4. Added comments.
Assorted code cleanups; e.g., sizeof(char) is 1 by definition, so there's
no need to do things like multiply by sizeof(char) in hairy malloc
arguments.  Fixed an undetected-overflow bug in readline_file().

longobject.c:  Fixed a really stupid bug in the new _PyLong_NumBits.

pickle.py:  Fixed stupid bug in save_long():  When proto is 2, it
wrote LONG1 or LONG4, but forgot to return then -- it went on to
append the proto 1 LONG opcode too.
Fixed equally stupid cancelling bugs in load_long1() and
load_long4():  they *returned* the unpickled long instead of pushing
it on the stack.  The return values were ignored.  Tests passed
before only because save_long() pickled the long twice.

Fixed bugs in encode_long().

Noted that decode_long() is quadratic-time despite our hopes,
because long(string, 16) is still quadratic-time in len(string).
It's hex() that's linear-time.  I don't know a way to make decode_long()
linear-time in Python, short of maybe transforming the 256's-complement
bytes into marshal's funky internal format, and letting marshal decode
that.  It would be more valuable to make long(string, 16) linear time.

pickletester.py:  Added a global "protocols" vector so tests can try
all the protocols in a sane way.  Changed test_ints() and test_unicode()
to do so.  Added a new test_long(), but the tail end of it is disabled
because it "takes forever" under pickle.py (but runs very quickly under
cPickle:  cPickle proto 2 for longs is linear-time).
2003-02-02 02:57:53 +00:00
Tim Peters 08a1d9cafc Squash compiler wng about signed/unsigned comparison mismatch. 2003-01-31 21:45:13 +00:00
Tim Peters 5b8132ffa3 _PyLong_NumBits(): The definition of this was too specific to the quirky
needs of pickling longs.  Backed off to a definition that's much easier
to understand.  The pickler will have to work a little harder, but other
uses are more likely to be correct <0.5 wink>.

_PyLong_Sign():  New teensy function to characterize a long, as to <0, ==0,
or >0.
2003-01-31 15:52:05 +00:00
Guido van Rossum 5d9113d8be Implement appropriate __getnewargs__ for all immutable subclassable builtin
types.  The special handling for these can now be removed from save_newobj().
Add some testing for this.

Also add support for setting the 'fast' flag on the Python Pickler class,
which suppresses use of the memo.
2003-01-29 17:58:45 +00:00
Tim Peters baefd9e552 Added new private API function _PyLong_NumBits. This will be used at the
start for the C implemention of new pickle LONG1 and LONG4 opcodes (the
linear-time way to pickle a long is to call _PyLong_AsByteArray, but
the caller has no idea how big an array to allocate, and correct
calculation is a bit subtle).
2003-01-28 20:37:45 +00:00
Neil Schemenauer 0df295889c Consolidate the int and long sequence repeat code. Before the change,
integers checked for integer overflow but longs did not.
2002-12-30 20:19:02 +00:00
Walter Dörwald f171540ab8 Change int() so that passing a string, unicode, float or long argument
that is outside the integer range no longer raises OverflowError, but
returns a long object instead.

This fixes SF bug http://www.python.org/sf/635115
2002-11-19 20:49:15 +00:00
Walter Dörwald 07e147667c Make int("...") return a long if an int would overflow.
Also remove the 512 character limitation for int(u"...") and long(u"...").

This closes SF bug #629989.
2002-11-06 16:15:14 +00:00
Skip Montanaro d581d7792b replace thread state objects' ticker and checkinterval fields with two
globals, _Py_Ticker and _Py_CheckInterval.  This also implements Jeremy's
shortcut in Py_AddPendingCall that zeroes out _Py_Ticker.  This allows the
test in the main loop to only test a single value.

The gory details are at

    http://python.org/sf/602191
2002-09-03 20:10:45 +00:00