Commit Graph

168 Commits

Author SHA1 Message Date
Armin Rigo 7ccbca93a2 Forward-port of r52136,52138: a review of overflow-detecting code.
* unified the way intobject, longobject and mystrtoul handle
  values around -sys.maxint-1.

* in general, trying to entierely avoid overflows in any computation
  involving signed ints or longs is extremely involved.  Fixed a few
  simple cases where a compiler might be too clever (but that's all
  guesswork).

* more overflow checks against bad data in marshal.c.

* 2.5 specific: fixed a number of places that were still confusing int
  and Py_ssize_t.  Some of them could potentially have caused
  "real-world" breakage.

* list.pop(x): fixing overflow issues on x was messy.  I just reverted
  to PyArg_ParseTuple("n"), which does the right thing.  (An obscure
  test was trying to give a Decimal to list.pop()... doesn't make
  sense any more IMHO)

* trying to write a few tests...
2006-10-04 12:17:45 +00:00
Neal Norwitz 99dfe3c411 Prevent memory leak on error.
Reported by Klocwork #36
2006-08-02 04:27:11 +00:00
Neal Norwitz 98a96004f9 Fix memory leaks spotted by Klocwork #37. 2006-07-23 07:57:11 +00:00
Neal Norwitz b59d08c2fb Fix more memory allocation issues found with failmalloc. 2006-07-22 16:20:49 +00:00
Neal Norwitz 0f8b31a2da Fix bug #1512695: cPickle.loads could crash if it was interrupted with
a KeyboardInterrupt since PyTuple_Pack was passed a NULL.

Will backport.
2006-06-28 06:28:31 +00:00
Georg Brandl 96a8c3954c Make use of METH_O and METH_NOARGS where possible.
Use Py_UnpackTuple instead of PyArg_ParseTuple where possible.
2006-05-29 21:04:52 +00:00
Richard Jones 7b9558d37d Conversion of exceptions over from faked-up classes to new-style C types. 2006-05-27 12:29:24 +00:00
Georg Brandl 684fd0c8ec Replace PyObject_CallFunction calls with only object args
with PyObject_CallFunctionObjArgs, which is 30% faster.
2006-05-25 19:15:31 +00:00
Georg Brandl de9b624fb9 Bug #1473625: stop cPickle making float dumps locale dependent in protocol 0.
On the way, add a decorator to test_support to facilitate running single
test functions in different locales with automatic cleanup.
2006-04-30 11:13:56 +00:00
Thomas Wouters c6e55068ca Use Py_VISIT in all tp_traverse methods, instead of traversing manually or
using a custom, nearly-identical macro. This probably changes how some of
these functions are compiled, which may result in fractionally slower (or
faster) execution. Considering the nature of traversal, visiting much of the
address space in unpredictable patterns, I'd argue the code readability and
maintainability is well worth it ;P
2006-04-15 21:47:09 +00:00
Thomas Wouters edf17d8798 Use Py_CLEAR instead of in-place DECREF/XDECREF or custom macros, for
tp_clear methods.
2006-04-15 17:28:34 +00:00
Neal Norwitz b183a25c29 Fix some warnings on HP-UX when using cc/aCC 2006-04-10 01:03:32 +00:00
Martin v. Löwis 15e62742fa Revert backwards-incompatible const changes. 2006-02-27 16:46:16 +00:00
Martin v. Löwis 18e165558b Merge ssize_t branch. 2006-02-15 17:27:45 +00:00
Neal Norwitz 1ac754fa10 Check return result from Py_InitModule*(). This API can fail.
Probably should be backported.
2006-01-19 06:09:39 +00:00
Jeremy Hylton af68c874a6 Add const to several API functions that take char *.
In C++, it's an error to pass a string literal to a char* function
without a const_cast().  Rather than require every C++ extension
module to put a cast around string literals, fix the API to state the
const-ness.

I focused on parts of the API where people usually pass literals:
PyArg_ParseTuple() and friends, Py_BuildValue(), PyMethodDef, the type
slots, etc.  Predictably, there were a large set of functions that
needed to be fixed as a result of these changes.  The most pervasive
change was to make the keyword args list passed to
PyArg_ParseTupleAndKewords() to be a const char *kwlist[].

One cast was required as a result of the changes:  A type object
mallocs the memory for its tp_doc slot and later frees it.
PyTypeObject says that tp_doc is const char *; but if the type was
created by type_new(), we know it is safe to cast to char *.
2005-12-10 18:50:16 +00:00
Raymond Hettinger a6b45cc31d Eliminate the deprecated option to return None instead of a tuple of arguments in __reduce__(). 2004-12-07 07:05:57 +00:00
Martin v. Löwis 544f1195b1 Patch #995766: Keyword argument support in cPickle. 2004-07-27 05:22:33 +00:00
Martin v. Löwis 737ea82a5a Patch #774665: Make Python LC_NUMERIC agnostic. 2004-06-08 18:52:54 +00:00
Christian Tismer 2460c62152 made cPickle fall back to the copy_reg/reduce protocol,
if a function cannot be stored as global.
This is for compatibility with pickle.py .
2004-02-26 16:21:45 +00:00
Raymond Hettinger 8ae4689657 Simplify and speedup uses of Py_BuildValue():
* Py_BuildValue("(OOO)",a,b,c)  -->  PyTuple_Pack(3,a,b,c)
* Py_BuildValue("()",a)         -->  PyTuple_New(0)
* Py_BuildValue("O", a)         -->  Py_INCREF(a)
2003-10-12 19:09:37 +00:00
Jeremy Hylton fff093fa7f The Unpickler forget about its find_class attribute. 2003-07-11 19:42:49 +00:00
Jeremy Hylton f00368f9ed Remove many blanket try/except clauses.
SF bug [ 751276 ] cPickle doesn't raise error, pickle does (recursiondepth)

Most of the calls to PyErr_Clear() were intended to catch & clear an
attribute error and try something different.  Guard all those cases
with a PyErr_ExceptionMatches() and fail if some other error
occurred.  The other error is likely a bug in the user code.

This is basically the C equivalent of changing "except:" to
"except AttributeError:"
2003-06-16 20:19:49 +00:00
Tim Peters 3cfe75470d PyType_Ready(): Complain if the type is a base type, and gc'able, and
tp_free is NULL or PyObject_Del at the end.  Because it's a base type
it must call tp_free in its dealloc function, and because it's gc'able
it must not call PyObject_Del.

inherit_slots():  Don't inherit tp_free unless the type and its base
agree about whether they're gc'able.  If the type is gc'able and the
base is not, and the base uses the default PyObject_Del for its
tp_free, give the type PyObject_GC_Del for its tp_free (the appropriate
default for a gc'able type).

cPickle.c:  The Pickler and Unpickler types claim to be base classes
and gc'able, but their dealloc functions didn't call tp_free.
Repaired that.  Also call PyType_Ready() on these typeobjects, so
that the correct (PyObject_GC_Del) default memory-freeing function
gets plugged into these types' tp_free slots.
2003-05-21 21:29:48 +00:00
Jeremy Hylton 7b5ce7f25a Make Unpickler objects colletable.
Bugfix candidate.
2003-04-09 21:25:30 +00:00
Jeremy Hylton 4cf6319cd2 Make Picklers collectable.
Bug fix candidate.
2003-04-09 21:05:12 +00:00
Tim Peters 9905b943f7 New private API functions _PyFloat_{Pack,Unpack}(4,8}. This is a
refactoring to get all the duplicates of this delicate code out of the
cPickle and struct modules.
2003-03-20 20:53:32 +00:00
Tim Peters d50ade68ec SF bug 705836: struct.pack of floats in non-native endian order
pack_float, pack_double, save_float:  All the routines for creating
IEEE-format packed representations of floats and doubles simply ignored
that rounding can (in rare cases) propagate out of a long string of
1 bits.  At worst, the end-off carry can (by mistake) interfere with
the exponent value, and then unpacking yields a result wrong by a factor
of 2.  In less severe cases, it can end up losing more low-order bits
than intended, or fail to catch overflow *caused* by rounding.

Bugfix candidate, but I already backported this to 2.2.

In 2.3, this code remains in severe need of refactoring.
2003-03-20 18:32:13 +00:00
Guido van Rossum b289b87a37 Use __reduce_ex__. 2003-02-19 01:45:13 +00:00
Tim Peters b9ce7cd8b8 save_global(): Trying to resolve module.name can fail for two
reasons:  importing module can fail, or the attribute lookup
module.name can fail.  We were giving the same error msg for
both cases, making it needlessly hard to guess what went wrong.
These cases give different error msgs now.
2003-02-18 20:50:45 +00:00
Neal Norwitz 0ae4c4a823 Make 2 module variables static. Assuming this is correct. 2003-02-15 15:07:17 +00:00
Tim Peters 080c88b912 cPickle.c, load_build(): Taught cPickle how to pick apart
the optional proto 2 slot state.

pickle.py, load_build():  CAUTION:  Noted that cPickle's
load_build and pickle's load_build really don't do the same
things with the state, and didn't before this patch either.
cPickle never tries to do .update(), and has no backoff if
instance.__dict__ can't be retrieved.  There are no tests
that can tell the difference, and part of what cPickle's
load_build() did looked accidental to me, so I don't know
what the true intent is here.

pickletester.py, test_pickle.py:  Got rid of the hack for
exempting cPickle from running some of the proto 2 tests.

dictobject.c, PyDict_Next():  documented intended use.
2003-02-15 03:01:11 +00:00
Tim Peters 71fcda5ba7 cPickle produces NEWOBJ appropriately now. It still doesn't know
how to unpickle the new slot-full state tuples.
2003-02-14 23:05:28 +00:00
Tim Peters 92c8bb3abf Minor assorted cleanups; no semantic changes. 2003-02-13 23:00:26 +00:00
Tim Peters aae53d47d6 The version of PyImport_Import() in cPickle is no longer needed (an edited
version was moved into import.c long ago), so squashed the duplication.
2003-02-13 22:17:05 +00:00
Tim Peters 5aa3da6495 save(): Reformat tail end just for clarity. 2003-02-13 21:03:57 +00:00
Tim Peters eab7db3cee Taught cPickle how to read pickles containing NEWOBJ. This won't get
exercised by the test suite before cPickle knows how to create NEWOBJ
too.  For now, it was just tried once by hand (via loading a NEWOBJ
pickle created by pickle.py).
2003-02-13 18:24:14 +00:00
Tim Peters 8587b3c073 Added a HIGHEST_PROTOCOL module attribute to pickle and cPickle. 2003-02-13 15:44:41 +00:00
Tim Peters 90975f1ff9 Minor cleanup of new batch-list/dict code. 2003-02-12 05:28:58 +00:00
Tim Peters 42f08ac1e3 Implemented batching for dicts in cPickle. This is after two failed
attempts to merge the C list-batch and dict-batch code -- they worked, but
it was a godawful mess to read.
2003-02-11 22:43:24 +00:00
Tim Peters 1092d64002 Implemented list batching in cPickle. 2003-02-11 21:06:20 +00:00
Tim Peters 6288e230d8 More typo repair. 2003-02-05 03:53:10 +00:00
Tim Peters 679201467b Typo repair. 2003-02-05 03:46:17 +00:00
Tim Peters 3e667d5452 cPickle: exempt two_tuple from GC -- it's a speed hack, and doesn't
guarantee to keep valid pointers in its slots.

tests:  Moved ExtensionSaver from test_copy_reg into pickletester, and
use it both places.  Once extension codes get assigned, it won't be
safe to overwrite them willy nilly in test suites, and ExtensionSaver
does a thorough job of undoing any possible damage.

Beefed up the EXT[124] tests a bit, to check the smallest and largest
codes in each opcode's range too.
2003-02-04 21:47:44 +00:00
Tim Peters 731098b3ff cPickle now generates proto 2 EXT[124] when appropriate.
Moved such EXT tests as currently exist from TempAbstractPickleTests to
AbstractPickleTests, so that test_cpickle runs them too.
2003-02-04 20:56:09 +00:00
Tim Peters fa05ce3c1d Typo in comment. 2003-02-04 05:20:32 +00:00
Tim Peters 2d62965734 cPickle can load pickles using proto 2 EXT[124] now, but can't yet
generate these opcodes.
2003-02-04 05:06:17 +00:00
Guido van Rossum d4b920c8bd Rename the extension registry variables to have leading underscores --
this clarifies that they are part of an internal API (albeit shared
between pickle.py, copy_reg.py and cPickle.c).

I'd like to do the same for copy_reg.dispatch_table, but worry that it
might be used by existing code.  This risk doesn't exist for the
extension registry.
2003-02-04 01:54:49 +00:00
Tim Peters 0dd23aa539 Typo repair. 2003-02-04 00:30:46 +00:00
Tim Peters 5b7da39f23 Brought some module variables into synch with pickle.py's current values.
Imported the extension-registry dicts from copy_reg.py, in preparation for
tackling EXT[124].
2003-02-04 00:21:07 +00:00