Commit Graph

155 Commits

Author SHA1 Message Date
Martin v. Löwis 18e165558b Merge ssize_t branch. 2006-02-15 17:27:45 +00:00
Neal Norwitz 1ac754fa10 Check return result from Py_InitModule*(). This API can fail.
Probably should be backported.
2006-01-19 06:09:39 +00:00
Jeremy Hylton af68c874a6 Add const to several API functions that take char *.
In C++, it's an error to pass a string literal to a char* function
without a const_cast().  Rather than require every C++ extension
module to put a cast around string literals, fix the API to state the
const-ness.

I focused on parts of the API where people usually pass literals:
PyArg_ParseTuple() and friends, Py_BuildValue(), PyMethodDef, the type
slots, etc.  Predictably, there were a large set of functions that
needed to be fixed as a result of these changes.  The most pervasive
change was to make the keyword args list passed to
PyArg_ParseTupleAndKewords() to be a const char *kwlist[].

One cast was required as a result of the changes:  A type object
mallocs the memory for its tp_doc slot and later frees it.
PyTypeObject says that tp_doc is const char *; but if the type was
created by type_new(), we know it is safe to cast to char *.
2005-12-10 18:50:16 +00:00
Raymond Hettinger a6b45cc31d Eliminate the deprecated option to return None instead of a tuple of arguments in __reduce__(). 2004-12-07 07:05:57 +00:00
Martin v. Löwis 544f1195b1 Patch #995766: Keyword argument support in cPickle. 2004-07-27 05:22:33 +00:00
Martin v. Löwis 737ea82a5a Patch #774665: Make Python LC_NUMERIC agnostic. 2004-06-08 18:52:54 +00:00
Christian Tismer 2460c62152 made cPickle fall back to the copy_reg/reduce protocol,
if a function cannot be stored as global.
This is for compatibility with pickle.py .
2004-02-26 16:21:45 +00:00
Raymond Hettinger 8ae4689657 Simplify and speedup uses of Py_BuildValue():
* Py_BuildValue("(OOO)",a,b,c)  -->  PyTuple_Pack(3,a,b,c)
* Py_BuildValue("()",a)         -->  PyTuple_New(0)
* Py_BuildValue("O", a)         -->  Py_INCREF(a)
2003-10-12 19:09:37 +00:00
Jeremy Hylton fff093fa7f The Unpickler forget about its find_class attribute. 2003-07-11 19:42:49 +00:00
Jeremy Hylton f00368f9ed Remove many blanket try/except clauses.
SF bug [ 751276 ] cPickle doesn't raise error, pickle does (recursiondepth)

Most of the calls to PyErr_Clear() were intended to catch & clear an
attribute error and try something different.  Guard all those cases
with a PyErr_ExceptionMatches() and fail if some other error
occurred.  The other error is likely a bug in the user code.

This is basically the C equivalent of changing "except:" to
"except AttributeError:"
2003-06-16 20:19:49 +00:00
Tim Peters 3cfe75470d PyType_Ready(): Complain if the type is a base type, and gc'able, and
tp_free is NULL or PyObject_Del at the end.  Because it's a base type
it must call tp_free in its dealloc function, and because it's gc'able
it must not call PyObject_Del.

inherit_slots():  Don't inherit tp_free unless the type and its base
agree about whether they're gc'able.  If the type is gc'able and the
base is not, and the base uses the default PyObject_Del for its
tp_free, give the type PyObject_GC_Del for its tp_free (the appropriate
default for a gc'able type).

cPickle.c:  The Pickler and Unpickler types claim to be base classes
and gc'able, but their dealloc functions didn't call tp_free.
Repaired that.  Also call PyType_Ready() on these typeobjects, so
that the correct (PyObject_GC_Del) default memory-freeing function
gets plugged into these types' tp_free slots.
2003-05-21 21:29:48 +00:00
Jeremy Hylton 7b5ce7f25a Make Unpickler objects colletable.
Bugfix candidate.
2003-04-09 21:25:30 +00:00
Jeremy Hylton 4cf6319cd2 Make Picklers collectable.
Bug fix candidate.
2003-04-09 21:05:12 +00:00
Tim Peters 9905b943f7 New private API functions _PyFloat_{Pack,Unpack}(4,8}. This is a
refactoring to get all the duplicates of this delicate code out of the
cPickle and struct modules.
2003-03-20 20:53:32 +00:00
Tim Peters d50ade68ec SF bug 705836: struct.pack of floats in non-native endian order
pack_float, pack_double, save_float:  All the routines for creating
IEEE-format packed representations of floats and doubles simply ignored
that rounding can (in rare cases) propagate out of a long string of
1 bits.  At worst, the end-off carry can (by mistake) interfere with
the exponent value, and then unpacking yields a result wrong by a factor
of 2.  In less severe cases, it can end up losing more low-order bits
than intended, or fail to catch overflow *caused* by rounding.

Bugfix candidate, but I already backported this to 2.2.

In 2.3, this code remains in severe need of refactoring.
2003-03-20 18:32:13 +00:00
Guido van Rossum b289b87a37 Use __reduce_ex__. 2003-02-19 01:45:13 +00:00
Tim Peters b9ce7cd8b8 save_global(): Trying to resolve module.name can fail for two
reasons:  importing module can fail, or the attribute lookup
module.name can fail.  We were giving the same error msg for
both cases, making it needlessly hard to guess what went wrong.
These cases give different error msgs now.
2003-02-18 20:50:45 +00:00
Neal Norwitz 0ae4c4a823 Make 2 module variables static. Assuming this is correct. 2003-02-15 15:07:17 +00:00
Tim Peters 080c88b912 cPickle.c, load_build(): Taught cPickle how to pick apart
the optional proto 2 slot state.

pickle.py, load_build():  CAUTION:  Noted that cPickle's
load_build and pickle's load_build really don't do the same
things with the state, and didn't before this patch either.
cPickle never tries to do .update(), and has no backoff if
instance.__dict__ can't be retrieved.  There are no tests
that can tell the difference, and part of what cPickle's
load_build() did looked accidental to me, so I don't know
what the true intent is here.

pickletester.py, test_pickle.py:  Got rid of the hack for
exempting cPickle from running some of the proto 2 tests.

dictobject.c, PyDict_Next():  documented intended use.
2003-02-15 03:01:11 +00:00
Tim Peters 71fcda5ba7 cPickle produces NEWOBJ appropriately now. It still doesn't know
how to unpickle the new slot-full state tuples.
2003-02-14 23:05:28 +00:00
Tim Peters 92c8bb3abf Minor assorted cleanups; no semantic changes. 2003-02-13 23:00:26 +00:00
Tim Peters aae53d47d6 The version of PyImport_Import() in cPickle is no longer needed (an edited
version was moved into import.c long ago), so squashed the duplication.
2003-02-13 22:17:05 +00:00
Tim Peters 5aa3da6495 save(): Reformat tail end just for clarity. 2003-02-13 21:03:57 +00:00
Tim Peters eab7db3cee Taught cPickle how to read pickles containing NEWOBJ. This won't get
exercised by the test suite before cPickle knows how to create NEWOBJ
too.  For now, it was just tried once by hand (via loading a NEWOBJ
pickle created by pickle.py).
2003-02-13 18:24:14 +00:00
Tim Peters 8587b3c073 Added a HIGHEST_PROTOCOL module attribute to pickle and cPickle. 2003-02-13 15:44:41 +00:00
Tim Peters 90975f1ff9 Minor cleanup of new batch-list/dict code. 2003-02-12 05:28:58 +00:00
Tim Peters 42f08ac1e3 Implemented batching for dicts in cPickle. This is after two failed
attempts to merge the C list-batch and dict-batch code -- they worked, but
it was a godawful mess to read.
2003-02-11 22:43:24 +00:00
Tim Peters 1092d64002 Implemented list batching in cPickle. 2003-02-11 21:06:20 +00:00
Tim Peters 6288e230d8 More typo repair. 2003-02-05 03:53:10 +00:00
Tim Peters 679201467b Typo repair. 2003-02-05 03:46:17 +00:00
Tim Peters 3e667d5452 cPickle: exempt two_tuple from GC -- it's a speed hack, and doesn't
guarantee to keep valid pointers in its slots.

tests:  Moved ExtensionSaver from test_copy_reg into pickletester, and
use it both places.  Once extension codes get assigned, it won't be
safe to overwrite them willy nilly in test suites, and ExtensionSaver
does a thorough job of undoing any possible damage.

Beefed up the EXT[124] tests a bit, to check the smallest and largest
codes in each opcode's range too.
2003-02-04 21:47:44 +00:00
Tim Peters 731098b3ff cPickle now generates proto 2 EXT[124] when appropriate.
Moved such EXT tests as currently exist from TempAbstractPickleTests to
AbstractPickleTests, so that test_cpickle runs them too.
2003-02-04 20:56:09 +00:00
Tim Peters fa05ce3c1d Typo in comment. 2003-02-04 05:20:32 +00:00
Tim Peters 2d62965734 cPickle can load pickles using proto 2 EXT[124] now, but can't yet
generate these opcodes.
2003-02-04 05:06:17 +00:00
Guido van Rossum d4b920c8bd Rename the extension registry variables to have leading underscores --
this clarifies that they are part of an internal API (albeit shared
between pickle.py, copy_reg.py and cPickle.c).

I'd like to do the same for copy_reg.dispatch_table, but worry that it
might be used by existing code.  This risk doesn't exist for the
extension registry.
2003-02-04 01:54:49 +00:00
Tim Peters 0dd23aa539 Typo repair. 2003-02-04 00:30:46 +00:00
Tim Peters 5b7da39f23 Brought some module variables into synch with pickle.py's current values.
Imported the extension-registry dicts from copy_reg.py, in preparation for
tackling EXT[124].
2003-02-04 00:21:07 +00:00
Tim Peters 0c7c48e02c load_counted_long(): Changed a ValueError to an UnpicklingError, just
because it seems more consistent with the rest of the code.

cPickle_PyMapping_HasKey():  This extern function isn't used anywhere in
Python or Zope, so got rid of it.
2003-02-03 22:07:24 +00:00
Tim Peters e0a3907904 PDATA_PUSH and PDATA_APPEND. documented, and reformatted for better
readability.

load_bool():  Now that I know the intended difference between _PUSH and
_APPEND, used the right one.

Pdata_grow():  Squashed out a redundant overflow test.
2003-02-03 15:45:56 +00:00
Tim Peters 1d63c9f151 cPickle support for TUPLE[123]. Incidentally plugged several undetected
overflow holes in Pdata_grow().
2003-02-02 20:29:39 +00:00
Tim Peters 0bc93f5c8b Massive edits. If p is a pointer to a struct, and p->f is a pointer to
a function, then

    p->f(arg1, arg2, ...)

is semantically the same as

    (*p->f)(arg1, arg2, ...)

Changed all instances of the latter into the former.  Given how often
the code embeds this kind of expression in an if test, the unnecessary
parens and dereferening operator were a real drag on readability.
2003-02-02 18:29:33 +00:00
Tim Peters ac5687a515 Minor cleanup, mostly adding horizontal whitespace, and breaking apart
embedded assignments, for readability.
2003-02-02 18:08:34 +00:00
Tim Peters 3c67d795ef Implemented proto 2 NEWTRUE and NEWFALSE in cPickle. 2003-02-02 17:59:11 +00:00
Tim Peters 70b02d79f9 Beefed up the tests by putting in more "for proto in protocols:" outer
loops.  Renamed DATA and BINDATA to DATA0 and DATA1.  Included
disassemblies, but noted why we can't test them.  Added XXX comment to
cPickle about a mysterious comment, where pickle and cPickle diverge
in how they number PUT indices.
2003-02-02 17:26:40 +00:00
Neal Norwitz 529baf2b57 Fix compiler warning 2003-02-02 17:08:33 +00:00
Tim Peters 87482ea055 dump(): Added asserts that self->proto is sane. 2003-02-02 16:16:30 +00:00
Tim Peters 9b1e03ff4b dump(): Fixed a stupid bug in new code. It wasn't possible for the bug
to have an effect before protocol 3 is invented, so no test can be
written for this (yet).
2003-02-02 16:14:23 +00:00
Tim Peters 4190fb8429 Add cPickle support for PROTO. Duplicated PROTO/LONG1/LONG4 code in
the hitherto unknown (to me) noload() cPickle function, which is (a)
something we don't test at all, and (b) pickle.py doesn't have.
2003-02-02 16:09:05 +00:00
Tim Peters ee1a53cbb1 cPickle.c: Full support for the new LONG1 and LONG4. Added comments.
Assorted code cleanups; e.g., sizeof(char) is 1 by definition, so there's
no need to do things like multiply by sizeof(char) in hairy malloc
arguments.  Fixed an undetected-overflow bug in readline_file().

longobject.c:  Fixed a really stupid bug in the new _PyLong_NumBits.

pickle.py:  Fixed stupid bug in save_long():  When proto is 2, it
wrote LONG1 or LONG4, but forgot to return then -- it went on to
append the proto 1 LONG opcode too.
Fixed equally stupid cancelling bugs in load_long1() and
load_long4():  they *returned* the unpickled long instead of pushing
it on the stack.  The return values were ignored.  Tests passed
before only because save_long() pickled the long twice.

Fixed bugs in encode_long().

Noted that decode_long() is quadratic-time despite our hopes,
because long(string, 16) is still quadratic-time in len(string).
It's hex() that's linear-time.  I don't know a way to make decode_long()
linear-time in Python, short of maybe transforming the 256's-complement
bytes into marshal's funky internal format, and letting marshal decode
that.  It would be more valuable to make long(string, 16) linear time.

pickletester.py:  Added a global "protocols" vector so tests can try
all the protocols in a sane way.  Changed test_ints() and test_unicode()
to do so.  Added a new test_long(), but the tail end of it is disabled
because it "takes forever" under pickle.py (but runs very quickly under
cPickle:  cPickle proto 2 for longs is linear-time).
2003-02-02 02:57:53 +00:00
Tim Peters 5bd2a79b22 The C pickle now knows how to deal with a proto= argument. Assorted
code cleanups, and purged more references to text-vs-binary modes.
2003-02-01 16:45:06 +00:00