Commit Graph

168 Commits

Author SHA1 Message Date
Tim Peters 0c7c48e02c load_counted_long(): Changed a ValueError to an UnpicklingError, just
because it seems more consistent with the rest of the code.

cPickle_PyMapping_HasKey():  This extern function isn't used anywhere in
Python or Zope, so got rid of it.
2003-02-03 22:07:24 +00:00
Tim Peters e0a3907904 PDATA_PUSH and PDATA_APPEND. documented, and reformatted for better
readability.

load_bool():  Now that I know the intended difference between _PUSH and
_APPEND, used the right one.

Pdata_grow():  Squashed out a redundant overflow test.
2003-02-03 15:45:56 +00:00
Tim Peters 1d63c9f151 cPickle support for TUPLE[123]. Incidentally plugged several undetected
overflow holes in Pdata_grow().
2003-02-02 20:29:39 +00:00
Tim Peters 0bc93f5c8b Massive edits. If p is a pointer to a struct, and p->f is a pointer to
a function, then

    p->f(arg1, arg2, ...)

is semantically the same as

    (*p->f)(arg1, arg2, ...)

Changed all instances of the latter into the former.  Given how often
the code embeds this kind of expression in an if test, the unnecessary
parens and dereferening operator were a real drag on readability.
2003-02-02 18:29:33 +00:00
Tim Peters ac5687a515 Minor cleanup, mostly adding horizontal whitespace, and breaking apart
embedded assignments, for readability.
2003-02-02 18:08:34 +00:00
Tim Peters 3c67d795ef Implemented proto 2 NEWTRUE and NEWFALSE in cPickle. 2003-02-02 17:59:11 +00:00
Tim Peters 70b02d79f9 Beefed up the tests by putting in more "for proto in protocols:" outer
loops.  Renamed DATA and BINDATA to DATA0 and DATA1.  Included
disassemblies, but noted why we can't test them.  Added XXX comment to
cPickle about a mysterious comment, where pickle and cPickle diverge
in how they number PUT indices.
2003-02-02 17:26:40 +00:00
Neal Norwitz 529baf2b57 Fix compiler warning 2003-02-02 17:08:33 +00:00
Tim Peters 87482ea055 dump(): Added asserts that self->proto is sane. 2003-02-02 16:16:30 +00:00
Tim Peters 9b1e03ff4b dump(): Fixed a stupid bug in new code. It wasn't possible for the bug
to have an effect before protocol 3 is invented, so no test can be
written for this (yet).
2003-02-02 16:14:23 +00:00
Tim Peters 4190fb8429 Add cPickle support for PROTO. Duplicated PROTO/LONG1/LONG4 code in
the hitherto unknown (to me) noload() cPickle function, which is (a)
something we don't test at all, and (b) pickle.py doesn't have.
2003-02-02 16:09:05 +00:00
Tim Peters ee1a53cbb1 cPickle.c: Full support for the new LONG1 and LONG4. Added comments.
Assorted code cleanups; e.g., sizeof(char) is 1 by definition, so there's
no need to do things like multiply by sizeof(char) in hairy malloc
arguments.  Fixed an undetected-overflow bug in readline_file().

longobject.c:  Fixed a really stupid bug in the new _PyLong_NumBits.

pickle.py:  Fixed stupid bug in save_long():  When proto is 2, it
wrote LONG1 or LONG4, but forgot to return then -- it went on to
append the proto 1 LONG opcode too.
Fixed equally stupid cancelling bugs in load_long1() and
load_long4():  they *returned* the unpickled long instead of pushing
it on the stack.  The return values were ignored.  Tests passed
before only because save_long() pickled the long twice.

Fixed bugs in encode_long().

Noted that decode_long() is quadratic-time despite our hopes,
because long(string, 16) is still quadratic-time in len(string).
It's hex() that's linear-time.  I don't know a way to make decode_long()
linear-time in Python, short of maybe transforming the 256's-complement
bytes into marshal's funky internal format, and letting marshal decode
that.  It would be more valuable to make long(string, 16) linear time.

pickletester.py:  Added a global "protocols" vector so tests can try
all the protocols in a sane way.  Changed test_ints() and test_unicode()
to do so.  Added a new test_long(), but the tail end of it is disabled
because it "takes forever" under pickle.py (but runs very quickly under
cPickle:  cPickle proto 2 for longs is linear-time).
2003-02-02 02:57:53 +00:00
Tim Peters 5bd2a79b22 The C pickle now knows how to deal with a proto= argument. Assorted
code cleanups, and purged more references to text-vs-binary modes.
2003-02-01 16:45:06 +00:00
Tim Peters 040e033796 Removed needless include of errno.h. 2003-02-01 06:30:12 +00:00
Tim Peters 64c04d1df3 The module docstring had an RCS ID from 1999 embedded in it. Enough
already <wink>.
2003-02-01 06:27:59 +00:00
Tim Peters cba30e2c96 Trimmed trailing whitespace. 2003-02-01 06:24:36 +00:00
Tim Peters 797ec24e87 Added #defines for proto 2 opcodes; gave the Pickler a proto member;
removed woefully inadequate opcode docs and pointed to pickletools.py
instead.
2003-02-01 06:22:36 +00:00
Tim Peters 1f1b2d2e68 Removed all uses of the out-of-favor __safe_for_unpickling__ magic
attr, and copy_reg.safe_constructors.
2003-02-01 02:16:37 +00:00
Guido van Rossum 8e0ad0cb0b Ignore the state returned by __reduce__ if it is Py_None. 2003-01-31 21:10:31 +00:00
Jason Tishler fb8595df4f Patch #661760: Cygwin auto-import module patch
The attached patch enables shared extension
modules to build cleanly under Cygwin without
moving the static initialization of certain function
pointers (i.e., ones exported from the Python
DLL core) to a module initialization function.

Additionally, this patch fixes the modules that
have been changed in the past to accommodate
Cygwin.
2003-01-06 12:41:26 +00:00
Guido van Rossum 75bfd0585b Add an XXX comment about relative imports. 2002-12-24 18:10:07 +00:00
Tim Peters 4e52ca82ae A patch from Kevin Jacobs, plugging several leaks discovered when
running the sandbox datetime tests.

Bugfix candidate.
2002-12-07 02:43:28 +00:00
Martin v. Löwis 658009afdb Make BadPickleGet a class. Fixes #609164. 2002-09-16 17:26:24 +00:00
Martin v. Löwis 8a8da798a5 Patch #505705: Remove eval in pickle and cPickle. 2002-08-14 07:46:28 +00:00
Neal Norwitz 200788ce45 Allow more docstrings to be removed during compilation in some modules 2002-08-13 22:20:41 +00:00
Martin v. Löwis 5a39530274 Add recursion counter for pickling. Fixes #576084.
2.2 bugfix candidate (may cause RuntimeError for applications that
currently work fine).
2002-08-04 08:20:23 +00:00
Mark Hammond fe51c6d66e Excise DL_EXPORT/DL_IMPORT from Modules/*. Required adding a prototype
for Py_Main().

Thanks to Kalle Svensson and Skip Montanaro for the patches.
2002-08-02 02:27:13 +00:00
Jeremy Hylton 7fa4bfa173 Fix indentation. 2002-07-18 20:58:57 +00:00
Jeremy Hylton 938ace69a0 staticforward bites the dust.
The staticforward define was needed to support certain broken C
compilers (notably SCO ODT 3.0, perhaps early AIX as well) botched the
static keyword when it was used with a forward declaration of a static
initialized structure.  Standard C allows the forward declaration with
static, and we've decided to stop catering to broken C compilers.  (In
fact, we expect that the compilers are all fixed eight years later.)

I'm leaving staticforward and statichere defined in object.h as
static.  This is only for backwards compatibility with C extensions
that might still use it.

XXX I haven't updated the documentation.
2002-07-17 16:30:39 +00:00
Jeremy Hylton 39c6116483 Given the persistent id code a shot at a class before calling save_global().
Some persistent picklers (well, probably, the *only* persistent
pickler) would like to pickle some classes in a special way.
2002-07-16 19:47:43 +00:00
Jeremy Hylton 9ee91f11b3 remove decl of unused variable 2002-07-11 22:02:33 +00:00
Jeremy Hylton 0e1f7a82e9 Do more robust test of whether global objects are accessible.
PyImport_ImportModule() is not guaranteed to return a module object.
When another type of object was returned, the PyModule_GetDict() call
return NULL and the subsequent GetItem() seg faulted.

Bug fix candidate.
2002-07-11 22:01:40 +00:00
Jack Jansen 3a96702b2b Undefine TRUE and FALSE before redefining them. 2002-06-26 20:40:42 +00:00
Martin v. Löwis 14f8b4cfcb Patch #568124: Add doc string macros. 2002-06-13 20:33:02 +00:00
Fred Drake 0ebacc8b38 Pickler_clear_memo(): convert to METH_NOARGS. 2002-05-01 20:36:39 +00:00
Tim Peters 5de9842b34 Repair widespread misuse of _PyString_Resize. Since it's clear people
don't understand how this function works, also beefed up the docs.  The
most common usage error is of this form (often spread out across gotos):

	if (_PyString_Resize(&s, n) < 0) {
		Py_DECREF(s);
		s = NULL;
		goto outtahere;
	}

The error is that if _PyString_Resize runs out of memory, it automatically
decrefs the input string object s (which also deallocates it, since its
refcount must be 1 upon entry), and sets s to NULL.  So if the "if"
branch ever triggers, it's an error to call Py_DECREF(s):  s is already
NULL!  A correct way to write the above is the simpler (and intended)

	if (_PyString_Resize(&s, n) < 0)
		goto outtahere;

Bugfix candidate.
2002-04-27 18:44:32 +00:00
Neal Norwitz 8ee3cd47a9 #546156, Remove load_false()/load_true(), they are not used 2002-04-21 23:44:34 +00:00
Guido van Rossum e276339cea Implement an idea by Paul Rubin:
Change pickling format for bools to use a backwards compatible
encoding.  This means you can pickle True or False on Python 2.3
and Python 2.2 or before will read it back as 1 or 0.  The code
used for pickling bools before would create pickles that could
not be read in previous Python versions.
2002-04-05 19:30:08 +00:00
Guido van Rossum 2e1c09c1fd Removed old Digital Creations copyright/license notices (with
permission from Paul Everitt).  Also removed a few other references to
Digital Creations and changed the remaining ones to Zope Corporation.
2002-04-04 17:52:50 +00:00
Guido van Rossum 77f6a65eb0 Add the 'bool' type and its values 'False' and 'True', as described in
PEP 285.  Everything described in the PEP is here, and there is even
some documentation.  I had to fix 12 unit tests; all but one of these
were printing Boolean outcomes that changed from 0/1 to False/True.
(The exception is test_unicode.py, which did a type(x) == type(y)
style comparison.  I could've fixed that with a single line using
issubtype(x, type(y)), but instead chose to be explicit about those
places where a bool is expected.

Still to do: perhaps more documentation; change standard library
modules to return False/True from predicates.
2002-04-03 22:41:51 +00:00
Martin v. Löwis 2f6ef4c630 Reindent. Break long lines. Move comments before the statements. 2002-04-01 17:40:08 +00:00
Martin v. Löwis 43c9d8ad23 Remove UNLESS. 2002-04-01 12:34:33 +00:00
Neal Norwitz b049325e92 Use symbolic METH_VARARGS/METH_OLDARGS instead of 1/0 for ml_flags 2002-03-31 14:44:22 +00:00
Neil Schemenauer fa79c65235 Match behavior of the pickle.py module more closely. 2002-03-22 23:02:53 +00:00
Barry Warsaw 52acb49298 Merge of the release22 branch changes back into the trunk. 2001-12-21 20:04:22 +00:00
Guido van Rossum 950dce6f01 save(): Fix for SF bug #494904: Cannot pickle a class with a
metaclass, reported by Dan Parisien.

Objects that are instances of custom metaclasses, i.e. whose ob_type
is a subclass of PyType_Type, should be pickled the same as new-style
classes (objects whose ob_type is PyType_Type).  This can't be done
through the existing dispatch switches, and the __reduce__ trick
doesn't work for these, since it finds the unbound __reduce__ for
instances of the class (inherited from PyBaseObject_Type).  So check
explicitly using PyType_IsSubtype().
2001-12-19 16:56:54 +00:00
Guido van Rossum 146483964e Patch supplied by Burton Radons for his own SF bug #487390: Modifying
type.__module__ behavior.

This adds the module name and a dot in front of the type name in every
type object initializer, except for built-in types (and those that
already had this).  Note that it touches lots of Mac modules -- I have
no way to test these but the changes look right.  Apologies if they're
not.  This also touches the weakref docs, which contains a sample type
object initializer.  It also touches the mmap test output, because the
mmap type's repr is included in that output.  It touches object.h to
put the correct description in a comment.
2001-12-08 18:02:58 +00:00
Jeremy Hylton 179c48c60e Use PyOS_snprintf() instead of sprintf(). 2001-11-28 21:49:51 +00:00
Barry Warsaw 9b481ff3d6 A change to sync with pickle.py:
find_class(): We no longer mask all exceptions[1] by transforming them
into SystemError.  The latter is definitely not the right thing to do,
so we let any exceptions that occur in the PyObject_GetAttr() call to
simply propagate up if they occur.

[1] Note that pickle only masked ImportError, KeyError, and
AttributeError, but cPickle masked all exceptions.
2001-11-15 23:45:26 +00:00
Tim Peters bf5ca65c2d load_string(): Force use of unsigned compare in a context that was
clearly (but incorrectly) assuming it.
2001-11-12 22:26:10 +00:00
Michael W. Hudson 03f96bd8f5 Fixes to compile cPickle.c & socketmodule.c on cygwin and possibly
other platforms that have funny ideas about whether addresses of
functions in dlls are compile-time constants.
2001-11-09 10:06:23 +00:00
Jeremy Hylton 3eb46f3a5d Must terminate the Pickler_members[] and Pickler_getsets with NULL. 2001-10-16 17:10:49 +00:00
Jeremy Hylton 499ab6a653 Better fix for core dumps on recursive objects in fast mode.
Raise ValueError when an object contains an arbitrarily nested
reference to itself.  (The previous fix just produced invalid
pickles.)

Solution is very much like Py_ReprEnter() and Py_ReprLeave():
fast_save_enter() and fast_save_leave() that tracks the fast_container
limit and keeps a fast_memo of objects currently being pickled.

The cost of the solution is moderately expensive for deeply nested
structures, but it still seems to be faster than normal pickling,
based on tests with deeply nested lists.

Once FAST_LIMIT is exceeded, the new code is about twice as slow as
fast-mode code that doesn't check for recursion.  It's still twice as
fast as the normal pickling code.  In the absence of deeply nested
structures, I couldn't measure a difference.
2001-10-15 21:37:58 +00:00
Jeremy Hylton a0fb177be8 Progress on SF bug #466175 and general cleanup.
Add a fast_container member to Picklerobject.  If fast is true, then
fast_container counts the depth of nested container calls.  If the
depth exceeds FAST_LIMIT (2000), the fast flag is ignored and the
normal checks occur.  This approach is much like the approach for
prevent stack overflow for comparison and reprs of recursive objects
(e.g. [[...]]).

    - Fast container used for save_list(), save_dict(), and
      save_inst().

      XXX Not clear which other save_xxx() functions should use it.

Make Picklerobject into new-style types, using PyObject_GenericGetAttr()
and PyObject_GenericSetAttr().

    - Use PyMemberDef for binary and fast members

    - Use PyGetSetDef for persistent_id, inst_persistent_id, memo, and
      PicklingError.

      XXX Not all of these seem like they need to use getset, but it's
      not clear why the old getattr() and setattr() had such odd
      semantics.  One change is that the getvalue() attribute will
      exist on all Picklers, not just list-based picklers; I think
      this is a more rationale interface.

There is a long laundry list of other changes:

    - Remove unused #defines for PyList_SET_ITEM() etc.

    - Make some of the indentation consistent

    - Replace uses of cPickle_PyMapping_HasKey() where the first
      argument is self->memo with calls to PyDict_GetItem(), because
      self->memo must be a dictionary.

    - Don't bother to check if cPickle_PyMapping_HasKey() returns < 0,
      because it can only return 0 or 1.

    - Replace uses of PyObject_CallObject() with PyObject_Call(), when
      we can guarantee that the argument tuple is really a tuple.

Performance impacts of these changes:

    - 5% speedup for normal pickling

    - No change to fast-mode pickling.

XXX Really need tests for all the features in cPickle that aren't in
pickle.
2001-10-12 04:11:06 +00:00
Tim Peters 12778e314b load_int: The fallback to long ints was coded in such a way that it
couldn't succeed.  Fixed.
2001-08-28 22:08:34 +00:00
Guido van Rossum a92d16aaec SF patch #452239 by Gordon McMillan, to fix SF bug #451547.
This patch attempts to do to cPickle what Guido did
   for pickle.py v 1.50. That is: save_global tries
   importing the module, and fetching the name from the
   module. If that fails, or the returned object is not
   the same one we started with, it raises a
   PicklingError. (All this so pickling a lambda will
   fail at save time, rather than load time).
2001-08-18 21:22:07 +00:00
Martin v. Löwis 339d0f720e Patch #445762: Support --disable-unicode
- Do not compile unicodeobject, unicodectype, and unicodedata if Unicode is disabled
- check for Py_USING_UNICODE in all places that use Unicode functions
- disables unicode literals, and the builtin functions
- add the types.StringTypes list
- remove Unicode literals from most tests.
2001-08-17 18:39:25 +00:00
Tim Peters 6d6c1a35e0 Merge of descr-branch back into trunk. 2001-08-02 04:15:00 +00:00
Fred Drake 2c7a6851ed Remove code to initialize globals that are never used.
Add some casts to quiet warnings from an unspecified non-GCC compiler.

This closes SF patch #436258.
2001-07-17 18:34:03 +00:00
Tim Peters d8ae7c2999 Ack -- this module mixes tabs and spaces, and what appears to be a mix
of 2-space and 4-space indents.  Whatever, when I saw the checkin diff it
was clear that what my editor thinks a tab means didn't match this module's
belief.  Removed all the tabs from the lines I added and changed, left
everything else alone.
2001-04-10 04:35:28 +00:00
Tim Peters 3906eb877a On a sizeof(long)==8 machine, ints in range(2**31, 2**32) were getting
pickled into the signed(!) 4-byte BININT format, so were getting unpickled
again as negative ints.  Repaired that.
Added some minimal docs at the top about what I've learned about the pickle
format codes (little of which was obvious from staring at the code,
although that's partly because all the size-related bugs greatly obscured
the true intent of the code).
Happy side effect:  because save_int() needed to grow a *proper* range
check in order to fix this bug, it can now use the more-efficient BININT1,
BININT2 and BININT formats when the long's value is small enough to fit
in a signed 4-byte int (before this, on a sizeof(long)==8 box it always
used the general INT format for negative ints).
test_cpickle works again on sizeof(long)==8 machines.  test_pickle is
still busted big-time.
2001-04-10 04:22:00 +00:00
Tim Peters bfa18f711f Critical fix: if cPickle on a sizeof(long)==8 box is used to read a
binary pickle, and the latter contains a pickle of a negative Python
int i written on a sizeof(long)==4 box (and whether by cPickle or
pickle.py), it's read incorrectly as i + 2**32.  The patch repairs that,
and allows test_cpickle.py (to which I added a relevant test case earlier
today) to work again on sizeof(long)==8 boxes.
There's another (at least one) sizeof(long)==8 binary pickle bug, but in
pickle.py instead.  That bug is still there, and test_pickle.py doesn't
catch it yet (try pickling and unpickling, e.g., 1 << 46).
2001-04-10 01:54:42 +00:00
Fred Drake 2c77355937 Make cPickle use the recently-added PyInstance_NewRaw() API to create
instance objects without calling the constructor.  This is the same as
the new.instance() function.
2001-03-22 17:52:17 +00:00
Tim Peters 84e87f379e SF bug [ #233200 ] cPickle does not use Py_BEGIN_ALLOW_THREADS.
http://sourceforge.net/tracker/?func=detail&aid=233200&group_id=5470&atid=105470
Wrapped the fread/fwrite calls in thread BEGIN_ALLOW/END_ALLOW brackets
Afraid I hit the "delete trailing whitespace key" too!  Only two "real" sections
of code changed here.
2001-03-17 04:50:51 +00:00
Guido van Rossum fb10c3f664 Minimal fix for the complaints about pickling Unicode objects. (SF
bugs #126161 and 123634).

The solution doesn't use the unicode-escape encoding; that has other
problems (it seems not 100% reversible).  Rather, it transforms the
input Unicode object slightly before encoding it using
raw-unicode-escape, so that the decoding will reconstruct the original
string: backslash and newline characters are translated into their
\uXXXX counterparts.

This is backwards incompatible for strings containing backslashes, but
for some of those strings, the pickling was already broken.

Note that SF bug #123634 complains specifically that cPickle fails to
unpickle the pickle for u'' (the empty Unicode string) correctly.
This was an off-by-one error in load_unicode().

XXX Ugliness: in order to do the modified raw-unicode-escape, I've
cut-and-pasted a copy of PyUnicode_EncodeRawUnicodeEscape() into this
file that also encodes '\\' and '\n'.  It might be nice to migrate
this into the Unicode implementation and give this encoding a new name
('half-raw-unicode-escape'? 'pickle-unicode-escape'?); that would help
pickle.py too.  But right now I can't be bothered with the necessary
infrastructural changes.
2000-12-19 02:08:38 +00:00
Neil Schemenauer 5196c586bb - Fix a GC bug caused by PyDict_New() failing. 2000-10-04 16:22:26 +00:00
Guido van Rossum ebba420285 Oops. Jim's fix didn't. This one does -- I tested it a bit better
this time!
2000-09-07 14:35:37 +00:00
Guido van Rossum c84d8bd7be Simple fix from Jin Fulton to avoid returning a half-initialized
module when e.g. copy_reg.py doesn't exist.  This caused a core dump.

This closes SF bug 112944.
2000-09-07 00:11:40 +00:00
Trent Mick 6c116dd56b Use safer comparisons (only matters when sizeof(int) != sizeof(size_t)). fread
and fwrite return size_t, so it is safer to cast up to the largest type for the
comparison. I believe the cast is required at all to remove compiler warnings.
2000-08-12 20:58:11 +00:00
Peter Schneider-Kamp 7e01890986 merge Include/my*.h into Include/pyport.h
marked my*.h as obsolete
2000-07-31 15:28:04 +00:00
Thomas Wouters 58d0510245 ANSIfy some more forward declarations. 2000-07-24 14:43:35 +00:00
Thomas Wouters 4789b3ae05 ... and yet more ANSIfications... 2000-07-24 11:36:47 +00:00
Thomas Wouters 3b6448fbae ANSIfication: add proper prototypes to function-pointers and declarations.
Also, fix a bug found by said declarations, where a string was defined as
unsigned char*, but used as signed.
2000-07-22 23:56:07 +00:00
Jeremy Hylton 03657cfdb0 replace PyXXX_Length calls with PyXXX_Size calls 2000-07-12 13:05:33 +00:00
Jeremy Hylton c5007aa5c3 final patches from Neil Schemenauer for garbage collection 2000-06-30 05:02:53 +00:00
Guido van Rossum 534b7c5c96 Trent Mick:
This patch fixes cPickle.c for 64-bit platforms.

- The false assumption sizeof(long) == size(void*) exists where
PyInt_FromLong is used to represent a pointer. The safe Python call
for this is PyLong_FromVoidPtr. (On platforms where the above
assumption *is* true a PyInt is returned as before so there is no
effective change.)

- use size_t instead of int for some variables
2000-06-28 22:23:56 +00:00
Guido van Rossum ea2b7157ab New version from Jim Fulton to fix a problem that Eric Raymond ran
into.  Jim writes:

The core dump was due to a C decrement operation
in a macro invocation in load_pop.  (BAD)

I fixed this by moving the decrement outside
the macro call.

I added a comment to load_pop and load_mark
to document the fact that cPickle separates the
unpickling stack into two separate stacks, one for
objects and one for marks.

I also moved some increments out of some macro
calls (PyTuple_SET_ITEM and PyList_SET_ITEM).
This wasn't necessary, but made me feel better. :)

I tested these changes in *my* cPickle, which
doesn't have the new Unicode stuff.
2000-05-09 18:14:50 +00:00
Guido van Rossum b18618dab7 Vladimir Marangozov's long-awaited malloc restructuring.
For more comments, read the patches@python.org archives.
For documentation read the comments in mymalloc.h and objimpl.h.

(This is not exactly what Vladimir posted to the patches list; I've
made a few changes, and Vladimir sent me a fix in private email for a
problem that only occurs in debug mode.  I'm also holding back on his
change to main.c, which seems unnecessary to me.)
2000-05-03 23:44:39 +00:00
Guido van Rossum 83addc7a0f Charles Waldman writes:
"""
Problem description:

	Run the following script:

import test.test_cpickle
for x in xrange(1000000):
    reload(test.test_cpickle)

Watch Python's memory use go up up and away!

In the course of debugging this I also saw that cPickle is
inconsistent with pickle - if you attempt a pickle.load or pickle.dump
on a closed file, you get a ValueError, whereas the corresponding
cPickle operations give an IOError.  Since cPickle is advertised as
being compatible with pickle, I changed these exceptions to match.
"""
2000-04-21 20:49:36 +00:00
Guido van Rossum 5fccb7c58e Marc-Andre Lemburg: support pickling Unicode objects, both in text
mode ('V') and in binary mode ('X').
2000-03-10 23:11:40 +00:00
Guido van Rossum 43713e5a28 Massive patch by Skip Montanaro to add ":name" to as many
PyArg_ParseTuple() format string arguments as possible.
2000-02-29 13:59:29 +00:00
Guido van Rossum 2f80d96c04 Patch by Stephen Turner, who writes:
"""
It fixes a memory corruption error resulting from BadPickleGet
exceptions in load_get, load_binget and load_long_binget.  This was
initially reported on c.l.py as a problem with Cookie.py; see the thread
titled "python core dump (SIGBUS) on Solaris" for more details.

If PyDict_GetItem(self->memo, py_key) call failed, then py_key was being
Py_DECREF'd out of existence before call was made to
PyErr_SetObject(BadPickleGet, py_key).

The bug can be duplicated as follows:

import cPickle
cPickle.loads('garyp')

This raises a BadPickleGet exception whose value is a freed object.  A
core dump will soon follow.
"""

Jim Fulton approves of the patch.
1999-07-13 15:18:58 +00:00
Guido van Rossum c3be1a3ca1 New version from Jim:
- Don't call Py_FatalError() when initialization fails.

- Fix bogus use of return value from PyRun_String().

- Fix misc. compiler errors on some platforms.
1999-06-15 14:36:59 +00:00
Guido van Rossum c03158bfc7 Jim Fulton writes:
I've updated cPickle.c to use class exceptions:

Changed pickle error types to classes:

  PickleError
     PicklingError
        UnpickleableError
     UnpicklingError

And change the handling of unpickleable objects so that an UnpickleableError
is raised with the unpickleable object as the argument.  UnpickleableError
has a reasonable string representation and provides access to the problem
object, which is useful during debugging.

[I'm still waiting for patches to do the same to pickle.py.]
1999-06-09 15:23:31 +00:00
Guido van Rossum 1b9e0aae6e Jim Fulton writes:
I have attached a new cPickle that adds a new control attribute
to unpicklers:

  Added new Unpickler attribute, find_global.  If set to None, then
  global and instance pickles are disabled.  Otherwise, it should be set to
  a callable object that takes two arguments, a module name and an
  object name, and returns an object.  If the attribute is unset, then
  the default mechanism is used.

  This feature provides an additional mechanism for controlling which
  classes can be used for unpickling.
1999-04-19 17:58:18 +00:00
Guido van Rossum 761fcd03aa Fix accidentally reversed NULL test in load_mark(). Suggested by
Tamito Kajiyama.  (This caused a bug only on platforms where malloc(0)
returns NULL.)
1999-04-12 22:51:20 +00:00
Guido van Rossum c91fcaa43b Protection against picling to/from closed (real) file.
The problem was reported by Moshe Zadka.
1999-03-29 20:00:14 +00:00
Guido van Rossum d1f66dc198 Fix buglet in load_put -- the test for bad readline result tested the
wrong variable.
1999-02-08 22:38:25 +00:00
Guido van Rossum f9ffb03c35 Jim Fulton: this fixes seg faults with bad pickles like "c". 1999-02-04 14:54:04 +00:00
Guido van Rossum aa8d16761b Make sure not to call realloc() with a NULL pointer -- call malloc()
in that case.  Tamito Kajiyama.
1999-01-25 21:43:51 +00:00
Guido van Rossum 21ef088265 Need to initialize self->safe_constructors early on to prevent crash
in early dealloc.  Patch by Andrew Dalke.
1998-12-11 03:20:00 +00:00
Guido van Rossum e94e3fbb72 Make VC++ 5.0 compiler happy. 1998-12-08 17:37:19 +00:00
Guido van Rossum 50f385c197 Fix two small bugs; add DL_EXPORT() to initcPickle decl. 1998-12-04 18:48:44 +00:00
Guido van Rossum 053b8dfcde New version from Jim Fulton:
- New copyright. (Open source)

  - Added new protocol for binary string pickles that
    takes out unneeded puts:

      p=Pickler()
      p.dump(x)
      p.dump(y)
      thePickle=p.getvalue()

    This has little or no impact on pickling time, but
    often reduces unpickling time and pickle size, sometimes
    significantly.

  - Changed unpickler to use internal data structure instead
    of list to reduce unpickling times by about a third.

  - Many cleanups to get rid of obfuscated error handling
    involving 'goto finally' and status variables.

  - Extensive reGuidofication. (formatting :)

  - Fixed binary floating-point pickling bug. 0.0 was not
    pickled correctly.

  - Now use binary floating point format when saving
    floats in binary mode.

  - Fixed some error message spelling error.
1998-11-25 16:18:00 +00:00
Jeremy Hylton ce616e4009 Enter Jim Fulton's latest version. He writes:
I had to make a slight diddle to work with Python 1.4, which
we and some of our customers are still using. :(

I've also made a few minor enhancements:

  - You can now both get and set the memo using a 'memo'
    attribute.  This is handy for certain advanced applications
    that we have.

  - Added a 'binary' attribute to get and set the binary
    mode for a pickler.

  - Added a somewhat experimental 'fast' attribute.  When this
    is set, objects are not placed in the memo during pickling.
    This should lead to faster pickling and smaller pickles in
    cases where:

      o you *know* there are no circular references, and

      o either you've:

        - preloaded the memo with class information
          by pickling classes in non-fast mode or by
          manipilating the memo directly, or

        - aren't pickling instances.
1998-08-13 23:13:52 +00:00
Jeremy Hylton d10552379d Two fixes to find_class:
1. Only DECREF the class's module when the module is retrieved via
PyImport_Import.  If it is retrieved from the modules dictionary with
PyDict_GetItem, it is using a borrowed reference.

2. If the module doesn't define the desired class, raise the same
SystemError that pickle.py does instead of returning an AttributeError
(which is cryptic at best).

Also, fix the PyArg_ParseTuple in cpm_loads (the externally visible
loads) function:  Use "S" instead of "O" because cStringIO will croak
with a "bad arguments to internal function" if passed anything other
than a string.
1998-08-11 19:52:51 +00:00
Guido van Rossum e2d81cd4d7 Jim Fulton's patches to get rid of the class_map(). 1998-08-08 19:40:10 +00:00
Fred Drake 764b984db5 Use PyErr_ExceptionMatches(...) instead of PyErr_Occurred() == ... in two
places.
1998-05-28 04:33:37 +00:00
Guido van Rossum ed33a3f415 whichmodule(): remove redundant PyErr_Clear(); add explicit setting
of error when sys.modules isn't there.
1998-05-14 02:34:46 +00:00
Guido van Rossum 104be4a4a7 Use %.17f to format floats/doubles 1998-04-03 21:13:02 +00:00