Commit Graph

248 Commits

Author SHA1 Message Date
Guido van Rossum 5aac4e6312 Move _better_reduce from copy.py to copy_reg.py, and also use it in
pickle.py, where it makes save_newobj() unnecessary.  Tests pass.
2003-02-06 22:57:00 +00:00
Guido van Rossum 98b922c0ec Remove a debug print statement. 2003-02-06 19:30:38 +00:00
Jeremy Hylton cc1fccbc1c No need for a continuation line. 2003-02-06 16:23:01 +00:00
Jeremy Hylton 656d4519b5 Replace hasattr() + getattr() with single getattr() and default value. 2003-02-06 16:22:01 +00:00
Guido van Rossum d4b920c8bd Rename the extension registry variables to have leading underscores --
this clarifies that they are part of an internal API (albeit shared
between pickle.py, copy_reg.py and cPickle.c).

I'd like to do the same for copy_reg.dispatch_table, but worry that it
might be used by existing code.  This risk doesn't exist for the
extension registry.
2003-02-04 01:54:49 +00:00
Guido van Rossum 93fe564217 _slotnames(): this is a fairly expensive calculation. Cache the
outcome as __slotnames__ on the class.  (Like __slots__, it's not safe
to ask for this as an attribute -- you must look for it in the
specific class's __dict__.  But it must be set using attribute
notation, because __dict__ is a read-only proxy.)
2003-02-03 19:46:54 +00:00
Guido van Rossum 868ecc22ab _slotnames(): exclude __dict__ and __weakref__; these aren't real
slots even though they can be listed in __slots__.
2003-02-03 18:10:09 +00:00
Guido van Rossum 795ea89cb5 Support keyword argument 'bin', with a pending deprecation warning. 2003-02-03 16:59:48 +00:00
Tim Peters 1d63c9f151 cPickle support for TUPLE[123]. Incidentally plugged several undetected
overflow holes in Pdata_grow().
2003-02-02 20:29:39 +00:00
Tim Peters bf2674be0e long(string, base) now takes time linear in len(string) when base is a
power of 2.  Enabled the tail end of test_long() in pickletester.py
because it no longer takes forever when run from test_pickle.py.
2003-02-02 07:51:32 +00:00
Tim Peters ee1a53cbb1 cPickle.c: Full support for the new LONG1 and LONG4. Added comments.
Assorted code cleanups; e.g., sizeof(char) is 1 by definition, so there's
no need to do things like multiply by sizeof(char) in hairy malloc
arguments.  Fixed an undetected-overflow bug in readline_file().

longobject.c:  Fixed a really stupid bug in the new _PyLong_NumBits.

pickle.py:  Fixed stupid bug in save_long():  When proto is 2, it
wrote LONG1 or LONG4, but forgot to return then -- it went on to
append the proto 1 LONG opcode too.
Fixed equally stupid cancelling bugs in load_long1() and
load_long4():  they *returned* the unpickled long instead of pushing
it on the stack.  The return values were ignored.  Tests passed
before only because save_long() pickled the long twice.

Fixed bugs in encode_long().

Noted that decode_long() is quadratic-time despite our hopes,
because long(string, 16) is still quadratic-time in len(string).
It's hex() that's linear-time.  I don't know a way to make decode_long()
linear-time in Python, short of maybe transforming the 256's-complement
bytes into marshal's funky internal format, and letting marshal decode
that.  It would be more valuable to make long(string, 16) linear time.

pickletester.py:  Added a global "protocols" vector so tests can try
all the protocols in a sane way.  Changed test_ints() and test_unicode()
to do so.  Added a new test_long(), but the tail end of it is disabled
because it "takes forever" under pickle.py (but runs very quickly under
cPickle:  cPickle proto 2 for longs is linear-time).
2003-02-02 02:57:53 +00:00
Tim Peters 5bd2a79b22 The C pickle now knows how to deal with a proto= argument. Assorted
code cleanups, and purged more references to text-vs-binary modes.
2003-02-01 16:45:06 +00:00
Guido van Rossum 7eff63abce Change the default protocol back to 0.
Add a feature suggested by Tim: a negative protocol value means to use
the largest protocol value supported.
2003-01-31 19:42:31 +00:00
Guido van Rossum 25cb7dfb0f Another extension to reduce(). It can return a 4- or 5-tuple now.
The 4th item can be None or an iterator yielding list items, which are
used to append() or extend() the object.  The 5th item can be None or
an iterator yielding a dict's (key, value) pairs, which are stuffed
into the object using __setitem__.

Also (as a separate, though related, feature) add "batching" for list
and dict items.  If you pickled a dict or list with a million items in
the past, it would push a million items onto the stack.  It now pushes
only 1000 items at a time on the stack, using repeated APPENDS or
SETITEMS opcodes.  (For lists, I hope that using many short extend()
calls doesn't exhibit quadratic behavior.)
2003-01-31 18:53:21 +00:00
Jeremy Hylton 4f0dcc9a9a Provide __module__ attributes for functions defined in C and Python.
__module__ is the string name of the module the function was defined
in, just like __module__ of classes.  In some cases, particularly for
C functions, the __module__ may be None.

Change PyCFunction_New() from a function to a macro, but keep an
unused copy of the function around so that we don't change the binary
API.

Change pickle's save_global() to use whichmodule() if __module__ is
None, but add the __module__ logic to whichmodule() since it might be
used outside of pickle.
2003-01-31 18:33:18 +00:00
Guido van Rossum f7f4517fae Pass the object to save_reduce(), so the memoize() call can go into
save_reduce(), before the state is pickled.  This makes it possible
for an object to be referenced from its own (mutable) state.
2003-01-31 17:17:49 +00:00
Guido van Rossum d053b4b416 Add a magical feature to save_reduce so that __reduce__ can cause
NEWOBJ to be generated.
2003-01-31 16:51:45 +00:00
Tim Peters 4b23f2b44b It's Official: for LONG1/LONG4, a "byte count" of 0 is taken as a
shortcut meaning 0L.  This allows LONG1 to encode 0L in two bytes
total.
2003-01-31 16:43:39 +00:00
Neal Norwitz d17406830c Fix typo 2003-01-31 04:04:23 +00:00
Tim Peters 91149821d3 Linear-time implementations of {encode,decode}_long. 2003-01-31 03:43:58 +00:00
Tim Peters d01c1e91c4 load_inst(), load_obj(): Put the bulk of these into a common new
_instantiate() method.
2003-01-30 15:41:46 +00:00
Guido van Rossum 9b40e804c7 There was a subtle big in save_newobj(): it used self.save_global(t)
on the type instead of self.save(t).  This defeated the purpose of
NEWOBJ, because it didn't generate a BINGET opcode when t was already
memoized; but moreover, it would generate multiple BINPUT opcodes for
the same type!  pickletools.dis() doesn't like this.

How I found this?  I was playing with picklesize.py in the datetime
sandbox, and noticed that protocol 2 pickles for multiple objects were
in fact larger than protocol 1 pickles!  That was suspicious, so I
decided to disassemble one of the pickles.

This really needs a unit test, but I'm exhausted.  I'll be late for
work as it is. :-(
2003-01-30 06:37:41 +00:00
Guido van Rossum 4fba220f4a Slight code rearrangement to avoid testing getstate twice. 2003-01-30 05:41:19 +00:00
Guido van Rossum 45486176ea In save_newobj(), if an object's __getnewargs__ and __getstate__ are
the same function, don't save the state or write a BUILD opcode.  This
is so that a type (e.g. datetime :-) can support protocol 2 using
__getnewargs__ while also supporting protocol 0 and 1 using
__getstate__.  (Without this, the state would be pickled twice with
protocol 2, unless __getstate__ is defined to return None, which
breaks protocol 0 and 1.)
2003-01-30 05:39:04 +00:00
Guido van Rossum ba884f3d22 Use %c rather than chr() to turn some ints into chars. 2003-01-29 20:14:23 +00:00
Guido van Rossum 5d9113d8be Implement appropriate __getnewargs__ for all immutable subclassable builtin
types.  The special handling for these can now be removed from save_newobj().
Add some testing for this.

Also add support for setting the 'fast' flag on the Python Pickler class,
which suppresses use of the memo.
2003-01-29 17:58:45 +00:00
Guido van Rossum 586c9e813c Declare Protocol 2 as implemented. 2003-01-29 06:16:12 +00:00
Guido van Rossum 255f3ee0a5 Support for extension codes. (By accident I checked in the tests first.) 2003-01-29 06:14:11 +00:00
Tim Peters c0c12b5707 pickle: Comment repair.
pickletools:  Import decode_long from pickle instead of duplicating it.
2003-01-29 00:56:17 +00:00
Guido van Rossum 4e2491dbb1 Add a comment about how some built-in types should grow a
__getnewargs__ method.
2003-01-28 22:31:25 +00:00
Guido van Rossum b26a97aa50 Get rid of __safe_for_unpickling__ and safe_constructors.
Also tidied up a few lines, got rid of apply(), added a comment.
2003-01-28 22:29:13 +00:00
Guido van Rossum ac5b5d2e8b Instead of bad hacks trying to worm around the inherited
object.__reduce__, do a getattr() on the class so we can explicitly
test for it.  The reduce()-calling code becomes a bit more regular as
a result.

Also add support slots: if an object has slots, the default state is
(dict, slots) where dict is the __dict__ or None, and slots is a dict
mapping slot names to slot values.  We do a best-effort approach to
find slot names, assuming the __slots__ fields of classes aren't
modified after class definition time to misrepresent the actual list
of slots defined by a class.
2003-01-28 22:01:16 +00:00
Guido van Rossum 3d8c01b31c The default __reduce__ on the base object type obscured any
possibility of calling save_reduce().  Add a special hack for this.
The tests for this are much simpler now (no __getstate__ or
__getnewargs__ needed).
2003-01-28 19:48:18 +00:00
Guido van Rossum 54fb192508 Move the NEWOBJ-generating code to a separate function, and invoke it
after checking for __reduce__.
2003-01-28 18:22:35 +00:00
Guido van Rossum 533dbcf250 Some experimental support for generating NEWOBJ with proto=2, and
fixed a bug in load_newobj().
2003-01-28 17:55:05 +00:00
Tim Peters a6ae9a2128 save_empty_tuple(): Comment on why we can't get rid of this. 2003-01-28 16:58:41 +00:00
Tim Peters 82ca59e002 save_dict(): Added a comment about the control flow NealN missed. 2003-01-28 16:47:59 +00:00
Tim Peters 13a25fb8e6 _is_string_secure(): This method is no longer used; removed it. (It
was used before string-escape codecs were added to the core.)
2003-01-28 16:42:22 +00:00
Guido van Rossum bc64e22ed6 Made save() fit on a page, while adding comments. (I moved some type
checks to save_reduce(), which can also be called from a subclass.)

Also tweaked some more comments.
2003-01-28 16:34:19 +00:00
Tim Peters ad5a771fae Got rid of the _quotes global. Used only once, and is trivial. 2003-01-28 16:23:33 +00:00
Guido van Rossum 1be3175992 Add a few comments. Change the way the protocol is checked (it must
be one of 0, 1 or 2).

I should note that the previous checkin also added NEWOBJ support to
the unpickler -- but there's nothing yet that generates this.
2003-01-28 15:19:53 +00:00
Guido van Rossum 3a41c61dd4 Rename all variables 'object' to 'obj' to avoid conflicts with the
type 'object'.  Also minor docstring tweakage, and rearranged a few
lines in save().
2003-01-28 15:10:22 +00:00
Guido van Rossum cbe2dbddda Don't memoize the empty tuple in protocol 0. 2003-01-28 14:40:16 +00:00
Tim Peters d97da80dd5 save_tuple(): So long as the charter is rewriting for clarity, the snaky
control flow had to be simplified.
2003-01-28 05:48:29 +00:00
Tim Peters ff57bff16e save_tuple(): I believe the new code for TUPLE{1,2,3} in proto 2 was
incorrect for recursive tuples.  Tried to repair; seems to work OK, but
there are no checked-in tests for this yet.
2003-01-28 05:34:53 +00:00
Guido van Rossum 7d97d31a1b OK, this is really the last one tonight!
NEWFALSE and NEWTRUE.
2003-01-28 04:25:27 +00:00
Guido van Rossum 44f0ea5f73 More protocol 2: TUPLE1, TUPLE2, TUPLE3.
Also moved the special case for empty tuples from save() to save_tuple().
2003-01-28 04:14:51 +00:00
Tim Peters 3b769835ca save_inst(): Rewrote to have only one branch on self.bin. Also got rid
of my recent XXX comment, taking a (what appears to be vanishingly small)
chance and calling self.memoize() instead.
2003-01-28 03:51:36 +00:00
Guido van Rossum d6c9e63af9 First baby steps towards implementing protocol 2: PROTO, LONG1 and LONG4. 2003-01-28 03:49:52 +00:00
Tim Peters d95c2df3a9 Fixed odd whitespace after "if", which I believe I introduced long ago. 2003-01-28 03:41:54 +00:00
Tim Peters 8fda7bc48d save_int(): Fixed two new off-by-1 glitches. 2003-01-28 03:40:52 +00:00
Guido van Rossum e0b904232f Add a comment explaining that struct.pack() beats marshal.dumps(), but
marshal.loads() beats struct.unpack()!  Possibly because the latter
creates a one-tuple. :-(
2003-01-28 03:17:21 +00:00
Guido van Rossum 5c938d00a1 Got rid of mdumps; I timed it, and struct.pack("<i", x) is more than
40% faster than marshal.dumps(x)[1:]!  (That's not counting the
module attribute lookups, which can be avoided in either case.)
2003-01-28 03:03:08 +00:00
Tim Peters f558da0f90 save_tuple(): Minor rewriting, and added a comment about the subtlety
created by recursive tuples.
2003-01-28 02:09:55 +00:00
Tim Peters 209ad95b00 load_appends(): replaced .append() loop with an .extend(). 2003-01-28 01:44:45 +00:00
Tim Peters c23d18a955 Comments. 2003-01-28 01:41:51 +00:00
Tim Peters 064567e41a save_dict(): Untangled most of the bin-vs-not-bin logic. Also used
iteritems() instead of materializing a (possibly giant) list of the
items.
2003-01-28 01:34:43 +00:00
Tim Peters 21c18f0bf5 save_list(): Rewrote, to untangle the proto 0 from the proto 1 cases.
The code is much easier to follow now, and I bet it's faster too.
2003-01-28 01:15:46 +00:00
Tim Peters 22dc6f4f7a save_list(): removed unused local "d". 2003-01-28 01:07:48 +00:00
Tim Peters 58d7846e68 save_list(): removed unused local "memo". 2003-01-28 01:07:10 +00:00
Tim Peters bd1cdb9227 save_pers(): Switched the order of cases, to get rid of a "not", and to
make the bin-vs-not-bin order consistent with what other routines try to
do (they almost all handle the bin case first).
2003-01-28 01:03:10 +00:00
Tim Peters 518df0dae4 Several routines appeared to inline the guts of memoize(), possibly for
some notion of low-level efficiency.  Undid that, but left one routine
alone:  save_inst() claims it has a reason for not using memoize().
I don't understand that comment, so added an XXX comment there.
2003-01-28 01:00:38 +00:00
Tim Peters b32a8317d7 save(): Fiddled the control flow to put the normal case where it
belongs.  This is a much smaller change than it may appear:  the bulk
of the function merely got unindented by one level.
2003-01-28 00:48:09 +00:00
Tim Peters c9d7c4a656 Added XXX about save()'s special-casing of tuples -- I don't get it. 2003-01-28 00:43:26 +00:00
Tim Peters 22987e3cbd save_bool(): simplified. 2003-01-28 00:26:14 +00:00
Tim Peters cbd0a32d09 Repaired grammar in new comment. 2003-01-28 00:24:43 +00:00
Tim Peters b377f8ac3e clear_memo(): Repaired grammar in docstring. 2003-01-28 00:23:36 +00:00
Tim Peters e105478a0c Added one-line comments to the proto 2 opcodes. 2003-01-28 00:22:12 +00:00
Tim Peters fdb8cfab08 Removed the new LONG2 opcode: it's extravagant. If LONG1 isn't enough,
then the embedded argument consumes at least 256 bytes.  The difference
between a 3-byte prefix (LONG2 + 2 bytes) and a 5-byte prefix (LONG4 +
4 bytes) is at worst less than 1%.  Note that binary strings and binary
Unicode strings also have only "size is 1 byte, or size is 4 bytes?"
flavors, and I expect for the same reason.  The only place a 2-byte
thingie was used was in BININT2, where the 2 bytes make up the *entire*
embedded argument (and now EXT2 also does this); that's a large savings
over 4 bytes, because the total opcode+argument size is so small in
the BININT2/EXT2 case.

Removed the TAKEN_FROM_ARGUMENT "number of bytes" code, and bifurcated it
into TAKEN_FROM_ARGUMENT1 and TAKEN_FROM_ARGUMENT4.  Now there's enough
info in ArgumentDescriptor objects to deduce the # of bytes consumed by
each opcode.

Rearranged the order in which proto2 opcodes are listed in pickle.py.
2003-01-28 00:13:19 +00:00
Guido van Rossum f29d3d6011 Begin the change from 'binary vs. text mode' to 'protocol 0, 1, 2'.
The protocol now defaults to 1.  Protocol 2 is still unimplemented.
2003-01-27 22:47:53 +00:00
Guido van Rossum 5a2d8f5e9a Begin documenting protocol 2. 2003-01-27 21:44:25 +00:00
Tim Peters bb38e306ae Added some comments. 2003-01-27 21:25:41 +00:00
Tim Peters e46b73f5b3 memoize(): Reworded the docs to try to disentangle the Pickler's memo
dict from the Unpickler's memo (which is a different beast!).
2003-01-27 21:22:10 +00:00
Tim Peters bbf63cd05c Using marshal functions to pack & unpack 1-byte ints is an obscure &
expensive way to spell chr() and ord().
2003-01-27 21:15:36 +00:00
Tim Peters 22a449aacc Added a brief comment to each pickle opcode declaration. 2003-01-27 20:16:36 +00:00
Jeremy Hylton 3422c99de1 Raise PicklingError when __reduce__() fails, and
add memoize() helper function to update the memo.

The first element of the tuple returned by __reduce__() must be a
callable.  If it isn't the Unpickler will raise an error.  Catch this
error in the pickler and raise the error there.

The memoize() helper also has a comment explaining how the memo
works.  So methods can't use memoize() because the write funny codes.
2003-01-24 19:29:52 +00:00
Jeremy Hylton 5e0f4e73a9 Remove inst_persistent_id() WANNI (we ain't never needed it).
Add some simple tests of the persistence hooks.
2002-11-13 22:01:27 +00:00
Jeremy Hylton f0cfdf7314 Fiddle comments and variable names in whichmodule(). 2002-09-19 23:00:12 +00:00
Jeremy Hylton 065a5ab8fb whichmodule() should skip dummy package entries in sys.modules.
This fixes the charming, but unhelpful error message for
>>> pickle.dumps(type.__new__)
Can't pickle <built-in method __new__ of type object at 0x812a440>: it's not the same object as datetime.math.__new__

Bugfix candidate.
2002-09-19 22:57:26 +00:00
Martin v. Löwis 8a8da798a5 Patch #505705: Remove eval in pickle and cPickle. 2002-08-14 07:46:28 +00:00
Jeremy Hylton 39c6116483 Given the persistent id code a shot at a class before calling save_global().
Some persistent picklers (well, probably, the *only* persistent
pickler) would like to pickle some classes in a special way.
2002-07-16 19:47:43 +00:00
Raymond Hettinger 46ac8eb3c8 Code modernization. Replace v=s[i]; del s[i] with single lookup v=s.pop(i) 2002-06-30 03:39:14 +00:00
Raymond Hettinger 54f0222547 SF 563203. Replaced 'has_key()' with 'in'. 2002-06-01 14:18:47 +00:00
Neal Norwitz efbb67b1a7 Remove comment about inheritance, look one line up 2002-05-30 12:12:04 +00:00
Raymond Hettinger aef22fb9cd Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). 2002-05-29 16:18:42 +00:00
Tim Peters 8ac1495a6a Whitespace normalization. 2002-05-23 15:15:30 +00:00
Raymond Hettinger 97394bc795 Patch 533291. Deprecate None return form of __reduce__. 2002-05-21 17:22:02 +00:00
Fred Drake 7f781c9aab Add Pickler.clear_memo() so the pickle and cPickle modules are more similar. 2002-05-01 20:33:53 +00:00
Guido van Rossum e276339cea Implement an idea by Paul Rubin:
Change pickling format for bools to use a backwards compatible
encoding.  This means you can pickle True or False on Python 2.3
and Python 2.2 or before will read it back as 1 or 0.  The code
used for pickling bools before would create pickles that could
not be read in previous Python versions.
2002-04-05 19:30:08 +00:00
Guido van Rossum 77f6a65eb0 Add the 'bool' type and its values 'False' and 'True', as described in
PEP 285.  Everything described in the PEP is here, and there is even
some documentation.  I had to fix 12 unit tests; all but one of these
were printing Boolean outcomes that changed from 0/1 to False/True.
(The exception is test_unicode.py, which did a type(x) == type(y)
style comparison.  I could've fixed that with a single line using
issubtype(x, type(y)), but instead chose to be explicit about those
places where a bool is expected.

Still to do: perhaps more documentation; change standard library
modules to return False/True from predicates.
2002-04-03 22:41:51 +00:00
Guido van Rossum 85ee491b3a Fix for SF 502085.
Don't die when issubclass(t, TypeType) fails.

Bugfix candidate (but I think it's too late for 2.2.1).
2002-03-26 00:51:56 +00:00
Neal Norwitz d5ba4aeab0 SF #515018, delete global variable that was apparently used only
in a list comprehension.
2002-02-11 18:12:06 +00:00
Guido van Rossum f048a8f6d7 Pickler.save(): Fix for SF bug #494904: Cannot pickle a class with a
metaclass, reported by Dan Parisien.

Objects that are instances of custom metaclasses, i.e. whose class is
a subclass of 'type', should be pickled the same as new-style classes
(objects whose class is 'type').  This can't be done through a
dispatch table entry, and the __reduce__ trick doesn't work for these,
since it finds the unbound __reduce__ for instances of the class
(inherited from 'object').  So check explicitly using issubclass().
2001-12-19 16:55:02 +00:00
Barry Warsaw bf4d959d28 Two changes:
load_inst(): Implement the security hook that cPickle already had.
When unpickling callables which are not classes, we look to see if the
object has an attribute __safe_for_unpickling__.  If this exists and
has a true value, then we can call it to create the unpickled object.
Otherwise we raise an UnpicklingError.

find_class(): We no longer mask ImportError, KeyError, and
AttributeError by transforming them into SystemError.  The latter is
definitely not the right thing to do, so we let the former three
exceptions simply propagate up if they occur, i.e. we remove the
try/except!
2001-11-15 23:42:58 +00:00
Jeremy Hylton 20747fa167 A better new, unique object 2001-11-09 16:15:04 +00:00
Jeremy Hylton abe2c62bdb Use cStringIO when available.
Remove test code.  It's available in Lib/test/picklertester.py.
2001-10-15 21:29:28 +00:00
Guido van Rossum dbb718fa87 Make these modules work when Python is compiled without Unicode support. 2001-09-21 19:22:34 +00:00
Tim Peters 19ef62d5a9 pickle.py, load_int(): Match cPickle's just-repaired ability to unpickle
64-bit INTs on 32-bit boxes (where they become longs).  Also exploit that
int(str) and long(str) will ignore a trailing newline (saves creating a
new string at the Python level).

pickletester.py:  Simulate reading a pickle produced by a 64-bit box.
2001-08-28 22:21:18 +00:00
Guido van Rossum b0a98e9c94 Address SF #451547. The approach is a bit draconian: any object that
is pickled as a global must now exist by the name under which it is
pickled, otherwise the pickling fails.  Previously, such things would
fail on unpickling, or unpickle as the wrong global object.  I'm
hoping that this won't break existing code that is playing tricks with
this.

I need a volunteer to do this for cPickle too.
2001-08-17 18:49:52 +00:00
Tim Peters 6d6c1a35e0 Merge of descr-branch back into trunk. 2001-08-02 04:15:00 +00:00
Tim Peters 44714007e8 test_pickle works on sizeof(long)==8 boxes again.
pickle.py
    The code implicitly assumed that all ints fit in 4 bytes, causing all
    sorts of mischief (from nonsense results to corrupted pickles).
    Repaired that.
marshal.c
    The int marshaling code assumed that right shifts of signed longs
    sign-extend.  Repaired that.
2001-04-10 05:02:52 +00:00
Tim Peters c32d824467 Mechanical fiddling for easier reading: every "if" test was enclosed in
parens, but no "while" test.  Removed the former.
2001-04-10 02:48:53 +00:00
Skip Montanaro 23bafc6fcd add module-level constants to __all__ 2001-02-18 03:10:09 +00:00
Tim Peters 658cba6706 Whitespace normalization. 2001-02-09 20:06:00 +00:00
Skip Montanaro 352674d01c a few more __all__ lists 2001-02-07 23:14:30 +00:00
Guido van Rossum 31584cb43d Finn Bock (SF patch #103349):
Allow pickle.py to be using with Jython unicode strings
2001-01-22 14:53:29 +00:00
Tim Peters 2344fae6d0 Whitespace normalization. 2001-01-15 00:50:52 +00:00
Guido van Rossum a31b4ef7c5 Minimal fix for the complaints about pickling Unicode objects. (SF
bugs #126161 and 123634).

The solution doesn't use the unicode-escape encoding; that has other
problems (it seems not 100% reversible).  Rather, it transforms the
input Unicode object slightly before encoding it using
raw-unicode-escape, so that the decoding will reconstruct the original
string: backslash and newline characters are translated into their
\uXXXX counterparts.

This is backwards incompatible for strings containing backslashes, but
for some of those strings, the pickling was already broken.
2000-12-19 01:29:00 +00:00
Guido van Rossum ff87174db1 Get rid of string module and string exceptions. 2000-12-13 18:11:56 +00:00
Jeremy Hylton be467e5c69 Fix Bug #114293:
Strings are unpickled by calling eval on the string's repr. This
    change makes pickle work like cPickle; it checks if the pickled
    string is safe to eval and raises ValueError if it is not.

test suite modifications:
    Verify that pickle catches a variety of insecure string pickles
    Make test_pickle and test_cpickle use exactly the same test suite
    Add test for pickling recursive object
2000-09-15 15:14:51 +00:00
Guido van Rossum 26e89d0185 Change string exceptions to class exceptions. 2000-06-29 16:15:52 +00:00
Guido van Rossum b5f2f1bb6f Marc-Andre Lemburg: support pickling Unicode objects, both in text
mode ('V') and in binary mode ('X').
2000-03-10 23:20:09 +00:00
Guido van Rossum 54f22ed30b More trivial comment -> docstring transformations by Ka-Ping Yee,
who writes:

Here is batch 2, as a big collection of CVS context diffs.
Along with moving comments into docstrings, i've added a
couple of missing docstrings and attempted to make sure more
module docstrings begin with a one-line summary.

I did not add docstrings to the methods in profile.py for
fear of upsetting any careful optimizations there, though
i did move class documentation into class docstrings.

The convention i'm using is to leave credits/version/copyright
type of stuff in # comments, and move the rest of the descriptive
stuff about module usage into module docstrings.  Hope this is
okay.
2000-02-04 15:10:34 +00:00
Guido van Rossum 08a92cb568 Jeremy writes:
I found the following patch helpful in tracking down a bug in some
code.  I had appended time, the module, instead of time.time().  Not
sure if it is generally true that printing the repr of the object is
good, but I expect that most unpicklable things will have fairly
information and concise reprs (like files or sockets or modules).
1999-10-10 21:14:25 +00:00
Guido van Rossum 397032aa46 Don't use "exec" in find_class(). It's slow, unnecessary, and (as AMK
points out) it doesn't work in JPython Applets.
1999-03-25 21:58:59 +00:00
Guido van Rossum d370379186 Jim Fulton writes:
"""
I've attached a long overdue patch to pickle.py to bring it to format
1.3, which is the same as 1.2 except that the binary float format
is supported.  This is done using the new platform-indepent format
features of struct.

This patch also gets rid of the undocumented obsolete Pickler
dump_special method.
"""
1998-10-22 20:15:36 +00:00
Guido van Rossum 743d17e3aa In load_inst(), when instantiating an instance the old way (i.e. when
there's an __getinitargs__() method), if a TypeError occurs, catch and
reraise it but add info to the error about the class name being
instantiated.  This makes debugging a lot easier if __getinitargs__()
returns something bogus (e.g. a string instead of a singleton tuple).
1998-09-15 20:25:57 +00:00
Jeremy Hylton 2b9d029308 add handler for JPython's org.python.core.PyStringMap object, which
walks and quacks like a dictionary.
1998-05-27 22:38:22 +00:00
Guido van Rossum b19e2a383c Whoops! Add a missing 'instantiated = 1' to load_inst(); otherwise it
would still try to call the class...
1998-04-13 18:08:45 +00:00
Guido van Rossum b1062fc9d2 Correct dumb typo found by kjpylint (stack should be self.stack). 1998-03-31 17:00:46 +00:00
Guido van Rossum 599174f74d Correct a definite typo ('mem' should be 'memo'). 1998-03-31 16:30:28 +00:00
Guido van Rossum 45e2fbc2e7 Mass check-in after untabifying all files that need it. 1998-03-26 21:13:24 +00:00
Fred Drake fe82acc3b1 Fixed typo in docstring: "__ version__" --> "__version__" 1998-02-13 03:24:48 +00:00
Barry Warsaw abe2a457de Unpickler.load_inst(), Unpickler.load_obj(), Unpickler.load_build():
Fixed problems when unpickling in restricted execution environments.
These methods try to assign to an instance's __class__ attribute, or
access the instances __dict__, which are prohibited in REE.  For the
first two methods, I re-implemented the old behavior when assignment
to value.__class__ fails.

For the load_build() I also re-implemented the old behavior when
inst.__dict__.update() fails but this means that unpickling in REE is
semantically different than unpickling in unrestricted mode.
1998-01-26 22:47:35 +00:00
Guido van Rossum d1f4984a9b Jim Fulton writes:
The attached patch adds the following behavior to the handling
of REDUCE codes:

- A user-defined type may have a __reduce__ method that returns
  a string rather than a tuple, in which case the object is
  saved as a global object with a name given by the string returned
  by reduce.

  This was a feature added to cPickle a long time ago.


- User-defined types can now support unpickling without
  executing a constructor.

  The second value returned from '__reduce__' can now be None,
  rather than an argument tuple.  On unpickling, if the
  second value returned from '__reduce__' during pickling was
  None, then rather than calling the first value returned from
  '__reduce__', directly, the '__basicnew__' method of the
  first value returned from '__reduce__' is called without
  arguments.

I also got rid of a few of Chris' extra ()s, which he used
to make python ifs look like C ifs.
1997-12-10 23:40:18 +00:00
Guido van Rossum e680546894 Don't specify base 0 to string.atoi when unpickling integers in text
mode.  The pickler always uses base 10 so the default base should be
fine.  (The base gets us in trouble when there's no strop module, as
the atoi() in string.py only supports base 10.  This is for JPython.)
1997-12-10 19:36:41 +00:00
Guido van Rossum e467be6511 When instantiating a class with no arguments and where the class does
not define __getinitargs__, bypass the __init__ constructor
completely.  This uses the trick of instantiating an empty dummy class
and then changing inst.__class__ to the real class.  This is done in
two places: once for the INST and once for the OBJ format code.

Also replaced the much outdated long doc string with a short summary
of the module; the information of that doc string is already
incorporated in the library reference manual.
1997-12-05 19:42:42 +00:00
Guido van Rossum 4fb5b28dfc Three independent changes:
- Don't use "from copy_reg import *".

- Use cls.__module__ instead of calling whichobject(cls, cls.__name__);
  also try __module__ in whichmodule(), just in case.

- After calling save_reduce(), add the object to the memo.
1997-09-12 20:07:24 +00:00
Guido van Rossum d6ead328b3 *Semantic change*: when unpickling the instance variables of an
instance, use inst.__dict__.update(value) instead of a for loop with
setattr() over the value.keys().  This is more consistent (the
pickling doesn't use getattr() either but pickles inst.__dict__) and
avoids problems with instances that have a __setattr__ hook.

But it *is* a semantic change (because the setattr hook is no longer
used).  So beware!
1997-09-08 02:08:11 +00:00
Guido van Rossum 5ed5c4c364 Add the same _keep_alive patch (by Michael Scharff) that was added to
copy.deepcopy() a while ago.  Can't reproduce this but it doesn't
break anything and it looks like the code could have the same problem.
1997-09-03 00:23:54 +00:00
Guido van Rossum 8be9a11617 Restore two features of the original 1.4 pickle:
- which_module() search __main__ last;

- load_inst() no longer checks that the classname really refers to a class.
1997-04-25 19:52:29 +00:00
Guido van Rossum 5a56649e09 Remove Digital Creations copyright (Jim sez it's okay). 1997-04-11 22:31:56 +00:00
Guido van Rossum b72cf2d697 New version by Digital Creations, supports binary format compatible
with cPickle.
1997-04-09 17:32:51 +00:00
Guido van Rossum efd3a3a843 Implement find_class() without exec statement. 1997-03-14 04:21:10 +00:00
Guido van Rossum 36fc11e894 Compromise on test in find_class(): a user-defined function is okay,
but a built-in function is not.
1996-10-07 14:34:20 +00:00
Guido van Rossum abfd8064d8 This is the third time I check in this change :-(
Don't use assignments into inst.__dict__ to restore instance
variables; use setattr() instead.
1996-08-08 20:26:45 +00:00
Guido van Rossum 37a6f16d2e Slight update of doc string -- suggest default args for __init__, no
longer complain that __getinitargs__ is an ugly name.
1996-08-08 18:35:22 +00:00
Guido van Rossum c7c5e697c3 Optimizations and one intentional loophole by Jim Fulton.
The optimizations consist mostly of using local variables to cache
methods or instance variables used a lot (e.g. "self.write").

The loopholes allows marshalling extension types as long as they have
a __class__ attribute (in which case they may support the rest of the
class piclking protocol as well).  This allows pickling MESS extension
types.
1996-07-22 22:26:07 +00:00
Guido van Rossum 955c5d156d Set the base for atoi() and atol() to 0, since we're reading Python
numbers here, and so that atol() doesn't barf on the trailing 'L'.
Add a test case involving a long integer.
1996-05-15 22:49:57 +00:00
Guido van Rossum 7853647dd1 Don't atoi() the memo key.
Use atoi(), atol(), atof() instead of load_atomic for seed-up.
1996-04-12 13:36:27 +00:00
Guido van Rossum e0bfd50d76 correct typo (persis*ent) 1995-08-07 20:16:58 +00:00
Guido van Rossum f71c79bb85 test other name variable 1995-06-22 18:51:23 +00:00
Guido van Rossum 48aa82e2df correct typo in example 1995-04-10 11:34:46 +00:00
Guido van Rossum 0c891ce61a pickle classes; add format_version, load(s)/dump(s) shortcuts 1995-03-14 15:09:05 +00:00
Guido van Rossum 7849da8631 added PicklingError exception 1995-03-09 14:08:35 +00:00
Guido van Rossum 7b5430f2e8 raise EOFError when load() hits EOF, instead of KeyError 1995-03-04 22:25:21 +00:00
Guido van Rossum 256cbd7487 minute comment changes 1995-02-16 16:30:50 +00:00
Guido van Rossum a48061a580 shelve.py: database of persistent objects, on top of pickle.py and anydbm.py
pickle.py: new low-level persistency module (used to be called flatten)
dbmac.py: stupid dbm clone for the Mac
anydbm.py: generic dbm interface (should be extended to support gdbm)
1995-01-10 00:31:14 +00:00