Commit Graph

82 Commits

Author SHA1 Message Date
Phillip J. Eby 0d6615fd29 PEP 342 implementation. Per Guido's comments, the generator throw()
method still needs to support string exceptions, and allow None for the
third argument.  Documentation updates are needed, too.
2005-08-02 00:46:46 +00:00
Neil Schemenauer 3b1cbf9ba0 Add missing INCREF.
Backport candidate.
2005-06-18 17:37:06 +00:00
Tim Peters bc1d1b80d1 gc_list_move(): Make this truly equivalent to remove+append. While
nothing in gc currently cares, the original coding could screw up if,
e.g., you tried to move a node to the list it's already in, and the node
was already the last in its list.
2004-11-01 16:39:57 +00:00
Tim Peters e2d591847c gc list function cleanup.
Introduced gc_list_move(), which captures the common gc_list_remove() +
gc_list_append() sequence.  In fact, no uses of gc_list_append() remained
(they were all in a gc_list_move() sequence), so commented that one out.

gc_list_merge():  assert that `from` != `to`; that was an implicit
precondition, now verified in a debug build.

Others:  added comments about their purpose.
2004-11-01 01:39:08 +00:00
Tim Peters cc2a866cb7 handle_weakrefs(): Simplification -- there's no need to make a second
pass over the unreachable weakrefs-with-callbacks to unreachable objects.
2004-10-31 22:12:43 +00:00
Tim Peters ead8b7ab30 SF 1055820: weakref callback vs gc vs threads
In cyclic gc, clear weakrefs to unreachable objects before allowing any
Python code (weakref callbacks or __del__ methods) to run.

This is a critical bugfix, affecting all versions of Python since weakrefs
were introduced.  I'll backport to 2.3.
2004-10-30 23:09:22 +00:00
Raymond Hettinger 674d56b82e Convert return value to boolean. 2004-01-04 04:00:13 +00:00
Guido van Rossum 0bba722fff Silence GCC warning when asserts are turned off. 2003-11-24 04:02:13 +00:00
Tim Peters 403a203223 SF bug 839548: Bug in type's GC handling causes segfaults.
Also SF patch 843455.

This is a critical bugfix.
I'll backport to 2.3 maint, but not beyond that.  The bugs this fixes
have been there since weakrefs were introduced.
2003-11-20 21:21:46 +00:00
Tim Peters 780c497972 update_refs(): assert that incoming refcounts aren't 0. The comment
for this function has always claimed that was true, but it wasn't
verified before.  For the latest batch of "double deallocation" bugs
(stemming from weakref callbacks invoked by way of subtype_dealloc),
this assert would have triggered (instead of waiting for
_Py_ForgetReference to die with a segfault later).
2003-11-14 00:01:17 +00:00
Jason Tishler 6bc06eca70 Bug #794140: cygwin builds do not embed
The embed2.diff patch solves the user's problem by exporting the missing
symbols from the Python core so Python can be embedded in another Cygwin
application (well, at lest vim).
2003-09-04 11:59:50 +00:00
Guido van Rossum e13ddc9ec8 - New C API PyGC_Collect(), same as calling gc.collect().
- Call this in Py_Finalize().
- Expand the Misc/NEWS text on PY_LONG_LONG.
2003-04-17 17:29:22 +00:00
Tim Peters 730f5535ba s/referrents/referents/g. Gotta love that referrers remains rife with rs. 2003-04-08 17:17:17 +00:00
Tim Peters 0f81ab6d88 Finished implementing gc.get_referrents(): dealt with error and end
cases, wrote docs, added a test.
2003-04-08 16:39:48 +00:00
Tim Peters fb2ab4d5ae Comment repair; no semantic changes. 2003-04-07 22:41:24 +00:00
Tim Peters f6b8045ca5 Reworked has_finalizer() to use the new _PyObject_Lookup() instead
of PyObject_HasAttr(); the former promises never to execute
arbitrary Python code.  Undid many of the changes recently made to
worm around the worst consequences of that PyObject_HasAttr() could
execute arbitrary Python code.

Compatibility is hard to discuss, because the dangerous cases are
so perverse, and much of this appears to rely on implementation
accidents.

To start with, using hasattr() to check for __del__ wasn't only
dangerous, in some cases it was wrong:  if an instance of an old-
style class didn't have "__del__" in its instance dict or in any
base class dict, but a getattr hook said __del__ existed, then
hasattr() said "yes, this object has a __del__".  But
instance_dealloc() ignores the possibility of getattr hooks when
looking for a __del__, so while object.__del__ succeeds, no
__del__ method is called when the object is deleted.  gc was
therefore incorrect in believing that the object had a finalizer.

The new method doesn't suffer that problem (like instance_dealloc(),
_PyObject_Lookup() doesn't believe __del__ exists in that case), but
does suffer a somewhat opposite-- and even more obscure --oddity:
if an instance of an old-style class doesn't have "__del__" in its
instance dict, and a base class does have "__del__" in its dict,
and the first base class with a "__del__" associates it with a
descriptor (an object with a __get__ method), *and* if that
descriptor raises an exception when __get__ is called, then
(a) the current method believes the instance does have a __del__,
but (b) hasattr() does not believe the instance has a __del__.

While these disagree, I believe the new method is "more correct":
because the descriptor *will* be called when the object is
destructed, it can execute arbitrary Python code at the time the
object is destructed, and that's really what gc means by "has a
finalizer":  not specifically a __del__ method, but more generally
the possibility of executing arbitrary Python code at object
destruction time.  Code in a descriptor's __get__() executed at
destruction time can be just as problematic as code in a
__del__() executed then.

So I believe the new method is better on all counts.

Bugfix candidate, but it's unclear to me how all this differs in
the 2.2 branch (e.g., new-style and old-style classes already
took different gc paths in 2.3 before this last round of patches,
but don't in the 2.2 branch).
2003-04-07 19:21:15 +00:00
Tim Peters 1155887a74 initgc(): Rewrote to use the PyModule_AddXYZ API; cuts code size. 2003-04-06 23:30:52 +00:00
Tim Peters 259272b7a0 handle_finalizers(): Rewrote to call append_objects() and gc_list_merge()
instead of looping.  Smaller and clearer.  Faster, too, when we're not
appending to gc.garbage:  gc_list_merge() takes constant time, regardless
of the lists' sizes.

append_objects():  Moved up to live with the other list manipulation
utilities.
2003-04-06 19:41:39 +00:00
Tim Peters 50c61d5a6c Switched from METH_VARARGS to METH_NOARGS for the 7 module functions that
take no arguments; cuts generated code size.
2003-04-06 01:50:50 +00:00
Tim Peters bf384c256e Reworked move_finalizer_reachable() to create two distinct lists:
externally unreachable objects with finalizers, and externally unreachable
objects without finalizers reachable from such objects.  This allows us
to call has_finalizer() at most once per object, and so limit the pain of
nasty getattr hooks.  This fixes the failing "boom 2" example Jeremy
posted (a non-printing variant of which is now part of test_gc), via never
triggering the nasty part of its __getattr__ method.
2003-04-06 00:11:39 +00:00
Tim Peters f6ae7a43eb move_finalizers(): Rewrote. It's not necessary for this routine
to special-case classic classes, or to worry about refcounts;
has_finalizer() deleted the current object iff the first entry in
the unreachable list has changed.  I don't believe it was correct
to check for ob_refcnt == 1, either:  the dealloc routine would get
called by Py_DECREF then, but there's nothing to stop the dealloc
routine from ressurecting the object, and then gc would remain at
the head of the unreachable list despite that its refcount temporarily
fell to 0 (and that would lead to an infinite loop in move_finalizers()).

I'm still worried about has_finalizer() resurrecting other objects
in the unreachable list:  what's to stop them from getting collected?
2003-04-05 18:40:50 +00:00
Tim Peters 86b993b6cf New comments. Rewrote has_finalizer() as a sequence of ifs instead of
squashed-together conditional operators; makes it much easier to step
thru in the debugger, and to set a breakpoint on the only dangerous
path.
2003-04-05 17:35:54 +00:00
Tim Peters 93ad66dea9 Fixed new seemingly random segfaults, by moving the initialization of
delstr from initgc() into collect().  initgc() isn't called unless the
user explicitly imports gc, so can be used only for initialization of
user-visible module features; delstr needs to be initialized for proper
internal operation, whether or not gc is explicitly imported.

Bugfix candidate?  I don't know whether the new bug was backported to
2.2 already.
2003-04-05 17:15:44 +00:00
Jeremy Hylton ce136e985a Fix Tim's boom example.
move_finalizers() moves every object from unreachable to collectable
or finalizers, unless the object is deallocated first.
2003-04-04 19:59:06 +00:00
Jeremy Hylton 059b094e29 Add get_referrents() helper function. 2003-04-03 16:29:13 +00:00
Jeremy Hylton 5bd378bfca Add get_referrents() helper function. 2003-04-03 16:28:38 +00:00
Martin v. Löwis 774348c8d6 Fix typo. 2002-11-09 19:54:06 +00:00
Guido van Rossum 4030714a93 For new-style classes, we can now test for tp_del instead of asking
for a __del__ attribute, to see if there's a finalizer.
2002-08-09 17:39:14 +00:00
Tim Peters 803526b9e2 Trashcan cleanup: Now that cyclic gc is always there, the trashcan
mechanism is no longer evil:  it no longer plays dangerous games with
the type pointer or refcounts, and objects in extension modules can play
along too without needing to edit the core first.

Rewrote all the comments to explain this, and (I hope) give clear
guidance to extension authors who do want to play along.  Documented
all the functions.  Added more asserts (it may no longer be evil, but
it's still dangerous <0.9 wink>).  Rearranged the generated code to
make it clearer, and to tolerate either the presence or absence of a
semicolon after the macros.  Rewrote _PyTrash_destroy_chain() to call
tp_dealloc directly; it was doing a Py_DECREF again, and that has all
sorts of obscure distorting effects in non-release builds (Py_DECREF
was already called on the object!).  Removed Christian's little "embedded
change log" comments -- that's what checkin messages are for, and since
it was impossible to correlate the comments with the code that changed,
I found them merely distracting.
2002-07-07 05:13:56 +00:00
Tim Peters 943382c8e5 Removed WITH_CYCLE_GC #ifdef-ery. Holes:
+ I'm not sure what to do about configure.in.  Left it alone.

+ Ditto pyexpat.c.  Fred or Martin will know what to do.
2002-07-07 03:59:34 +00:00
Michael W. Hudson 8b7f131f8b gc_list_move defined but not used. 2002-07-04 17:11:36 +00:00
Tim Peters aab713bdf7 visit_decref(): Added another assert. 2002-07-02 22:15:28 +00:00
Tim Peters 6fc13d9595 Finished transitioning to using gc_refs to track gc objects' states.
This was mostly a matter of adding comments and light code rearrangement.
Upon untracking, gc_next is still set to NULL.  It's a cheap way to
provoke memory faults if calling code is insane.  It's also used in some
way by the trashcan mechanism.
2002-07-02 18:12:35 +00:00
Tim Peters ea405639bf Reserved another gc_refs value for untracked objects. Every live gc
object should now have a well-defined gc_refs value, with clear transitions
among gc_refs states.  As a result, none of the visit_XYZ traversal
callbacks need to check IS_TRACKED() anymore, and those tests were removed.
(They were already looking for objects with specific gc_refs states, and
the gc_refs state of an untracked object can no longer match any other
gc_refs state by accident.)
Added more asserts.
I expect that the gc_next == NULL indicator for an untracked object is
now redundant and can also be removed, but I ran out of time for this.
2002-07-02 00:52:30 +00:00
Tim Peters 19b74c7868 OK, I couldn't stand it <0.5 wink>: removed all uncertainty about what's
in gc_refs, even at the cost of putting back a test+branch in
visit_decref.

The good news:  since gc_refs became utterly tame then, it became
clear that another special value could be useful.  The move_roots() and
move_root_reachable() passes have now been replaced by a single
move_unreachable() pass.  Besides saving a pass over the generation, this
has a better effect:  most of the time everything turns out to be
reachable, so we were breaking the generation list apart and moving it
into into the reachable list, one element at a time.  Now the reachable
stuff stays in the generation list, and the unreachable stuff is moved
instead.  This isn't quite as good as it sounds, since sometimes we
guess wrongly that a thing is unreachable, and have to move it back again.

Still, overall, it yields a significant (but not dramatic) boost in
collection speed.
2002-07-01 03:52:19 +00:00
Tim Peters 93cd83e4ae visit_decref(): Two optimizations.
1. You're not supposed to call this with a NULL argument, although the
   docs could be clearer about that.  The other visit_XYZ() functions
   don't bother to check.  This doesn't either now, although it does
   assert non-NULL-ness now.

2. It doesn't matter whether the object is currently tracked, so don't
   bother checking that either (if it isn't currently tracked, it may
   have some nonsense value in gc_refs, but it doesn't hurt to
   decrement gibberish, and it's cheaper to do so than to make everyone
   test for trackedness).

It would be nice to get rid of the other tests on IS_TRACKED.  Perhaps
trackedness should not be a matter of not being in any gc list, but
should be a matter of being in a new "untracked" gc list.  This list
simply wouldn't be involved in the collection mechanism.  A newly
created object would be put in the untracked list.  Tracking would
simply unlink it and move it into the gen0 list.  Untracking would do
the reverse.  No test+branch needed then.  visit_move() may be vulnerable
then, though, and I don't know how this would work with the trashcan.
2002-06-30 21:31:03 +00:00
Tim Peters 8839617cc9 SF bug #574132: Major GC related performance regression
"The regression" is actually due to that 2.2.1 had a bug that prevented
the regression (which isn't a regression at all) from showing up.  "The
regression" is actually a glitch in cyclic gc that's been there forever.

As the generation being collected is analyzed, objects that can't be
collected (because, e.g., we find they're externally referenced, or
are in an unreachable cycle but have a __del__ method) are moved out
of the list of candidates.  A tricksy scheme uses negative values of
gc_refs to mark such objects as being moved.  However, the exact
negative value set at the start may become "more negative" over time
for objects not in the generation being collected, and the scheme was
checking for an exact match on the negative value originally assigned.
As a result, objects in generations older than the one being collected
could get scanned too, and yanked back into a younger generation.  Doing
so doesn't lead to an error, but doesn't do any good, and can burn an
unbounded amount of time doing useless work.

A test case is simple (thanks to Kevin Jacobs for finding it!):

x = []
for i in xrange(200000):
    x.append((1,))

Without the patch, this ends up scanning all of x on every gen0 collection,
scans all of x twice on every gen1 collection, and x gets yanked back into
gen1 on every gen0 collection.  With the patch, once x gets to gen2, it's
never scanned again until another gen2 collection, and stays in gen2.

Bugfix candidate, although the code has changed enough that I think I'll
need to port it by hand.  2.2.1 also has a different bug that causes
bound method objects not to get tracked at all (so the test case doesn't
burn absurd amounts of time in 2.2.1, but *should* <wink>).
2002-06-30 17:56:40 +00:00
Neil Schemenauer c9051640f8 Fix small bug. The count of objects in all generations younger then the
collected one should be zeroed.
2002-06-28 19:16:04 +00:00
Martin v. Löwis 14f8b4cfcb Patch #568124: Add doc string macros. 2002-06-13 20:33:02 +00:00
Jeremy Hylton 8a13518d25 Remove casts to PyObject * when declaration is for PyObject * 2002-06-06 23:23:55 +00:00
Neil Schemenauer a2b11ecb08 Add IS_TRACKED and IS_MOVED macros. This makes the logic a little more clear. 2002-05-21 15:53:24 +00:00
Neil Schemenauer 2880ae53e6 Move all data for a single generation into a structure. The set of
generations is now an array.  This cleans up some code and makes it easy
to change the number of generations.  Also, implemented a
gc_list_is_empty() function.  This makes the logic a little clearer in
places.  The performance impact of these changes should be negligible.

One functional change is that allocation/collection counters are always
zeroed at the start of a collection.  This should fix SF bug #551915.
This change is too big for back-porting but the minimal patch on SF
looks good for a bugfix release.
2002-05-04 05:35:20 +00:00
Tim Peters fa8efab30f _PyObject_GC_New: Could call PyObject_INIT with a NULL 1st argument.
_PyObject_GC_NewVar:  Could call PyObject_INIT_VAR likewise.

Bugfix candidate.
2002-04-28 01:57:25 +00:00
Neil Schemenauer fec4eb1be1 Allow PyObject_Del to be used as a function designator. Provide binary
compatibility function.

Make PyObject_GC_Track and PyObject_GC_UnTrack functions instead of
trivial macros wrapping functions.  Provide binary compatibility
functions.
2002-04-12 02:41:03 +00:00
Neil Schemenauer b883310d59 Make _PyObject_GC_UnTrack do nothing if WITH_CYCLE_GC is not defined. 2002-03-29 03:04:25 +00:00
Guido van Rossum ff413af605 This is Neil's fix for SF bug 535905 (Evil Trashcan and GC interaction).
The fix makes it possible to call PyObject_GC_UnTrack() more than once
on the same object, and then move the PyObject_GC_UnTrack() call to
*before* the trashcan code is invoked.

BUGFIX CANDIDATE!
2002-03-28 20:34:59 +00:00
Neil Schemenauer 1b0e4fcc29 Use pymalloc for realloc() as well. 2002-03-22 15:41:03 +00:00
Neil Schemenauer dcc819a5c9 Use pymalloc if it's enabled. 2002-03-22 15:33:15 +00:00
Neal Norwitz 2a47c0fa23 Fix spelling mistakes. Bugfix candidates. 2002-01-29 00:53:41 +00:00
Martin v. Löwis f8a6f241b3 Check for NULL return value of PyList_New (follow-up to patch #486743). 2001-12-02 18:31:02 +00:00