Commit Graph

39 Commits

Author SHA1 Message Date
Antoine Pitrou 58098a77e6 Issue #13992: The trashcan mechanism is now thread-safe. This eliminates
sporadic crashes in multi-thread programs when several long deallocator
chains ran concurrently and involved subclasses of built-in container
types.

Because of this change, a couple extension modules compiled for 2.7.4
(those which use the trashcan mechanism, despite it being undocumented)
will not be loadable by 2.7.3 and earlier. However, extension modules
compiled for 2.7.3 and earlier will be loadable by 2.7.4.
2012-09-06 00:59:49 +02:00
Gregory P. Smith 2839985c7e The unittest.TestCase.assertEqual() now displays the differences in lists,
tuples, dicts and sets on failure.

Many new handy type and comparison specific assert* methods have been added
that fail with error messages actually useful for debugging.  Contributed in
by Google and completed with help from mfoord and GvR at PyCon 2009 sprints.

Discussion lives in http://bugs.python.org/issue2578.
2009-03-31 16:54:10 +00:00
Antoine Pitrou f8387af262 Issue #4688: Add a heuristic so that tuples and dicts containing only
untrackable objects are not tracked by the garbage collector. This can
reduce the size of collections and therefore the garbage collection overhead
on long-running programs, depending on their particular use of datatypes.

(trivia: this makes the "binary_trees" benchmark from the Computer Language
Shootout 40% faster)
2009-03-23 18:41:45 +00:00
Amaury Forgeot d'Arc d8bcbf2b2e Re-enable tests, they were failing since gc.collect() clears the various freelists.
They still remain fragile.

For example, a call to assertEqual currently does not make any allocation
(which surprised me at first).
But this can change when gc.collect also deletes the numerous "zombie frames"
attached to each function.
2008-02-15 22:44:20 +00:00
Amaury Forgeot d'Arc f67abccf64 Temporarily let these tests pass 2008-02-15 21:27:44 +00:00
Neal Norwitz 0d4c06e06e Whitespace normalization. Ugh, we really need to do this more often.
You might want to review this change as it's my first time.  Be gentle. :-)
2007-04-25 06:30:05 +00:00
Collin Winter fef1dcf433 Convert test_gc to use unittest. 2007-04-06 20:00:05 +00:00
Barry Warsaw d3c38ff7f8 SF patch #1443865; gc.get_count() added and optional argument 'generation'
added to gc.collect().  Updated docs, unit test, and NEWS entry.

(Also, fixed a typo in NEWS.)
2006-03-07 09:46:03 +00:00
Tim Peters ead8b7ab30 SF 1055820: weakref callback vs gc vs threads
In cyclic gc, clear weakrefs to unreachable objects before allowing any
Python code (weakref callbacks or __del__ methods) to run.

This is a critical bugfix, affecting all versions of Python since weakrefs
were introduced.  I'll backport to 2.3.
2004-10-30 23:09:22 +00:00
Tim Peters 8805e66ec8 New tests identical to boom and boom2, except using new-style classes.
These never failed in 2.3, and the tests confirm it.  They still blow up
in the 2.2 branch, despite that all the gc-vs-__del__ fixes from 2.3
have been backported (and this is expected -- 2.2 needs more work than
2.3 needed).
2003-04-08 19:44:13 +00:00
Tim Peters 730f5535ba s/referrents/referents/g. Gotta love that referrers remains rife with rs. 2003-04-08 17:17:17 +00:00
Tim Peters 0f81ab6d88 Finished implementing gc.get_referrents(): dealt with error and end
cases, wrote docs, added a test.
2003-04-08 16:39:48 +00:00
Tim Peters f6b8045ca5 Reworked has_finalizer() to use the new _PyObject_Lookup() instead
of PyObject_HasAttr(); the former promises never to execute
arbitrary Python code.  Undid many of the changes recently made to
worm around the worst consequences of that PyObject_HasAttr() could
execute arbitrary Python code.

Compatibility is hard to discuss, because the dangerous cases are
so perverse, and much of this appears to rely on implementation
accidents.

To start with, using hasattr() to check for __del__ wasn't only
dangerous, in some cases it was wrong:  if an instance of an old-
style class didn't have "__del__" in its instance dict or in any
base class dict, but a getattr hook said __del__ existed, then
hasattr() said "yes, this object has a __del__".  But
instance_dealloc() ignores the possibility of getattr hooks when
looking for a __del__, so while object.__del__ succeeds, no
__del__ method is called when the object is deleted.  gc was
therefore incorrect in believing that the object had a finalizer.

The new method doesn't suffer that problem (like instance_dealloc(),
_PyObject_Lookup() doesn't believe __del__ exists in that case), but
does suffer a somewhat opposite-- and even more obscure --oddity:
if an instance of an old-style class doesn't have "__del__" in its
instance dict, and a base class does have "__del__" in its dict,
and the first base class with a "__del__" associates it with a
descriptor (an object with a __get__ method), *and* if that
descriptor raises an exception when __get__ is called, then
(a) the current method believes the instance does have a __del__,
but (b) hasattr() does not believe the instance has a __del__.

While these disagree, I believe the new method is "more correct":
because the descriptor *will* be called when the object is
destructed, it can execute arbitrary Python code at the time the
object is destructed, and that's really what gc means by "has a
finalizer":  not specifically a __del__ method, but more generally
the possibility of executing arbitrary Python code at object
destruction time.  Code in a descriptor's __get__() executed at
destruction time can be just as problematic as code in a
__del__() executed then.

So I believe the new method is better on all counts.

Bugfix candidate, but it's unclear to me how all this differs in
the 2.2 branch (e.g., new-style and old-style classes already
took different gc paths in 2.3 before this last round of patches,
but don't in the 2.2 branch).
2003-04-07 19:21:15 +00:00
Tim Peters bf384c256e Reworked move_finalizer_reachable() to create two distinct lists:
externally unreachable objects with finalizers, and externally unreachable
objects without finalizers reachable from such objects.  This allows us
to call has_finalizer() at most once per object, and so limit the pain of
nasty getattr hooks.  This fixes the failing "boom 2" example Jeremy
posted (a non-printing variant of which is now part of test_gc), via never
triggering the nasty part of its __getattr__ method.
2003-04-06 00:11:39 +00:00
Tim Peters 2f74fddfc1 test_boom: More comments. Also check that len(gc.garbage) doesn't
change (it would be another kind of bug if the trash cycle weren't
reclaimed).
2003-04-05 17:46:04 +00:00
Jeremy Hylton dee38ac7dd Add Tim's gc boom test to the test suite. 2003-04-04 20:00:04 +00:00
Tim Peters d92ae840e9 test_saveall(): Another small simplification; plus s/l/L/g.
test_del(), test_del_newclass():  No need to use apply() in these.
2002-08-11 04:15:09 +00:00
Tim Peters a1ad3f08ad And one more simplification to test_saveall(). 2002-08-10 21:32:16 +00:00
Tim Peters 4803c126a6 test_saveall(): Simplified a little, given that we only expect one item
in gc.garbage (so no need to loop looking for it -- it's there or it's
not).
2002-08-10 21:29:56 +00:00
Tim Peters c708c0a8c4 If any trash happened to be sitting around waiting to get collected at
the time it's called, test_saveall() made it look a leak, triggering
bogus warnings from regrtest's -l (findleaks) mode.
2002-08-10 21:20:54 +00:00
Guido van Rossum 4aa21aa5b3 Test finalizers and GC from inside __del__ for new classes. 2002-08-09 17:38:16 +00:00
Barry Warsaw 04f357cffe Get rid of relative imports in all unittests. Now anything that
imports e.g. test_support must do so using an absolute package name
such as "import test.test_support" or "from test import test_support".

This also updates the README in Lib/test, and gets rid of the
duplicate data dirctory in Lib/test/data (replaced by
Lib/email/test/data).

Now Tim and Jack can have at it. :)
2002-07-23 19:04:11 +00:00
Tim Peters c62b95e550 test_trashcan() and supporting class Ouch(): Jeremy noted that this test
takes much longer to run in the context of the test suite than when run in
isolation.  That's because it forces a large number of full collections,
which take time proportional to the total number of gc'ed objects in the
whole system.

But since the dangerous implementation trickery that caused this test to
fail in 2.0, 2.1 and 2.2 doesn't exist in 2.3 anymore (the trashcan
mechanism stopped doing evil things when the possibility for compiling
without cyclic gc was taken away), such an expensive test is no longer
justified.  This checkin leaves the test intact, but fiddles the
constants to reduce the runtime by about a factor of 5.
2002-07-11 19:07:45 +00:00
Guido van Rossum 3fcd45230f Whitespace nit. 2002-06-13 11:53:12 +00:00
Guido van Rossum 05e01ee114 Add a testcase to ensure that cycles going through the __class__ link
of a new-style instance are detected by the garbage collector.
2002-06-12 14:38:04 +00:00
Tim Peters d4ce758505 test_trashcan: reword obscure code.
Bugfix candidate.
2002-03-28 21:22:25 +00:00
Tim Peters d222559cdb New test_traschcan() test in test_gc, which reliably provokes segfaults
under 2.0, 2.1 and 2.2.

Bugfix candidate.
2002-03-28 21:08:30 +00:00
Tim Peters 1ce150c675 Remove obsolete __static__/__dynamic__ distinction. 2001-10-15 22:49:27 +00:00
Guido van Rossum 9475a2310d Enable GC for new-style instances. This touches lots of files, since
many types were subclassable but had a xxx_dealloc function that
called PyObject_DEL(self) directly instead of deferring to
self->ob_type->tp_free(self).  It is permissible to set tp_free in the
type object directly to _PyObject_Del, for non-GC types, or to
_PyObject_GC_Del, for GC types.  Still, PyObject_DEL was a tad faster,
so I'm fearing that our pystone rating is going down again.  I'm not
sure if doing something like

void xxx_dealloc(PyObject *self)
{
	if (PyXxxCheckExact(self))
		PyObject_DEL(self);
	else
		self->ob_type->tp_free(self);
}

is any faster than always calling the else branch, so I haven't
attempted that -- however those types whose own dealloc is fancier
(int, float, unicode) do use this pattern.
2001-10-05 20:51:39 +00:00
Guido van Rossum 048eb75c2d Add Garbage Collection support to new-style classes (not yet to their
instances).

Also added GC support to various auxiliary types: super, property,
descriptors, wrappers, dictproxy.  (Only type objects have a tp_clear
field; the other types are.)

One change was necessary to the GC infrastructure.  We have statically
allocated type objects that don't have a GC header (and can't easily
be given one) and heap-allocated type objects that do have a GC
header.  Giving these different metatypes would be really ugly: I
tried, and I had to modify pickle.py, cPickle.c, copy.py, add a new
invent a new name for the new metatype and make it a built-in, change
affected tests...  In short, a mess.  So instead, we add a new type
slot tp_is_gc, which is a simple Boolean function that determines
whether a particular instance has GC headers or not.  This slot is
only relevant for types that have the (new) GC flag bit set.  If the
tp_is_gc slot is NULL (by far the most common case), all instances of
the type are deemed to have GC headers.  This slot is called by the
PyObject_IS_GC() macro (which is only used twice, both times in
gcmodule.c).

I also changed the extern declarations for a bunch of GC-related
functions (_PyObject_GC_Del etc.): these always exist but objimpl.h
only declared them when WITH_CYCLE_GC was defined, but I needed to be
able to reference them without #ifdefs.  (When WITH_CYCLE_GC is not
defined, they do the same as their non-GC counterparts anyway.)
2001-10-02 21:24:57 +00:00
Guido van Rossum c907bd89de The error reporting here was a bit sparse. In verbose mode, the code
in run_test() referenced two non-existent variables, and in
non-verbose mode, the tests didn't report the actual number, when it
differed from the expected number.  Fixed this.

Also added an extra call to gc.collect() at the start of test_all().
This will be needed when I check in the changes to add GC to new-style
classes.
2001-10-02 19:49:47 +00:00
Neil Schemenauer 88c761a117 Test GC of frame objects. 2001-07-12 13:25:53 +00:00
Marc-André Lemburg 3661908a6a This patch removes all uses of "assert" in the regression test suite
and replaces them with a new API verify(). As a result the regression
suite will also perform its tests in optimization mode.

Written by Marc-Andre Lemburg. Copyright assigned to Guido van Rossum.
2001-01-17 19:11:13 +00:00
Fred Drake 004d5e6880 Make reindent.py happy (convert everything to 4-space indents!). 2000-10-23 17:22:08 +00:00
Neil Schemenauer faae266e89 - Add test for new SAVEALL debugging flag
- Use exceptions rather than asserts for failing tests.
- Reorganize tests and produce some output if verbose option is set.
2000-09-22 15:26:20 +00:00
Neil Schemenauer a53cf79275 - add a new test
- document some of the tricky tests (hopefully correctly :)
2000-09-15 22:32:29 +00:00
Vladimir Marangozov f9d20c3786 Neil Schemenauer: GC enable(), disable(), isenabled() interface.
Small stylistic changes by VM:
- is_enabled() -> isenabled()
- static ... Py_<func> -> static ... gc_<func>
2000-08-06 22:45:31 +00:00
Jeremy Hylton 094c9e587c remove all prints (since the prints where of memory locations) 2000-06-30 15:13:15 +00:00
Jeremy Hylton c5007aa5c3 final patches from Neil Schemenauer for garbage collection 2000-06-30 05:02:53 +00:00