Commit Graph

1504 Commits

Author SHA1 Message Date
Tim Peters 29c0afcfec Just added comments, and cleared some XXX questions, related to int
memory management.
2002-04-28 16:57:34 +00:00
Tim Peters 449b5a8da1 _PyObject_DebugCheckAddress(): If the leading pad bytes are corrupt,
display a msg warning that the count of bytes requested may be bogus,
and that a segfault may happen next.
2002-04-28 06:14:45 +00:00
Tim Peters 5de9842b34 Repair widespread misuse of _PyString_Resize. Since it's clear people
don't understand how this function works, also beefed up the docs.  The
most common usage error is of this form (often spread out across gotos):

	if (_PyString_Resize(&s, n) < 0) {
		Py_DECREF(s);
		s = NULL;
		goto outtahere;
	}

The error is that if _PyString_Resize runs out of memory, it automatically
decrefs the input string object s (which also deallocates it, since its
refcount must be 1 upon entry), and sets s to NULL.  So if the "if"
branch ever triggers, it's an error to call Py_DECREF(s):  s is already
NULL!  A correct way to write the above is the simpler (and intended)

	if (_PyString_Resize(&s, n) < 0)
		goto outtahere;

Bugfix candidate.
2002-04-27 18:44:32 +00:00
Tim Peters 602f740bc2 SF patch 549375: Compromise PyUnicode_EncodeUTF8
This implements ideas from Marc-Andre, Martin, Guido and me on Python-Dev.

"Short" Unicode strings are encoded into a "big enough" stack buffer,
then exactly as much string space as they turn out to need is allocated
at the end.  This should have speed benefits akin to Martin's "measure
once, allocate once" strategy, but without needing a distinct measuring
pass.

"Long" Unicode strings allocate as much heap space as they could possibly
need (4 x # Unicode chars), and do a realloc at the end to return the
untouched excess.  Since the overallocation is likely to be substantial,
this shouldn't burden the platform realloc with unusably small excess
blocks.

Also simplified uses of the PyString_xyz functions.  Also added a release-
build check that 4*size doesn't overflow a C int.  Sooner or later, that's
going to happen.
2002-04-27 18:03:26 +00:00
Guido van Rossum 7dab2426ca - New builtin function enumerate(x), from PEP 279. Example:
enumerate("abc") is an iterator returning (0,"a"), (1,"b"), (2,"c").
  The argument can be an arbitrary iterable object.
2002-04-26 19:40:56 +00:00
Guido van Rossum 517c7d4fd3 PyNumber_CoerceEx: this took a shortcut (not doing anything) when the
left and right type were of the same type and not classic instances.

This shortcut is dangerous for proxy types, because it means that
coerce(Proxy(1), Proxy(2.1)) leaves Proxy(1) unchanged rather than
turning it into Proxy(1.0).

In an ever-so-slight change of semantics, I now only take the shortcut
when the left and right types are of the same type and don't have the
CHECKTYPES feature.  It so happens that classic instances have this
flag, so the shortcut is still skipped in this case (i.e. nothing
changes for classic instances).  Proxies also have this flag set
(otherwise implementing numeric operations on proxies would become
nightmarish) and this means that the shortcut is also skipped there,
as desired.  It so happens that int, long and float also have this
flag set; that means that e.g. coerce(1, 1) will now invoke
int_coerce().  This is fine: int_coerce() can deal with this, and I'm
not worried about the performance; int_coerce() is only invoked when
the user explicitly calls coerce(), which should be rarer than rare.
2002-04-26 02:49:14 +00:00
Guido van Rossum 9364698101 Make sure that tp_free frees the int the same way as tp_dealloc would.
This fixes the problem that Barry reported on python-dev:
   >>> 23000 .__class__ = bool
crashes in the deallocator.  This was because int inherited tp_free
from object, which uses the default allocator.

2.2. Bugfix candidate.
2002-04-26 00:53:34 +00:00
Guido van Rossum c95cc87447 Clean up the layout of the bool_as_number struct initializer. 2002-04-25 20:01:10 +00:00
Barry Warsaw f16951cffe abstract_get_bases(): Clarify exactly what the return values and
states can be for this function, and ensure that only AttributeErrors
are masked.  Any other exception raised via the equivalent of
getattr(cls, '__bases__') should be propagated up.

abstract_issubclass(): If abstract_get_bases() returns NULL, we must
call PyErr_Occurred() to see if an exception is being propagated, and
return -1 or 0 as appropriate.  This is the specific fix for a problem
whereby if getattr(derived, '__bases__') raised an exception, an
"undetected error" would occur (under a debug build).  This nasty
situation was uncovered when writing a security proxy extension type
for the Zope3 project, where the security proxy raised a Forbidden
exception on getattr of __bases__.

PyObject_IsInstance(), PyObject_IsSubclass(): After both calls to
abstract_get_bases(), where we're setting the TypeError if the return
value is NULL, we must first check to see if an exception occurred,
and /not/ mask an existing exception.

Neil Schemenauer should double check that these changes don't break
his ExtensionClass examples (there aren't any test cases for those
examples and abstract_get_bases() was added by him in response to
problems with ExtensionClass).  Neil, please add test cases if
possible!

I belive this is a bug fix candidate for Python 2.2.2.
2002-04-23 22:45:44 +00:00
Jeremy Hylton c4ad0bcbe5 Clarify return value of PyLong_AsLongLong().
The function is documented to return -1 on error.  If res was < 0, it
returned res.  It wasn't clear that the invariant was res < 0 iff res
== -1.
2002-04-23 20:01:20 +00:00
Tim Peters 030a5cebf4 unicode_memchr(): Squashed gratuitous int-vs-size_t mismatch (which
gives a compiler wng under MSVC because of the resulting signed-vs-
unsigned comparison).
2002-04-22 19:00:10 +00:00
Walter Dörwald de02bcb265 Apply patch diff.txt from SF feature request
http://www.python.org/sf/444708

This adds the optional argument for str.strip
to unicode.strip too and makes it possible
to call str.strip with a unicode argument
and unicode.strip with a str argument.
2002-04-22 17:42:37 +00:00
Tim Peters e1682a80fa Py_UniversalNewlineFread(): small speed boost on non-Windows boxes. 2002-04-21 18:15:20 +00:00
Tim Peters 0eca65c4c5 PyUnicode_EncodeUTF8(): tightened the memory asserts a bit, and at least
tried to catch some possible arithmetic overflows in the debug build.
2002-04-21 17:28:06 +00:00
Martin v. Löwis 2a7ff35a07 Back out 2.140. 2002-04-21 09:59:45 +00:00
Tim Peters 058b141ef7 Py_UniversalNewlineFread(): Many changes.
+ Continued looping until n bytes in the buffer have been filled, not
  just when n bytes have been read from the file.  This repairs the
  bug that f.readlines() only sucked up the first 8192 bytes of the file
  on Windows when universal newlines was enabled and f was opened in
  U mode (see Python-Dev -- this was the ultimate cause of the
  test_inspect.py failure).

+ Changed prototye to take a char* buffer (void* doesn't make much sense).

+ Squashed size_t vs int mismatches (in particular, besides the unsigned
  vs signed distinction, size_t may be larger than int).

+ Gets out under all error conditions now (it's possible for fread() to
  suffer an error even if it returns a number larger than 0 -- any
  "short read" is an error or EOF condition).

+ Rearranged and simplified declarations.
2002-04-21 07:29:14 +00:00
Tim Peters 7e3d961fc1 PyUnicode_EncodeUTF8: squash compiler wng. The difference of two
pointers is a signed type.  Changing "allocated" to a signed int makes
undetected overflow more likely, but there was no overflow detection
before either.
2002-04-21 03:26:37 +00:00
Martin v. Löwis a4eb14b7a4 Patch #495401: Count number of required bytes for encoding UTF-8 before
allocating the target buffer.
2002-04-20 13:44:01 +00:00
Jeremy Hylton 24ea8d3d9c Fix SF bug #505315: Make free and cell vars show up consistently in locals().
PyFrame_FastToLocals() and PyFrame_LocalsToFast() had a return if
f_nlocals was 0.  I think this was a holdover from the pre 2.1 days
when regular locals were the only kind of local variables.

The change makes it possible to use a free variable in eval or exec if
it the variable is also used elsewhere in the same block, which is
what the documentation says.
2002-04-20 04:46:55 +00:00
Tim Peters 08d821582f _PyObject_DebugMallocStats(): Added some potentially expensive internal
consistency checks, enabled only in a debug (Py_DEBUG) build.  Note that
this never gets called automatically unless PYMALLOC_DEBUG is #define'd
too, and the envar PYTHONMALLOCSTATS exists.
2002-04-18 22:25:03 +00:00
Tim Peters 64d80c9f40 PyObject_Malloc: make a tiny bit faster for platforms where malloc(0)
doesn't return NULL.

PyObject_Realloc:  better comment for why we don't call PyObject_Malloc(0).
2002-04-18 21:58:56 +00:00
Tim Peters 3e12071dbe Remove some long-disabled debugging boilerplate. 2002-04-18 21:37:03 +00:00
Tim Peters 2b85897189 type_get_doc(): Squash compiler wng about incompatible ptr types. 2002-04-18 04:12:28 +00:00
Guido van Rossum 6ca7d41c1f SF bug 542984.
Change type_get_doc (the get function for __doc__) to look in tp_dict
more often, and if it finds a descriptor in tp_dict, to call it (with
a NULL instance).  This means you can add a __doc__ descriptor to a
new-style class that returns instance docs when called on an instance,
and class docs when called on a class -- or the same docs in either
case, but lazily computed.

I'll also check this into the 2.2 maintenance branch.
2002-04-18 00:22:00 +00:00
Guido van Rossum e8fc640349 SF bug 544647.
PyNumber_InPlaceMultiply insisted on calling sq_inplace_repeat if it
existed, even if nb_inplace_multiply also existed and the arguments
weren't right for sq_inplace_repeat.  Change this to only use
sq_inplace_repeat if nb_inplace_multiply isn't defined.

Bugfix candidate.
2002-04-16 16:44:51 +00:00
Guido van Rossum 7766091e04 Whitespace normalization and fold some long lines. 2002-04-16 16:32:50 +00:00
Walter Dörwald 0fe940c862 Return the orginal string only if it's a real str or unicode
instance, otherwise make a copy.
2002-04-15 18:42:15 +00:00
Guido van Rossum 3aa3fc46c8 Remove 'const' from local variable declaration in string_zfill() -- it
isn't constant, so why bother.

Folded long lines.

Whitespace normalization.
2002-04-15 13:48:52 +00:00
Walter Dörwald 068325ef92 Apply the second version of SF patch http://www.python.org/sf/536241
Add a method zfill to str, unicode and UserString and change
Lib/string.py accordingly.

This activates the zfill version in unicodeobject.c that was
commented out and implements the same in stringobject.c. It also
adds the test for unicode support in Lib/string.py back in and
uses repr() instead() of str() (as it was before Lib/string.py 1.62)
2002-04-15 13:36:47 +00:00
Guido van Rossum 69cf3c7641 Deprecate % as well. The message for deprecation of //, % and divmod
is the same in all three cases (mostly because // calls divmod :-).
2002-04-15 12:39:12 +00:00
Guido van Rossum 9ec4c78a54 SF bug #543387.
Complex numbers implement divmod() and //, neither of which makes one
lick of sense.  Unfortunately this is documented, so I'm adding a
deprecation warning now, so we can delete this silliness, oh, around
2005 or so.

Bugfix candidate (At least for 2.2.2, I think.)
2002-04-15 01:41:56 +00:00
Guido van Rossum b6b8942f53 SF bug #541883 (Vincent Fiack).
A stupid bug in object_set_class(): didn't check for value==NULL
before checking its type.

Bugfix candidate.
2002-04-15 01:03:30 +00:00
Tim Peters 077f27141f SF bug 543840: complex(string) accepts strings with \0
complex_subtype_from_string():  this stopped parsing at the first 0
byte, as if that were the end of the input string.

Bugfix candidate.
2002-04-14 22:04:03 +00:00
Jack Jansen 7b8c7546eb Mass checkin of universal newline support.
Highlights: import and friends will understand any of \r, \n and \r\n
as end of line. Python file input will do the same if you use mode 'U'.
Everything can be disabled by configuring with --without-universal-newlines.

See PEP278 for details.
2002-04-14 20:12:41 +00:00
Guido van Rossum 5961042061 Fold long lines. (Walter, please take note! :-) 2002-04-13 14:06:36 +00:00
Tim Peters 0e871188e8 _PyObject_DebugDumpStats: renamed to _PyObject_DebugMallocStats.
Added code to call this when PYMALLOC_DEBUG is enabled, and envar
PYTHONMALLOCSTATS is set, whenever a new arena is obtained and once
late in the Python shutdown process.
2002-04-13 08:29:14 +00:00
Tim Peters b7ba743312 SF bug 543148: Memory leak with stackframes + inspect.
Put a bound on the number of frameobjects that can live in the
frameobject free_list.

Am also backporting to 2.2.  I don't intend to backport to 2.1 (too
much work -- lots of cyclic structures leak there, and the GC API).
2002-04-13 05:21:47 +00:00
Guido van Rossum 018b0eb0f5 Partially implement SF feature request 444708.
Add optional arg to string methods strip(), lstrip(), rstrip().
The optional arg specifies characters to delete.

Also for UserString.

Still to do:

- Misc/NEWS
- LaTeX docs (I did the docstrings though)
- Unicode methods, and Unicode support in the string methods.
2002-04-13 00:56:08 +00:00
Tim Peters 8a8cdfd0f5 Small anal correctness tweaks:
_PyObject_DebugMalloc:  explicitly cast PyObject_Malloc's result to the
target pointer type.

_PyObject_DebugDumpStats:  change decl of arena_alignment from unsigned
int to unsigned long.

This is for the 2.3 release only (it's new code).
2002-04-12 20:49:36 +00:00
Guido van Rossum e027d9818f Add Raymond Hettinger's d.pop(). See SF patch 539949. 2002-04-12 15:11:59 +00:00
Tim Peters 85cc1c4368 _PyObject_DebugRealloc(): rewritten to let the underlying realloc do
most of the work.  In particular, if the underlying realloc is able to
grow the memory block in place, great (this routine used to do a fresh
malloc + memcpy every time a block grew).  BTW, I'm not so keen here on
avoiding possible quadratic-time realloc patterns as I am on making
the debug pymalloc more invisible (the more it uses memory "just like"
the underlying allocator, the better the chance that a suspected memory
corruption bug won't vanish when the debug malloc is turned on).
2002-04-12 08:52:50 +00:00
Tim Peters f539c68ccd _PyObject_DebugDumpAddress(): clarify an output message. 2002-04-12 07:43:07 +00:00
Tim Peters f6fb501c57 PYMALLOC_{CLEAN, DEAD, FORBIDDEN}BYTE symbols: remove the PYMALLOC_
prefix.  These symbols are private to the file, and the PYMALLOC_ gets
in the way (overly long code lines, comments, and error messages).
2002-04-12 07:38:53 +00:00
Tim Peters af3e8de580 First stab at rationalizing the PyMem_ API. Mixing PyObject_xyz with
PyMem_{Del, DEL} doesn't work yet (compilation problems).

pyport.h:  _PyMem_EXTRA is gone.

pmem.h:  Repaired comments.  PyMem_{Malloc, MALLOC} and
PyMem_{Realloc, REALLOC} now make the same x-platform guarantees when
asking for 0 bytes, and when passing a NULL pointer to the latter.

object.c:  PyMem_{Malloc, Realloc} just call their macro versions
now, since the latter take care of the x-platform 0 and NULL stuff
by themselves now.

pypcre.c, grow_stack():  So sue me.  On two lines, this called
PyMem_RESIZE to grow a "const" area.  It's not legit to realloc a
const area, so the compiler warned given the new expansion of
PyMem_RESIZE.  It would have gotten the same warning before if it
had used PyMem_Resize() instead; the older macro version, but not the
function version, silently cast away the constness.  IMO that was a wrong
thing to do, and the docs say the macro versions of PyMem_xyz are
deprecated anyway.  If somebody else is resizing const areas with the
macro spelling, they'll get a warning when they recompile now too.
2002-04-12 07:22:56 +00:00
Neil Schemenauer d2560cd37c Move PyObject_Malloc and PyObject_Free here from object.c. Remove
PyMalloc_ prefix and use PyObject_ instead.  I'm not sure about the
debugging functions.  Perhaps they should stay as PyMalloc_.
2002-04-12 03:10:20 +00:00
Neil Schemenauer bdf0eedb68 Move PyObject_Malloc and PyObject_Free to obmalloc.c. 2002-04-12 03:08:42 +00:00
Neil Schemenauer 58aa861fa2 Remove PyMalloc_*. 2002-04-12 03:07:20 +00:00
Neil Schemenauer 09a2ae5882 Change signature of _PyObject_GC_Malloc to match PyObject_MALLOC.
PyObject_Del and PyObject_GC_Del can now be used as a function
designators.
2002-04-12 03:06:53 +00:00
Neil Schemenauer 626d774df6 PyObject_GC_Del can now be used as a function designator. 2002-04-12 03:05:52 +00:00
Neil Schemenauer 7465ad2fc9 Remove PyMalloc_New and PyMalloc_Del. 2002-04-12 03:05:37 +00:00