Commit Graph

1027 Commits

Author SHA1 Message Date
Marc-André Lemburg 6c6bfb7c70 Make the unicode-escape and the UTF-16 codecs handle surrogates
correctly and thus roundtrip-safe.

Some minor cleanups of the code.

Added tests for the roundtrip-safety.
2001-07-20 17:39:11 +00:00
Guido van Rossum 0d42e0c54a #ifdef out generation of \U escapes unless Py_UNICODE_WIDE. This
#caused warnings with the VMS C compiler.  (SF bug #442998, in part.)
On a narrow system the current code should never be executed since ch
will always be < 0x10000.

Marc-Andre: you may end up fixing this a different way, since I
believe you have plans to generate \U for surrogate pairs.  I'll leave
that to you.
2001-07-20 16:36:21 +00:00
Fred Drake 1bc8fab0e7 Kill more warnings from the SGI compiler.
Part of SF patch #434992.
2001-07-19 21:49:38 +00:00
Tim Peters 0d5dd68692 Python.h: Don't attempt to redefine NDEBUG if it's already defined.
Others:  Remove redundant includes of assert.h.
2001-07-15 18:38:47 +00:00
Tim Peters 586b2e3f3d long_format: Simplify the overly elaborate base-is-a-power-of-2 code. 2001-07-15 09:11:14 +00:00
Guido van Rossum 3c29602d74 _Py_GetObjects(): GCC suggests to add () around && within || for some
code only compiled in debug mode, and I dutifully comply.
2001-07-14 17:58:00 +00:00
Tim Peters 212e614f60 divrem1 & long_format: found a clean way to factor divrem1 so that
long_format can reuse a scratch area for its repeated divisions (instead
of malloc/free for every digit produced); speeds str(long)/repr(long).
2001-07-14 12:23:19 +00:00
Tim Peters c8a6b9b6d6 long_format(): Simplify new code a bit. 2001-07-14 11:01:28 +00:00
Tim Peters fad225f1ba long_format(): Easy speedup for output bases that aren't a power of 2 (in
particular, str(long) and repr(long) use base 10, and that gets a factor
of 4 speedup).  Another factor of 2 can be gotten by refactoring divrem1 to
support in-place division, but that started getting messy so I'm leaving
that out.
2001-07-13 02:59:26 +00:00
Neil Schemenauer 10c6692774 GC for method objects. 2001-07-12 13:27:35 +00:00
Neil Schemenauer 7eac9b72d4 GC for iterator objects. 2001-07-12 13:27:25 +00:00
Neil Schemenauer 19cd292bbc GC for frame objects. 2001-07-12 13:27:11 +00:00
Guido van Rossum 0ec9abaa2b On long to the negative long power, let float handle it instead of
raising an error.  This was one of the two issues that the VPython
folks were particularly problematic for their students.  (The other
one was integer division...)  This implements (my) SF patch #440487.
2001-07-12 11:21:17 +00:00
Guido van Rossum b82fedc7d8 On int to the negative integral power, let float handle it instead of
raising an error.  This was one of the two issues that the VPython
folks were particularly problematic for their students.  (The other
one was integer division...)  This implements (my) SF patch #440487.
2001-07-12 11:19:45 +00:00
Thomas Wouters efafcea280 Re-add 'advanced' xrange features, adding DeprecationWarnings as discussed
on python-dev. The features will still vanish, however, just one release
later.
2001-07-09 12:30:54 +00:00
Tim Peters 6ee4234802 SF bug #439104: Tuple richcompares has code-typo.
Symptom:  (1, 2, 3) <= (1, 2) returned 1.
This was already fixed in CVS for tuples, but an isomorphic error was in
the list richcompare code.
2001-07-06 17:45:43 +00:00
Guido van Rossum 3f56166b1a Rip out the fancy behaviors of xrange that nobody uses: repeat, slice,
contains, tolist(), and the start/stop/step attributes.  This includes
removing the 4th ('repeat') argument to PyRange_New().
2001-07-05 13:27:48 +00:00
Fredrik Lundh 72b068566a removed "register const" from scalar arguments to the unicode
predicates
2001-06-27 22:08:26 +00:00
Fredrik Lundh 8f4558583f use Py_UNICODE_WIDE instead of USE_UCS4_STORAGE and Py_UNICODE_SIZE
tests.
2001-06-27 18:59:43 +00:00
Martin v. Löwis ce9b5a55e1 Encode surrogates in UTF-8 even for a wide Py_UNICODE.
Implement sys.maxunicode.
Explicitly wrap around upper/lower computations for wide Py_UNICODE.
When decoding large characters with UTF-8, represent expected test
results using the \U notation.
2001-06-27 06:28:56 +00:00
Martin v. Löwis ac93bc2501 When decoding UTF-16, don't assume that the buffer is in native endianness
when checking surrogates.
2001-06-26 22:43:40 +00:00
Martin v. Löwis 0ba70cc3c8 Support using UCS-4 as the Py_UNICODE type:
Add configure option --enable-unicode.
Add config.h macros Py_USING_UNICODE, PY_UNICODE_TYPE, Py_UNICODE_SIZE,
                    SIZEOF_WCHAR_T.
Define Py_UCS2.
Encode and decode large UTF-8 characters into single Py_UNICODE values
for wide Unicode types; likewise for UTF-16.
Remove test whether sizeof Py_UNICODE is two.
2001-06-26 22:22:37 +00:00
Fredrik Lundh ee13dba1aa more unicode tweaks: fix unicodectype for sizeof(Py_UNICODE) >
sizeof(int)
2001-06-26 20:36:12 +00:00
Barry Warsaw 66a0d1d9b9 dict_update(): Generalize this method so {}.update() accepts any
"mapping" object, specifically one that supports PyMapping_Keys() and
PyObject_GetItem().  This allows you to say e.g. {}.update(UserDict())

We keep the special case for concrete dict objects, although that
seems moderately questionable.  OTOH, the code exists and works, so
why change that?

.update()'s docstring already claims that D.update(E) implies calling
E.keys() so it's appropriate not to transform AttributeErrors in
PyMapping_Keys() to TypeErrors.

Patch eyeballed by Tim.
2001-06-26 20:08:32 +00:00
Fredrik Lundh 1294ad0c59 experimental UCS-4 support: added USE_UCS4_STORAGE define to
unicodeobject.h, which forces sizeof(Py_UNICODE) == sizeof(Py_UCS4).
(this may be good enough for platforms that doesn't have a 16-bit
type.  the UTF-16 codecs don't work, though)
2001-06-26 17:17:07 +00:00
Fredrik Lundh 45714e9ecb experimental UCS-4 support: made compare a bit more robust, in case
sizeof(Py_UNICODE) >= sizeof(long).  also changed surrogate expansion
to work if sizeof(Py_UNICODE) > 2.
2001-06-26 16:39:36 +00:00
Fredrik Lundh 3083163dc1 experimental UCS-4 support: don't assume that MS_WIN32 implies
HAVE_USABLE_WCHAR_T
2001-06-26 15:11:00 +00:00
Tim Peters 8c96369513 PyFrameObject: rename f_stackbottom to f_stacktop, since it points to
the next free valuestack slot, not to the base (in America, stacks push
and pop at the top -- they mutate at the bottom in Australia <winK>).
eval_frame():  assert that f_stacktop isn't NULL upon entry.
frame_delloc():  avoid ordered pointer comparisons involving f_stacktop
when f_stacktop is NULL.
2001-06-23 05:26:56 +00:00
Tim Peters 5ca576ed0a Merging the gen-branch into the main line, at Guido's direction. Yay!
Bugfix candidate in inspect.py:  it was referencing "self" outside of
a method.
2001-06-18 22:08:13 +00:00
Tim Peters 1dad6a86de SF bug 434186: 0x80000000/2 != 0x80000000>>1
i_divmod:  New and simpler algorithm.  Old one returned gibberish on most
boxes when the numerator was -sys.maxint-1.  Oddly enough, it worked in the
release (not debug) build on Windows, because the compiler optimized away
some tricky sign manipulations that were incorrect in this case.
Makes you wonder <wink> ...
Bugfix candidate.
2001-06-18 19:21:11 +00:00
Tim Peters 70128a1ba6 PyLong_{As, From}VoidPtr: cleanup, replacing assumptions in comments with
#if/#error constructs.
2001-06-16 08:48:40 +00:00
Tim Peters c605784174 dict_repr: Reuse one of the int vars (minor code simplification). 2001-06-16 07:52:53 +00:00
Tim Peters 52e155e31b Reformat decl of new _PyString_Join. Add NEWS blurb about repr() speedup. 2001-06-16 05:42:57 +00:00
Tim Peters a7259597f1 SF bug 433228: repr(list) woes when len(list) big.
Gave Python linear-time repr() implementations for dicts, lists, strings.
This means, e.g., that repr(range(50000)) is no longer 50x slower than
pprint.pprint() in 2.2 <wink>.

I don't consider this a bugfix candidate, as it's a performance boost.

Added _PyString_Join() to the internal string API.  If we want that in the
public API, fine, but then it requires runtime error checks instead of
asserts.
2001-06-16 05:11:17 +00:00
Tim Peters cf37dfc3db Change IS_LITTLE_ENDIAN macro -- a little faster now. 2001-06-14 18:42:50 +00:00
Guido van Rossum ad98db1d9e Fix a mis-indentation in _PyUnicode_New() that caused me to stare at
some code for longer than needed.
2001-06-14 17:52:02 +00:00
Tim Peters ede0509111 _PyLong_AsByteArray: simplify the logic for dealing with the most-
significant digits sign bits.  Again no change in semantics.
2001-06-14 08:53:38 +00:00
Tim Peters ce9de2f79a PyLong_From{Unsigned,}Long: count the # of digits first, so no more space
is allocated than needed (used to allocate 80 bytes of digit space no
matter how small the long input).  This also runs faster, at least on 32-
bit boxes.
2001-06-14 04:56:19 +00:00
Tim Peters f251d06a66 _PyLong_FromByteArray: changed decl of "carry" to match "thisbyte". No
semantic change, but a bit clearer and may help a really stupid compiler
avoid pointless runtime length conversions.
2001-06-13 21:09:15 +00:00
Tim Peters 05607ad4fd _PyLong_AsByteArray: Don't do the "delicate overflow" check unless it's
truly needed; usually saves a little time, but no change in semantics.
2001-06-13 21:01:27 +00:00
Tim Peters 898cf85c25 _PyLong_AsByteArray: added assert that the input is normalized. This is
outside the function's control, but is crucial to correct operation.
2001-06-13 20:50:08 +00:00
Tim Peters 9cb0c38fff PyLong_As{Unsigned,}LongLong: fiddled final result casting. 2001-06-13 20:45:17 +00:00
Tim Peters d1a7da6c0d longobject.c:
Replaced PyLong_{As,From}{Unsigned,}LongLong guts with calls
    to _PyLong_{As,From}ByteArray.
_testcapimodule.c:
    Added strong tests of PyLong_{As,From}{Unsigned,}LongLong.

Fixes SF bug #432552 PyLong_AsLongLong() problems.
Possible bugfix candidate, but the fix relies on code added to longobject
to support the new q/Q structmodule format codes.
2001-06-13 00:35:57 +00:00
Tim Peters 8bc84b4391 _PyLong_{As,From}ByteArray: Minor code rearrangement aimed at improving
clarity.  Should have no effect visible to callers.
2001-06-12 19:17:03 +00:00
Marc-André Lemburg 8c2133da7b Fix for bug #432384: Recursion in PyString_AsEncodedString? 2001-06-12 13:14:10 +00:00
Tim Peters 7a3bfc3a47 Added q/Q standard (x-platform 8-byte ints) mode in struct module.
This completes the q/Q project.

longobject.c _PyLong_AsByteArray:  The original code had a gross bug:
the most-significant Python digit doesn't necessarily have SHIFT
significant bits, and you really need to count how many copies of the sign
bit it has else spurious overflow errors result.

test_struct.py:  This now does exhaustive std q/Q testing at, and on both
sides of, all relevant power-of-2 boundaries, both positive and negative.

NEWS:  Added brief dict news while I was at it.
2001-06-12 01:22:22 +00:00
Tim Peters 2a9b367385 Two new private longobject API functions,
_PyLong_FromByteArray
    _PyLong_AsByteArray
Untested and probably buggy -- they compile OK, but nothing calls them
yet.  Will soon be called by the struct module, to implement x-platform
'q' and 'Q'.
If other people have uses for them, we could move them into the public API.
See longobject.h for usage details.
2001-06-11 21:23:58 +00:00
Jack Jansen fcc54cab10 Added a missing cast to the hashfunc initializer. 2001-06-10 21:43:28 +00:00
Martin v. Löwis 0163d6d6ef Patch #424475: Speed-up tp_compare usage, by special-casing the common
case of objects with equal types which support tp_compare. Give
type objects a tp_compare function.
Also add c<0 tests before a few PyErr_Occurred tests.
2001-06-09 07:34:05 +00:00
Marc-André Lemburg 8879a33613 Fixes [ #430986 ] Buglet in PyUnicode_FromUnicode. 2001-06-07 12:26:56 +00:00