Commit Graph

336 Commits

Author SHA1 Message Date
Neal Norwitz 419fd49abe Issue 2321: reduce memory usage (increase the memory that is returned
to the system) by using pymalloc for the data of unicode objects.

Will backport.
2008-03-17 20:22:43 +00:00
Guido van Rossum 5bdff60617 Fix the overflows in expandtabs(). "This time for sure!"
(Exploit at request.)
2008-03-11 21:18:06 +00:00
Facundo Batista c11cecf3d0 Issue 1742669. Now %d accepts very big float numbers.
Thanks Gabriel Genellina.
2008-02-24 03:17:21 +00:00
Eric Smith a9f7d62480 Backport of PEP 3101, Advanced String Formatting, from py3k.
Highlights:
 - Adding PyObject_Format.
 - Adding string.Format class.
 - Adding __format__ for str, unicode, int, long, float, datetime.
 - Adding builtin format.
 - Adding ''.format and u''.format.
 - str/unicode fixups for formatters.

The files in Objects/stringlib that implement PEP 3101 (stringdefs.h,
unicodedefs.h, formatter.h, string_format.h) are identical in trunk
and py3k.  Any changes from here on should be made to trunk, and
changes will propogate to py3k).
2008-02-17 19:46:49 +00:00
Christian Heimes 3b718a79af Implemented Martin's suggestion to clear the free lists during the garbage collection of the highest generation. 2008-02-14 12:47:33 +00:00
Christian Heimes 5b970ad483 Unified naming convention for free lists and their limits. All free lists
in Object/ are named ``free_list``, the counter ``numfree`` and the upper
limit is a macro ``PyName_MAXFREELIST`` inside an #ifndef block.

The chances should make it easier to adjust Python for platforms with
less memory, e.g. mobile phones.
2008-02-06 13:33:44 +00:00
Christian Heimes 4d4f270941 Patch #1970 by Antoine Pitrou: Speedup unicode whitespace and linebreak detection. The speedup is about 25% for split() (571 / 457 usec) and 35% (175 / 127 usec )for splitlines() 2008-01-30 11:32:37 +00:00
Christian Heimes 7f39c9fcbb Backport of several functions from Python 3.0 to 2.6 including PyUnicode_FromString, PyUnicode_Format and PyLong_From/AsSsize_t. The functions are partly required for the backport of the bytearray type and _fileio module. They should also make it easier to port C to 3.0.
First chapter of the Python 3.0 io framework back port: _fileio
The next step depends on a working bytearray type which itself depends on a backport of the nwe buffer API.
2008-01-25 12:18:43 +00:00
Christian Heimes 000a074c95 Modified PyImport_Import and PyImport_ImportModule to always use absolute imports by calling __import__ with an explicit level of 0
Added a new API function PyImport_ImportModuleNoBlock. It solves the problem with dead locks when mixing threads and imports
2008-01-03 22:16:32 +00:00
Christian Heimes e93237dfcc #1629: Renamed Py_Size, Py_Type and Py_Refcnt to Py_SIZE, Py_TYPE and Py_REFCNT. Macros for b/w compatibility are available. 2007-12-19 02:37:44 +00:00
Amaury Forgeot d'Arc 5087980c1e The incremental decoder for utf-7 must preserve its state between calls.
Solves issue1460.

Might not be a backport candidate: a new API function was added,
and some code may rely on details in utf-7.py.
2007-11-20 23:31:27 +00:00
Facundo Batista 6f7e6fb7a2 Made _ParseTupleFinds only defined to unicodeobject.c 2007-11-16 19:16:15 +00:00
Facundo Batista 57d5669f4b Now in find, rfind, index, and rindex, you can use None as defaults,
as usual with slicing (both with str and unicode strings).  This
fixes issue 1259.

For str only the stringobject.c file was modified.  But for unicode,
I needed to repeat in the four functions a lot of code, so created
a new function that does part of the job for them (and placed it in
find.h, following a suggestion of Barry).

Also added tests for this behaviour.
2007-11-16 18:04:14 +00:00
Guido van Rossum 1c1ac38157 Backport fixes for the code that decodes octal escapes (and for PyString
also hex escapes) -- this was reaching beyond the end of the input string
buffer, even though it is not supposed to be \0-terminated.
This has no visible effect but is clearly the correct thing to do.
(In 3.0 it had a visible effect after removing ob_sstate from PyString.)
2007-10-29 22:15:05 +00:00
Walter Dörwald 9d04542cc9 Set startinpos before calling the error handler. 2007-08-30 15:34:55 +00:00
Walter Dörwald 8757878b12 Rewrap line. 2007-08-30 15:30:09 +00:00
Thomas Wouters 3ccec68a05 Improve extended slicing support in builtin types and classes. Specifically:
- Specialcase extended slices that amount to a shallow copy the same way as
   is done for simple slices, in the tuple, string and unicode case.

 - Specialcase step-1 extended slices to optimize the common case for all
   involved types.

 - For lists, allow extended slice assignment of differing lengths as long
   as the step is 1. (Previously, 'l[:2:1] = []' failed even though
   'l[:2] = []' and 'l[:2:None] = []' do not.)

 - Implement extended slicing for buffer, array, structseq, mmap and
   UserString.UserString.

 - Implement slice-object support (but not non-step-1 slice assignment) for
   UserString.MutableString.

 - Add tests for all new functionality.
2007-08-28 15:28:19 +00:00
Walter Dörwald 9ab80a9fb4 Move another variable declaration up. 2007-08-17 16:58:43 +00:00
Walter Dörwald 20b40d3bce Move variable declaration up. 2007-08-17 16:52:50 +00:00
Walter Dörwald 6e39080649 Backport r57105 and r57145 from the py3k branch: UTF-32 codecs. 2007-08-17 16:41:28 +00:00
Georg Brandl 9efd9b6fa4 Bug #1763149: use proper slice syntax in docstring.
(backport)
2007-07-29 17:38:35 +00:00
Martin v. Löwis 6819210b9e PEP 3123: Provide forward compatibility with Python 3.0, while keeping
backwards compatibility. Add Py_Refcnt, Py_Type, Py_Size, and
PyVarObject_HEAD_INIT.
2007-07-21 06:55:02 +00:00
Georg Brandl 7c3b50db66 Patch #1673759: add a missing overflow check when formatting floats
with %G.
2007-07-12 08:38:00 +00:00
Neal Norwitz 5c9a81a3d8 Fix a bug when there was a newline in the string expandtabs was called on.
This also catches another condition that can overflow.

Will backport.
2007-06-11 02:16:10 +00:00
Neal Norwitz 7dbd2a3720 Prevent expandtabs() on string and unicode objects from causing a segfault when
a large width is passed on 32-bit platforms.  Found by Google.

It would be good for people to review this especially carefully and verify
I don't have an off by one error and there is no other way to cause overflow.
2007-06-09 03:36:34 +00:00
Neal Norwitz ee3a1b5244 Variation of patch # 1624059 to speed up checking if an object is a subclass
of some of the common builtin types.

Use a bit in tp_flags for each common builtin type.  Check the bit
to determine if any instance is a subclass of these common types.
The check avoids a function call and O(n) search of the base classes.
The check is done in the various Py*_Check macros rather than calling
PyType_IsSubtype().

All the bits are set in tp_flags when the type is declared
in the Objects/*object.c files because PyType_Ready() is not called
for all the types.  Should PyType_Ready() be called for all types?
If so and the change is made, the changes to the Objects/*object.c files
can be reverted (remove setting the tp_flags).  Objects/typeobject.c
would also have to be modified to add conditions
for Py*_CheckExact() in addition to each the PyType_IsSubtype check.
2007-02-25 19:44:48 +00:00
Armin Rigo 7ccbca93a2 Forward-port of r52136,52138: a review of overflow-detecting code.
* unified the way intobject, longobject and mystrtoul handle
  values around -sys.maxint-1.

* in general, trying to entierely avoid overflows in any computation
  involving signed ints or longs is extremely involved.  Fixed a few
  simple cases where a compiler might be too clever (but that's all
  guesswork).

* more overflow checks against bad data in marshal.c.

* 2.5 specific: fixed a number of places that were still confusing int
  and Py_ssize_t.  Some of them could potentially have caused
  "real-world" breakage.

* list.pop(x): fixing overflow issues on x was messy.  I just reverted
  to PyArg_ParseTuple("n"), which does the right thing.  (An obscure
  test was trying to give a Decimal to list.pop()... doesn't make
  sense any more IMHO)

* trying to write a few tests...
2006-10-04 12:17:45 +00:00
Raymond Hettinger a0c95fa4d8 Fix endcase for str.rpartition() 2006-09-04 15:32:48 +00:00
Neal Norwitz 17753ecbfa Patch #1541585: fix buffer overrun when performing repr() on
a unicode string in a build with wide unicode (UCS-4) support.

This code could be improved, so add an XXX comment.
2006-08-21 22:21:19 +00:00
Marc-André Lemburg 3a457790c7 Correct an accidentally removed previous patch. 2006-08-14 12:57:27 +00:00
Marc-André Lemburg 040f76b79c Slightly revised version of patch #1538956:
Replace UnicodeDecodeErrors raised during == and !=
compares of Unicode and other objects with a new
UnicodeWarning.

All other comparisons continue to raise exceptions.
Exceptions other than UnicodeDecodeErrors are also left
untouched.
2006-08-14 10:55:19 +00:00
Neal Norwitz 8a87f5d37e Patch #1538606, Patch to fix __index__() clipping.
I modified this patch some by fixing style, some error checking, and adding
XXX comments.  This patch requires review and some changes are to be expected.
I'm checking in now to get the greatest possible review and establish a
baseline for moving forward.  I don't want this to hold up release if possible.
2006-08-12 17:03:09 +00:00
Neal Norwitz e1fdb32ff2 Handle allocation failures gracefully. Found with failmalloc.
Many (all?) of these could be backported.
2006-07-21 05:32:28 +00:00
Martin v. Löwis d825143be1 Patch #1455898: Incremental mode for "mbcs" codec. 2006-06-14 05:21:04 +00:00
Neal Norwitz de4c78a1d7 Initialize the type object so pychecker can't crash the interpreter. 2006-06-13 08:28:19 +00:00
Georg Brandl 90e27d38f5 Apply perky's fix for #1503157: "/".join([u"", u""]) raising OverflowError.
Also improve error message on overflow.
2006-06-10 06:40:50 +00:00
Georg Brandl 242508160e RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. 2006-06-09 18:45:48 +00:00
Georg Brandl 9f16760666 Repair refleaks in unicodeobject. 2006-06-04 21:46:16 +00:00
Martin v. Löwis 3f767795f6 Patch #1359618: Speed-up charmap encoder. 2006-06-04 19:36:28 +00:00
Fredrik Lundh 60d8b18831 needforspeed: stringlib refactoring: changed find_obj to find_slice,
to enable use from stringobject
2006-05-27 15:20:22 +00:00
Fredrik Lundh c2d29c5a6d needforspeed: replace improvements, changed to Py_LOCAL_INLINE
where appropriate
2006-05-27 14:58:20 +00:00
Martin v. Löwis 2e3f6b77d5 Revert bogus change committed in 46432 to this file. 2006-05-27 11:07:49 +00:00
Andrew Dalke e0df762719 fixed typo 2006-05-27 11:04:36 +00:00
Fredrik Lundh 2d23d5bf2e needforspeed: more stringlib refactoring 2006-05-27 10:05:10 +00:00
Martin v. Löwis d004fc810a Patch 1494554: Update numeric properties to Unicode 4.1. 2006-05-27 08:36:52 +00:00
Neal Norwitz d1b6cd7bfb Fix Coverity warnings.
- Check the correct variable (str_obj, not str) for NULL
 - sep_len was already verified it wasn't 0
2006-05-27 05:21:30 +00:00
Andrew M. Kuchling 07bbfc6a51 Comment typo 2006-05-26 19:51:10 +00:00
Fredrik Lundh e6e43c867d needforspeed: stringlib refactoring: use stringlib/find for string find 2006-05-26 19:48:07 +00:00
Fredrik Lundh c816281304 needforspeed: use a macro to fix slice indexes 2006-05-26 19:33:03 +00:00
Fredrik Lundh ce4eccb0c4 needforspeed: stringlib refactoring: use stringlib/find for unicode
find
2006-05-26 19:29:05 +00:00