Commit Graph

188 Commits

Author SHA1 Message Date
Benjamin Peterson d455ce4fd4 merge 3.3 2014-03-30 19:52:39 -04:00
Benjamin Peterson 0ad6098b67 merge 3.2 2014-03-30 19:52:22 -04:00
Benjamin Peterson 23cf403ca1 fix expandtabs overflow detection to be consistent and not rely on signed overflow 2014-03-30 19:47:57 -04:00
Serhiy Storchaka 3079328d29 Reverted changeset b72c5573c5e7 (issue #15027). 2014-01-04 22:44:01 +02:00
Serhiy Storchaka 583a93943c Issue #15027: Rewrite the UTF-32 encoder. It is now 1.6x to 3.5x faster. 2014-01-04 19:25:37 +02:00
Benjamin Peterson 0ee22bf774 fix format spec recursive expansion (closes #19729) 2013-11-26 19:22:36 -06:00
Serhiy Storchaka dc2fd5101a Remove dead code committed in issue #12892. 2013-11-19 15:56:05 +02:00
Serhiy Storchaka 58cf607d13 Issue #12892: The utf-16* and utf-32* codecs now reject (lone) surrogates.
The utf-16* and utf-32* encoders no longer allow surrogate code points
(U+D800-U+DFFF) to be encoded.
The utf-32* decoders no longer decode byte sequences that correspond to
surrogate code points.
The surrogatepass error handler now works with the utf-16* and utf-32* codecs.

Based on patches by Victor Stinner and Kang-Hao (Kenny) Lu.
2013-11-19 11:32:41 +02:00
Ezio Melotti 745d54d2fa #17806: Added keyword-argument support for "tabsize" to str/bytes.expandtabs(). 2013-11-16 19:10:57 +02:00
Victor Stinner cc64eb5b9f Issue #18408: Fix bytearrayiter.partition()/rpartition(), handle
PyByteArray_FromStringAndSize() failure (ex: on memory allocation failure)
2013-10-29 03:15:37 +01:00
Serhiy Storchaka 8fa8ee3970 Issue #18701: Remove support of old CPython versions (<3.0) from C code. 2013-08-17 00:48:02 +03:00
Raymond Hettinger d06eeb4a24 merge 2013-08-13 18:20:55 -07:00
Raymond Hettinger b1b915c796 Issue 18719: Remove a false optimization
Remove an unused early-out test from the critical path for
dict and set lookups.

When the strings already have matching lengths, kinds, and hashes,
there is no additional information gained by checking the first
characters (the probability of a mismatch is already known to
be less than 1 in 2**64).
2013-08-13 18:16:34 -07:00
Antoine Pitrou 9ed5f27266 Issue #18722: Remove uses of the "register" keyword in C code. 2013-08-13 20:18:52 +02:00
Benjamin Peterson d2b58a9880 only recursively expand in the format spec (closes #17644) 2013-05-17 17:34:30 -05:00
Benjamin Peterson 4d94474ba3 rewrite the parsing of field names to be more consistent wrt recursive expansion 2013-05-17 18:22:31 -05:00
Benjamin Peterson 48953632df merge 3.3 2013-05-17 17:35:28 -05:00
Ezio Melotti 5263c13801 Merge removal of trailing whitespace from 3.3. 2013-04-21 04:08:18 +03:00
Ezio Melotti 6b02772c13 Remove trailing whitespace. 2013-04-21 04:07:51 +03:00
Victor Stinner 8f674ccd64 Close #17694: Add minimum length to _PyUnicodeWriter
* Add also min_char attribute to _PyUnicodeWriter structure (currently unused)
 * _PyUnicodeWriter_Init() has no more argument (except the writer itself):
   min_length and overallocate must be set explicitly
 * In error handlers, only enable overallocation if the replacement string
   is longer than 1 character
 * CJK decoders don't use overallocation anymore
 * Set min_length, instead of preallocating memory using
   _PyUnicodeWriter_Prepare(), in many decoders
 * _PyUnicode_DecodeUnicodeInternal() checks for integer overflow
2013-04-17 23:02:17 +02:00
Victor Stinner 76b3b2726c stringlib: remove unused STRINGLIB_RESIZE macro 2013-04-14 16:29:09 +02:00
Serhiy Storchaka e2cef885a2 Issue #16061: Speed up str.replace() for replacing 1-character strings. 2013-04-13 22:45:04 +03:00
Victor Stinner 7efa3b8242 Close #13126: "Simplify" FASTSEARCH() code to help the compiler to emit more
efficient machine code. Patch written by Antoine Pitrou.

Without this change, str.find() was 10% slower than str.rfind() in the worst
case.
2013-04-08 00:26:43 +02:00
Victor Stinner cfc4c13b04 Add _PyUnicodeWriter_WriteSubstring() function
Write a function to enable more optimizations:

 * If the substring is the whole string and overallocation is disabled, just
   keep a reference to the string, don't copy characters
 * Avoid a call to the expensive _PyUnicode_FindMaxChar() function when
   possible
2013-04-03 01:48:39 +02:00
Serhiy Storchaka 06b16f879f Remove unused defines. 2013-02-23 14:49:09 +02:00
Serhiy Storchaka 18809fa94e Remove unused defines. 2013-02-23 14:48:16 +02:00
Antoine Pitrou 4de7457009 Issue #17173: Remove uses of locale-dependent C functions (isalpha() etc.) in the interpreter.
I've left a couple of them in: zlib (third-party lib), getaddrinfo.c
(doesn't include Python.h, and probably obsolete), _sre.c (legitimate
use for the re.LOCALE flag).
2013-02-09 23:11:27 +01:00
Serhiy Storchaka b946af5897 Check for NULL before the pointer aligning in fastsearch_memchr_1char.
There is no guarantee that NULL is aligned.
2013-01-15 13:32:41 +02:00
Serhiy Storchaka 18ba40b945 Check for NULL before the pointer aligning in fastsearch_memchr_1char.
There is no guarantee that NULL is aligned.
2013-01-15 13:27:28 +02:00
Christian Heimes 5f7e8dab11 Issue #16592: stringlib_bytes_join doesn't raise MemoryError on allocation failure 2012-12-02 07:56:42 +01:00
Victor Stinner 6caa6fb535 (Merge 3.3) Issue #8271: Fix compilation on Windows 2012-11-05 00:00:50 +01:00
Victor Stinner ab60de478d Issue #8271: Fix compilation on Windows 2012-11-04 23:59:15 +01:00
Ezio Melotti cfa9636404 #8271: merge with 3.3. 2012-11-04 23:23:09 +02:00
Ezio Melotti f7ed5d111b #8271: the utf-8 decoder now outputs the correct number of U+FFFD characters when used with the "replace" error handler on invalid utf-8 sequences. Patch by Serhiy Storchaka, tests by Ezio Melotti. 2012-11-04 23:21:38 +02:00
Antoine Pitrou 6f7b0da6bc Issue #12805: Make bytes.join and bytearray.join faster when the separator is empty.
Patch by Serhiy Storchaka.
2012-10-20 23:08:34 +02:00
Christian Heimes 743e0cd6b5 Issue #16166: Add PY_LITTLE_ENDIAN and PY_BIG_ENDIAN macros and unified
endianess detection and handling.
2012-10-17 23:52:17 +02:00
Antoine Pitrou cfc22b4a9b Issue #15958: bytes.join and bytearray.join now accept arbitrary buffer objects. 2012-10-16 21:07:23 +02:00
Antoine Pitrou ca8aa4acf6 Issue #15144: Fix possible integer overflow when handling pointers as integer values, by using Py_uintptr_t instead of size_t.
Patch by Serhiy Storchaka.
2012-09-20 20:56:47 +02:00
Victor Stinner b3f5501250 Close #15534: Fix a typo in the fast search function of the string library (_s => s)
Replace _s with ptr to avoid future confusion. Add also non regression tests.
2012-08-02 23:05:01 +02:00
Mark Dickinson fb90c0934c Issue #14700: Fix buggy overflow checks for large precision and width in new-style and old-style formatting. 2012-10-28 10:18:03 +00:00
Mark Dickinson 01ac8b6ab1 Use correct types for ASCII_CHAR_MASK integer constants. 2012-07-07 14:08:48 +02:00
Mark Dickinson 106c4145ff Issue #14923: Optimize continuation-byte check in UTF-8 decoding. Patch by Serhiy Storchaka. 2012-06-23 21:45:14 +01:00
Antoine Pitrou a759d4e9f4 Make private function static (from `make smelly`) 2012-06-21 17:26:28 +02:00
Antoine Pitrou 27f6a3b0bf Issue #15026: utf-16 encoding is now significantly faster (up to 10x).
Patch by Serhiy Storchaka.
2012-06-15 22:15:23 +02:00
Victor Stinner d7b7c7472b Issue #14993: Use standard "unsigned char" instead of a unsigned char bitfield 2012-06-04 22:52:12 +02:00
Victor Stinner d3f0882dfb Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
* Formatting string, int, float and complex use the _PyUnicodeWriter API. It
   avoids a temporary buffer in most cases.
 * Add _PyUnicodeWriter_WriteStr() to restore the PyAccu optimization: just
   keep a reference to the string if the output is only composed of one string
 * Disable overallocation when formatting the last argument of str%args and
   str.format(args)
 * Overallocation allocates at least 100 characters: add min_length attribute
   to the _PyUnicodeWriter structure
 * Add new private functions: _PyUnicode_FastCopyCharacters(),
   _PyUnicode_FastFill() and _PyUnicode_FromASCII()

The speed up is around 20% in average.
2012-05-29 12:57:52 +02:00
Antoine Pitrou 63065d761e Issue #14624: UTF-16 decoding is now 3x to 4x faster on various inputs.
Patch by Serhiy Storchaka.
2012-05-15 23:48:04 +02:00
Antoine Pitrou ca5f91b888 Issue #14738: Speed-up UTF-8 decoding on non-ASCII data. Patch by Serhiy Storchaka. 2012-05-10 16:36:02 +02:00
Victor Stinner 3b1a74a9c3 Rename unicode_write_t structure and its methods to "_PyUnicodeWriter" 2012-05-09 22:25:00 +02:00
Victor Stinner ee4544c920 Issue #14744: Inline unicode_writer_write_char() and unicode_write_str()
Optimize also PyUnicode_Format(): call unicode_writer_prepare() only once
per argument.
2012-05-09 22:24:08 +02:00