Commit Graph

4541 Commits

Author SHA1 Message Date
Serhiy Storchaka e58785b200 Issue #16975: Fix error handling bug in the escape-decode bytes decoder. 2013-01-25 23:32:41 +02:00
Serhiy Storchaka ace3ad3bf7 Issue #16975: Fix error handling bug in the escape-decode bytes decoder. 2013-01-25 23:31:43 +02:00
Serhiy Storchaka 6481bfb2b5 Issue #16335: Fix integer overflow in unicode-escape decoder. 2013-01-21 11:44:40 +02:00
Serhiy Storchaka c35f3a9f61 Issue #16335: Fix integer overflow in unicode-escape decoder. 2013-01-21 11:42:57 +02:00
Serhiy Storchaka 4f5f0e54e0 Issue #16335: Fix integer overflow in unicode-escape decoder. 2013-01-21 11:38:00 +02:00
Serhiy Storchaka 441d30fac7 Issue #15989: Fix several occurrences of integer overflow
when result of PyLong_AsLong() narrowed to int without checks.

This is a backport of changesets 13e2e44db99d and 525407d89277.
2013-01-19 12:26:26 +02:00
Serhiy Storchaka 9101e23ff6 Issue #15989: Fix several occurrences of integer overflow
when result of PyLong_AsLong() narrowed to int without checks.

This is a backport of changesets 13e2e44db99d and 525407d89277.
2013-01-19 12:41:45 +02:00
Serhiy Storchaka 55e2cb497b Issue #14850: Now a chamap decoder treates U+FFFE as "undefined mapping"
in any mapping, not only in an unicode string.
2013-01-15 15:30:04 +02:00
Serhiy Storchaka 45d16d9924 Issue #14850: Now a chamap decoder treates U+FFFE as "undefined mapping"
in any mapping, not only in an unicode string.
2013-01-15 15:01:20 +02:00
Serhiy Storchaka 4fb8caee87 Issue #14850: Now a chamap decoder treates U+FFFE as "undefined mapping"
in any mapping, not only in an unicode string.
2013-01-15 14:43:21 +02:00
Serhiy Storchaka b946af5897 Check for NULL before the pointer aligning in fastsearch_memchr_1char.
There is no guarantee that NULL is aligned.
2013-01-15 13:32:41 +02:00
Serhiy Storchaka 18ba40b945 Check for NULL before the pointer aligning in fastsearch_memchr_1char.
There is no guarantee that NULL is aligned.
2013-01-15 13:27:28 +02:00
Serhiy Storchaka 7898043868 Issue #15989: Fix several occurrences of integer overflow
when result of PyLong_AsLong() narrowed to int without checks.
2013-01-15 01:12:17 +02:00
Benjamin Peterson 0b32a480bd merge 3.3 (#16906) 2013-01-09 09:52:22 -06:00
Benjamin Peterson 0c270a8bb7 correct static string clearing loop (closes #16906) 2013-01-09 09:52:01 -06:00
Serhiy Storchaka 24a3ef6999 Issue #11461: Fix the incremental UTF-16 decoder. Original patch by
Amaury Forgeot d'Arc. Added tests for partial decoding of non-BMP
characters.
2013-01-08 23:41:55 +02:00
Serhiy Storchaka ae3b32ad6b Issue #11461: Fix the incremental UTF-16 decoder. Original patch by
Amaury Forgeot d'Arc. Added tests for partial decoding of non-BMP
characters.
2013-01-08 23:40:52 +02:00
Serhiy Storchaka 48e188e573 Issue #11461: Fix the incremental UTF-16 decoder. Original patch by
Amaury Forgeot d'Arc. Added tests for partial decoding of non-BMP
characters.
2013-01-08 23:14:24 +02:00
Serhiy Storchaka dec798eb46 Fix out of bound read in UTF-32 decoder on "narrow Unicode" builds. 2013-01-08 22:45:42 +02:00
Christian Heimes 34bdeb5d81 Add a comment about *not* caching the hash value. Issue #9685 suggested to memorize the hash value, but the feature request was rejected because no speed ups were found. 2013-01-07 21:24:18 +01:00
Serhiy Storchaka 4e02538bf3 Issue #16856: Fix a segmentation fault from calling repr() on a dict with
a key whose repr raise an exception.
2013-01-04 12:40:35 +02:00
Serhiy Storchaka 6c83e739d7 Issue #16856: Fix a segmentation fault from calling repr() on a dict with
a key whose repr raise an exception.
2013-01-04 12:39:34 +02:00
Victor Stinner 18aa4477d3 Close #16281: handle tailmatch() failure and remove useless comment
"honor direction and do a forward or backwards search": the runtime speed may
be different, but I consider that it doesn't really matter in practice. The
direction was never honored before: Python 2.7 uses memcmp() for the str type
for example.
2013-01-03 03:18:09 +01:00
Victor Stinner 7ae320d667 (Merge 3.2) Issue #16455: On FreeBSD and Solaris, if the locale is C, the
ASCII/surrogateescape codec is now used, instead of the locale encoding, to
decode the command line arguments. This change fixes inconsistencies with
os.fsencode() and os.fsdecode() because these operating systems announces an
ASCII locale encoding, whereas the ISO-8859-1 encoding is used in practice.
2013-01-03 01:21:07 +01:00
Victor Stinner 20b654acb5 Issue #16455: On FreeBSD and Solaris, if the locale is C, the
ASCII/surrogateescape codec is now used, instead of the locale encoding, to
decode the command line arguments. This change fixes inconsistencies with
os.fsencode() and os.fsdecode() because these operating systems announces an
ASCII locale encoding, whereas the ISO-8859-1 encoding is used in practice.
2013-01-03 01:08:58 +01:00
Antoine Pitrou a2678f3eb6 Fix the advertised size of PyCFunctionObjects in sys._debugmallocstats(). 2012-12-30 22:46:56 +01:00
Antoine Pitrou 0811f98e10 Fix the advertised size of PyCFunctionObjects in sys._debugmallocstats(). 2012-12-30 22:46:04 +01:00
Serhiy Storchaka c819b077bb Issue #16761: Raise TypeError when int() called with base argument only. 2012-12-28 10:09:54 +02:00
Serhiy Storchaka 00e2843115 Issue #16761: Raise TypeError when int() called with base argument only. 2012-12-28 10:02:42 +02:00
Serhiy Storchaka 0b386d5247 Issue #16761: Raise TypeError when int() called with base argument only. 2012-12-28 09:42:11 +02:00
Benjamin Peterson 513762fe9c use more specific type 2012-12-26 16:43:33 -06:00
Andrew Svetlov 4de2924dab Fix compilation error for #15422 2012-12-26 23:08:54 +02:00
Gregory P. Smith a689e524e7 Test for issue16772 and redoes the previous fix to accept __index__-aware
objects as the base by using PyNumber_AsSsize_t similar to round().
2012-12-25 22:38:32 -08:00
Gregory P. Smith 4fbbf8c0a3 Fixes issue #16772: int() constructor second argument (base) must be an int.
Consistent with the behavior in Python 2.
2012-12-25 13:05:31 -08:00
Andrew Svetlov 3ba3a3ee56 Issue #15422: get rid of PyCFunction_New macro 2012-12-25 13:32:35 +02:00
Andrew Svetlov 2cd8ce4690 Issue #9856: Replace deprecation warinigs to raising TypeError in object.__format__
Patch by Florent Xicluna.
2012-12-23 14:27:17 +02:00
Benjamin Peterson 7643c92cdd merge 3.3 (#16722) 2012-12-19 15:28:46 -06:00
Benjamin Peterson 5ff3f73d94 try to call __bytes__ before __index__ (closes #16722) 2012-12-19 15:27:41 -06:00
Andrew Svetlov 2606a6f197 Issue #16719: Get rid of WindowsError. Use OSError instead
Patch by Serhiy Storchaka.
2012-12-19 14:33:35 +02:00
Antoine Pitrou 928405303d Following issue #13390, fix compilation --without-pymalloc, and make sys.getallocatedblocks() return 0 in that situation. 2012-12-17 23:05:59 +01:00
Gregory P. Smith 27dc02e8c5 Fix the internals of our hash functions to used unsigned values during hash
computation as the overflow behavior of signed integers is undefined.

NOTE: This change is smaller compared to 3.2 as much of this cleanup had
already been done.  I added the comment that my change in 3.2 added so that the
code would match up.  Otherwise this just adds or synchronizes appropriate UL
designations on some constants to be pedantic.

In practice we require compiling everything with -fwrapv which forces overflow
to be defined as twos compliment but this keeps the code cleaner for checkers
or in the case where someone has compiled it without -fwrapv or their
compiler's equivalent.  We could work to get rid of the -fwrapv requirement
in 3.4 but that requires more planning.

Found by Clang trunk's Undefined Behavior Sanitizer (UBSan).

Cleanup only - no functionality or hash values change.
2012-12-10 19:51:29 -08:00
Gregory P. Smith a6be61ec71 Keep y a Py_hash_t instead of Py_uhash_t as it is compared with == -1 and the
compiler logic will do the right thing with just x as a Py_uhash_t.  This
matches what was already done in the 3.3 version.

cleanup only - no functionality or hash values change.
2012-12-10 18:34:09 -08:00
Gregory P. Smith c2176e46d7 Fix the internals of our hash functions to used unsigned values during hash
computation as the overflow behavior of signed integers is undefined.

NOTE: This change is smaller compared to 3.2 as much of this cleanup had
already been done.  I added the comment that my change in 3.2 added so that the
code would match up.  Otherwise this just adds or synchronizes appropriate UL
designations on some constants to be pedantic.

In practice we require compiling everything with -fwrapv which forces overflow
to be defined as twos compliment but this keeps the code cleaner for checkers
or in the case where someone has compiled it without -fwrapv or their
compiler's equivalent.

Found by Clang trunk's Undefined Behavior Sanitizer (UBSan).

Cleanup only - no functionality or hash values change.
2012-12-10 18:32:53 -08:00
Gregory P. Smith 27cbcd6241 Fix the internals of our hash functions to used unsigned values during hash
computation as the overflow behavior of signed integers is undefined.

In practice we require compiling everything with -fwrapv which forces overflow
to be defined as twos compliment but this keeps the code cleaner for checkers
or in the case where someone has compiled it without -fwrapv or their
compiler's equivalent.

Found by Clang trunk's Undefined Behavior Sanitizer (UBSan).

Cleanup only - no functionality or hash values change.
2012-12-10 18:15:46 -08:00
Antoine Pitrou f9d0b1256f Issue #13390: New function :func:`sys.getallocatedblocks()` returns the number of memory blocks currently allocated.
Also, the ``-R`` option to regrtest uses this function to guard against memory allocation leaks.
2012-12-09 14:28:26 +01:00
Antoine Pitrou 53f604c794 Issue #16602: When a weakref's target was part of a long deallocation chain, the object could remain reachable through its weakref even though its refcount had dropped to zero.
Thanks to Eugene Toder for diagnosing and reporting the issue.
2012-12-08 21:18:50 +01:00
Antoine Pitrou f93ed3fa67 Issue #16602: When a weakref's target was part of a long deallocation chain, the object could remain reachable through its weakref even though its refcount had dropped to zero.
Thanks to Eugene Toder for diagnosing and reporting the issue.
2012-12-08 21:17:03 +01:00
Antoine Pitrou 62a0d6ea40 Issue #16602: When a weakref's target was part of a long deallocation chain, the object could remain reachable through its weakref even though its refcount had dropped to zero.
Thanks to Eugene Toder for diagnosing and reporting the issue.
2012-12-08 21:15:26 +01:00
Chris Jerdonek e7f2186f99 Issue #16495: remove extraneous NULL encoding check from bytes_decode().
The NULL encoding check in bytes_decode() was unnecessary because this case
is already taken care of by the call to _Py_normalize_encoding() inside
PyUnicode_Decode().
2012-12-07 15:51:53 -08:00
Victor Stinner 8dbd421b4d Cleanup unicodeobject.c
* Remove micro-optization:
   (errors == "surrogateescape" || strcmp(errors, "surrogateescape") == 0).
   Only use strcmp()
 * Initialize 'arg' members in unicode_format_arg() to help the compiler to
   diagnose real bugs and also make the code simpler to read
2012-12-04 09:30:24 +01:00
Victor Stinner d45c7f8d74 Issue #16455: On FreeBSD and Solaris, if the locale is C, the
ASCII/surrogateescape codec is now used, instead of the locale encoding, to
decode the command line arguments. This change fixes inconsistencies with
os.fsencode() and os.fsdecode() because these operating systems announces an
ASCII locale encoding, whereas the ISO-8859-1 encoding is used in practice.
2012-12-04 01:34:47 +01:00
Victor Stinner 2660e427d1 (Merge 3.2) Issue #16416: On Mac OS X, operating system data are now always
encoded/decoded to/from UTF-8/surrogateescape, instead of the locale encoding
(which may be ASCII if no locale environment variable is set), to avoid
inconsistencies with os.fsencode() and os.fsdecode() functions which are
already using UTF-8/surrogateescape.
2012-12-03 12:48:53 +01:00
Victor Stinner 27b1ca29cc Issue #16416: On Mac OS X, operating system data are now always
encoded/decoded to/from UTF-8/surrogateescape, instead of the locale encoding
(which may be ASCII if no locale environment variable is set), to avoid
inconsistencies with os.fsencode() and os.fsdecode() functions which are
already using UTF-8/surrogateescape.
2012-12-03 12:47:59 +01:00
Antoine Pitrou 0e9958b543 Issue #16562: Optimize dict equality testing.
Patch by Serhiy Storchaka (reviewed by Martin and Raymond).
2012-12-02 19:10:07 +01:00
Christian Heimes 5f7e8dab11 Issue #16592: stringlib_bytes_join doesn't raise MemoryError on allocation failure 2012-12-02 07:56:42 +01:00
Antoine Pitrou 5439458a2a Issue #16215: Fix potential double memory free in str.replace().
Patch by Serhiy Storchaka.
2012-11-17 23:29:28 +01:00
Antoine Pitrou 6d5ad227a5 Issue #16215: Fix potential double memory free in str.replace().
Patch by Serhiy Storchaka.
2012-11-17 23:28:17 +01:00
Mark Dickinson ffdb2c21b3 Issue #16451: Refactor to remove duplication between range and slice in slice index computations. 2012-11-17 19:18:10 +00:00
Mark Dickinson d20fb82195 Issue #16290: __complex__ must now always return an instance of complex. 2012-11-14 17:08:31 +00:00
Victor Stinner 0d92c4f667 Issue #16416: Fix error handling in _Py_wchar2char() _Py_char2wchar() functions 2012-11-12 23:32:21 +01:00
Antoine Pitrou 898347056a Issue #16453: Fix equality testing of dead weakref objects.
Also add tests for ordering and hashing.
2012-11-11 19:39:35 +01:00
Antoine Pitrou f6a50cfa07 Issue #16453: Fix equality testing of dead weakref objects.
Also add tests for ordering and hashing.
2012-11-11 19:37:41 +01:00
Antoine Pitrou e11fecb5a9 Issue #16453: Fix equality testing of dead weakref objects.
Also add tests for ordering and hashing.
2012-11-11 19:36:51 +01:00
Mark Dickinson c8a6967ea8 Issue #14794: slice.indices no longer returns OverflowError for out-of-range start, stop, step or length. 2012-11-10 14:52:10 +00:00
Victor Stinner fc009eff9e Close #16311: Use the _PyUnicodeWriter API in text decoders
* Remove unicode_widen(): replaced with _PyUnicodeWriter_Prepare()
 * Remove unicode_putchar(): replaced with
   PyUnicodeWriter_Prepare() + PyUnicode_WRITER()
 * When handling an decoding error, only overallocate the buffer by +25%
   instead of +100%
2012-11-07 00:36:38 +01:00
Victor Stinner 6caa6fb535 (Merge 3.3) Issue #8271: Fix compilation on Windows 2012-11-05 00:00:50 +01:00
Victor Stinner ab60de478d Issue #8271: Fix compilation on Windows 2012-11-04 23:59:15 +01:00
Ezio Melotti cfa9636404 #8271: merge with 3.3. 2012-11-04 23:23:09 +02:00
Ezio Melotti f7ed5d111b #8271: the utf-8 decoder now outputs the correct number of U+FFFD characters when used with the "replace" error handler on invalid utf-8 sequences. Patch by Serhiy Storchaka, tests by Ezio Melotti. 2012-11-04 23:21:38 +02:00
Mark Dickinson c992fafddc Issue #16402: Merge fix from 3.3 2012-11-04 11:47:47 +00:00
Mark Dickinson 1321edaa55 Issue #16402: Merge fix from 3.2 2012-11-04 11:47:05 +00:00
Mark Dickinson 8cd1c7681d Issue #16402: In range slicing, fix shadowing of exceptions from __index__ method. 2012-11-04 11:46:17 +00:00
Christian Heimes e9d08cf450 Fix compilation on Windows 2012-11-03 23:08:27 +01:00
Christian Heimes d081fbba58 Fix compilation on Windows 2012-11-03 23:08:18 +01:00
Christian Heimes 6d26ade920 Fix compilation on Windows 2012-11-03 23:07:59 +01:00
Ezio Melotti 212843b29f #8401: merge with 3.3. 2012-11-03 21:24:47 +02:00
Ezio Melotti 7376801f61 #8401: merge with 3.2. 2012-11-03 21:22:41 +02:00
Ezio Melotti c64bcbec4b #8401: assigning an int to a bytearray slice (e.g. b[3:4] = 5) now raises an error. 2012-11-03 21:19:06 +02:00
Stefan Krah c38c816ea1 Merge 3.3. 2012-11-02 17:55:11 +01:00
Stefan Krah 4af77a0276 Issue #15814: Use hash function that is compatible with the equality
definition from #15573.
2012-11-02 17:49:22 +01:00
Benjamin Peterson 8781d4a84c merge 3.3 2012-10-31 14:22:31 -04:00
Benjamin Peterson 591c921411 merge 3.2 2012-10-31 14:22:25 -04:00
Benjamin Peterson 9892f52145 avoid a function call with redundant checks for dict size 2012-10-31 14:22:12 -04:00
Benjamin Peterson 7503e08588 merge 3.3 (#16345) 2012-10-31 14:10:04 -04:00
Benjamin Peterson d97eb0d338 merge 3.2 (#16345) 2012-10-31 14:09:11 -04:00
Benjamin Peterson d1f2cb37a2 only fast-path fromkeys() when the constructor returns a empty dict (closes #16345) 2012-10-31 14:05:55 -04:00
Benjamin Peterson 3cb90241fc merge 3.3 2012-10-31 00:04:42 -04:00
Benjamin Peterson 2c05a2e01b do safety checks on __qualname__ assignment 2012-10-31 00:01:15 -04:00
Benjamin Peterson 8afa7fa510 don't shadow the __qualname__ descriptor with __qualname__ in the class's __dict__ (closes #16271) 2012-10-30 23:51:03 -04:00
Benjamin Peterson 42124a727d initialize map/filter/zip in _PyBuiltin_Init rather than the catch-all function 2012-10-30 23:41:54 -04:00
Benjamin Peterson 7ff2094bc7 merge 3.3 (#16369) 2012-10-30 23:31:12 -04:00
Benjamin Peterson e8ea97fffb merge 3.2 (#16369) 2012-10-30 23:27:52 -04:00
Benjamin Peterson c43112823b initialize more global type objects (closes #16369) 2012-10-30 23:21:10 -04:00
Victor Stinner 7a6d7cf3db Issue #9566: Use the right type to fix a compiler warnings on Win64 2012-10-31 00:37:41 +01:00
Victor Stinner 4ca1cf35fb Issue #16086: PyTypeObject.tp_flags and PyType_Spec.flags are now unsigned
... (unsigned long and unsigned int) to avoid an undefined behaviour with
Py_TPFLAGS_TYPE_SUBCLASS ((1 << 31). PyType_GetFlags() result type is now
unsigned too (unsigned long, instead of long).
2012-10-30 23:40:45 +01:00
Victor Stinner e64322e034 Close #14625: Rewrite the UTF-32 decoder. It is now 3x to 4x faster
Patch written by Serhiy Storchaka.
2012-10-30 23:12:47 +01:00
Victor Stinner 76df43de30 Issue #16330: Use surrogate-related macros
Patch written by Serhiy Storchaka.
2012-10-30 01:42:39 +01:00
Mark Dickinson fb90c0934c Issue #14700: Fix buggy overflow checks for large precision and width in new-style and old-style formatting. 2012-10-28 10:18:03 +00:00
Victor Stinner c6cf1ba29e Replace usage of the deprecated Py_UNICODE_COPY() with Py_MEMCPY() in resize_copy() 2012-10-23 02:54:47 +02:00
Victor Stinner fe75fb4b3e Optimize _PyUnicode_HasNULChars(): use findchar() instead of PyUnicode_Contains() 2012-10-23 02:52:18 +02:00