Commit Graph

5702 Commits

Author SHA1 Message Date
Serhiy Storchaka 0ce7a3a34c Issue #25914: Fixed and simplified OrderedDict.__sizeof__. 2015-12-22 08:16:18 +02:00
Serhiy Storchaka 3182db356c Issue #25766: Special method __bytes__() now works in str subclasses. 2015-12-20 16:37:21 +02:00
Serhiy Storchaka 5aac3ed799 Issue #25766: Special method __bytes__() now works in str subclasses. 2015-12-20 16:36:34 +02:00
Serhiy Storchaka a9406e77fa Issue #25421: __sizeof__ methods of builtin types now use dynamic basic size.
This allows sys.getsize() to work correctly with their subclasses with
__slots__ defined.
2015-12-19 20:07:11 +02:00
Serhiy Storchaka 5c4064e8bd Issue #25421: __sizeof__ methods of builtin types now use dynamic basic size.
This allows sys.getsize() to work correctly with their subclasses with
__slots__ defined.
2015-12-19 20:05:25 +02:00
Serhiy Storchaka 42bb126f0a Issue #25899: Converted Objects/listsort.txt to UTF-8.
Original patch by Chris Angelico.
2015-12-18 13:23:33 +02:00
Serhiy Storchaka 9b3a2eec1c Issues #25890, #25891, #25892: Removed unused variables in Windows code.
Reported by Alexander Riccio.
2015-12-18 10:03:13 +02:00
Raymond Hettinger e4495877dd Minor tweek. Counting down rather than up reduces register pressure. 2015-12-15 00:42:30 -08:00
Raymond Hettinger 86d322f020 Undo inadvertent line swap 2015-12-13 19:27:17 -08:00
Raymond Hettinger 5088f6005f Hoist constant expressions (so->table and so->mask) out of the inner-loop. 2015-12-13 18:45:01 -08:00
Serhiy Storchaka 7c088a9b5c Issue #25709: Fixed problem with in-place string concatenation and utf-8 cache. 2015-12-03 01:05:52 +02:00
Serhiy Storchaka 6648bf5661 Issue #25709: Fixed problem with in-place string concatenation and utf-8 cache. 2015-12-03 01:04:37 +02:00
Serhiy Storchaka 7aa690860e Issue #25709: Fixed problem with in-place string concatenation and utf-8 cache. 2015-12-03 01:02:03 +02:00
Serhiy Storchaka 6a50e79130 Issue #24097: Fixed crash in object.__reduce__() if slot name is freed inside
__getattr__.  Original patch by Antoine Pitrou.
2015-11-25 18:35:15 +02:00
Serhiy Storchaka 670d78abc5 Issue #24097: Fixed crash in object.__reduce__() if slot name is freed inside
__getattr__.  Original patch by Antoine Pitrou.
2015-11-25 18:34:19 +02:00
Serhiy Storchaka d28bb624d1 Issue #24097: Fixed crash in object.__reduce__() if slot name is freed inside
__getattr__.  Original patch by Antoine Pitrou.
2015-11-25 18:33:29 +02:00
Serhiy Storchaka bb6e4a0b31 Issue #24731: Fixed crash on converting objects with special methods
__bytes__, __trunc__, and __float__ returning instances of subclasses of
bytes, int, and float to subclasses of bytes, int, and float correspondingly.
2015-11-25 15:53:19 +02:00
Serhiy Storchaka f9afda57ad Issue #24731: Fixed crash on converting objects with special methods
__bytes__, __trunc__, and __float__ returning instances of subclasses of
bytes, int, and float to subclasses of bytes, int, and float correspondingly.
2015-11-25 15:52:04 +02:00
Serhiy Storchaka 15095800a3 Issue #24731: Fixed crash on converting objects with special methods
__bytes__, __trunc__, and __float__ returning instances of subclasses of
bytes, int, and float to subclasses of bytes, int, and float correspondingly.
2015-11-25 15:47:01 +02:00
Victor Stinner b4efc963d6 Issue #25557: Refactor _PyDict_LoadGlobal()
Don't fallback to PyDict_GetItemWithError() if the hash is unknown: compute the
hash instead. Add also comments to explain the optimization a little bit.
2015-11-20 09:24:02 +01:00
Raymond Hettinger 66f6238fca Add assertion to verify the pre-condition in the comments. 2015-11-17 20:58:43 -08:00
Raymond Hettinger 6019c8ced0 Issue #25629: Move set fill/used updates out of inner loop 2015-11-17 08:28:07 -08:00
Benjamin Peterson d798dc1034 merge 3.5 (#25630) 2015-11-15 21:57:50 -08:00
Benjamin Peterson a4d33b3428 make the PyUnicode_FSConverter cleanup set the decrefed argument to NULL (closes #25630) 2015-11-15 21:57:39 -08:00
Serhiy Storchaka 413fdcea21 Issue #24821: Refactor STRINGLIB(fastsearch_memchr_1char) and split it on
STRINGLIB(find_char) and STRINGLIB(rfind_char) that can be used independedly
without special preconditions.
2015-11-14 15:42:17 +02:00
Serhiy Storchaka 1010921c71 Issue #25462: The hash of the key now is calculated only once in most
operations in C implementation of OrderedDict.
2015-11-13 15:18:26 +02:00
Serhiy Storchaka 19a70e7f5d Issue #25462: The hash of the key now is calculated only once in most
operations in C implementation of OrderedDict.
2015-11-13 14:48:36 +02:00
Serhiy Storchaka 12ab296f82 Issue #22995: Default implementation of __reduce__ and __reduce_ex__ now
rejects builtin types with not defined __new__.
Added tests for non-pickleable types.
2015-11-12 11:34:39 +02:00
Serhiy Storchaka 609a2e17ad Issue #22995: Default implementation of __reduce__ and __reduce_ex__ now
rejects builtin types with not defined __new__.
Added tests for non-pickleable types.
2015-11-12 11:31:51 +02:00
Serhiy Storchaka d7a4415599 Issue #22995: Default implementation of __reduce__ and __reduce_ex__ now
rejects builtin types with not defined __new__.
Added tests for non-pickleable types.
2015-11-12 11:23:04 +02:00
Stefan Krah e2cf984384 Merge 3.5. 2015-11-10 18:35:47 +01:00
Stefan Krah 7c3f3859a5 Fix Visual Studio warning. 2015-11-10 18:35:19 +01:00
Stefan Krah 3407cc26cb Merge 3.5. 2015-11-10 18:18:07 +01:00
Stefan Krah 0ce5b6e268 Iaaue #25598: Fix memory_hex from #9951 for non-contiguous buffers. 2015-11-10 18:17:22 +01:00
Martin Panter 9513ba3b8f Issue #24802: Merge null termination fixes from 3.5 2015-11-07 03:15:32 +00:00
Martin Panter 61d6e4ae9d Issue #24802: Merge null termination fixes from 3.4 into 3.5 2015-11-07 02:56:11 +00:00
Martin Panter eeb896c411 Issue #24802: Copy bytes-like objects to null-terminated buffers if necessary
This avoids possible buffer overreads when int(), float(), compile(), exec()
and eval() are passed bytes-like objects. Similar code is removed from the
complex() constructor, where it was not reachable.

Patch by John Leitch, Serhiy Storchaka and Martin Panter.
2015-11-07 02:32:21 +00:00
Serhiy Storchaka 0c009bf3b5 Issue #25410: Made testing that od_fast_nodes and dk_entries are in sync more
reliable.
2015-11-06 12:00:43 +02:00
Serhiy Storchaka 97f46db904 Issue #25410: Made testing that od_fast_nodes and dk_entries are in sync more
reliable.
2015-11-06 12:00:03 +02:00
Serhiy Storchaka 3a811f272a Issue #24726: Revert setting the value on the dict if
_odict_add_new_node() fails.
2015-11-06 11:07:31 +02:00
Serhiy Storchaka d5f353ec8d Issue #24726: Revert setting the value on the dict if
_odict_add_new_node() fails.
2015-11-06 11:07:11 +02:00
Serhiy Storchaka 6980be40bd Issue #25558: Refactoring OrderedDict iteration. 2015-11-06 10:40:05 +02:00
Serhiy Storchaka 9c967611e3 Issue #25558: Refactoring OrderedDict iteration. 2015-11-06 10:39:51 +02:00
Victor Stinner e20310fa19 Issue #25556: Add assertions to PyObject_GetItem() to ensure that an exception
is raised when it returns NULL.

Simplify also ceval.c: rely on the fact that PyObject_GetItem() raised an
exception when it returns NULL.
2015-11-05 13:56:58 +01:00
Serhiy Storchaka f83afa508f Issue #25449: Fixed a crash and leaking NULL in repr() of OrderedDict that
was mutated by direct calls of dict methods.
2015-11-04 22:33:33 +02:00
Serhiy Storchaka 710cd34bdb Issue #25449: Fixed a crash and leaking NULL in repr() of OrderedDict that
was mutated by direct calls of dict methods.
2015-11-04 22:33:07 +02:00
Serhiy Storchaka 161b6ff19c Issue #25449: Iterating OrderedDict with keys with unstable hash now raises
KeyError in C implementations as well as in Python implementation.

Added tests for OrderedDict subclasses.
2015-11-04 22:08:52 +02:00
Serhiy Storchaka b45b7b2137 Issue #25449: Iterating OrderedDict with keys with unstable hash now raises
KeyError in C implementations as well as in Python implementation.

Added tests for OrderedDict subclasses.
2015-11-04 22:05:38 +02:00
Serhiy Storchaka 4a7c03aab4 Issue #25523: Merge a-to-an corrections from 3.5. 2015-11-02 14:44:29 +02:00
Serhiy Storchaka a84f6c3dd3 Issue #25523: Merge a-to-an corrections from 3.4. 2015-11-02 14:39:05 +02:00
Serhiy Storchaka d65c9496da Issue #25523: Further a-to-an corrections. 2015-11-02 14:10:23 +02:00
Martin Panter e56a919100 Issue #25523: Merge a-to-an corrections from 3.5 2015-11-02 04:27:17 +00:00
Martin Panter 2eb819f7a8 Issue #25523: Merge "a" to "an" fixes from 3.4 into 3.5 2015-11-02 04:04:57 +00:00
Martin Panter 7462b64911 Issue #25523: Correct "a" article to "an" article
This changes the main documentation, doc strings, source code comments, and a
couple error messages in the test suite. In some cases the word was removed
or edited some other way to fix the grammar.
2015-11-02 03:37:02 +00:00
Serhiy Storchaka 9b69491901 Issue #25395: Fixed crash when highly nested OrderedDict structures were
garbage collected.
2015-11-01 16:13:45 +02:00
Serhiy Storchaka 14eefe353e Issue #25395: Fixed crash when highly nested OrderedDict structures were
garbage collected.
2015-11-01 16:12:34 +02:00
Serhiy Storchaka afbea7af02 Issue #25410: C implementation of OrderedDict now uses type(self) instead of
self.__class__ in __repr__() and __reduce__() for simplicity and reliability.
2015-10-22 20:19:14 +03:00
Serhiy Storchaka 4575beba4b Issue #25410: C implementation of OrderedDict now uses type(self) instead of
self.__class__ in __repr__() and __reduce__() for simplicity and reliability.
2015-10-22 20:18:24 +03:00
Serhiy Storchaka 60a08929af Issue #25410: Fixed a memory leak in OrderedDict in the case when key's hash
calculation fails.
2015-10-20 18:22:36 +03:00
Serhiy Storchaka d17427b7bd Issue #25410: Fixed a memory leak in OrderedDict in the case when key's hash
calculation fails.
2015-10-20 18:21:48 +03:00
Serhiy Storchaka 3e9f87782e Issue #25410: Cleaned up and fixed minor bugs in C implementation of OrderedDict. 2015-10-18 09:54:42 +03:00
Serhiy Storchaka 8003bafd7f Issue #25410: Cleaned up and fixed minor bugs in C implementation of OrderedDict. 2015-10-18 09:53:17 +03:00
Victor Stinner 91108f049f Issue #25210: Change error message of do_richcompare()
Don't add parenthesis to type names. Add also quotes around the type names.

Before:

  TypeError: unorderable types: int() < NoneType()

After:

  TypeError: '<' not supported between instances of 'int' and 'NoneType'
2015-10-14 18:25:31 +02:00
Serhiy Storchaka dbb98c1443 Issue #25406: Fixed a bug in C implementation of OrderedDict.move_to_end()
that caused segmentation fault or hang in iterating after moving several
items to the start of ordered dict.
2015-10-14 19:22:44 +03:00
Serhiy Storchaka 992ec46acc Issue #25406: Fixed a bug in C implementation of OrderedDict.move_to_end()
that caused segmentation fault or hang in iterating after moving several
items to the start of ordered dict.
2015-10-14 19:21:24 +03:00
Victor Stinner c3d2bc19e4 Use _PyBytesWriter in _PyBytes_FromIterator() 2015-10-14 14:15:49 +02:00
Victor Stinner c5c3ba4bec Add _PyBytesWriter_Resize() function
This function gives a control to the buffer size without using min_size.
2015-10-14 13:56:47 +02:00
Victor Stinner 3c50ce39bf Factorize _PyBytes_FromList() and _PyBytes_FromTuple() code using a C macro 2015-10-14 13:50:40 +02:00
Victor Stinner f2eafa323b Split PyBytes_FromObject() into subfunctions 2015-10-14 13:44:29 +02:00
Victor Stinner 2ec8063cc9 Modify _PyBytes_DecodeEscapeRecode() to use _PyBytesAPI
* Don't overallocate by 400% when recode is needed: only overallocate on demand
  using _PyBytesWriter.
* Use _PyLong_DigitValue to convert hexadecimal digit to int
* Create _PyBytes_DecodeEscapeRecode() subfunction
2015-10-14 13:32:13 +02:00
Victor Stinner 1285e5c805 Fix compiler warnings (uninitialized variables), false alarms in fact 2015-10-14 12:10:20 +02:00
Victor Stinner f6358a7e4c _PyBytesWriter_Alloc(): only use 10 bytes of the small buffer in debug mode to
enhance code to detect buffer under- and overflow.
2015-10-14 12:02:39 +02:00
Victor Stinner f091033b14 Issue #25401: Remove now unused hex_digit_to_int() function 2015-10-14 11:59:46 +02:00
Victor Stinner 2bf8993db9 Optimize bytes.fromhex() and bytearray.fromhex()
Issue #25401: Optimize bytes.fromhex() and bytearray.fromhex(): they are now
between 2x and 3.5x faster. Changes:

* Use a fast-path working on a char* string for ASCII string
* Use a slow-path for non-ASCII string
* Replace slow hex_digit_to_int() function with a O(1) lookup in
  _PyLong_DigitValue precomputed table
* Use _PyBytesWriter API to handle the buffer
* Add unit tests to check the error position in error messages
2015-10-14 11:25:33 +02:00
Victor Stinner 772b2b09f2 Optimize bytearray % args
Issue #25399: Don't create temporary bytes objects: modify _PyBytes_Format() to
create work directly on bytearray objects.

* Rename _PyBytes_Format() to _PyBytes_FormatEx() just in case if something
  outside CPython uses it
* _PyBytes_FormatEx() now uses (char*, Py_ssize_t) for the input string, so
  bytearray_format() doesn't need tot create a temporary input bytes object
* Add use_bytearray parameter to _PyBytes_FormatEx() which is passed to
  _PyBytesWriter, to create a bytearray buffer instead of a bytes buffer

Most formatting operations are now between 2.5 and 5 times faster.
2015-10-14 09:56:53 +02:00
Victor Stinner 661aaccf9d Add use_bytearray attribute to _PyBytesWriter
Issue #25399: Add a new use_bytearray attribute to _PyBytesWriter to use a
bytearray buffer, instead of using a bytes object.
2015-10-14 09:41:48 +02:00
Victor Stinner 199c9a6f4b Fix long_format_binary()
Issue #25399: Fix long_format_binary(), allocate bytes for the bytes writer.
2015-10-14 09:47:23 +02:00
Victor Stinner 03dab786b2 Rewrite PyBytes_FromFormatV() using _PyBytesWriter API
* Add much more unit tests on PyBytes_FromFormatV()
* Remove the first loop to compute the length of the output string
* Use _PyBytesWriter to handle the bytes buffer, use overallocation
* Cleanup the code to make simpler and easier to review
2015-10-14 00:21:35 +02:00
Victor Stinner 358af13526 Issue #25353: Optimize unicode escape and raw unicode escape encoders to use
the new _PyBytesWriter API.
2015-10-12 22:36:57 +02:00
Victor Stinner e9aa5950bb Fix compilation error in _PyBytesWriter_WriteBytes() on Windows 2015-10-12 13:57:47 +02:00
Victor Stinner 6c2cdae9e6 Writer APIs: use empty string singletons
Modify _PyBytesWriter_Finish() and _PyUnicodeWriter_Finish() to return the
empty bytes/Unicode string if the string is empty.
2015-10-12 13:29:43 +02:00
Victor Stinner c29e29bed1 Relax _PyBytesWriter API
Don't require _PyBytesWriter pointer to be a "char *". Same change for
_PyBytesWriter_WriteBytes() parameter.

For example, binascii uses "unsigned char*".
2015-10-12 13:12:54 +02:00
Serhiy Storchaka 0d554d7ef1 Issue #24164: Objects that need calling ``__new__`` with keyword arguments,
can now be pickled using pickle protocols older than protocol version 4.
2015-10-10 22:42:18 +03:00
Victor Stinner 0cdad1e2bc Issue #25349: Add fast path for b'%c' % int
Optimize also %% formater.
2015-10-09 22:50:36 +02:00
Victor Stinner be75b8cf23 Issue #25349: Optimize bytes % int
Optimize bytes.__mod__(args) for integere formats: %d (%i, %u), %o, %x and %X.
_PyBytesWriter is now used to format directly the integer into the writer
buffer, instead of using a temporary bytes object.

Formatting is between 30% and 50% faster on a microbenchmark.
2015-10-09 22:43:24 +02:00
Victor Stinner 6bd525b656 Optimize error handlers of ASCII and Latin1 encoders when the replacement
string is pure ASCII: use _PyBytesWriter_WriteBytes(), don't check individual
character.

Cleanup unicode_encode_ucs1():

* Rename repunicode to rep
* Clear rep object on error
* Factorize code between bytes and unicode path
2015-10-09 13:10:05 +02:00
Victor Stinner ce179bf6ba Add _PyBytesWriter_WriteBytes() to factorize the code 2015-10-09 12:57:22 +02:00
Victor Stinner ad7715891e _PyBytesWriter: simplify code to avoid "prealloc" parameters
Substract preallocate bytes from min_size before calling
_PyBytesWriter_Prepare().
2015-10-09 12:38:53 +02:00
Victor Stinner 53926a1ce2 _PyBytesWriter: rename size attribute to min_size 2015-10-09 12:37:03 +02:00
Victor Stinner fa7762ec06 Issue #25349: Optimize bytes % args using the new private _PyBytesWriter API
* Thanks to the _PyBytesWriter API, output smaller than 512 bytes are allocated
  on the stack and so avoid calling _PyBytes_Resize(). Because of that, change
  the default buffer size to fmtcnt instead of fmtcnt+100.
* Rely on _PyBytesWriter algorithm to overallocate the buffer instead of using
  a custom code. For example, _PyBytesWriter uses a different overallocation
  factor (25% or 50%) depending on the platform to get best performances.
* Disable overallocation for the last write.
* Replace C loops to fill characters with memset()
* Add also many comments to _PyBytes_Format()
* Remove unused FORMATBUFLEN constant
* Avoid the creation of a temporary bytes object when formatting a floating
  point number (when no custom formatting option is used)
* Fix also reference leaks on error handling
* Use Py_MEMCPY() to copy bytes between two formatters (%)
2015-10-09 11:48:06 +02:00
Victor Stinner b3653a3458 Issue #25318: cleanup code _PyBytesWriter
Rename "stack buffer" to "small buffer".

Add also an assertion in _PyBytesWriter_GetPos().
2015-10-09 03:38:24 +02:00
Victor Stinner 3fa36ff5e4 Issue #25318: Fix backslashreplace()
Fix code to estimate the needed space.
2015-10-09 03:37:11 +02:00
Victor Stinner 797485e101 Issue #25318: Avoid sprintf() in backslashreplace()
Rewrite backslashreplace() to be closer to PyCodec_BackslashReplaceErrors().

Add also unit tests for non-BMP characters.
2015-10-09 03:17:30 +02:00
Victor Stinner b13b97d3b8 Issue #25318: Fix compilation error
Replace "#if Py_DEBUG" with "#ifdef Py_DEBUG".
2015-10-09 02:52:16 +02:00
Victor Stinner 0016507c16 Issue #25318: Move _PyBytesWriter to bytesobject.c
Declare also the private API in bytesobject.h.
2015-10-09 01:53:21 +02:00
Victor Stinner e7bf86cd7d Optimize backslashreplace error handler
Issue #25318: Optimize backslashreplace and xmlcharrefreplace error handlers in
UTF-8 encoder. Optimize also backslashreplace error handler for ASCII and
Latin1 encoders.

Use the new _PyBytesWriter API to optimize these error handlers for the
encoders. It avoids to create an exception and call the slow implementation of
the error handler.
2015-10-09 01:39:28 +02:00
Victor Stinner fdfbf78114 Issue #25318: Add _PyBytesWriter API
Add a new private API to optimize Unicode encoders. It uses a small buffer
allocated on the stack and supports overallocation.

Use _PyBytesWriter API for UCS1 (ASCII and Latin1) and UTF-8 encoders. Enable
overallocation for the UTF-8 encoder with error handlers.

unicode_encode_ucs1(): initialize collend to collstart+1 to not check the
current character twice, we already know that it is not ASCII.
2015-10-09 00:33:49 +02:00
Martin Panter 585a6acfef Merge typo fixes from 3.5 2015-10-07 11:13:55 +00:00
Martin Panter ec1aa5c2a1 More typos in 3.5 documentation and comments 2015-10-07 11:03:53 +00:00
Martin Panter 3f930dcd87 Merge typo fixes from 3.4 into 3.5 2015-10-07 11:01:47 +00:00
Martin Panter 9955a373a8 Various minor typos in documentation and comments 2015-10-07 10:26:23 +00:00
Benjamin Peterson cdae2cb88a merge 3.5 (closes #24806) 2015-10-06 19:42:46 -07:00
Benjamin Peterson 59dc696821 merge 3.4 (#24806) 2015-10-06 19:42:02 -07:00
Benjamin Peterson bd6c41a185 prevent unacceptable bases from becoming bases through multiple inheritance (#24806) 2015-10-06 19:36:54 -07:00
Victor Stinner 74e8fac3c8 Issue #25301: Fix compatibility with ISO C90 2015-10-05 13:49:26 +02:00
Victor Stinner 1d65d9192d Issue #25301: The UTF-8 decoder is now up to 15 times as fast for error
handlers: ``ignore``, ``replace`` and ``surrogateescape``.
2015-10-05 13:43:50 +02:00
Victor Stinner eb36fdaad8 Fix _PyUnicodeWriter_PrepareKind()
Initialize kind to 0 (PyUnicode_WCHAR_KIND) to ensure that
_PyUnicodeWriter_PrepareKind() handles correctly read-only buffer: copy the
buffer.
2015-10-03 01:55:51 +02:00
Serhiy Storchaka 29e68edbf4 Issue #24848: Fixed bugs in UTF-7 decoding of misformed data:
1. Non-ASCII bytes were accepted after shift sequence.
2. A low surrogate could be emitted in case of error in high surrogate.
3. In some circumstances the '\xfd' character was produced instead of the
replacement character '\ufffd' (due to a bug in _PyUnicodeWriter).
2015-10-02 13:14:03 +03:00
Serhiy Storchaka 58c8f2bb6d Issue #24848: Fixed bugs in UTF-7 decoding of misformed data:
1. Non-ASCII bytes were accepted after shift sequence.
2. A low surrogate could be emitted in case of error in high surrogate.
3. In some circumstances the '\xfd' character was produced instead of the
replacement character '\ufffd' (due to a bug in _PyUnicodeWriter).
2015-10-02 13:13:14 +03:00
Serhiy Storchaka 28b21e50c8 Issue #24848: Fixed bugs in UTF-7 decoding of misformed data:
1. Non-ASCII bytes were accepted after shift sequence.
2. A low surrogate could be emitted in case of error in high surrogate.
2015-10-02 13:07:28 +03:00
Serhiy Storchaka 5dbe245ef2 Issue #24483: C implementation of functools.lru_cache() now calculates key's
hash only once.
2015-10-02 12:47:59 +03:00
Serhiy Storchaka b9d98d532c Issue #24483: C implementation of functools.lru_cache() now calculates key's
hash only once.
2015-10-02 12:47:11 +03:00
Victor Stinner 3222da26fe Make _PyUnicode_TranslateCharmap() symbol private
unicodeobject.h exposes PyUnicode_TranslateCharmap() and PyUnicode_Translate().
2015-10-01 22:07:32 +02:00
Victor Stinner 01ada3996b Issue #25267: The UTF-8 encoder is now up to 75 times as fast for error
handlers: ``ignore``, ``replace``, ``surrogateescape``, ``surrogatepass``.
Patch co-written with Serhiy Storchaka.
2015-10-01 21:54:51 +02:00
Victor Stinner d69dd8bd5e (Merge 3.5) Issue #25182: Fix compilation on Windows 2015-09-30 15:03:50 +02:00
Victor Stinner ae86da9b20 (Merge 3.4) Issue #25182: Fix compilation on Windows 2015-09-30 15:03:31 +02:00
Victor Stinner 89719e1daf Issue #25182: Fix compilation on Windows
Restore also errno value before calling PyErr_SetFromErrno().
2015-09-30 15:01:34 +02:00
Serhiy Storchaka 85c386dee4 Issue #25182: The stdprinter (used as sys.stderr before the io module is
imported at startup) now uses the backslashreplace error handler.
2015-09-30 15:51:01 +03:00
Serhiy Storchaka 008fc77e1e Issue #25182: The stdprinter (used as sys.stderr before the io module is
imported at startup) now uses the backslashreplace error handler.
2015-09-30 15:50:32 +03:00
Serhiy Storchaka a59018c7ab Issue #25182: The stdprinter (used as sys.stderr before the io module is
imported at startup) now uses the backslashreplace error handler.
2015-09-30 15:46:53 +03:00
Victor Stinner c3713e9706 Optimize ascii/latin1+surrogateescape encoders
Issue #25227: Optimize ASCII and latin1 encoders with the ``surrogateescape``
error handler: the encoders are now up to 3 times as fast.

Initial patch written by Serhiy Storchaka.
2015-09-29 12:32:13 +02:00
Victor Stinner 0030cd52da Issue #25227: Cleanup unicode_encode_ucs1() error handler
* Change limit type from unsigned int to Py_UCS4, to use the same type than the
  "ch" variable (an Unicode character).
* Reuse ch variable for _Py_ERROR_XMLCHARREFREPLACE
* Add some newlines for readability
2015-09-24 14:45:00 +02:00
Victor Stinner 54385b206d Issue #24870: revert unwanted change
Sorry, I pushed the patch on the UTF-8 decoder by mistake :-(
2015-09-22 10:46:52 +02:00
Victor Stinner 5ebae87628 Issue #25207, #14626: Fix my commit.
It doesn't work to use #define XXX defined(YYY)" and then "#ifdef XXX"
to check YYY.
2015-09-22 01:29:33 +02:00
Victor Stinner 6174474bea _PyUnicodeWriter_PrepareInternal(): make the assertion more strict 2015-09-22 01:01:17 +02:00
Victor Stinner ca9381ea01 Issue #24870: Add _PyUnicodeWriter_PrepareKind() macro
Add a macro which ensures that the writer has at least the requested kind.
2015-09-22 00:58:32 +02:00
Victor Stinner 5014920cb7 Issue #24870: Reuse the new _Py_error_handler enum
Factorize code with the new get_error_handler() function.

Add some empty lines for readability.
2015-09-22 00:26:54 +02:00
Victor Stinner f96418de05 Issue #24870: Optimize the ASCII decoder for error handlers: surrogateescape,
ignore and replace. Initial patch written by Naoki Inada.

The decoder is now up to 60 times as fast for these error handlers.

Add also unit tests for the ASCII decoder.
2015-09-21 23:06:27 +02:00
Victor Stinner 026977717e Merge 3.5 2015-09-19 13:39:16 +02:00
Victor Stinner 5783fd2c58 Issue #24999: In longobject.c, use two shifts instead of ">> 2*PyLong_SHIFT" to
avoid undefined behaviour when LONG_MAX type is smaller than 60 bits.

This change should fix a warning with the ICC compiler.
2015-09-19 13:39:03 +02:00
Victor Stinner 058258652a Merge 3.5 (pytime, odict) 2015-09-18 13:55:15 +02:00
Victor Stinner 4a0d1e7c36 odictobject.c: fix compiler warning
PyObject_Length() returns a P_ssize_t, not an int. Use a Py_ssize_t to avoid
overflow.
2015-09-18 13:44:11 +02:00
Serhiy Storchaka 56f6e76c68 Issue #15989: Fixed some scarcely probable integer overflows.
It is very unlikely that they can occur in real code for now.
2015-09-06 21:25:30 +03:00
Guido van Rossum ba5f59089a Issue #24912: Prevent __class__ assignment to immutable built-in objects. (Merge 3.5 -> 3.6) 2015-09-05 15:20:57 -07:00
Guido van Rossum 37fdcbc4c3 Issue #24912: Prevent __class__ assignment to immutable built-in objects. (Merge 3.5.0 -> 3.5) 2015-09-05 15:20:08 -07:00
Guido van Rossum 7d293ee97d Issue #24912: Prevent __class__ assignment to immutable built-in objects. 2015-09-04 20:54:07 -07:00
Victor Stinner fa9dfd4f82 Merge 3.5 (odict) 2015-09-03 17:50:30 +02:00
Victor Stinner ca30b02abe Issue #24992: Fix error handling and a race condition (related to garbage
collection) in collections.OrderedDict constructor.

Patch reviewed by Serhiy Storchaka.
2015-09-03 17:50:04 +02:00
Victor Stinner 99bb14bf0c type_call() now detect bugs in type new and init
* Call _Py_CheckFunctionResult() to check for bugs in type
  constructors (tp_new)
* Add assertions to ensure an exception was raised if tp_init failed
  or that no exception was raised if tp_init succeed

Refactor also the function to have less indentation.
2015-09-03 12:16:49 +02:00
Eric V. Smith ab2aa6dc91 Fixed an incorrect comment. 2015-08-26 14:10:32 -04:00
Stefan Krah 5f35725fb8 Merge #15944. 2015-08-08 13:38:59 +02:00
Stefan Krah 0c51595a78 Issue #15944: memoryview: Allow arbitrary formats when casting to bytes.
Original patch by Martin Panter.
2015-08-08 13:38:10 +02:00
Eric Snow 5060bc51ca Merge from 3.5 (issue #24667). 2015-08-07 17:47:35 -06:00
Eric Snow 8c7f9558eb Issue #24667: Resize odict in all cases that the underlying dict resizes. 2015-08-07 17:45:12 -06:00
Eric V. Smith 577d2069da Fix typo in comment. 2015-08-07 18:50:24 -04:00
Raymond Hettinger 4148195c45 Move the active entry multiplication to later in the hash calculation 2015-08-07 00:43:39 -07:00
Raymond Hettinger b501a27ad8 Restore frozenset hash caching removed in cf707dd190a9 2015-08-06 22:15:22 -07:00
Zachary Ware 070bd62cfa Closes #21279: Merge with 3.5 2015-08-06 00:05:13 -05:00
Zachary Ware d987a81d29 Issue #21279: Merge with 3.4 2015-08-06 00:04:23 -05:00
Zachary Ware 79b98df023 Issue #21279: Flesh out str.translate docs
Initial patch by Kinga Farkas, Martin Panter, and John Posner.
2015-08-05 23:54:15 -05:00
Raymond Hettinger a286a51ae1 Fix comment typo 2015-08-01 11:07:11 -07:00
Raymond Hettinger 36c0500990 Tweak the comments 2015-08-01 10:57:42 -07:00
Raymond Hettinger fbffdef47d Issue #24762: Speed-up frozenset_hash() and greatly beef-up the comments. 2015-08-01 09:53:00 -07:00
Raymond Hettinger daffc916aa Issue #24681: Move the most likely test first in set_add_entry(). 2015-07-31 07:58:56 -07:00
Raymond Hettinger 70559b5c20 Issue #24681: Move the store of so->table to the code block where it is used. 2015-07-23 07:42:23 -04:00
Serhiy Storchaka c8fe04484e Issue #23573: Restored optimization of bytes.rfind() and bytearray.rfind()
for single-byte argument on Linux.
2015-07-20 22:58:29 +03:00
Serhiy Storchaka d92d4efe3d Issue #23573: Restored optimization of bytes.rfind() and bytearray.rfind()
for single-byte argument on Linux.
2015-07-20 22:58:02 +03:00
Raymond Hettinger ff9e18a863 Issue #24583: Consolidate previous set object updates into a single function
with a single entry point, named exit points at the bottom, more self-evident
refcount adjustments, and a comment describing why the pre-increment was
necessary at all.
2015-07-20 07:34:05 -04:00
Raymond Hettinger 482c05cbb5 Issue #24583: Fix refcount leak. 2015-07-20 01:23:32 -04:00
Raymond Hettinger 061091a7c5 Issue #24583: Fix crash when set is mutated while being updated. 2015-07-15 23:54:02 -07:00
Raymond Hettinger d702044bcd merge 2015-07-15 23:52:29 -07:00
Raymond Hettinger a3626bc5bd Issue #24583: Fix crash when set is mutated while being updated. 2015-07-15 23:50:14 -07:00
Raymond Hettinger 5d2385ff6f Neaten-up a little bit. 2015-07-08 11:52:27 -07:00
Raymond Hettinger 9632a7d735 Issue 24581: Revert c9782a9ac031 pending a stronger test for mutation during iteration. 2015-07-07 15:29:24 -07:00
Raymond Hettinger 11ce8e6c37 Minor bit of factoring-out common code. 2015-07-06 19:08:49 -07:00
Raymond Hettinger 3dbc11cadd Tighten-up code in the set iterator to use an entry pointer rather than indexing. 2015-07-06 19:03:01 -07:00
Raymond Hettinger ef6bd7d963 Tighten-up code in set_next() to use an entry pointer rather than indexing. 2015-07-06 08:43:37 -07:00
Raymond Hettinger b48d6a63ff Bring related functions add/contains/discard together in the code. 2015-07-05 16:27:44 -07:00
Raymond Hettinger 73799b181e Change add/contains/discard calls to pass the key and hash instead of an entry struct. 2015-07-05 16:06:10 -07:00
Raymond Hettinger dc28d5a198 Clean-up call patterns for add/contains/discard to better match the caller's needs. 2015-07-05 10:03:20 -07:00
Benjamin Peterson 191a8f0a34 merge heads 2015-07-04 22:53:45 -05:00
Benjamin Peterson d34c246baf merge 3.5 (#24407) 2015-07-04 19:59:50 -05:00
Benjamin Peterson e54d5321cc merge 3.4 (#24407) 2015-07-04 19:59:24 -05:00
Benjamin Peterson 2a48a6eb33 merge 3.3 (#24407) 2015-07-04 19:58:11 -05:00
Benjamin Peterson a82f77fb00 protect against mutation of the dict during insertion (closes #24407) 2015-07-04 19:55:16 -05:00
Raymond Hettinger ac2ef65c32 Make the unicode equality test an external function rather than in-lining it.
The real benefit of the unicode specialized function comes from
bypassing the overhead of PyObject_RichCompareBool() and not
from being in-lined (especially since there was almost no shared
data between the caller and callee).  Also, the in-lining was
having a negative effect on code generation for the callee.
2015-07-04 16:04:44 -07:00
Raymond Hettinger e186c7674c Make sure the dummy percentage calculation won't overflow. 2015-07-04 11:28:35 -07:00
Raymond Hettinger c2480dc0c4 Minor cleanup. 2015-07-04 08:46:31 -07:00
Raymond Hettinger b322326f48 Minor nit: Make the style of checking error return values more consistent. 2015-07-03 23:37:16 -07:00
Raymond Hettinger 4897300276 Minor factoring: move redundant resize scaling logic into the resize function. 2015-07-03 20:00:03 -07:00
Raymond Hettinger 3c1f52e829 Call set_lookkey() directly to avoid unnecessary memory spills and reloads. 2015-07-03 18:31:09 -07:00
Raymond Hettinger 15f0869609 Move insertion resize logic into set_insert_key().
Simplifies the code a little bit and does the resize check
only when a new key is added (giving a small speed up in
the case where the key already exists).

Fixes possible bug in set_merge() where the set_insert_key()
call relies on a big resize at the start to make enough room
for the keys but is vulnerable to a comparision callback that
could cause the table to shrink in the middle of the merge.

Also, changed the resize threshold from two-thirds of the
mask+1 to just two-thirds.  The plus one offset gave no
real benefit (afterall, the two-thirds mark is just a
heuristic and isn't a precise cut-off).
2015-07-03 17:21:17 -07:00
Yury Selivanov bb215e2300 Merge 3.5 (Issue #19235) 2015-07-03 01:10:11 -04:00
Yury Selivanov f488fb422a Issue #19235: Add new RecursionError exception. Patch by Georg Brandl. 2015-07-03 01:04:23 -04:00
Yury Selivanov d6d0b5b1fa Merge 3.5 (Issue #24450) 2015-07-03 00:24:14 -04:00
Yury Selivanov e13f8f3cab Issue #24450: Add gi_yieldfrom to generators; cr_await to coroutines.
Patch by Benno Leslie and Yury Selivanov.
2015-07-03 00:23:30 -04:00
Serhiy Storchaka 81da944d61 Issue #24467: Fixed possible buffer over-read in bytearray. The bytearray
object now always allocates place for trailing null byte and it's buffer now
is always null-terminated.
2015-06-29 21:18:55 +03:00
Serhiy Storchaka bc9e75ed02 Issue #24467: Fixed possible buffer over-read in bytearray. The bytearray
object now always allocates place for trailing null byte and it's buffer now
is always null-terminated.
2015-06-29 21:18:01 +03:00
Serhiy Storchaka 7b6e3b91f5 Issue #24467: Fixed possible buffer over-read in bytearray. The bytearray
object now always allocates place for trailing null byte and it's buffer now
is always null-terminated.
2015-06-29 21:14:06 +03:00
Raymond Hettinger 2eff9e9441 Minor refactoring. Move reference count logic into function that adds entry. 2015-06-27 22:03:35 -07:00
Benjamin Peterson b5048d3938 merge 3.5 2015-06-27 15:46:31 -05:00
Benjamin Peterson 4801383c29 upgrade to Unicode 8.0.0 2015-06-27 15:45:56 -05:00
Raymond Hettinger 91672617d5 Minor tweeak to tighten the inner-loop. 2015-06-26 02:50:21 -07:00
Yury Selivanov b257b7993c Merge 3.5 (issue #24439) 2015-06-24 11:04:39 -04:00
Yury Selivanov 66f8828bfc Issue #24439: Improve PEP 492 related docs.
Patch by Martin Panter.
2015-06-24 11:04:15 -04:00
Raymond Hettinger 38bb95e49d Minor code cleanup. 2015-06-24 01:22:19 -07:00
Yury Selivanov 6edc2f7549 Issue #24400: Merge 3.5 2015-06-22 12:31:24 -04:00
Yury Selivanov 5376ba9630 Issue #24400: Introduce a distinct type for 'async def' coroutines.
Summary of changes:

1. Coroutines now have a distinct, separate from generators
   type at the C level: PyGen_Type, and a new typedef PyCoroObject.
   PyCoroObject shares the initial segment of struct layout with
   PyGenObject, making it possible to reuse existing generators
   machinery.  The new type is exposed as 'types.CoroutineType'.

   As a consequence of having a new type, CO_GENERATOR flag is
   no longer applied to coroutines.

2. Having a separate type for coroutines made it possible to add
   an __await__ method to the type.  Although it is not used by the
   interpreter (see details on that below), it makes coroutines
   naturally (without using __instancecheck__) conform to
   collections.abc.Coroutine and collections.abc.Awaitable ABCs.

   [The __instancecheck__ is still used for generator-based
   coroutines, as we don't want to add __await__ for generators.]

3. Add new opcode: GET_YIELD_FROM_ITER.  The opcode is needed to
   allow passing native coroutines to the YIELD_FROM opcode.

   Before this change, 'yield from o' expression was compiled to:

      (o)
      GET_ITER
      LOAD_CONST
      YIELD_FROM

   Now, we use GET_YIELD_FROM_ITER instead of GET_ITER.

   The reason for adding a new opcode is that GET_ITER is used
   in some contexts (such as 'for .. in' loops) where passing
   a coroutine object is invalid.

4. Add two new introspection functions to the inspec module:
   getcoroutinestate(c) and getcoroutinelocals(c).

5. inspect.iscoroutine(o) is updated to test if 'o' is a native
   coroutine object.  Before this commit it used abc.Coroutine,
   and it was requested to update inspect.isgenerator(o) to use
   abc.Generator; it was decided, however, that inspect functions
   should really be tailored for checking for native types.

6. sys.set_coroutine_wrapper(w) API is updated to work with only
   native coroutines.  Since types.coroutine decorator supports
   any type of callables now, it would be confusing that it does
   not work for all types of coroutines.

7. Exceptions logic in generators C implementation was updated
   to raise clearer messages for coroutines:

   Before: TypeError("generator raised StopIteration")
   After: TypeError("coroutine raised StopIteration")
2015-06-22 12:19:30 -04:00
Raymond Hettinger 7e3592dca6 Harmonize the bottom of the outer loop with its entry point
giving a small simplification.  Timings show that hash
pre-check seems only benefit the inner-loop (the linear probes).
2015-06-21 10:47:20 -07:00
Raymond Hettinger 6ee588f14e Restore quick exit (no freeslot check) for common case (found null on first probe). 2015-06-20 21:39:51 -07:00
Serhiy Storchaka ec466a15d9 Fixed indentation of Python examples in C comments. 2015-06-11 00:09:32 +03:00
Serhiy Storchaka 553e156921 Fixed indentation of Python examples in C comments. 2015-06-11 00:07:47 +03:00
Serhiy Storchaka d741a88049 Fixed indentation of Python examples in C comments. 2015-06-11 00:06:39 +03:00
Benjamin Peterson 239e90196e Merge 3.5 2015-06-07 00:00:48 -05:00
Benjamin Peterson 0718de9770 repair my irrational excuberance 2015-06-07 00:00:42 -05:00
Benjamin Peterson 4a931adef3 merge 3.5 2015-06-06 23:20:40 -05:00
Benjamin Peterson 99e96f2bb0 remove unnecessary braces and indentation 2015-06-06 23:20:32 -05:00
Benjamin Peterson 9c474172cb merge 3.5 2015-06-04 14:34:43 -05:00
Benjamin Peterson 2ad80f53c1 fix refleak when keys() fails 2015-06-04 14:34:20 -05:00
Eric Snow f02dcc8803 Merge from 3.5. 2015-06-04 00:12:14 -06:00
Eric Snow 4fabf02633 Issue #24369: Defend against key-changes during iteration. 2015-06-04 00:09:56 -06:00
Eric Snow d8e5d6c4a5 Merge from 3.5. 2015-06-03 11:11:22 -06:00
Eric Snow db4061cb9d Issue #24377: Fix a ref leak in OrderedDict.__repr__. 2015-06-03 11:09:48 -06:00
Eric Snow cbdb17f681 Merge from 3.5. 2015-06-03 10:54:05 -06:00
Eric Snow 4c72918a59 Issue #24362: Simplify the C OrderedDict fast nodes resize logic. 2015-06-03 10:50:37 -06:00
Eric Snow d14de5e22d Merge from 3.5. 2015-06-02 20:43:37 -06:00
Eric Snow ac02ef373f Issue #24368: Support keyword arguments in OrderedDict methods. 2015-06-02 20:42:14 -06:00
Eric Snow ec6cd162de Merge from 3.5. 2015-06-01 23:35:40 -06:00
Eric Snow b952ab43f2 Issue #24359: Check for changed OrderedDict size during iteration. 2015-06-01 23:35:13 -06:00
Eric Snow e9dd08d08c Merge from 3.5. 2015-06-01 23:14:26 -06:00
Eric Snow d171975609 Issue #24348: Drop superfluous increfs/decrefs. 2015-06-01 23:12:13 -06:00
Eric Snow c976b73002 Merge from 3.5. 2015-06-01 23:06:20 -06:00
Eric Snow a762af74b2 Issue #24347: Set KeyError if PyDict_GetItemWithError returns NULL. 2015-06-01 22:59:08 -06:00
Benjamin Peterson cae0658f09 merge 3.5 (#24345) 2015-06-01 10:14:35 -05:00
Benjamin Peterson 0969a9f8ab add Py_tp_finalize slot (closes #24345)
Patch from Petr Viktorin.
2015-06-01 10:12:48 -05:00
Serhiy Storchaka 442a894d1e Issue #24284: The startswith and endswith methods of the str class no longer
return True when finding the empty string and the indexes are completely out
of range.
2015-05-31 09:16:13 +03:00
Serhiy Storchaka d4ea03c785 Issue #24284: The startswith and endswith methods of the str class no longer
return True when finding the empty string and the indexes are completely out
of range.
2015-05-31 09:15:51 +03:00
Eric Snow 7a3414c201 Merge from 3.5. 2015-05-30 12:55:22 -06:00
Eric Snow e3dfa9e5ce Issue #16991: Fix a few leaks and other memory-related concerns in OrderedDict. 2015-05-30 12:51:15 -06:00
Eric Snow 3ef8958e40 Merge from 3.5. 2015-05-30 12:00:05 -06:00
Eric Snow 67fb92e8c6 Issue #16991: Do not return None from OrderedDict.__reversed__. 2015-05-30 11:43:36 -06:00
Eric Snow c5e59609ac Issue #16991: Properly handle return values in several places. 2015-05-30 11:28:56 -06:00
Yury Selivanov 7aa5341164 Reverting my previous commit.
Something went horribly wrong when I was doing `hg rebase`.
2015-05-30 10:57:56 -04:00
Serhiy Storchaka e90982111a Issue #24115: Update uses of PyObject_IsTrue(), PyObject_Not(),
PyObject_IsInstance(), PyObject_RichCompareBool() and _PyDict_Contains()
to check for and handle errors correctly.
2015-05-30 17:48:54 +03:00
Serhiy Storchaka ac5569b1fa Issue #24115: Update uses of PyObject_IsTrue(), PyObject_Not(),
PyObject_IsInstance(), PyObject_RichCompareBool() and _PyDict_Contains()
to check for and handle errors correctly.
2015-05-30 17:48:19 +03:00
Serhiy Storchaka fa494fd883 Issue #24115: Update uses of PyObject_IsTrue(), PyObject_Not(),
PyObject_IsInstance(), PyObject_RichCompareBool() and _PyDict_Contains()
to check for and handle errors correctly.
2015-05-30 17:45:22 +03:00
Serhiy Storchaka 2806518e0c Specify default values of semantic booleans in Argument Clinic generated signatures as booleans. 2015-05-30 11:32:58 +03:00
Serhiy Storchaka 8b2e8b6cce Specify default values of semantic booleans in Argument Clinic generated signatures as booleans. 2015-05-30 11:30:39 +03:00
Eric Snow 96c6af9b20 Issue #16991: Add a C implementation of collections.OrderedDict. 2015-05-29 22:21:39 -06:00
Eric Snow 47db71756d Issue #16991: Add a C implementation of collections.OrderedDict. 2015-05-29 22:21:39 -06:00
Yury Selivanov ac0bffb962 Issue 24017: Drop getawaitablefunc and friends in favor of unaryfunc. 2015-05-28 11:22:41 -04:00
Yury Selivanov 6ef059097c Issue 24017: Drop getawaitablefunc and friends in favor of unaryfunc. 2015-05-28 11:21:31 -04:00
Raymond Hettinger 8651a50475 Issue #23359: Specialize set_lookkey intoa lookup function and an insert function. 2015-05-27 10:37:20 -07:00
Serhiy Storchaka a7a0ad6f73 Issue #24276: Fixed optimization of property descriptor getter. 2015-05-24 21:38:06 +03:00
Nick Coghlan a48db2bc8b Issue #24268: Address some PEP 489 refleaks
- missing DECREF in PyModule_FromDefAndSpec2
- missing DECREF in PyType_FromSpecAndBases2
- missing DECREF in _testmultiphase module

Patch by Petr Viktorin
2015-05-24 01:03:46 +10:00
Nick Coghlan d5cacbb1d9 PEP 489: Multi-phase extension module initialization
Known limitations of the current implementation:

- documentation changes are incomplete
- there's a reference leak I haven't tracked down yet

The leak is most visible by running:

  ./python -m test -R3:3 test_importlib

However, you can also see it by running:

  ./python -X showrefcount

Importing the array or _testmultiphase modules, and
then deleting them from both sys.modules and the local
namespace shows significant increases in the total
number of active references each cycle. By contrast,
with _testcapi (which continues to use single-phase
initialisation) the global refcounts stabilise after
a couple of cycles.
2015-05-23 22:24:10 +10:00
Yury Selivanov 683333955a Issue 24237: Raise PendingDeprecationWarning per PEP 479
Raise PendingDeprecationWarning when generator raises StopIteration
and no __future__ import is used.  Fix offenders in the stdlib
and tests.

See also issue 22906.
Thanks to Nick Coghlan and Berker Peksag for reviews.
2015-05-22 11:16:47 -04:00
Serhiy Storchaka e79ec70801 Issue #24257: Fixed incorrect uses of PyObject_IsInstance().
Fixed segmentation fault in sqlite3.Row constructor with faked cursor type.
Fixed system error in the comparison of faked types.SimpleNamespace.
2015-05-22 11:13:20 +03:00
Serhiy Storchaka 08d230a540 Issue #24257: Fixed incorrect uses of PyObject_IsInstance().
Fixed segmentation fault in sqlite3.Row constructor with faked cursor type.
Fixed system error in the comparison of faked types.SimpleNamespace.
2015-05-22 11:02:49 +03:00
Serhiy Storchaka b2f3c2357c Issue #23985: Fixed integer overflow in iterator object. Patch by
Clement Rouault.
2015-05-21 20:51:53 +03:00
Serhiy Storchaka 4faf5c5655 Issue #23985: Fixed integer overflow in iterator object. Patch by
Clement Rouault.
2015-05-21 20:50:25 +03:00
Antoine Pitrou 873e0df946 Fix some compilation warnings when using gcc (-Wmaybe-uninitialized). 2015-05-19 21:06:04 +02:00
Antoine Pitrou f6d1f1fa8a Fix some compilation warnings when using gcc (-Wmaybe-uninitialized). 2015-05-19 21:04:33 +02:00
Antoine Pitrou ef64847f71 Issue #23985: Fix a possible buffer overrun when deleting a slice from the front of a bytearray and then appending some other bytes data.
Patch by Martin Panter.
2015-05-19 20:55:42 +02:00
Antoine Pitrou 2545411e28 Issue #23985: Fix a possible buffer overrun when deleting a slice from the front of a bytearray and then appending some other bytes data.
Patch by Martin Panter.
2015-05-19 20:52:27 +02:00
Serhiy Storchaka 9ce71a6475 Fixed typos in comments. 2015-05-18 22:20:18 +03:00
Serhiy Storchaka 7e29eea926 Fixed typos in comments. 2015-05-18 22:19:42 +03:00
Raymond Hettinger 740d6134f1 merge 2015-05-17 14:47:00 -07:00
Raymond Hettinger 610a51f364 Issue #23757: Only call the concrete list API for exact lists. 2015-05-17 14:45:58 -07:00
Raymond Hettinger d4be691494 Issue #24064: Help property() support GC 2015-05-13 11:12:33 -07:00
Raymond Hettinger 5af9e13c18 Minor stylistic and consistency cleanup. 2015-05-13 01:44:36 -07:00
Raymond Hettinger 1bd8d75be3 Issue #23290: Optimize set_merge() for cases where the target is empty.
(Contributed by Serhiy Storchaka.)
2015-05-13 01:26:14 -07:00
Raymond Hettinger eac503aeac Issue #24064: Property() docstrings are now writeable.
(Patch by Berker Peksag.)
2015-05-13 01:09:59 -07:00
Serhiy Storchaka 48e47aaa28 Issue #22486: Added the math.gcd() function. The fractions.gcd() function now is
deprecated.  Based on patch by Mark Dickinson.
2015-05-13 00:19:51 +03:00
Serhiy Storchaka 0d4df752ac Issue #15027: The UTF-32 encoder is now 3x to 7x faster. 2015-05-12 23:12:45 +03:00
Yury Selivanov df52e67d85 Fix warnings for gen_get_iter() 2015-05-11 23:23:05 -04:00
Yury Selivanov 7544508f02 PEP 0492 -- Coroutines with async and await syntax. Issue #24017. 2015-05-11 22:57:16 -04:00
Yury Selivanov 18c30a29f8 Issue #22906: Do incref before SetCause/SetContext 2015-05-10 15:09:46 -04:00
Yury Selivanov 80e4dbea0e Issue 22906: Increment refcount after PyException_SetContext 2015-05-09 14:04:17 -04:00
Yury Selivanov 8170e8c0d1 PEP 479: Change StopIteration handling inside generators.
Closes issue #22906.
2015-05-09 11:44:30 -04:00
Benjamin Peterson 025e9ebd0a PEP 448: additional unpacking generalizations (closes #2292)
Patch by Neil Girdhar.
2015-05-05 20:16:41 -04:00
Raymond Hettinger c4e335b67a Issue #23910: Optimize property() getter calls. Patch by Joe Jevnik 2015-04-30 08:08:13 -07:00
Andrew Svetlov 48050cbbe7 Merge 3.4 2015-04-27 17:49:16 +03:00
Andrew Svetlov 9df36c9456 Issue #21354: PyCFunction_New function is exposed by python DLL again. 2015-04-27 17:48:50 +03:00
Antoine Pitrou 7503509f19 Issue #23996: Avoid a crash when a delegated generator raises an unnormalized StopIteration exception. Patch by Stefan Behnel. 2015-04-26 18:48:16 +02:00
Antoine Pitrou 7403e91630 Issue #23996: Avoid a crash when a delegated generator raises an unnormalized StopIteration exception. Patch by Stefan Behnel. 2015-04-26 18:46:40 +02:00
Gregory P. Smith 8cb6569fe1 Implements issue #9951: Adds a hex() method to bytes, bytearray, & memoryview.
Also updates a few internal implementations of the same thing to use the
new built-in code.

Contributed by Arnon Yaari.
2015-04-25 23:22:26 +00:00
Serhiy Storchaka 247789cee9 Issue #24007: Argument Clinic now writes the format of PyArg_Parse*() at the
same line as function name.
2015-04-24 00:40:51 +03:00
Benjamin Peterson a30e2256f7 merge 3.4 (#24044) 2015-04-23 17:06:45 -04:00
Benjamin Peterson bd91ee9cd7 merge 3.3 (#24044) 2015-04-23 17:06:33 -04:00
Benjamin Peterson 51454a62e2 merge 3.2 (#24044) 2015-04-23 17:05:07 -04:00
Benjamin Peterson 0823ffb2fb properly handle malloc failure (closes #24044)
Patch by Christian Heimes.
2015-04-23 17:04:36 -04:00
Benjamin Peterson 4320f54ea3 merge 3.4 (#22785) 2015-04-22 09:16:27 -04:00
Benjamin Peterson ac22c6b923 improved range docstring (closes #22785)
Patch by Ned Batchelder.
2015-04-22 09:16:07 -04:00
Serhiy Storchaka 7e9d1d1a1b Issue #23908: os functions now reject paths with embedded null character
on Windows instead of silently truncate them.

Removed no longer used _PyUnicode_HasNULChars().
2015-04-20 10:12:28 +03:00
Christian Heimes 4e25913f9f Remove local dead code. In both blocks dir is always greater 0. 2015-04-18 05:54:02 +02:00
Steve Dower cb39d1f466 Issue 19933: Provide default argument for ndigits in round. Patch by Vajrasky Kok. 2015-04-15 16:10:59 -04:00
Larry Hastings 89964c48d1 Issue #23944: Argument Clinic now wraps long impl prototypes at column 78. 2015-04-14 18:07:59 -04:00
Zachary Ware 8ef887ce47 Issue #20586: Argument Clinic now ensures signatures on functions without docstrings. 2015-04-13 18:22:35 -05:00
Antoine Pitrou a63cc21234 Issue #23726: Don't enable GC for user subclasses of non-GC types that don't add any new fields.
Patch by Eugene Toder.
2015-04-13 20:10:06 +02:00
Serhiy Storchaka 92e8af67a8 Issue #23492: Argument Clinic now generates argument parsing code with
PyArg_Parse instead of PyArg_ParseTuple if possible.
2015-04-04 00:12:11 +03:00
Serhiy Storchaka 1009bf18b3 Issue #23501: Argumen Clinic now generates code into separate files by default. 2015-04-03 23:53:51 +03:00
Serhiy Storchaka 41525e31a5 Issue #23466: Raised OverflowError if %c argument is out of range. 2015-04-03 20:53:46 +03:00
Serhiy Storchaka be1c60c3f6 Issue #23783: Fixed memory leak in PyObject_ClearWeakRefs() in case of
MemoryError.
2015-03-30 09:52:53 +03:00
Serhiy Storchaka 4aa867959f Issue #23783: Fixed memory leak in PyObject_ClearWeakRefs() in case of
MemoryError.
2015-03-30 09:52:29 +03:00
Serhiy Storchaka 2c7b5a9d0d Issue #23466: %c, %o, %x, and %X in bytes formatting now raise TypeError on
non-integer input.
2015-03-30 09:19:08 +03:00
Serhiy Storchaka 48070c1248 Issue #23803: Fixed str.partition() and str.rpartition() when a separator
is wider then partitioned string.
2015-03-29 19:21:02 +03:00
Victor Stinner 1912b39def _PyUnicodeWriter_WriteStr() now checks that the input string is consistent
in debug mode to detect bugs earlier.

_PyUnicodeWriter_Finish() doesn't check if the read only string is consistent,
whereas it does check consistency for strings built by itself.
2015-03-26 09:37:23 +01:00
Serhiy Storchaka d357b89f0b Issue #22079: Deprecation warning now is issued in PyType_Ready() instead of
raising TypeError when statically allocated type subclasses dynamically
allocated type
2015-03-22 09:46:36 +02:00
Victor Stinner dabbfe7b30 Issue #23573: Fix bytes.rfind() and bytearray.rfind() on Windows
Windows has no memrchr() function.

This change is only a workaround, the optimization must be reenabled on other
platforms.
2015-03-25 03:16:32 +01:00
Serhiy Storchaka d9d769fcdd Issue #23573: Increased performance of string search operations (str.find,
str.index, str.count, the in operator, str.split, str.partition) with
arguments of different kinds (UCS1, UCS2, UCS4).
2015-03-24 21:55:47 +02:00
Victor Stinner 944fbcc478 Issue #23571: Enhance _Py_CheckFunctionResult()
Too bad, sometimes Py_FatalError() is unable to write the exception into
sys.stderr (on "AMD64 OpenIndiana 3.x" buildbot, the buildbot was probably out
of memory).

Call Py_FatalError() with a different message for the two cases (result+error,
or no result and no error).
2015-03-24 16:28:52 +01:00
Victor Stinner 454bd3a277 stdprinter_write(): mention the encoding 2015-03-24 13:40:29 +01:00
Victor Stinner 3b06dfb9d1 Issue #23571: In debug mode, _Py_CheckFunctionResult() now calls
Py_FatalError() instead of using an assertion in debug mode. Py_FatalError()
displays the current exception and the traceback which contain more information
than just the assertion error.
2015-03-24 12:53:59 +01:00
Victor Stinner efde146b0c Issue #23571: _Py_CheckFunctionResult() now gives the name of the function
which returned an invalid result (result+error or no result without error) in
the exception message.

Add also unit test to check that the exception contains the name of the
function.

Special case: the final _PyEval_EvalFrameEx() check doesn't mention the
function since it didn't execute a single function but a whole frame.
2015-03-21 15:04:43 +01:00
Serhiy Storchaka 1dd49824df Issue #23681: The -b option now affects comparisons of bytes with int. 2015-03-20 16:54:57 +02:00
Victor Stinner f50e187724 Fix compiler warnings: comparison between signed and unsigned numbers 2015-03-20 11:32:24 +01:00
Antoine Pitrou 484b9f3b99 Merge 2015-03-20 00:10:23 +01:00
Antoine Pitrou 31084ba528 Issue #23632: Memoryviews now allow tuple indexing (including for multi-dimensional memoryviews). 2015-03-19 23:29:36 +01:00
Victor Stinner 66aab0c4b5 Issue #23708: Add _Py_read() and _Py_write() functions to factorize code handle
EINTR error and special cases for Windows.

These functions now truncate the length to PY_SSIZE_T_MAX to have a portable
and reliable behaviour. For example, read() result is undefined if counter is
greater than PY_SSIZE_T_MAX on Linux.
2015-03-19 22:53:20 +01:00
Serhiy Storchaka 009b811d67 Removed unintentional trailing spaces in non-external and non-generated C files. 2015-03-18 21:53:15 +02:00
Victor Stinner 0c39b1b970 Initialize variables to prevent GCC warnings 2015-03-18 15:02:06 +01:00
Ethan Furman 62e977f1b6 Close issue23467: add %r compatibility to bytes and bytearray 2015-03-11 08:17:00 -07:00
Antoine Pitrou 63afdaa110 Issue #23629: Fix the default __sizeof__ implementation for variable-sized objects. 2015-03-10 22:35:24 +01:00
Antoine Pitrou a654510150 Issue #23629: Fix the default __sizeof__ implementation for variable-sized objects. 2015-03-10 22:32:00 +01:00
Victor Stinner 283f3f8ab3 Issue #23571: Oops, fix #ifdef assert()
assert() are noop when NDEBUG is defined. We want the opposite.
2015-03-09 15:55:37 +01:00
Victor Stinner 4a7cc88472 Issue #23571: PyObject_Call(), PyCFunction_Call() and call_function() now
raise a SystemError if a function returns a result and raises an exception.
The SystemError is chained to the previous exception.

Refactor also PyObject_Call() and PyCFunction_Call() to make them more readable.

Remove some checks which became useless (duplicate checks).

Change reviewed by Serhiy Storchaka.
2015-03-06 23:35:27 +01:00
Benjamin Peterson abda8ddd3e merge 3.4 (#23590) 2015-03-06 09:09:46 -05:00
Benjamin Peterson a915723dc7 fix potential refleak in PyFloat_AsDouble (closes #23590) 2015-03-06 09:08:44 -05:00
Benjamin Peterson 92ce1b4392 merge 3.3 (#23362) 2015-03-02 13:23:41 -05:00
Benjamin Peterson e5a853c390 use PyMem_NEW to detect overflow (closes #23362) 2015-03-02 13:23:25 -05:00
Larry Hastings 42fc0eb827 Issue #21293: Remove unnecessary "capsule hack". 2015-03-02 08:44:51 -08:00
Steve Dower 3e96f324dc Issue #23451: Update pyconfig.h for Windows to require Vista headers and remove unnecessary version checks. 2015-03-02 08:01:10 -08:00
Serhiy Storchaka 71c6f445cd Fixed a typo. 2015-03-01 14:39:20 +02:00
Serhiy Storchaka 490055a167 Issue #20204: Deprecation warning is now raised for builtin type without the
__module__ attribute.
2015-03-01 10:03:02 +02:00
Benjamin Peterson c8fd9ce73d merge 3.4 (#23515) 2015-02-25 10:16:56 -05:00
Benjamin Peterson b808d590a2 fix merge_collapse to actually maintain the invariant it purports to (closes #23515)
See
de Gouw, Stijn and Rot, Jurriaan and de Boer, Frank S and Bubel, Richard and Hähnle, Reiner
"OpenJDK’s java.utils.Collection.sort() is broken: The good, the bad and the worst case"
2015-02-25 10:12:26 -05:00
Serhiy Storchaka 78a8249127 Issue #23490: Fixed possible crashes related to interoperability between
old-style and new API for string with 2**30-1 characters.
2015-02-20 21:34:39 +02:00
Serhiy Storchaka e55181f517 Issue #23490: Fixed possible crashes related to interoperability between
old-style and new API for string with 2**30-1 characters.
2015-02-20 21:34:06 +02:00
Serhiy Storchaka 26861b0b29 Issue #23450: Fixed possible integer overflows. 2015-02-16 20:52:17 +02:00
Serhiy Storchaka 4d0d982985 Issue #23446: Use PyMem_New instead of PyMem_Malloc to avoid possible integer
overflows.  Added few missed PyErr_NoMemory().
2015-02-16 13:33:32 +02:00
Serhiy Storchaka 1a1ff29659 Issue #23446: Use PyMem_New instead of PyMem_Malloc to avoid possible integer
overflows.  Added few missed PyErr_NoMemory().
2015-02-16 13:28:22 +02:00
Serhiy Storchaka ea5ce5a15e Issue #23383: Cleaned up bytes formatting. 2015-02-10 23:23:12 +02:00
Raymond Hettinger 438f9134cf Mirco-optimizations to reduce register spills and reloads observed on CLANG and GCC. 2015-02-09 06:48:29 -06:00
Steve Dower d327d8d9a4 Fix definition mismatch for type_is_subtype_base_chain. 2015-02-06 08:51:26 -08:00
Steve Dower b4e20bb588 Fix definition mismatch for type_is_subtype_base_chain. 2015-02-06 08:50:23 -08:00
Benjamin Peterson 6c62ac1a01 merge 3.4 (#22735) 2015-02-05 22:29:46 -05:00
Benjamin Peterson 104b9e0cca fix many custom mro() edge cases and improve code quality (#22735)
Patch by Eldar Abusalimov.
2015-02-05 22:29:14 -05:00
Raymond Hettinger 8249282622 Minor code clean up. 2015-02-04 08:37:02 -08:00
Raymond Hettinger 06bb1226d1 Issue 23359: Reduce size of code in set_lookkey. Only do linear probes when there is no wrap-around.
Nice simplification contributed by Serhiy Storchaka :-)
2015-02-03 08:15:30 -08:00
Stefan Krah 5178d91be0 Issue #14203: Remove obsolete support for view==NULL in PyBuffer_FillInfo()
and bytearray_getbuffer().  Both functions now raise BufferError in that
case.
2015-02-03 16:57:21 +01:00
Serhiy Storchaka 83848704f5 Issue #22896: Fixed using _getbuffer() in recently added _PyBytes_Format(). 2015-02-03 01:49:18 +02:00
Serhiy Storchaka 3dd3e26680 Issue #22896: Avoid to use PyObject_AsCharBuffer(), PyObject_AsReadBuffer()
and PyObject_AsWriteBuffer().
2015-02-03 01:25:42 +02:00
Serhiy Storchaka 4fdb68491e Issue #22896: Avoid to use PyObject_AsCharBuffer(), PyObject_AsReadBuffer()
and PyObject_AsWriteBuffer().
2015-02-03 01:21:08 +02:00
Raymond Hettinger c658d85487 Issue 23359: Tighten inner search loop for sets (don't and-mask every entry lookup). 2015-02-02 08:35:00 -08:00
Stefan Krah 2e6658b4a7 Merge from 3.4. 2015-02-01 19:19:49 +01:00
Stefan Krah 7213fcc27e Issue #23370: Fix off-by-one error for non-contiguous buffers. 2015-02-01 16:19:23 +01:00
Stefan Krah 363af44a4a Issue #22445: PyBuffer_IsContiguous() now implements precise contiguity
tests, compatible with NumPy's NPY_RELAXED_STRIDES_CHECKING compilation
flag.  Previously the function reported false negatives for corner cases.
2015-02-01 14:53:54 +01:00
Raymond Hettinger 59ecabd12a Keep the definition of i consistent between set_lookkey() and set_insert_clean(). 2015-01-31 02:45:12 -08:00
Raymond Hettinger 9edd753229 Minor tweak to improve code clarity. 2015-01-30 20:09:23 -08:00