Commit Graph

5820 Commits

Author SHA1 Message Date
Martin Panter b93d8637a6 Issue #1621: Avoid signed overflow in list and tuple operations
Patch by Xiang Zhang.
2016-07-25 02:39:20 +00:00
Martin Panter 32d2ce3561 Issue #27581: Merge overflow fix from 3.5 2016-07-25 03:31:29 +00:00
Martin Panter e8db861f47 Issue #27581: Don’t rely on overflow wrapping in PySequence_Tuple()
Patch by Xiang Zhang.
2016-07-25 02:30:05 +00:00
Berker Peksag ced8d4c6eb Issue #27454: Use PyDict_SetDefault in PyUnicode_InternInPlace
Patch by INADA Naoki.
2016-07-25 04:40:39 +03:00
Martin Panter bd09f15b69 Issue #27507: Merge overflow check from 3.5 2016-07-18 08:18:32 +00:00
Martin Panter 371731ebfd Issue #27507: Check for integer overflow in bytearray.extend()
Patch by Xiang Zhang.
2016-07-18 07:53:13 +00:00
Benjamin Peterson ab078e9ed1 Backed out changeset af29d89083b3 (closes #25548) (closes #27498) 2016-07-13 21:13:29 -07:00
Serhiy Storchaka ca0da9b0a3 Issue #27473: Fixed possible integer overflow in bytes and bytearray
concatenations.  Patch by Xiang Zhang.
2016-07-10 20:51:35 +03:00
Serhiy Storchaka 06cfb0cd70 Issue #27473: Fixed possible integer overflow in bytes and bytearray
concatenations.  Patch by Xiang Zhang.
2016-07-10 20:48:43 +03:00
R David Murray ce85acff3a Merge: #20647: Update dictobject.c comments to account for randomized string hashes. 2016-07-10 12:40:03 -04:00
R David Murray 537ad7ad9f #20647: Update dictobject.c comments to account for randomized string hashes.
Patch by Jaysinh Shukla.
2016-07-10 12:33:18 -04:00
Serhiy Storchaka f9efb8b18b Issue #27474: Unified error messages in the __contains__ method of bytes and
bytearray for integers in and out of the Py_ssize_t range.
Patch by Xiang Zhang.
2016-07-10 12:37:30 +03:00
Benjamin Peterson c13d85bf8d merge 3.5 (#27248)
Patch from Xiang Zhang.
2016-07-06 23:28:19 -07:00
Benjamin Peterson 27007dccec fix refleaks in PyDict_SetItem error cases (closes #27248) 2016-07-06 23:26:32 -07:00
Serhiy Storchaka f54d7811c3 Issue #27460: Unified error messages in bytes constructor for integers
in and out of the Py_ssize_t range.  Patch by Xiang Zhang.
2016-07-06 21:39:44 +03:00
Serhiy Storchaka 7e160ce356 Issue #23034: The output of a special Python build with defined COUNT_ALLOCS,
SHOW_ALLOC_COUNT or SHOW_TRACK_COUNT macros is now off by  default.  It can
be re-enabled using the "-X showalloccount" option.  It now outputs to stderr
instead of stdout.
2016-07-03 21:03:53 +03:00
Serhiy Storchaka 6c94d10a19 Issue #27443: __length_hint__() of bytearray itearator no longer return
negative integer for resized bytearray.
2016-07-03 14:42:17 +03:00
Serhiy Storchaka af65872da2 Issue #27443: __length_hint__() of bytearray itearator no longer return
negative integer for resized bytearray.
2016-07-03 14:41:36 +03:00
Serhiy Storchaka e09132f2c7 Backed out changeset b0087e17cd5e (issue #26765)
For unknown reasons it perhaps caused a crash on 32-bit Windows (issue #).
2016-07-03 13:57:48 +03:00
Serhiy Storchaka ab8bcb34dc Issue #26765: Fixed parsing Py_ssize_t arguments on 32-bit Windows. 2016-07-03 13:26:52 +03:00
Serhiy Storchaka 355048970b Issue #26765: Moved wrappers for bytes and bytearray methods to common header
file.
2016-07-01 17:57:30 +03:00
Serhiy Storchaka 0855e706aa Issue #27007: The fromhex() class methods of bytes and bytearray subclasses
now return an instance of corresponding subclass.
2016-07-01 17:22:31 +03:00
Martin Panter 590dcab95a Merge spelling fixes from 3.5 2016-06-20 07:55:14 +00:00
Martin Panter 2275e626b1 Fix spelling errors in code comments 2016-06-20 07:52:50 +00:00
Serhiy Storchaka f95de0e8cc Issue #26754: PyUnicode_FSDecoder() accepted a filename argument encoded as
an iterable of integers. Now only strings and byte-like objects are accepted.
2016-06-18 13:56:16 +03:00
Serhiy Storchaka 9305d83425 Issue #26754: PyUnicode_FSDecoder() accepted a filename argument encoded as
an iterable of integers. Now only strings and byte-like objects are accepted.
2016-06-18 13:53:36 +03:00
Serhiy Storchaka 5d062d7ba3 Issue #27333: Simplified testing step on 0. 2016-06-18 09:51:55 +03:00
Serhiy Storchaka cfdfbb4d3c Issue #27342: Replaced some Py_XDECREFs with Py_DECREFs.
Patch by Xiang Zhang.
2016-06-18 09:44:03 +03:00
Serhiy Storchaka 5697c4b641 Comment fixes extracted from patch by Demur Rumed. 2016-06-12 17:02:10 +03:00
Yury Selivanov 711d25db48 Merge 3.5 (issue #27243) 2016-06-09 15:13:16 -04:00
Yury Selivanov a6f6edbda8 Issue #27243: Fix __aiter__ protocol 2016-06-09 15:08:31 -04:00
Serhiy Storchaka 5dee6551e2 Issue #26305: Argument Clinic now uses braces in C code as required by PEP 7. 2016-06-09 16:16:06 +03:00
Victor Stinner e18719509f odict: Remove useless ";" after function definition
Fix a "gcc -pendatic" warning.
2016-06-08 10:18:18 +02:00
Serhiy Storchaka 841de28035 Issue #27225: Fixed a reference leak in type_new when setting __new__ fails.
Patch by Xiang Zhang.
2016-06-05 10:50:16 +03:00
Serhiy Storchaka 484c913ed9 Issue #27225: Fixed a reference leak in type_new when setting __new__ fails.
Patch by Xiang Zhang.
2016-06-05 10:48:36 +03:00
Kushal Das 5801ecb440 Issue #25548: Showing memory address of class objects in repl 2016-06-04 16:21:13 -07:00
Serhiy Storchaka 64c26e5f60 Issue #20041: Fixed TypeError when frame.f_trace is set to None.
Patch by Xavier de Gaye.
2016-06-04 20:33:33 +03:00
Serhiy Storchaka 64a263a1ce Issue #20041: Fixed TypeError when frame.f_trace is set to None.
Patch by Xavier de Gaye.
2016-06-04 20:32:36 +03:00
Serhiy Storchaka e63e5d60ee Issue #27073: Removed redundant checks in long_add and long_sub.
Patch by Oren Milman.
2016-06-04 00:06:45 +03:00
Serhiy Storchaka 16931c3559 Issue #26983: float() now always return an instance of exact float.
The deprecation warning is emitted if __float__ returns an instance of
a strict subclass of float.  In a future versions of Python this can
be an error.
2016-06-03 21:42:55 +03:00
Martin Panter 0b7d84de6b Issue #27171: Merge typo fixes from 3.5 2016-06-02 10:11:18 +00:00
Martin Panter e26da7c03a Issue #27171: Fix typos in documentation, comments, and test function names 2016-06-02 10:07:09 +00:00
Martin Panter e514093a2f Issue #27125: Merge typo fixes from 3.5 2016-05-30 05:24:49 +00:00
Martin Panter a90a4a9651 Issue #27125: Remove duplicated words from documentation and comments 2016-05-30 04:04:50 +00:00
Martin Panter e501a93c18 Issue #27125: Merge typo fixes from 3.5
Also merge changes from Issue #27117; no actual code changes to 3.6.
2016-05-29 09:05:06 +00:00
Martin Panter 8d56c026a5 Issue #27125: Fix various errors like “will [be] inherited” 2016-05-29 04:13:35 +00:00
Benjamin Peterson 6ca4260219 merge 3.5 2016-05-28 14:05:11 -07:00
Benjamin Peterson c352620c92 backout 3c9512d8ac0d 2016-05-28 14:04:40 -07:00
Serhiy Storchaka 27ba8864a4 Issue #27118: Clean up Py_XINCREF/Py_XDECREF in typeobject.c.
Patch by Xiang Zhang.
2016-05-25 16:14:55 +03:00
Serhiy Storchaka b0f80b0312 Issue #26647: Python interpreter now uses 16-bit wordcode instead of bytecode.
Patch by Demur Rumed.
2016-05-24 09:15:14 +03:00
Serhiy Storchaka 4b23494ded Issue #27039: Fixed bytearray.remove() for values greater than 127.
Based on patch by Joe Jevnik.
2016-05-16 22:24:03 +03:00
Serhiy Storchaka 7bf36dace8 Issue #27039: Fixed bytearray.remove() for values greater than 127.
Patch by Joe Jevnik.
2016-05-16 22:15:38 +03:00
Serhiy Storchaka bcde10aa7e Issue #26765: Ensure that bytes- and unicode-specific stringlib files are used
with correct type.
2016-05-16 09:42:29 +03:00
Benjamin Peterson 99740925ed merge 3.5 2016-05-12 23:14:28 -07:00
Benjamin Peterson 07451ddd4f fix indentation of comments in cellobject.c (closes #27011) 2016-05-12 23:12:21 -07:00
Serhiy Storchaka ea36c941a1 Issue #23640: int.from_bytes() no longer bypasses constructors for subclasses. 2016-05-12 10:37:58 +03:00
Serhiy Storchaka 1f364438ad Issue #23640: int.from_bytes() no longer bypasses constructors for subclasses. 2016-05-12 10:39:32 +03:00
Serhiy Storchaka 5787ef621a Issue #27005: Fixed the call of PyObject_CallFunctionObjArgs(). 2016-05-12 10:32:30 +03:00
Serhiy Storchaka 25885d1dc5 Issue #27005: Optimized the float.fromhex() class method for exact float. 2016-05-12 10:21:14 +03:00
Serhiy Storchaka 17e22959a4 Issue #17765: weakref.ref() no longer silently ignores keyword arguments.
Patch by Georg Brandl.
2016-05-07 15:43:59 +03:00
Serhiy Storchaka 21eb48764c Issue #17765: weakref.ref() no longer silently ignores keyword arguments.
Patch by Georg Brandl.
2016-05-07 15:41:09 +03:00
Serhiy Storchaka fb81d3cbe7 Issue #26765: Moved common code for the replace() method of bytes and bytearray
to a template file.
2016-05-05 09:26:07 +03:00
Serhiy Storchaka dd40fc3e57 Issue #26765: Moved common code and docstrings for bytes and bytearray methods
to bytes_methods.c.
2016-05-04 22:23:26 +03:00
Serhiy Storchaka fcce462e9c Issue #26811: gc.get_objects() no longer contains a broken tuple with NULL
pointer.
2016-05-04 21:42:52 +03:00
Serhiy Storchaka 7822f151b6 Issue #26811: gc.get_objects() no longer contains a broken tuple with NULL
pointer.
2016-05-04 21:42:05 +03:00
Serhiy Storchaka 7a9579c0ce Got rid of redundand "self" parameter declarations.
Argument Clinic is now able to infer all needed information.
2016-05-02 13:45:20 +03:00
Berker Peksag 0ac70c0e90 Fix typos. Reported by andportnoy on GitHub. 2016-04-29 16:54:10 +03:00
Raymond Hettinger 8421d714d0 Removed unused initialization and the uninteresting comment. 2016-04-29 01:37:05 -07:00
Serhiy Storchaka fa070298e9 Issue #26880: Removed redundant checks in set.__init__. 2016-04-29 11:31:52 +03:00
Victor Stinner 71dc3d878a Issue #25349, #26249: Fix memleak in formatfloat() 2016-04-26 12:35:13 +02:00
Victor Stinner 15932593ba Issue #26249: Try test_capi on Windows 2016-04-22 18:52:22 +02:00
Victor Stinner f5c4b99034 PyMem_Malloc() now uses the fast pymalloc allocator
Issue #26249: PyMem_Malloc() allocator family now uses the pymalloc allocator
rather than system malloc(). Applications calling PyMem_Malloc() without
holding the GIL can now crash: use PYTHONMALLOC=debug environment variable to
validate the usage of memory allocators in your application.
2016-04-22 16:26:23 +02:00
Martin Panter abe4d52a1a Merge doc and comment fixes from 3.5 2016-04-19 23:23:16 +00:00
Victor Stinner 0621e0ea86 Don't define _PyMem_PymallocEnabled() if pymalloc is disabled
Isse #26516.
2016-04-19 17:02:55 +02:00
Martin Panter 8f26565ba9 Fix spelling (inital), grammar (may translates) in documentation, comments 2016-04-19 04:03:41 +00:00
Serhiy Storchaka 55c861f637 Issue #26745: Removed redundant code in _PyObject_GenericSetAttrWithDict.
Based on patch by Xiang Zhang.
2016-04-17 20:31:51 +03:00
Serhiy Storchaka b6a9c9761c Issue #26778: Fixed "a/an/and" typos in code comment, documentation and error
messages.
2016-04-17 09:39:28 +03:00
Serhiy Storchaka 6a7b3a77b4 Issue #26778: Fixed "a/an/and" typos in code comment and documentation. 2016-04-17 08:32:47 +03:00
Benjamin Peterson a4ed2daea3 merge 3.5 2016-04-16 15:12:37 -07:00
Benjamin Peterson b0c04cb98c fix slice cache finalization to use gc del 2016-04-16 15:12:29 -07:00
Benjamin Peterson 259d247b31 merge 3.5 (#26659) 2016-04-16 14:54:27 -07:00
Benjamin Peterson 2b601d3905 add gc support to slice (closes #26659) 2016-04-16 14:47:12 -07:00
Berker Peksag 43de36d2c7 Issue #26766: Remove redundant bytearray_format() from bytearrayobject.c 2016-04-16 01:20:47 +03:00
Victor Stinner e914d41312 Issue #26766: Fix _PyBytesWriter_Finish()
Return a bytearray object when bytearray is requested and when the small buffer
is used.

Fix also test_bytes: bytearray%args must return a bytearray type.
2016-04-15 17:52:27 +02:00
Serhiy Storchaka c9a59e6e4f Issue #26764: Fixed SystemError in bytes.__rmod__. 2016-04-15 14:11:10 +03:00
Martin Panter cda80940ed Issue #15984: Merge PyUnicode doc from 3.5 2016-04-15 02:27:11 +00:00
Martin Panter 6245cb3c01 Correct “an” → “a” with “Unicode”, “user”, “UTF”, etc
This affects documentation, code comments, and a debugging messages.
2016-04-15 02:14:19 +00:00
Serhiy Storchaka 21a663ea28 Issue #26057: Got rid of nonneeded use of PyUnicode_FromObject(). 2016-04-13 15:37:23 +03:00
Serhiy Storchaka 131b8f8eee Issue #26718: super.__init__ no longer leaks memory if called multiple times.
NOTE: A direct call of super.__init__ is not endorsed!
2016-04-13 15:28:53 +03:00
Serhiy Storchaka 3d7497608b Issue #26718: super.__init__ no longer leaks memory if called multiple times.
NOTE: A direct call of super.__init__ is not endorsed!
2016-04-13 15:27:33 +03:00
Serhiy Storchaka 59865e7fe1 Issue #26200: Restored more safe usages of Py_SETREF. 2016-04-11 09:57:37 +03:00
Serhiy Storchaka f01e408c16 Issue #26200: Added Py_SETREF and replaced Py_XSETREF with Py_SETREF
in places where Py_DECREF was used.
2016-04-10 18:12:01 +03:00
Serhiy Storchaka 57a01d3a0e Issue #26200: Added Py_SETREF and replaced Py_XSETREF with Py_SETREF
in places where Py_DECREF was used.
2016-04-10 18:05:40 +03:00
Serhiy Storchaka 03f17f8671 Issue #17339: Improved TypeError message in bytes constructor. 2016-04-10 14:44:59 +03:00
Serhiy Storchaka ec39756960 Issue #22570: Renamed Py_SETREF to Py_XSETREF. 2016-04-06 09:50:03 +03:00
Serhiy Storchaka 48842714b9 Issue #22570: Renamed Py_SETREF to Py_XSETREF. 2016-04-06 09:45:48 +03:00
Serhiy Storchaka ab479c49d3 Issue #26494: Fixed crash on iterating exhausting iterators.
Affected classes are generic sequence iterators, iterators of str, bytes,
bytearray, list, tuple, set, frozenset, dict, OrderedDict, corresponding
views and os.scandir() iterator.
2016-03-30 20:41:15 +03:00
Serhiy Storchaka fbb1c5ee06 Issue #26494: Fixed crash on iterating exhausting iterators.
Affected classes are generic sequence iterators, iterators of str, bytes,
bytearray, list, tuple, set, frozenset, dict, OrderedDict, corresponding
views and os.scandir() iterator.
2016-03-30 20:40:02 +03:00
Raymond Hettinger 3625af5f21 Moved misplaced functions to the section for C API functions. 2016-03-27 01:15:07 -07:00
Raymond Hettinger 2c257ab0f8 Responsibility for argument checking belongs in set.__init__() rather than set.__new__().
See dict.__new__() and list.__new__() for comparison.  Neither of those examine or touch
args or kwds.  That work is done in the __init__() methods.
2016-03-26 04:10:11 -07:00
Raymond Hettinger b72e21b9ab Speed-up construction of empty sets by approx 12-14%. 2016-03-25 02:29:59 -07:00
Victor Stinner f963c13597 longobject.c: fix compilation warning on Windows 64-bit
We know that Py_SIZE(b) is -1 or 1 an so fits into the sdigit type.
2016-03-23 18:36:54 +01:00
Victor Stinner 0aed3a4ebc _PyMem_DebugFree(): fix compiler warning on Windows
Don't return a void value.
2016-03-23 11:30:43 +01:00
Victor Stinner fac395681f Optimize bytes.replace(b'', b'.')
Issue #26574: Optimize bytes.replace(b'', b'.') and
bytearray.replace(b'', b'.'): up to 80% faster. Patch written by Josh Snider.
2016-03-21 10:38:58 +01:00
Victor Stinner ad524375af Fail if PyMem_Malloc() is called without holding the GIL
Issue #26563: Debug hooks on Python memory allocators now raise a fatal error
if functions of the PyMem_Malloc() family are called without holding the GIL.
2016-03-16 12:12:53 +01:00
Victor Stinner ffcf1a54d3 Oops, revert unwanted change used to create an example
Issue #26564.
2016-03-15 22:49:40 +01:00
Victor Stinner 0611c26a58 On memory error, dump the memory block traceback
Issue #26564: _PyObject_DebugDumpAddress() now dumps the traceback where a
memory block was allocated on memory block. Use the tracemalloc module to get
the traceback.
2016-03-15 22:22:13 +01:00
Victor Stinner c4aec3628b Check the GIL in PyObject_Malloc()
Issue #26558: The debug hook of PyObject_Malloc() now checks that the GIL is
held when the function is called.
2016-03-14 22:26:53 +01:00
Victor Stinner 34be807ca4 Add PYTHONMALLOC env var
Issue #26516:

* Add PYTHONMALLOC environment variable to set the Python memory
  allocators and/or install debug hooks.
* PyMem_SetupDebugHooks() can now also be used on Python compiled in release
  mode.
* The PYTHONMALLOCSTATS environment variable can now also be used on Python
  compiled in release mode. It now has no effect if set to an empty string.
* In debug mode, debug hooks are now also installed on Python memory allocators
  when Python is configured without pymalloc.
2016-03-14 12:04:26 +01:00
Benjamin Peterson 0f04bc7959 merge 3.5 (closes #26478) 2016-03-03 22:10:52 -08:00
Benjamin Peterson cfc2a1fc70 merge 3.4 (closes #26478) 2016-03-03 22:08:01 -08:00
Benjamin Peterson f11b25b081 properly use the ObjArgs variant of CallMethod in dictview binary operations (closes #26478) 2016-03-03 22:05:36 -08:00
Yury Selivanov d2dc15b26b Merge 3.5 (issue #25888) 2016-03-02 11:31:06 -05:00
Yury Selivanov c724bae51c coroutines: Error when awaiting on coroutine that's being awaited
Issue #25888
2016-03-02 11:30:46 -05:00
Victor Stinner f2192855dd Merge 3.5 2016-03-01 22:07:53 +01:00
Victor Stinner 337986740f Issue #26464: Fix unicode_fast_translate() again
Initialize i variable if the string is non-ASCII.
2016-03-01 21:59:58 +01:00
Victor Stinner 3d9d77a3dc Merge 3.5 2016-03-01 21:30:50 +01:00
Victor Stinner 6c9aa8f2bf Fix str.translate()
Issue #26464: Fix str.translate() when string is ASCII and first replacements
removes character, but next replacement uses a non-ASCII character or a string
longer than 1 character. Regression introduced in Python 3.5.0.
2016-03-01 21:30:30 +01:00
Yury Selivanov 3e48b38dff Merge 3.5 (issue #25887) 2016-02-13 18:00:31 -05:00
Yury Selivanov 77c96813ab Issue #25887: Raise a RuntimeError when a coroutine is awaited more than once. 2016-02-13 17:59:05 -05:00
Yury Selivanov e0b23095ee Issues #26289 and #26315: Optimize floor/modulo div for single-digit longs
Microbenchmarks show 2-2.5x improvement.  Built-in 'divmod' function
is now also ~10% faster.

-m timeit -s "x=22331" "x//2;x//-3;x//4;x//5;x//-6;x//7;x//8;x//-99;x//100;"
with patch: 0.321          without patch: 0.633

-m timeit -s "x=22331" "x%2;x%3;x%-4;x%5;x%6;x%-7;x%8;x%99;x%-100;"
with patch: 0.224          without patch: 0.66

Big thanks to Serhiy Storchaka, Mark Dickinson and Victor Stinner for
thorow code reviews and algorithms improvements.
2016-02-11 10:26:27 -05:00
Martin Panter 7e3a91a5fc Issue #26136: Upgrade the generator_stop warning to DeprecationWarning
Patch by Anish Shah.
2016-02-10 04:40:48 +00:00
Serhiy Storchaka d2962f145a Issue #25949: __dict__ for an OrderedDict instance is now created only when
needed.
2016-02-08 16:39:05 +02:00
Yury Selivanov a0fcaca4e1 Issue #26288: Fix comment 2016-02-06 12:21:33 -05:00
Yury Selivanov 186c30b7ae Issue #26288: Optimize PyLong_AsDouble. 2016-02-05 19:40:01 -05:00
Raymond Hettinger f50215412c Add early-out for the common case where kwds is NULL (gives 1.1% speedup). 2016-02-04 02:46:16 -08:00
Raymond Hettinger ce5179fcba Issue #23601: Use small object allocator for dict key objects 2016-01-31 08:56:21 -08:00
Victor Stinner 5b96f17b1c Merge 3.5 2016-01-27 17:01:13 +01:00
Victor Stinner 5bc03a6d4d Fix resize_compact()
Issue #26217: resize_compact() must set wstr_length to 0 after freeing the wstr
string. Otherwise, an assertion fails in _PyUnicode_CheckConsistency().
2016-01-27 16:56:53 +01:00
Raymond Hettinger 0dceb91866 Tests versus zero are more compact than tests versus -1. 2016-01-25 10:33:30 -08:00
Victor Stinner 3cdd5fb970 code_richcompare() now uses the constants types
Issue #25843: When compiling code, don't merge constants if they are equal but
have a different types. For example, "f1, f2 = lambda: 1, lambda: 1.0" is now
correctly compiled to two different functions: f1() returns 1 (int) and f2()
returns 1.0 (int), even if 1 and 1.0 are equal.

Add a new _PyCode_ConstantKey() private function.
2016-01-22 12:33:12 +01:00
Victor Stinner efb2413ce8 code_richcompare() now uses the constants types
Issue #25843: When compiling code, don't merge constants if they are equal but
have a different types. For example, "f1, f2 = lambda: 1, lambda: 1.0" is now
correctly compiled to two different functions: f1() returns 1 (int) and f2()
returns 1.0 (int), even if 1 and 1.0 are equal.

Add a new _PyCode_ConstantKey() private function.
2016-01-22 12:33:12 +01:00
Victor Stinner 9f78939552 Issue #26107: Fix typo in Objects/lnotab_notes.txt
Double parenthesis
2016-01-21 18:12:29 +01:00
Victor Stinner f3914eb16d co_lnotab supports negative line number delta
Issue #26107: The format of the co_lnotab attribute of code objects changes to
support negative line number delta.

Changes:

* assemble_lnotab(): if line number delta is less than -128 or greater than
  127, emit multiple (offset_delta, lineno_delta) in co_lnotab
* update functions decoding co_lnotab to use signed 8-bit integers

  - dis.findlinestarts()
  - PyCode_Addr2Line()
  - _PyCode_CheckLineNumber()
  - frame_setlineno()

* update lnotab_notes.txt
* increase importlib MAGIC_NUMBER to 3361
* document the change in What's New in Python 3.6
* cleanup also PyCode_Optimize() to use better variable names
2016-01-20 12:16:21 +01:00
Victor Stinner b56837a033 Merge 3.5
Issue #26154: Add a new private _PyThreadState_UncheckedGet() function.
2016-01-20 11:19:46 +01:00
Victor Stinner bfd316e750 Add _PyThreadState_UncheckedGet()
Issue #26154: Add a new private _PyThreadState_UncheckedGet() function which
gets the current thread state, but don't call Py_FatalError() if it is NULL.

Python 3.5.1 removed the _PyThreadState_Current symbol from the Python C API to
no more expose complex and private atomic types. Atomic types depends on the
compiler or can even depend on compiler options. The new function
_PyThreadState_UncheckedGet() allows to get the variable value without having
to care of the exact implementation of atomic types.

Changes:

* Replace direct usage of the _PyThreadState_Current variable with a call to
  _PyThreadState_UncheckedGet().
* In pystate.c, replace direct usage of the _PyThreadState_Current variable
  with the PyThreadState_GET() macro for readability.
* Document also PyThreadState_Get() in pystate.h
2016-01-20 11:12:38 +01:00
Serhiy Storchaka 4918b47c64 Issue #25935: Garbage collector now breaks reference loops with OrderedDict. 2016-01-19 14:48:42 +02:00
Serhiy Storchaka d205d0145c Issue #25935: Garbage collector now breaks reference loops with OrderedDict. 2016-01-19 14:46:25 +02:00
Benjamin Peterson 2c028492d4 merge 3.5 (#25731) 2016-01-18 21:18:35 -08:00
Benjamin Peterson 9ad11544bf set tp_new from the class in the hierarchy that actually owns the descriptor (closes #25731)
Debugging by Eryk Sun.
2016-01-18 21:11:18 -08:00
doko@ubuntu.com 3b63dc3aa9 - merge 3.5 2016-01-14 14:05:21 +01:00
doko@ubuntu.com 17210f50d2 dictobject.c(dict_sizeof): Make it static again. 2016-01-14 14:04:59 +01:00
Barry Warsaw 3d220cc4d7 As per further discussion, re-enable the typeobject.c guard for picklability.
upstreams such as Cython will have to adjust.
2016-01-11 15:51:27 -05:00
Barry Warsaw 09880c89e9 Comment out some tests that won't pass now that we've reverted the
picklability regression.  Also, as per further discussion, remove the
regressing code.
2016-01-11 15:14:53 -05:00
Barry Warsaw ec71f1779f Issue #22995: [UPDATE] Comment out the one of the pickleability tests in
_PyObject_GetState() due to regressions observed in Cython-based projects.
2016-01-11 13:49:40 -05:00
Barry Warsaw 885e1939be Issue #22995: [UPDATE] Comment out the one of the pickleability tests in
_PyObject_GetState() due to regressions observed in Cython-based projects.
2016-01-11 13:24:02 -05:00
Serhiy Storchaka 576f132b98 Issue #20440: Cleaning up the code by using Py_SETREF. 2016-01-05 21:27:54 +02:00
Benjamin Peterson 3e47a1337c merge 3.5 2016-01-01 11:56:35 -06:00
Benjamin Peterson 4e3dd51396 merge 3.4 2016-01-01 11:56:16 -06:00
Benjamin Peterson 630329e4ea merge 3.3 2016-01-01 11:55:47 -06:00
Benjamin Peterson 0e617e22f0 remove some copyright notices supserseded by the toplevel ones 2016-01-01 11:53:47 -06:00
Serhiy Storchaka 288cb25f1a Issue #25961: Disallowed null characters in the type name.
Simplified testing for null characters in __name__ setter.
2015-12-30 21:41:53 +02:00
Serhiy Storchaka 42bf8fc901 Issue #25961: Disallowed null characters in the type name.
Simplified testing for null characters in __name__ setter.
2015-12-30 21:40:49 +02:00
Serhiy Storchaka 4e6aad1f7a Clean up float.as_integer_ratio(). 2015-12-29 22:55:48 +02:00
Serhiy Storchaka 0d250bc119 Issue #25971: Optimized creating Fractions from floats by 2 times and from
Decimals by 3 times.
Unified error messages in float.as_integer_ratio(), Decimal.as_integer_ratio(),
and Fraction constructors.
2015-12-29 22:34:23 +02:00
Serhiy Storchaka 1ed017ae92 Issue #20440: Cleaning up the code by using Py_SETREF and Py_CLEAR.
Old code is correct, but with Py_SETREF and Py_CLEAR it can be cleaner.
This patch doesn't fix bugs and hence there is no need to backport it.
2015-12-27 15:51:32 +02:00
Serhiy Storchaka 726fc139a5 Issue #20440: More use of Py_SETREF.
This patch is manually crafted and contains changes that couldn't be handled
automatically.
2015-12-27 15:44:33 +02:00
Serhiy Storchaka 191321d11b Issue #20440: More use of Py_SETREF.
This patch is manually crafted and contains changes that couldn't be handled
automatically.
2015-12-27 15:41:34 +02:00
Serhiy Storchaka bdb908ea54 Issue #20440: Applied yet one patch for using Py_SETREF.
The patch is automatically generated, it replaces the code that uses Py_CLEAR.
2015-12-27 12:38:28 +02:00
Serhiy Storchaka 4a1e70fc31 Issue #20440: Applied yet one patch for using Py_SETREF.
The patch is automatically generated, it replaces the code that uses Py_CLEAR.
2015-12-27 12:36:18 +02:00
Serhiy Storchaka f9253c96fd Issue #22995: Instances of extension types with a state that aren't
subclasses of list or dict and haven't implemented any pickle-related
methods (__reduce__, __reduce_ex__, __getnewargs__, __getnewargs_ex__,
or __getstate__), can no longer be pickled.  Including memoryview.
2015-12-25 21:05:35 +02:00
Serhiy Storchaka f81be8aa3f Issue #22995: Instances of extension types with a state that aren't
subclasses of list or dict and haven't implemented any pickle-related
methods (__reduce__, __reduce_ex__, __getnewargs__, __getnewargs_ex__,
or __getstate__), can no longer be pickled.  Including memoryview.
2015-12-25 21:04:29 +02:00
Serhiy Storchaka ef1585eb9a Issue #25923: Added more const qualifiers to signatures of static and private functions. 2015-12-25 20:01:53 +02:00
Serhiy Storchaka 2d06e84455 Issue #25923: Added the const qualifier to static constant arrays. 2015-12-25 19:53:18 +02:00
Serhiy Storchaka f006940351 Issue #20440: Massive replacing unsafe attribute setting code with special
macro Py_SETREF.
2015-12-24 10:39:57 +02:00
Serhiy Storchaka 5a57ade58e Issue #20440: Massive replacing unsafe attribute setting code with special
macro Py_SETREF.
2015-12-24 10:35:59 +02:00
Serhiy Storchaka 3987e26e75 Issue #25914: Fixed and simplified OrderedDict.__sizeof__. 2015-12-22 08:22:05 +02:00
Serhiy Storchaka 0ce7a3a34c Issue #25914: Fixed and simplified OrderedDict.__sizeof__. 2015-12-22 08:16:18 +02:00
Serhiy Storchaka 3182db356c Issue #25766: Special method __bytes__() now works in str subclasses. 2015-12-20 16:37:21 +02:00
Serhiy Storchaka 5aac3ed799 Issue #25766: Special method __bytes__() now works in str subclasses. 2015-12-20 16:36:34 +02:00
Serhiy Storchaka a9406e77fa Issue #25421: __sizeof__ methods of builtin types now use dynamic basic size.
This allows sys.getsize() to work correctly with their subclasses with
__slots__ defined.
2015-12-19 20:07:11 +02:00
Serhiy Storchaka 5c4064e8bd Issue #25421: __sizeof__ methods of builtin types now use dynamic basic size.
This allows sys.getsize() to work correctly with their subclasses with
__slots__ defined.
2015-12-19 20:05:25 +02:00
Serhiy Storchaka 42bb126f0a Issue #25899: Converted Objects/listsort.txt to UTF-8.
Original patch by Chris Angelico.
2015-12-18 13:23:33 +02:00
Serhiy Storchaka 9b3a2eec1c Issues #25890, #25891, #25892: Removed unused variables in Windows code.
Reported by Alexander Riccio.
2015-12-18 10:03:13 +02:00
Raymond Hettinger e4495877dd Minor tweek. Counting down rather than up reduces register pressure. 2015-12-15 00:42:30 -08:00
Raymond Hettinger 86d322f020 Undo inadvertent line swap 2015-12-13 19:27:17 -08:00
Raymond Hettinger 5088f6005f Hoist constant expressions (so->table and so->mask) out of the inner-loop. 2015-12-13 18:45:01 -08:00
Serhiy Storchaka 7c088a9b5c Issue #25709: Fixed problem with in-place string concatenation and utf-8 cache. 2015-12-03 01:05:52 +02:00
Serhiy Storchaka 6648bf5661 Issue #25709: Fixed problem with in-place string concatenation and utf-8 cache. 2015-12-03 01:04:37 +02:00
Serhiy Storchaka 31b9410654 Issue #25709: Fixed problem with in-place string concatenation and utf-8 cache. 2015-12-03 01:02:03 +02:00
Serhiy Storchaka 7aa690860e Issue #25709: Fixed problem with in-place string concatenation and utf-8 cache. 2015-12-03 01:02:03 +02:00
Serhiy Storchaka 6a50e79130 Issue #24097: Fixed crash in object.__reduce__() if slot name is freed inside
__getattr__.  Original patch by Antoine Pitrou.
2015-11-25 18:35:15 +02:00
Serhiy Storchaka 670d78abc5 Issue #24097: Fixed crash in object.__reduce__() if slot name is freed inside
__getattr__.  Original patch by Antoine Pitrou.
2015-11-25 18:34:19 +02:00
Serhiy Storchaka d28bb624d1 Issue #24097: Fixed crash in object.__reduce__() if slot name is freed inside
__getattr__.  Original patch by Antoine Pitrou.
2015-11-25 18:33:29 +02:00
Serhiy Storchaka bb6e4a0b31 Issue #24731: Fixed crash on converting objects with special methods
__bytes__, __trunc__, and __float__ returning instances of subclasses of
bytes, int, and float to subclasses of bytes, int, and float correspondingly.
2015-11-25 15:53:19 +02:00
Serhiy Storchaka f9afda57ad Issue #24731: Fixed crash on converting objects with special methods
__bytes__, __trunc__, and __float__ returning instances of subclasses of
bytes, int, and float to subclasses of bytes, int, and float correspondingly.
2015-11-25 15:52:04 +02:00
Serhiy Storchaka 15095800a3 Issue #24731: Fixed crash on converting objects with special methods
__bytes__, __trunc__, and __float__ returning instances of subclasses of
bytes, int, and float to subclasses of bytes, int, and float correspondingly.
2015-11-25 15:47:01 +02:00
Victor Stinner b4efc963d6 Issue #25557: Refactor _PyDict_LoadGlobal()
Don't fallback to PyDict_GetItemWithError() if the hash is unknown: compute the
hash instead. Add also comments to explain the optimization a little bit.
2015-11-20 09:24:02 +01:00
Raymond Hettinger 66f6238fca Add assertion to verify the pre-condition in the comments. 2015-11-17 20:58:43 -08:00
Raymond Hettinger 6019c8ced0 Issue #25629: Move set fill/used updates out of inner loop 2015-11-17 08:28:07 -08:00
Benjamin Peterson d798dc1034 merge 3.5 (#25630) 2015-11-15 21:57:50 -08:00
Benjamin Peterson a4d33b3428 make the PyUnicode_FSConverter cleanup set the decrefed argument to NULL (closes #25630) 2015-11-15 21:57:39 -08:00
Serhiy Storchaka 413fdcea21 Issue #24821: Refactor STRINGLIB(fastsearch_memchr_1char) and split it on
STRINGLIB(find_char) and STRINGLIB(rfind_char) that can be used independedly
without special preconditions.
2015-11-14 15:42:17 +02:00
Serhiy Storchaka 1010921c71 Issue #25462: The hash of the key now is calculated only once in most
operations in C implementation of OrderedDict.
2015-11-13 15:18:26 +02:00
Serhiy Storchaka 19a70e7f5d Issue #25462: The hash of the key now is calculated only once in most
operations in C implementation of OrderedDict.
2015-11-13 14:48:36 +02:00
Serhiy Storchaka 12ab296f82 Issue #22995: Default implementation of __reduce__ and __reduce_ex__ now
rejects builtin types with not defined __new__.
Added tests for non-pickleable types.
2015-11-12 11:34:39 +02:00
Serhiy Storchaka 609a2e17ad Issue #22995: Default implementation of __reduce__ and __reduce_ex__ now
rejects builtin types with not defined __new__.
Added tests for non-pickleable types.
2015-11-12 11:31:51 +02:00
Serhiy Storchaka d7a4415599 Issue #22995: Default implementation of __reduce__ and __reduce_ex__ now
rejects builtin types with not defined __new__.
Added tests for non-pickleable types.
2015-11-12 11:23:04 +02:00
Stefan Krah e2cf984384 Merge 3.5. 2015-11-10 18:35:47 +01:00
Stefan Krah 7c3f3859a5 Fix Visual Studio warning. 2015-11-10 18:35:19 +01:00
Stefan Krah 3407cc26cb Merge 3.5. 2015-11-10 18:18:07 +01:00
Stefan Krah 0ce5b6e268 Iaaue #25598: Fix memory_hex from #9951 for non-contiguous buffers. 2015-11-10 18:17:22 +01:00
Martin Panter 9513ba3b8f Issue #24802: Merge null termination fixes from 3.5 2015-11-07 03:15:32 +00:00
Martin Panter 61d6e4ae9d Issue #24802: Merge null termination fixes from 3.4 into 3.5 2015-11-07 02:56:11 +00:00
Martin Panter eeb896c411 Issue #24802: Copy bytes-like objects to null-terminated buffers if necessary
This avoids possible buffer overreads when int(), float(), compile(), exec()
and eval() are passed bytes-like objects. Similar code is removed from the
complex() constructor, where it was not reachable.

Patch by John Leitch, Serhiy Storchaka and Martin Panter.
2015-11-07 02:32:21 +00:00
Serhiy Storchaka 0c009bf3b5 Issue #25410: Made testing that od_fast_nodes and dk_entries are in sync more
reliable.
2015-11-06 12:00:43 +02:00
Serhiy Storchaka 97f46db904 Issue #25410: Made testing that od_fast_nodes and dk_entries are in sync more
reliable.
2015-11-06 12:00:03 +02:00
Serhiy Storchaka 3a811f272a Issue #24726: Revert setting the value on the dict if
_odict_add_new_node() fails.
2015-11-06 11:07:31 +02:00
Serhiy Storchaka d5f353ec8d Issue #24726: Revert setting the value on the dict if
_odict_add_new_node() fails.
2015-11-06 11:07:11 +02:00
Serhiy Storchaka 6980be40bd Issue #25558: Refactoring OrderedDict iteration. 2015-11-06 10:40:05 +02:00
Serhiy Storchaka 9c967611e3 Issue #25558: Refactoring OrderedDict iteration. 2015-11-06 10:39:51 +02:00
Victor Stinner e20310fa19 Issue #25556: Add assertions to PyObject_GetItem() to ensure that an exception
is raised when it returns NULL.

Simplify also ceval.c: rely on the fact that PyObject_GetItem() raised an
exception when it returns NULL.
2015-11-05 13:56:58 +01:00
Serhiy Storchaka f83afa508f Issue #25449: Fixed a crash and leaking NULL in repr() of OrderedDict that
was mutated by direct calls of dict methods.
2015-11-04 22:33:33 +02:00
Serhiy Storchaka 710cd34bdb Issue #25449: Fixed a crash and leaking NULL in repr() of OrderedDict that
was mutated by direct calls of dict methods.
2015-11-04 22:33:07 +02:00
Serhiy Storchaka 161b6ff19c Issue #25449: Iterating OrderedDict with keys with unstable hash now raises
KeyError in C implementations as well as in Python implementation.

Added tests for OrderedDict subclasses.
2015-11-04 22:08:52 +02:00
Serhiy Storchaka b45b7b2137 Issue #25449: Iterating OrderedDict with keys with unstable hash now raises
KeyError in C implementations as well as in Python implementation.

Added tests for OrderedDict subclasses.
2015-11-04 22:05:38 +02:00
Serhiy Storchaka 4a7c03aab4 Issue #25523: Merge a-to-an corrections from 3.5. 2015-11-02 14:44:29 +02:00
Serhiy Storchaka a84f6c3dd3 Issue #25523: Merge a-to-an corrections from 3.4. 2015-11-02 14:39:05 +02:00
Serhiy Storchaka d65c9496da Issue #25523: Further a-to-an corrections. 2015-11-02 14:10:23 +02:00
Martin Panter e56a919100 Issue #25523: Merge a-to-an corrections from 3.5 2015-11-02 04:27:17 +00:00
Martin Panter 2eb819f7a8 Issue #25523: Merge "a" to "an" fixes from 3.4 into 3.5 2015-11-02 04:04:57 +00:00
Martin Panter 7462b64911 Issue #25523: Correct "a" article to "an" article
This changes the main documentation, doc strings, source code comments, and a
couple error messages in the test suite. In some cases the word was removed
or edited some other way to fix the grammar.
2015-11-02 03:37:02 +00:00
Serhiy Storchaka 9b69491901 Issue #25395: Fixed crash when highly nested OrderedDict structures were
garbage collected.
2015-11-01 16:13:45 +02:00
Serhiy Storchaka 14eefe353e Issue #25395: Fixed crash when highly nested OrderedDict structures were
garbage collected.
2015-11-01 16:12:34 +02:00
Serhiy Storchaka afbea7af02 Issue #25410: C implementation of OrderedDict now uses type(self) instead of
self.__class__ in __repr__() and __reduce__() for simplicity and reliability.
2015-10-22 20:19:14 +03:00
Serhiy Storchaka 4575beba4b Issue #25410: C implementation of OrderedDict now uses type(self) instead of
self.__class__ in __repr__() and __reduce__() for simplicity and reliability.
2015-10-22 20:18:24 +03:00
Serhiy Storchaka 60a08929af Issue #25410: Fixed a memory leak in OrderedDict in the case when key's hash
calculation fails.
2015-10-20 18:22:36 +03:00
Serhiy Storchaka d17427b7bd Issue #25410: Fixed a memory leak in OrderedDict in the case when key's hash
calculation fails.
2015-10-20 18:21:48 +03:00
Serhiy Storchaka 3e9f87782e Issue #25410: Cleaned up and fixed minor bugs in C implementation of OrderedDict. 2015-10-18 09:54:42 +03:00
Serhiy Storchaka 8003bafd7f Issue #25410: Cleaned up and fixed minor bugs in C implementation of OrderedDict. 2015-10-18 09:53:17 +03:00
Victor Stinner 91108f049f Issue #25210: Change error message of do_richcompare()
Don't add parenthesis to type names. Add also quotes around the type names.

Before:

  TypeError: unorderable types: int() < NoneType()

After:

  TypeError: '<' not supported between instances of 'int' and 'NoneType'
2015-10-14 18:25:31 +02:00
Serhiy Storchaka dbb98c1443 Issue #25406: Fixed a bug in C implementation of OrderedDict.move_to_end()
that caused segmentation fault or hang in iterating after moving several
items to the start of ordered dict.
2015-10-14 19:22:44 +03:00
Serhiy Storchaka 992ec46acc Issue #25406: Fixed a bug in C implementation of OrderedDict.move_to_end()
that caused segmentation fault or hang in iterating after moving several
items to the start of ordered dict.
2015-10-14 19:21:24 +03:00
Victor Stinner c3d2bc19e4 Use _PyBytesWriter in _PyBytes_FromIterator() 2015-10-14 14:15:49 +02:00
Victor Stinner c5c3ba4bec Add _PyBytesWriter_Resize() function
This function gives a control to the buffer size without using min_size.
2015-10-14 13:56:47 +02:00
Victor Stinner 3c50ce39bf Factorize _PyBytes_FromList() and _PyBytes_FromTuple() code using a C macro 2015-10-14 13:50:40 +02:00
Victor Stinner f2eafa323b Split PyBytes_FromObject() into subfunctions 2015-10-14 13:44:29 +02:00
Victor Stinner 2ec8063cc9 Modify _PyBytes_DecodeEscapeRecode() to use _PyBytesAPI
* Don't overallocate by 400% when recode is needed: only overallocate on demand
  using _PyBytesWriter.
* Use _PyLong_DigitValue to convert hexadecimal digit to int
* Create _PyBytes_DecodeEscapeRecode() subfunction
2015-10-14 13:32:13 +02:00
Victor Stinner 1285e5c805 Fix compiler warnings (uninitialized variables), false alarms in fact 2015-10-14 12:10:20 +02:00
Victor Stinner f6358a7e4c _PyBytesWriter_Alloc(): only use 10 bytes of the small buffer in debug mode to
enhance code to detect buffer under- and overflow.
2015-10-14 12:02:39 +02:00
Victor Stinner f091033b14 Issue #25401: Remove now unused hex_digit_to_int() function 2015-10-14 11:59:46 +02:00
Victor Stinner 2bf8993db9 Optimize bytes.fromhex() and bytearray.fromhex()
Issue #25401: Optimize bytes.fromhex() and bytearray.fromhex(): they are now
between 2x and 3.5x faster. Changes:

* Use a fast-path working on a char* string for ASCII string
* Use a slow-path for non-ASCII string
* Replace slow hex_digit_to_int() function with a O(1) lookup in
  _PyLong_DigitValue precomputed table
* Use _PyBytesWriter API to handle the buffer
* Add unit tests to check the error position in error messages
2015-10-14 11:25:33 +02:00
Victor Stinner 772b2b09f2 Optimize bytearray % args
Issue #25399: Don't create temporary bytes objects: modify _PyBytes_Format() to
create work directly on bytearray objects.

* Rename _PyBytes_Format() to _PyBytes_FormatEx() just in case if something
  outside CPython uses it
* _PyBytes_FormatEx() now uses (char*, Py_ssize_t) for the input string, so
  bytearray_format() doesn't need tot create a temporary input bytes object
* Add use_bytearray parameter to _PyBytes_FormatEx() which is passed to
  _PyBytesWriter, to create a bytearray buffer instead of a bytes buffer

Most formatting operations are now between 2.5 and 5 times faster.
2015-10-14 09:56:53 +02:00
Victor Stinner 661aaccf9d Add use_bytearray attribute to _PyBytesWriter
Issue #25399: Add a new use_bytearray attribute to _PyBytesWriter to use a
bytearray buffer, instead of using a bytes object.
2015-10-14 09:41:48 +02:00
Victor Stinner 199c9a6f4b Fix long_format_binary()
Issue #25399: Fix long_format_binary(), allocate bytes for the bytes writer.
2015-10-14 09:47:23 +02:00
Victor Stinner 03dab786b2 Rewrite PyBytes_FromFormatV() using _PyBytesWriter API
* Add much more unit tests on PyBytes_FromFormatV()
* Remove the first loop to compute the length of the output string
* Use _PyBytesWriter to handle the bytes buffer, use overallocation
* Cleanup the code to make simpler and easier to review
2015-10-14 00:21:35 +02:00
Victor Stinner 358af13526 Issue #25353: Optimize unicode escape and raw unicode escape encoders to use
the new _PyBytesWriter API.
2015-10-12 22:36:57 +02:00
Victor Stinner e9aa5950bb Fix compilation error in _PyBytesWriter_WriteBytes() on Windows 2015-10-12 13:57:47 +02:00
Victor Stinner 6c2cdae9e6 Writer APIs: use empty string singletons
Modify _PyBytesWriter_Finish() and _PyUnicodeWriter_Finish() to return the
empty bytes/Unicode string if the string is empty.
2015-10-12 13:29:43 +02:00
Victor Stinner c29e29bed1 Relax _PyBytesWriter API
Don't require _PyBytesWriter pointer to be a "char *". Same change for
_PyBytesWriter_WriteBytes() parameter.

For example, binascii uses "unsigned char*".
2015-10-12 13:12:54 +02:00
Serhiy Storchaka 0d554d7ef1 Issue #24164: Objects that need calling ``__new__`` with keyword arguments,
can now be pickled using pickle protocols older than protocol version 4.
2015-10-10 22:42:18 +03:00
Victor Stinner 0cdad1e2bc Issue #25349: Add fast path for b'%c' % int
Optimize also %% formater.
2015-10-09 22:50:36 +02:00
Victor Stinner be75b8cf23 Issue #25349: Optimize bytes % int
Optimize bytes.__mod__(args) for integere formats: %d (%i, %u), %o, %x and %X.
_PyBytesWriter is now used to format directly the integer into the writer
buffer, instead of using a temporary bytes object.

Formatting is between 30% and 50% faster on a microbenchmark.
2015-10-09 22:43:24 +02:00
Victor Stinner 6bd525b656 Optimize error handlers of ASCII and Latin1 encoders when the replacement
string is pure ASCII: use _PyBytesWriter_WriteBytes(), don't check individual
character.

Cleanup unicode_encode_ucs1():

* Rename repunicode to rep
* Clear rep object on error
* Factorize code between bytes and unicode path
2015-10-09 13:10:05 +02:00
Victor Stinner ce179bf6ba Add _PyBytesWriter_WriteBytes() to factorize the code 2015-10-09 12:57:22 +02:00
Victor Stinner ad7715891e _PyBytesWriter: simplify code to avoid "prealloc" parameters
Substract preallocate bytes from min_size before calling
_PyBytesWriter_Prepare().
2015-10-09 12:38:53 +02:00
Victor Stinner 53926a1ce2 _PyBytesWriter: rename size attribute to min_size 2015-10-09 12:37:03 +02:00
Victor Stinner fa7762ec06 Issue #25349: Optimize bytes % args using the new private _PyBytesWriter API
* Thanks to the _PyBytesWriter API, output smaller than 512 bytes are allocated
  on the stack and so avoid calling _PyBytes_Resize(). Because of that, change
  the default buffer size to fmtcnt instead of fmtcnt+100.
* Rely on _PyBytesWriter algorithm to overallocate the buffer instead of using
  a custom code. For example, _PyBytesWriter uses a different overallocation
  factor (25% or 50%) depending on the platform to get best performances.
* Disable overallocation for the last write.
* Replace C loops to fill characters with memset()
* Add also many comments to _PyBytes_Format()
* Remove unused FORMATBUFLEN constant
* Avoid the creation of a temporary bytes object when formatting a floating
  point number (when no custom formatting option is used)
* Fix also reference leaks on error handling
* Use Py_MEMCPY() to copy bytes between two formatters (%)
2015-10-09 11:48:06 +02:00
Victor Stinner b3653a3458 Issue #25318: cleanup code _PyBytesWriter
Rename "stack buffer" to "small buffer".

Add also an assertion in _PyBytesWriter_GetPos().
2015-10-09 03:38:24 +02:00
Victor Stinner 3fa36ff5e4 Issue #25318: Fix backslashreplace()
Fix code to estimate the needed space.
2015-10-09 03:37:11 +02:00
Victor Stinner 797485e101 Issue #25318: Avoid sprintf() in backslashreplace()
Rewrite backslashreplace() to be closer to PyCodec_BackslashReplaceErrors().

Add also unit tests for non-BMP characters.
2015-10-09 03:17:30 +02:00
Victor Stinner b13b97d3b8 Issue #25318: Fix compilation error
Replace "#if Py_DEBUG" with "#ifdef Py_DEBUG".
2015-10-09 02:52:16 +02:00
Victor Stinner 0016507c16 Issue #25318: Move _PyBytesWriter to bytesobject.c
Declare also the private API in bytesobject.h.
2015-10-09 01:53:21 +02:00
Victor Stinner e7bf86cd7d Optimize backslashreplace error handler
Issue #25318: Optimize backslashreplace and xmlcharrefreplace error handlers in
UTF-8 encoder. Optimize also backslashreplace error handler for ASCII and
Latin1 encoders.

Use the new _PyBytesWriter API to optimize these error handlers for the
encoders. It avoids to create an exception and call the slow implementation of
the error handler.
2015-10-09 01:39:28 +02:00
Victor Stinner fdfbf78114 Issue #25318: Add _PyBytesWriter API
Add a new private API to optimize Unicode encoders. It uses a small buffer
allocated on the stack and supports overallocation.

Use _PyBytesWriter API for UCS1 (ASCII and Latin1) and UTF-8 encoders. Enable
overallocation for the UTF-8 encoder with error handlers.

unicode_encode_ucs1(): initialize collend to collstart+1 to not check the
current character twice, we already know that it is not ASCII.
2015-10-09 00:33:49 +02:00
Martin Panter 585a6acfef Merge typo fixes from 3.5 2015-10-07 11:13:55 +00:00
Martin Panter ec1aa5c2a1 More typos in 3.5 documentation and comments 2015-10-07 11:03:53 +00:00
Martin Panter 3f930dcd87 Merge typo fixes from 3.4 into 3.5 2015-10-07 11:01:47 +00:00
Martin Panter 9955a373a8 Various minor typos in documentation and comments 2015-10-07 10:26:23 +00:00
Benjamin Peterson cdae2cb88a merge 3.5 (closes #24806) 2015-10-06 19:42:46 -07:00
Benjamin Peterson 59dc696821 merge 3.4 (#24806) 2015-10-06 19:42:02 -07:00
Benjamin Peterson bd6c41a185 prevent unacceptable bases from becoming bases through multiple inheritance (#24806) 2015-10-06 19:36:54 -07:00
Victor Stinner 74e8fac3c8 Issue #25301: Fix compatibility with ISO C90 2015-10-05 13:49:26 +02:00
Victor Stinner 1d65d9192d Issue #25301: The UTF-8 decoder is now up to 15 times as fast for error
handlers: ``ignore``, ``replace`` and ``surrogateescape``.
2015-10-05 13:43:50 +02:00
Victor Stinner eb36fdaad8 Fix _PyUnicodeWriter_PrepareKind()
Initialize kind to 0 (PyUnicode_WCHAR_KIND) to ensure that
_PyUnicodeWriter_PrepareKind() handles correctly read-only buffer: copy the
buffer.
2015-10-03 01:55:51 +02:00
Serhiy Storchaka 29e68edbf4 Issue #24848: Fixed bugs in UTF-7 decoding of misformed data:
1. Non-ASCII bytes were accepted after shift sequence.
2. A low surrogate could be emitted in case of error in high surrogate.
3. In some circumstances the '\xfd' character was produced instead of the
replacement character '\ufffd' (due to a bug in _PyUnicodeWriter).
2015-10-02 13:14:03 +03:00
Serhiy Storchaka 58c8f2bb6d Issue #24848: Fixed bugs in UTF-7 decoding of misformed data:
1. Non-ASCII bytes were accepted after shift sequence.
2. A low surrogate could be emitted in case of error in high surrogate.
3. In some circumstances the '\xfd' character was produced instead of the
replacement character '\ufffd' (due to a bug in _PyUnicodeWriter).
2015-10-02 13:13:14 +03:00
Serhiy Storchaka 28b21e50c8 Issue #24848: Fixed bugs in UTF-7 decoding of misformed data:
1. Non-ASCII bytes were accepted after shift sequence.
2. A low surrogate could be emitted in case of error in high surrogate.
2015-10-02 13:07:28 +03:00
Serhiy Storchaka 5dbe245ef2 Issue #24483: C implementation of functools.lru_cache() now calculates key's
hash only once.
2015-10-02 12:47:59 +03:00
Serhiy Storchaka b9d98d532c Issue #24483: C implementation of functools.lru_cache() now calculates key's
hash only once.
2015-10-02 12:47:11 +03:00
Victor Stinner 3222da26fe Make _PyUnicode_TranslateCharmap() symbol private
unicodeobject.h exposes PyUnicode_TranslateCharmap() and PyUnicode_Translate().
2015-10-01 22:07:32 +02:00
Victor Stinner 01ada3996b Issue #25267: The UTF-8 encoder is now up to 75 times as fast for error
handlers: ``ignore``, ``replace``, ``surrogateescape``, ``surrogatepass``.
Patch co-written with Serhiy Storchaka.
2015-10-01 21:54:51 +02:00
Victor Stinner d69dd8bd5e (Merge 3.5) Issue #25182: Fix compilation on Windows 2015-09-30 15:03:50 +02:00
Victor Stinner ae86da9b20 (Merge 3.4) Issue #25182: Fix compilation on Windows 2015-09-30 15:03:31 +02:00
Victor Stinner 89719e1daf Issue #25182: Fix compilation on Windows
Restore also errno value before calling PyErr_SetFromErrno().
2015-09-30 15:01:34 +02:00
Serhiy Storchaka 85c386dee4 Issue #25182: The stdprinter (used as sys.stderr before the io module is
imported at startup) now uses the backslashreplace error handler.
2015-09-30 15:51:01 +03:00
Serhiy Storchaka 008fc77e1e Issue #25182: The stdprinter (used as sys.stderr before the io module is
imported at startup) now uses the backslashreplace error handler.
2015-09-30 15:50:32 +03:00
Serhiy Storchaka a59018c7ab Issue #25182: The stdprinter (used as sys.stderr before the io module is
imported at startup) now uses the backslashreplace error handler.
2015-09-30 15:46:53 +03:00
Victor Stinner c3713e9706 Optimize ascii/latin1+surrogateescape encoders
Issue #25227: Optimize ASCII and latin1 encoders with the ``surrogateescape``
error handler: the encoders are now up to 3 times as fast.

Initial patch written by Serhiy Storchaka.
2015-09-29 12:32:13 +02:00
Victor Stinner 0030cd52da Issue #25227: Cleanup unicode_encode_ucs1() error handler
* Change limit type from unsigned int to Py_UCS4, to use the same type than the
  "ch" variable (an Unicode character).
* Reuse ch variable for _Py_ERROR_XMLCHARREFREPLACE
* Add some newlines for readability
2015-09-24 14:45:00 +02:00
Victor Stinner 54385b206d Issue #24870: revert unwanted change
Sorry, I pushed the patch on the UTF-8 decoder by mistake :-(
2015-09-22 10:46:52 +02:00
Victor Stinner 5ebae87628 Issue #25207, #14626: Fix my commit.
It doesn't work to use #define XXX defined(YYY)" and then "#ifdef XXX"
to check YYY.
2015-09-22 01:29:33 +02:00
Victor Stinner 6174474bea _PyUnicodeWriter_PrepareInternal(): make the assertion more strict 2015-09-22 01:01:17 +02:00
Victor Stinner ca9381ea01 Issue #24870: Add _PyUnicodeWriter_PrepareKind() macro
Add a macro which ensures that the writer has at least the requested kind.
2015-09-22 00:58:32 +02:00
Victor Stinner 5014920cb7 Issue #24870: Reuse the new _Py_error_handler enum
Factorize code with the new get_error_handler() function.

Add some empty lines for readability.
2015-09-22 00:26:54 +02:00
Victor Stinner f96418de05 Issue #24870: Optimize the ASCII decoder for error handlers: surrogateescape,
ignore and replace. Initial patch written by Naoki Inada.

The decoder is now up to 60 times as fast for these error handlers.

Add also unit tests for the ASCII decoder.
2015-09-21 23:06:27 +02:00
Victor Stinner 026977717e Merge 3.5 2015-09-19 13:39:16 +02:00
Victor Stinner 5783fd2c58 Issue #24999: In longobject.c, use two shifts instead of ">> 2*PyLong_SHIFT" to
avoid undefined behaviour when LONG_MAX type is smaller than 60 bits.

This change should fix a warning with the ICC compiler.
2015-09-19 13:39:03 +02:00
Victor Stinner 058258652a Merge 3.5 (pytime, odict) 2015-09-18 13:55:15 +02:00
Victor Stinner 4a0d1e7c36 odictobject.c: fix compiler warning
PyObject_Length() returns a P_ssize_t, not an int. Use a Py_ssize_t to avoid
overflow.
2015-09-18 13:44:11 +02:00